Filling in a blank puzzle: What it takes to produce a mallowstreet Insights report
Pardon the Interruption
This article is just an example of the content available to mallowstreet members.
On average over 150 pieces of new content are published from across the industry per month on mallowstreet. Members get access to the latest developments, industry views and a range of in-depth research.
All the content on mallowstreet is accredited for CPD by the PMI and is available to trustees for free.
Imagine a puzzle with all blank pieces. All you know is that when it’s done, it’ll have a castle on it, a moat and some mountains in the background. But you have no idea what they would look like, or what else would be in the puzzle, until you colour in the pieces one by one.
This is a great analogy for how the research process works at mallowstreet Insights. Here are the steps we take and how we do things differently.
Initial research helps us decide what puzzles we want to work on
My day starts and ends with research:
Every morning, I look at the previous day’s market activity – it frames everything else I read that day.
During my lunch break, I go through 30+ articles about the institutional market, including everything that our clients post on mallowstreet. I log any asset class specific coverage in a database, which helps me identify trends.
I also look out for new fund launches and hires in institutional asset management and try to understand what they are telling us about the strategic focus of the companies involved.
Every few weeks, I tally up the topics discussed at our digital and in-person events, as well as those requested by trustees.
I also log the top 10 monthly search terms on mallowstreet and keep an eye out for changes.
Most importantly, I listen – to the research conversations and casual chats I have with everyone who has given me the gift of their time and thoughts so far (thank you!).
Getting the questions right is vital to getting puzzle pieces that fit together
As we prepare to launch a survey, I have to take everything I know about the topic and distil it to the 30 most vital questions I believe our clients should be asking UK pension funds.
I take input from everyone, including our CEO Stuart Breyer and the client, but it is ultimately my responsibility to ensure the questions are:
Unbiased – they do not fish for a newsworthy statistic but prompt a meaningful response, which often means we shy away from ‘should’ and ‘yes/no’ questions
Unambiguous – the questions should tackle a single topic and the answer choices should match it, without grouping related topics together
Relevant – the questions are tailored to the UK pensions industry and its specific challenges
All this is vital to ensuring the blank puzzle pieces we end up with fit together – there is nothing more defeating than having 100 qualified responses, but to the wrong questions.
We then test the questionnaire rigorously, both from a tech and a timing perspective. We want to ensure it is easy to navigate for our respondents and does not take more than 10-15 minutes.
This is where the mallowstreet community gets to work. The relationships we have built with 2,600+ trustees and advisors allow us to ask for their help, and we are humbled by their support.
We typically don’t have to contact everyone. We rely on detailed data to decide how many people we need to contact via each channel, in order to guarantee the target number of responses.
Knowing our audience means that almost all responses we get are qualified. That does not stop us from validating them: ensuring everyone is who they say they are and have provided meaningful responses.
Protecting the anonymity of our respondents is paramount, so we anonymise the data set after cleaning it. This means we remove any information that identifies our respondents or the organisations they work for and replace it with anonymous IDs. We store the personal data separately, behind multiple passwords.
Data analysis allows us to colour in the pieces of the blank puzzle
It’s then time for the data to tell me its story, and I approach it without any pre-conceptions. With thousands of primary data points, there is a lot of noise initially, so I tally up the answers to each question to get a sense of opinions.
I then segment the data in various ways and look for the most meaningful differences between groups. This approach has been so useful that we are currently developing a proprietary data analysis tool powered by Python.
Each finding helps me colour in a piece of the blank puzzle, but I still don’t know how it will come together. About halfway in this process, I start trying to put the puzzle together. I might get one side done or find that a few pieces fit together without knowing where they belong.
The more pieces I have coloured in, the clearer it becomes which pieces are missing, so that allows me to ask the data some very precise questions. Sometimes I get such an important piece of the puzzle that way that it makes me rearrange the rest.
Piece by piece, a final picture appears. I then show it to my colleagues and listen to their feedback. Often, I have to swap some puzzle pieces to make the image sharper – other times, it’s fine as it is.
Writing a report means others can put the puzzle together too
Once the puzzle is complete, I write a report to tell its story to a broader audience:
The client’s marketing team, who want to know the crucial needs of their target market
The sales team, who want to prioritise opportunities and strengthen their pitch
The product development team, who want more relevant and differentiated products
The PR team, who want to tell the broad market about the company’s unique strengths
I make specific recommendations for each team and back them up with report data. I test different charts to find the best way to present the most relevant data points from hundreds of data tables.
Sometimes, the client has additional questions that we can answer with the existing data, so we create a supplementary report for them. Other times, the report raises new questions, so we conduct a series of interviews to add more colour to the findings. But more on that later!
How does your company go about understanding the needs of UK pension schemes? Commentwithin our online community or send us an email on email@example.com.