Monday, 11th December 2017
We answer a customer query on how to evaluate the value for money and quality of off-the-shelf market research reports.
Jinfo Consulting recently answered a customer query about a publisher of market research reports.
Our customer had never heard of this particular provider and needed to make a quick decision about the value of the report in question - a niche report in the pharmaceutical field, projected at 30 pages and costing around US$4,500. For that amount, the publisher was also offering up to 40 hours of analyst time to customise the data.
"We've had these questions from [internal] customers before, and it's very hard for us to evaluate the reports if we don't know the provider," she explained. "The customer wants the report so we purchase it, but then it turns out to be worthless. The customer is upset with us for not knowing this ahead of time, and we're out a fair bit of money.
"How do we make a sound decision, quickly, when we don't have time to do a lot of research?"
She sent me the materials the publisher had provided, along with copies of emails they exchanged about methodology and coverage.
Asking about methodology is certainly the first step in vetting a research provider; the publisher's description of the methodology was pretty vague, however, with claims of "proprietary algorithms" preventing provision of more detail.
So I took a closer look at the samples, marketing materials, website and other online resources.
Here's our quick list of tasks for getting an initial sense of the value of a market research provider:
In this case, I found red flags everywhere:
The website did not list analysts or provide any validating information about the quality of the people doing the research. I found numerous typos and broken links when I dug beyond the initial pages.
Employee comments and listings on Glassdoor and LinkedIn were primarily for marketing staff, most of whom described their role as relating to search engine optimisation: they represent a significant investment in making the website discoverable via search engine.
The few analysts I found as employees on LinkedIn were relatively junior - in the first three years of their careers. Most employees I found on either site indicated they had been with the company less than a year.
The collateral and sample materials included typos but not bylines. There were no testimonials from satisfied customers. One brochure claimed that "advanced analysis using Excel" was part of their process - an odd claim to make in 2017, when advanced market research techniques would be relying on more sophisticated tools.
Finally, my gut reaction on pricing was negatively impacted by the promise of 40 hours of analyst time. If all they provided was the analyst time for the quoted price of $4,500, the hourly rate comes to US$112.50. Either they don't expect many customers to use this time, or these are extremely junior analysts who will be conducting the work.
This quick review took around 20 minutes to complete, with a bit more time to document the results for our client. She was able to bring this information to her internal customer, who was grateful to not waste the money.
Further discussion on the topic
At the Content Buyers' Breakfast hosted by the Leadership and Management Division at the SLA annual conference last June, we had a very interesting discussion about exactly these types of publishers and the concerns of content buyers.
Some of the concerns raised:
Bringing these concerns to an internal customer can kick off valuable discussions about what they truly need and what they're willing to spend in order to get it. What would make US$4,500 "money well-spent"? You might be able to come up with a different source or approach to meet their needs, thus performing as a strategic partner rather than a transaction centre.
Have a question for Jinfo Consulting? Learn more about our capabilities, then start the conversation today.
Related Subscription Reports:
"Remote Information Teams"
If you're newly remote, you may feel like you've been thrown into a mysterious culture where you sort of speak the language but sort of not. Use Jinfo's "Focus on Remote Information Teams" to become fluent:
Our proven processes, resources and guidance will help your team make the shift from transaction centre to strategic asset.
Designed around the most common challenges and pain points for time- and resource-strapped information teams
Supercharge remote productivity and value
Holistic content portfolio management
Future-proof your information service
A tailored overview of our research and active discussion with your Jinfo analyst.
Measure your starting point to articulate your strengths and set priorities for future improvements. Assessments gauge risk, capacity, value and more.
Read case studies, and start the conversation:
Connect your team with the practical tools, original research and expertise to build and support information strategy in your organisation.
A Jinfo Subscription gives access to all Content (articles, reports, webinars) and Community.