On Questionnaires and Briefings: Explaining the GigaOm Policy Change

A stitch in time saves nine, they say, and so can receiving information in the right order.

We at GigaOm are constantly looking to make our research processes more efficient and more effective. Vendors often tell us what it’s like to work with us—we welcome these interactions and look to address every comment (so thank you for these!). We spent a good part of 2023 on driving far-reaching improvements in our processes, and we’re building on that in the knowledge that better efficiency leads to higher quality research at lower cost, as well as happier analysts and vendors!

That’s why we’re making a small yet necessary change to our briefings process. Historically, we’ve asked vendors to complete a questionnaire and/or schedule a briefing call, and we haven’t specified the order these should take place. The small tweak is to request that vendors first complete a questionnaire, THEN participate in a call to clarify details.

In practice, this means we will enforce receiving a completed questionnaire 24 hours before a scheduled briefing call. Should we not receive it within this timeframe, we will reschedule the briefing so the questionnaire can be completed and reviewed prior to the call. Analysts need time to review vendor responses before a briefing, so getting the questionnaire five minutes before won’t cut it.

As well as fostering efficiency on both sides, the broader reasons for this change are founded in our engineering-led evaluation approach, which reflects how an end-user organization might conduct an RFP process. What we do is to set out a number of decision criteria we expect products to possess, then ask for evidence to show these features are in fact present.

Briefings are actually an inefficient mechanism for delivering that information; the questionnaire is far better at giving us what we need to know to assess whether and how a product delivers on our criteria. Briefings should supplement the questionnaire, giving analysts an opportunity to ask follow-up questions about vendor responses which will cut down on unnecessary back-and-forth during fact check.

Briefings also have their own distractions. Keep in mind that we care less about market positioning and more about product capability. General briefings (outside of the research cycle) are a great place to set out strategy, have the logo slide, run through case studies, and all that. We love those general briefings, but the research cycle is the wrong moment for the big tent stuff (which often exists as a prerecorded video that we’d be happy to review, just not as part of a report briefing call).

I’ve often told vendors we’re not looking for all the bling during briefings. In the best cases, our engineers engage with your engineers about the key features of your products. We don’t need trained spokespeople as much as an honest conversation about functionality and use cases—10 minutes on a video call can clarify something that reams of marketing material, and user documentation cannot. Hence the change.

This shouldn’t add any extra time to the process—the opposite, in fact, as briefings are more productive when the questionnaire is already in place. We can reduce costly errors, decrease back-and-forth clarifications, and minimize misinterpretation (with the consequent potential backlash on AR, “how did you let them write that?”).

So, there you have it. We’ll be rolling out this change in early June for our September reports, so nothing will happen in a rush. Any questions or concerns, please do let us know—we’re constantly adjusting timeframes based on national holidays, industry conferences, and competitor cycles, and we welcome all input on events that might impact delivery.

We are looking at other ways we can improve efficiency, notably simplifying or reformatting the questionnaire, so watch this space for details—and we welcome any thoughts you may have! We also understand that logistics can be tough: we are all juggling time, resources, and people to enable research to happen.

We absolutely recognize the symbiosis between analysts and vendors, and we thoroughly appreciate the efforts made by AR teams on our behalf, to enable these interactions to happen—from familiarization with GigaOm and explaining our value, through negotiating the minefield of operational logistics! Our door is always open if you need anyone to help support your endeavors, as we work toward a win-win for all.

The post On Questionnaires and Briefings: Explaining the GigaOm Policy Change appeared first on Gigaom.