In an Age When ‘Story Telling’ is king, Is Yours Based on Substance?
As researchers, we love a good quote. We use them to emphasize a point in the findings. They allow us to tell stories and dimensionalize our implications. But sometimes quotes taken from research can take on a life of their own. Without the right context, they can be misinterpreted. It might be difficult for clients to trust that a verbatim is representative of the sample (let alone the greater population). And then there’s bias … how easy it is for anyone human to create a biased narrative with just a little data and some imagination.
Yes, a biased narrative. The plague that can hit anyone in the research field. Researchers must mitigate the risk of biases in their research. They work hard to avoid selection bias, especially challenging within segmentation research but necessary. They avoid cognitive bias, analytical bias. And then there’s confirmation bias.
Need a refresher? Confirmation bias is the tendency to interpret new evidence as confirmation of one’s existing beliefs or theories. With little information, an observer can create a very complex narrative about motivations and emotions that might not be accurately representative of the research. (An example of how easy it is to create narratives is the Heider-Simmel experiment [Heider-Simmel Animation]. Go ahead, watch and reflect).
How does this relate to open ended survey questions?
Traditionally, researchers must read — or must rely on others to read and report to them — every open-ended response in a study and create a narrative of the key point and themes contained in that unstructured text. Their reports include verbatims or quotes from respondents that support their findings and conclusions. That’s why text analytics is so helpful. A researcher, who knows all about biases and does the best they can to mitigate them, can rely on the very statistical measures they tout to help them do their work better. With text analytics researchers, can find the important themes in the open-ended responses and quantify/validate that data.
Text analytics – and in particular OdinText – allows researchers that want to add both power and validity to their selected verbatims. Avoid bias by confirming that the stories they tell from the unstructured data are actually supported by the data.
In an age where so much emphasis is put on story telling, let’s not forget that stories are just that unless they are supported by data.
Next time you find yourself overwhelmed with unstructured data, and are tempted to just “tell a story”, please reach out. I’d love to show you with your own data how modern text analytics can ensure your story is based on fact!
Tim Lynch
