Solving Problems With Insights

I believe that often to solve a problem we need new insights. But what are insights? I’ve looked up this word in a number of different sources and this is what I came up with:

Insights are the hidden nature of things; the cause and effect relationship that is not obvious to the naked eye.


That’s a bit abstract, I know, but my main takeaway from all of my research is that for someone to have had ‘insight’ into something, they must have seen something that the rest of us don’t see when looking at the same situation. What they see may not necessarily be the insight itself, but the situation may be sufficiently puzzling to warrant research until you discover the underlying cause: the insight.

I found that Freakonomics presented a good example of this. The authors described a situation where juvenile crime was declining despite experts’ beliefs that it would increase. The experts concluded that crime was declining because the economy was booming. The authors, on the other hand, studied the same data and concluded that the economy had boomed before and juvenile crime did not decline so something else must be at work. What they found was that years prior, legislation was passed that allowed women to have abortions. This was particular important for women that were drug addicts, or were social outcasts for other reasons, whose children often were the ones becoming juvenile criminals. The country had now reached a point where these children would be juveniles, but since they were never born, there was a significant decline in potential criminals. Without this insight, we may have believed that the solution to crime was to grow the economy.

So how is it that experts looked at the same data as the authors but did not get the same insights? My theory is that sometimes we see what we expect to see.

Consider this example: profits in your business are declining at the same time that the economy is in a recession. All over the news everyone is blaming lower profits on the decline in economic activity. So, you conclude that we’re doing the best that we can given that the economy is tanking. You looked at your data and you saw what you expected to see: a relationship between lower economic activity and lower profits. Unless the economy improves and your business does not, you may never dig deeper.

Or what about tourism in Barbados? We were told back in 2009 that the tourism industry was declining because of the global economic slowdown. We said okay, that makes sense. But here we are in 2014, the industry is still struggling and the global economy is not. What now?

So how do we know that what we believe to be the cause of a situation really is the cause? Test your assumptions. You may be right, in which case you can speak with more conviction. You may be wrong, in which case you dig deeper.

The unfortunate thing with this solution is that it assumes that you have the data you need to test the assumption. In our business example, if your business does not collect information on revenue by product, location, sales agent, etc. it may not be possible to see if for example you lost a key sales agent and that is why profits are falling. And if you only collect information on tourist arrivals by market and there is no further segmentation into customer preferences, you may never discover that closing a key hotel was the main reason why the number of tourists declined.

The data challenge is not only what is collected, it is also the length of time for which it has been collected. Few companies can boast that they have detailed customer and accounting data from the time they started the company. Some systems simply do not allow that type of storage. It’s hard to pick up patterns with short histories. Suppose your company existed back in 2002 when the world’s economic growth slipped in the wake of 9/11 but you have no data from that far back. How would you be able to tell whether profits slipped during the last recession too?

And so, here are my conclusions:

  • ‘Insights’ can only be ‘insights’ if they are correct
  • We can only prove that they are correct if we conduct the necessary tests
  • We can only test our assumptions if we have data
  • And that brings me to my fundamental challenge: how do we get new insights in the Caribbean with no data?