The key challenge in business intelligence (BI), as in life, is to reconcile opposites. In life, we need to know when to follow our head and when to heed our heart; when to resist change and when to accept it; and when to admonish our friends and family and when to comfort them. And so on. Every situation calls for a different tactic, and sometimes, we must apply both tactics at once. This is not schizophrenic; it's just common sense.
Things are not much different in the world of BI. We are often faced with seemingly contradictory needs and desires, such as: speed versus standards; flexibility versus security; business versus IT; departments versus enterprise; and ad hoc access versus highly tailored views. (See figure 1.)
Reconciling these opposites is not easy; in fact, it's quite stressful, both emotionally and intellectually. It's much easier to take a "principled" stand and choose one position over the other. With this approach, the arguments are clear and the rationale valid, but the results are always suboptimal. Fixating on one option in a continuous spectrum may work for a while and gain a lot of attention, but it eventually makes things worse than when you started.
Analytical Silos versus Enterprise BI. For instance, an organization may consolidate all departmental data marts and reporting systems into a central, corporate BI group to eliminate data silos and deliver inconsistent data. But if it's not careful, the corporate BI team can easily become a development bottleneck, causing frustrated department heads to hire their own BI staff under the radar and start building analytical silos. In the end, the organization pays double the amount for BI than previously and still doesn't eliminate analytical silos. In other words, embracing a rigid, single-track approach to solving a problem makes things worse. The remedy is worse than the disease.
Hedgehogs Versus Foxes
In his book "The Signal and the Noise: Why So Many Predictions Fail--But Some Don't", Nate Silver borrows an analogy to describe the way two types of pundits forecast the future.
"Hedgehogs", he says, are "type A personalities who believe in Big Ideas--in governing principles about the world that behave as though they were physical laws and undergird virtually every interaction in society. Think Karl Marx ... or Sigmund Freud...." In contrast, Silver says that "foxes" are "scrappy creatures who believe in ... taking a multitude of approaches toward a problem. They tend to be more tolerant of nuance, uncertainty, complexity, and dissenting opinion."
Research shows that "foxes" are considerably better at forecasting than hedgehogs," says Silver. In fact, hedgehogs' forecasts are barely any better than random chance. But "foxes", who see both sides of the coin and recognize how noisy the data can be, make much better predictions, he says.
The moral of the story is that the more fixed and rigid we are in solving problems, the less we succeed. The hedgehog may claim that he abides by long-held "principles"--which justifies his intransigence in the face of new information that indicate the need for a new approach or way of thinking. A fox, on the other hand, who takes a more flexible and sees both sides of an issue--as difficult and stressful as that is--often fares better in the long run.
As BI managers, we need to embrace the opposites that threaten our BI programs. Rather than make compromises between conflicting requirements and groups, we need to think outside of the box and reconcile disparate needs with a new approach that is attune with the realities on the ground. Rather than see the world in black and white, we need to recognize the value of black, white, and everything in between. The way to address challenges is to keep an open mind and remember that the world doesn't operate in absolutes, but shades of gray. Opposites do attract.
Posted September 4, 2013 12:22 PM
Permalink | No Comments |