We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

The State of Healthcare Business Intelligence

Originally published January 19, 2011

When I was working in a healthcare organization and had to hire consultants, I would seek out consultants that had healthcare experience. I had learned the hard way that teaching someone the nuances of healthcare while trying to get them to be efficient was not a hill I wanted to climb. I didn’t want to have to pay them to learn on my time. I just wanted to get the work done. The learning curve is significant, and trying to explain all of the pressures associated with this work to someone not familiar with healthcare adds significantly to the length of any project.

Caution: Learning Curve Ahead

So, what has to be learned? First, there are significant regulatory pressures. I have a “Summary of Health Insurance Plan Regulation” (see Figure 1) from America’s Health Insurance Plans (AHIP) hanging in my office. According to AHIP, in 2008 there were 18 agencies to which health insurance companies had to report. Each of these agencies requires a myriad of reports with their own deadlines and requisites, and that doesn’t include the latest flood of regulatory requirements from the current administration. On the provider side, a client recently told me that his department was responsible for more than 1,200 reports that went out to different regulatory agencies. Those are just the reports that they have to complete to meet base requirements of these agencies, and things like reimbursements and accreditations are held hostage by each agency until requirements are met. Again, these are just the basic reports that must be completed to be in compliance with various regulations. What about the reports and analysis that an organization can use to create additional value? How do you find time to do that when you have 1,200 regulatory reports to complete?

Figure 1: Summary of Health Insurance Plan Regulation
Source: AHIP
(Note: For a more readable version of Figure 1, click here to download the PDF from AHIP.)

Now let’s talk about the data itself. If I work at a retailer and misinterpret data, I might miss a sales opportunity or place the wrong product on an endcap. A similar mistake in healthcare can have broad implications for payments and reimbursements.

Trying to get everyone on the same page in healthcare is an enormous task. Because of all of the regulatory requirements, when you start talking data governance and a common corporate lexicon, you also have to remember that how, for example, CMS (Centers for Medicare and Medicaid Services) defines “patient” or “member” meets their needs first, and those needs do change. An organization can attempt to align with whatever CMS says (and many do), only to find that it really doesn’t align with how they see the rest of their business. That can have huge ramifications if you have a large commercial population (e.g., patients that are reimbursed through a payer such as Aetna).

This merely scratches the surface of the demands placed on healthcare organizations and the downstream effects these demands have on their day-to-day work. As far as I can tell, there are no other industries that have the same market pressures as healthcare. It is critical when working on business intelligence in healthcare to know and understand the multiplicity and uniqueness of the demands placed on these organizations. Just because someone has visited a doctor or read the benefits guide from his employer, that doesn’t make him an expert in healthcare.

Healthcare Business Intelligence: State of the Union

To be clear, many healthcare payers have been using business intelligence for years. In the realm of healthcare business intelligence (BI), payer data is relatively easy because it’s primarily claims data. Claims data is repeatable; and although it’s messy, it beats clinical data from a quality perspective. The challenges with clinical data are that much of it is qualitative, such as nurse notes. In order to glean value from their data, the organization has to put the data in context. If the data is qualitative, it takes a person with a clinical background to put that data into context. Therefore, extract, transform and load (ETL) is a challenge for many healthcare organizations. Because there are many implications to ETL for healthcare, we will address the best practices for ETL in a future article.

Another complication is the data model itself. The power of a really well conceived data model is significant; it can make a significant difference not only for query performance but also for data availability. The challenge is a big one, though, when you start to consider relationships among entities. What’s the relationship between a physician, member, patient, facility, provider, encounter and event? This is so easy to get wrong, yet so few healthcare organizations take the time to get it right.

A number of years ago, I was working for a large payer when we found an error in our data model. It resulted in a set of reports that failed to include applicable events. Without going into too much detail, I can tell you that I will never again underestimate the power of a good data model.

Finally, as healthcare organizations look to put the information into the end-users’ hands, they focus on static reports, something they are very familiar with in the regulatory realm. However, they miss a very big opportunity to take advantage of many of the latest data visualization and mobile capabilities of today’s BI tools. All in all, healthcare organizations are behind the times from a BI industry perspective, but that’s not too surprising when you consider the obstacles that healthcare companies have to address in order to get the work done.

Top Ten To-Do’s for Healthcare Organizations

With the tsunami of data, an already over-burdened IT staff and more regulatory pressures on the way, what can be done to stay ahead of the fray? It’s more important than ever to stay focused on the things that add business value to the organization. That is no different in healthcare. Business intelligence has the ability to bring market differentiation to any organization, if done right. Here are the ten things you should focus on to build out a world-class BI organization:
  1. Team Up
    To give your BI program any teeth, you have to fund it. First, create a leadership position that will assist in building out the BI program. This doesn’t always have to add full-time employee (FTE) overhead. I have always been a big proponent of small, agile teams. Investing in this effort will send a clear message to your organization that the data is important to you.

  2. Data, Data Everywhere—But Not a Drop of Knowledge
    Sure, disk space is cheap compared to a few years ago, but you still have to ask yourself if you really need that data in a warehouse environment. There are new ways of managing external and extraneous data without creating additional work for your IT staff. Analytic sandboxes are a great way to bring in data that’s relevant for analysis in the short term but doesn’t require a sophisticated ETL process to move or maintain over time.

  3. Business Intelligence = Business Value
    The focus should always be on providing value. If you aren’t doing that, then pack up and go home. Everything that you do, from creating ETL scripts to visualizing data, should have an end goal of empowering your users to make better business decisions. That’s where the real return on investment resides. Keep your eye on the value ball; everything else will fall into place.

  4. Failing to Plan is Planning to Fail
    Gartner indicated in a research note a few years back that many failures of BI programs were attributable to the lack of a strategic vision for business intelligence. Create a roadmap for your program, with accompanying detail for the next 12 months’ worth of projects. Leave room for changes to your organization’s goals, but always have a method of planning in place.

  5. Semper Gumby—Always Flexible
    Remember that organizations—including healthcare organizations— are dynamic. The days where we could take 8 to 10 months to complete a project are gone. Agile methods for data warehousing and BI projects are gaining popularity because they drive tangible business value to the users in shorter increments, reducing costly rework cycles. This isn’t just a trend, and it doesn’t mean a diminished quality product. Agile methods are here to stay—learn them.

  6. ETL, ELT, but Never EL
    Healthcare data will always require a fair amount of transformation to be usable by the end users. Spend your time on your business requirements and test often (agile methods!). However you cut it, ETL processes are not the “sexy” part of BI (assuming there is a “sexy” part of BI) but the truth is that for most BI and data warehousing projects, success resides in the ETL, and the vast majority of effort is there. Hire a really good ETL architect, and don’t skimp on the time it takes to do this work well. Remember, if you are only bringing in the data you need, you should have time to do it right the first time.

  7. Data Model!
    I alluded to the importance of the data model earlier in thid article. I am a big fan of spending the time (and money) to create a flexible model that will grow with the organization. But data modeling is no different than any other project, and it’s easy to get stuck in analysis paralysis. Remember to focus on business value. You don’t have to deliver everything immediately, and your business will thank you if you take the time to deliver smaller projects more frequently without rework.

  8. Data Must Be Governed
    If you are serious about your data being a corporate asset, then you had better plan on managing it as a corporate asset. Data is incredibly fluid, dynamic and often—almost by definition—flawed. In order to have a BI program that provides reliable data to end users, you must have a method of governing that data.

  9. End Users Are Not Stupid
    End users are often overworked, frustrated and flustered, but they aren’t stupid. Their job isn’t to figure out the data (unless it is their job). Their job is to take the information and make key business decisions with context. Create the content that is most relevant to them in a way that makes them want to consume it. In other words, create dashboards, allow for mobile BI, and create an analytic sandbox. If you have happy and engaged end users, you are more likely to have a well-funded BI program.

  10. Manage Your Program
    BI programs do not manage themselves. With all the moving parts of data governance, ETL, data models, reports, dashboards, etc., these programs can be quite complex. Have a plan in place to manage each aspect of your BI program, including the operational aspects. For example, how long will it take you to respond to a new request? What if the server goes down? The management of the BI program is also responsible for the roadmap, ROI and budget.
Nothing is foolproof, but if you understand the fundamental reasons why healthcare BI is unique and take these steps as you build out your program, you will be set for success.

  • Laura MadsenLaura Madsen
    Laura leads the healthcare practice for Lancet, where she brings more than a decade of experience in business intelligence (BI) and data warehousing for healthcare, and a passion for engaging and educating the BI community.  At Lancet, she spearheads strategy and product development for the healthcare sector. She also works with key accounts across the country in the provider, payer, and healthcare manufacturing markets. Laura is the founder of the Healthcare Business Intelligence Summit, an annual event that brings together top hospitals, insurers, and suppliers in the healthcare business intelligence space. Laura is also the author of the popular book, Healthcare Business Intelligence: A Guide to Empowering Successful Data Reporting and Analytics (Wiley, 2012). You may reach her at lmadsen@lancetsoftware.com.

    Editor's note: More healthcare articles, resources, news and events are available in the BeyeNETWORK's Healthcare Channel featuring Laura Madsen and Scott Wanless. Be sure to visit today!

Recent articles by Laura Madsen



Want to post a comment? Login or become a member today!

Posted January 25, 2011 by Laura Madsen


Thanks for reading!  I am not a data security expert by any stretch.  But, what I do know for sure is that security in these instances is required at every part of the data process chain.  You have to have application level security, and of course, row level security at the database.   The final level of security happens at the BI Presentation layer, where you can have group and role level security.  Encryption is a key component of security for healthcare.

Talking from the perspective of a person that has had to send reports and analysis out regarding inadvertant PHI disclosures, each one of these levels of security makes a difference.  If you protect each level you reduce your risk. 

Thanks, Laura


Is this comment inappropriate? Click here to flag this comment.

Posted January 21, 2011 by Sadiq Shaik

Loved the article!  Could you comment on best practices on how information sourced from different applications but is merged in one DWH is secured to ensure patient privacy, etc?  I look forward to your next article.

Is this comment inappropriate? Click here to flag this comment.

Posted January 19, 2011 by Laura Madsen

True, the 'commandments' are consistent across industries.  I think the differences lie in the implications.  If a retail organization has a data quality issue it can be expensive but there are no clinical implications.  Or, as I indicated in the data model section, a missed join in a query can mean missing a regulatory requirement and or worse (as in my example).

Thanks for reading!

Is this comment inappropriate? Click here to flag this comment.

Posted January 19, 2011 by Anonymous

Very relevant one and candid too.  But the beauty lies in the fact that the 'Top Ten To-Dos' are relevant to all the other domains too. Truly 'Ten Commandments of BI'

Is this comment inappropriate? Click here to flag this comment.