Blog: Lou Agosta Subscribe to this blog's RSS feed!

Lou Agosta

Greetings and welcome to my blog focusing on reengineering healthcare using information technology. The commitment is to provide an engaging mixture of brainstorming, blue sky speculation and business intelligence vision with real world experiences – including those reported by you, the reader-participant – about what works and what doesn't in using healthcare information technology (HIT) to optimize consumer, provider and payer processes in healthcare. Keeping in mind that sometimes a scalpel, not a hammer, is the tool of choice, the approach is to be a stand for new possibilities in the face of entrenched mediocrity, to do so without tilting windmills and to follow the line of least resistance to getting the job done – a healthcare system that works for us all. So let me invite you to HIT me with your best shot at

About the author >

Lou Agosta is an independent industry analyst, specializing in data warehousing, data mining and data quality. A former industry analyst at Giga Information Group, Agosta has published extensively on industry trends in data warehousing, business and information technology. He is currently focusing on the challenge of transforming America’s healthcare system using information technology (HIT). He can be reached at

Editor's Note: More articles, resources, and events are available in Lou's BeyeNETWORK Expert Channel. Be sure to visit today!

Recently in workflow (work flow) Category

Datameer takes its name from the sea - the sea of data - as in the French la mer or German, das Meer.


I caught up with Ajay Anand, CEO, and Stefan Groschupf, CTO. Ajay earned his stripes as Director of Cloud Computing and Hadoop at Yahoo. Stefan is a long-time open source consultant, and advocate, and cloud computing architect from EMI Music.


Datameer is aligning with datameerlogo.JPGthe two trends of Big Data and Open Source. You do not need an industry analyst to tell you that data volumes continue to grow, with unstructured data growing at a rate of almost 62% CAGR and structured less, but a still substantial 22% (according to IDC). Meanwhile, open source has never looked better as a cost effective enabler of infrastructure.


The product beta is launched with McAfee, nurago, a leading financial services company and a major telecommunications service provider  in April with the summer promising to deliver early adopters with the gold product shipping in the autumn. (Schedule is subject to changes without notice.) 


The value proposition of Datameer Analytics Solution (DAS) is  helping users perform advanced analytics and data mining with the same level of expertise required for a reasonably competent user of an Excel spreadsheet.


As is often the case, the back story is the story. The underlying technology is Hadoop. Hadoop is an open source standard for highly distributed systems of data. It includes both storage technology and execution capabilities, making it a kind of distributed operating system, providing a high level of virtualization. Unlike a relational database where search requires chasing up and down a binary tree, Hadoop performs some of the work upfront, sorting the data and performing streaming data manipulation. This is definitely not efficient for small gigabyte volumes of data. But when the data gets big - really big - like multiple terabytes and petabytes, then the search and data manipulation functions enjoy an order of magnitude performance improvement. The search and manipulation are enabled by the MapReduce algorithm.  MapReduce has been made famous by the Google implementation as well as the Aster Data implementation of it. Of course, Hadoop is open source. MapReduce takes a user defined mapping function and a user defined reduce function and performs key pair exchange, executing a process of grouping, reducing, and aggregation at a low level that you do not want to have to code yourself. Hence, the need for and value in a tool such as DAS. It generates the assembly level code required to answer business and data mining questions that business wants to ask of the data. In this regards, DAS functions rather like a Cognos or BusinessObjects front-end in that it presents a simple interface in comparison to all the work being done "under the hood". Clients who have to deal with a sea of data now have another option for boiling the ocean without getting steamed up over it.

Posted April 15, 2010 9:21 AM
Permalink | No Comments |

To be sure, the USA is not the Sudan. The USA does have a system. Medical innovations are redefining the limits of what is possible in healthcare - biologic "miracle drugs," minimally invasive fiber optic procedures, robotic surgery, powerful magnetic resonance imaging (MRI) machines, and emerging prospects of personalized medicine. These innovations are enormously valuable, incredibly expensive, and enjoy wide public support. Count me in! Yet there are significant "buts"... Much of the healthcare technology is isolated, under utilized, and mis-directed by being located behind information technology (IT) systems that are years if not decades behind the curve. Even when the information technology is upgraded - say, in order to capitalize on federal incentives - it is sometimes used to "pave the cow path." In other words, lay down a veneer of GUI-based modernity over a workflow that is otherwise distinctly 1950s in its organizational and informational design.

When the patient looks at the doctor taking handwritten, paper-based notes; when she tries to get an x-ray sent from the facility to a specialist - and finds it is easier to walk it over in person; when a member of the family is out of town and requires a record from his healthcare file; when a patient or provider tries to get an accurate, committed answer about whether a treatment is reimbursable by a payer - then the modern operating room and laboratory seem less modern and indeed are held in check by a back office, workflow, and organization design that is distinctly prior to the invention of automation. In some cases, these are symptoms of a non-existent information infrastructure, and, in others, of one firmly rooted in the past. Unless explicitly addressed, they represent the default future - more of what we are already doing. We have highly trained doctors and equipment trapped on islands of information separated by manual processes, IT systems with robust 1980s architecture, policy-based turf delimiters, and sneaker networks - remember that? Get on your sneakers and carry the diskette (now a CD) over the connecting bridge between buildings on foot. A bold statement of the obvious: The IT system is due for an upgrade. IT is a lightening rod for policy and organizational transformation that is on the horizon and closing fast. However, the emphasis in this article is on the IT aspects. You will know we are making progress when the IT system supports -

Coordination of care (COC): Note that here "coordination of care" refers to multiple medical specialists in different locations accessing the same patient information. It is important to make this distinction since "coordination of care" also came to mean "reduction or denial of service" by payers, taking on a politically-charged meaning that it should not really have. This time COC does not mean reduced payments for doctors or services for patient like it did in the 1990s. This time it means doctors and care-givers work together using high bandwidth networks and accurate databases to mange complex conditions presented by individuals in context. In a commercial business context, this kind of coordination would be called "customer relationship management" (CRM), including a single view of the customer. Doctors and healthcare providers are smart people - very smart - and this means they will readily learn and adapt when a new possibility is presented that enables improved patient care, accountability, transparency, and reduced risk.

Computational resources available at the point of service: The goal is to access and use information to reduce uncertainty in the examining room, patient bedside, or ambulatory facility. Nevertheless, where even a single life is at stake, automation requires supervision and approval by human (medical) intelligence. This is the value of the collaboration between bar-coded prescriptions, nurse identifiers, and clinical decision support software "double check" prior to administering drugs. A few high profile cases where potentially fatal medication mix ups were caught and prevented - or, unfortunately, not prevented - is a powerful justification. The drill requires that patients, medications, and providers be unambiguously identified and authenticated by a unique, barcode-like id. Then when the nurse goes to change the intravenous drip, he or she must scan the medication, scan their own id, and scan the patient's id - as easy as 1-2-3. If potassium chloride has been mistaken for sodium chloride, then the system catches the error, and the lethal mistake is avoided. Note also the upstream implications for physician order entry and physician accountability. The doctor (or her or his assistant) must have entered the order for the medicine into the clinical management system and it must be available in a real time or near real time way. Anything that reduces risk in a system already stressed out by the need for tort reform in an overly litigious society will be embraced and exploited.

The value of database resources is recognized and implemented. This is taken for granted in many business contexts. Thinking in healthcare is still catching up. A case in point. When Hurricane Katrina put the city of New Orleans under nine feet of water, including the medical records, the need for disaster recovery was demonstrated in spades. Of course, both paper and electronic records were submerged. However, in some cases - such as the Veterans Administration - an off site, distributed back up of the electronic records was available. Those veterans were able to receive up-to-the-minute care in Houston or other evacuation site, since the electronic version was actually located in a different city or it was restored and available within days. So some things are working right, albeit intermittently across the system. The recommendation? Get to a standard relation database. A centralized repository or a distributed one that appears and functions in a centralized way is critical path for coordination of care. Maybe I need to get out more, but the number of so-called modern, run-your-hospital systems that implement the admittedly powerful b-trieve database, Mumps, is a significant symptom - and data point. Mumps is not bad, but it is a curiosity. There is a clear cost reduction opportunity in surmounting the technology lock in such a "one off," proprietary (legacy) choice in favor of an open approach where "open" extends to de facto standard market offerings from IBM, Microsoft, Oracle (to name just the "big three").

Personalized medicine gets traction. In a commercial business context, this would be called "business intelligence," using data to track trends, influence markets, and create opportunities. I hasten to add that the challenge is to use comparative effectiveness research (CER), which is related to pay-for-performance, in an accurate and timely way to identify winning treatments rather than to find an excuse to deny service. For example, Genentech/Roche's Avastin costs $50-$100,000 per year of treatment but works in fewer than 50% of patients. A relatively simple upfront test for Avastin response could save as much as $6 billion per year if all nonresponders could be identified and not targeted. The potential cumulative savings are substantial. (Data taken from - ) This is a high bar, and there is still a lot of data modeling, data collection, data cleansing, and system integration to be done before meaningful predictions can be made about treatment effectiveness.

Social factors and usability are integrated. Short cuts for expert and advanced users of the system to fast forward through steps that are already familiar are required. Expect healthcare providers make for demanding users - and they should be. They require support accordingly. We are learning a lot from practices at the Veteran Administration (VA) and at the Mayo Clinics, including the stunning insight that automation is most useful only if it is used; that automation can be inflexible, requiring doctors and providers to accommodate the rules. The latter can be misperceived as an affront to the sovereign authority of physicians operating in the "command and control" practice model. Usability testing and user (physician) buy in remain on the critical path.

Patients and consumers take responsibility, and do this regardless of the work-in-progress nature of the new and existing IT systems. Lest the reader imagine that IT is a silver bullet, there are lessons here for healthcare patients (consumers) too. Regardless of what comes out the US Congress, this quarter or in the future, consumers are held accountable for their health. The default is to be held accountable by life - getting sick. We simply cannot afford unhealthy lifestyles. Eating right, regular exercise, following the advice of one's doctor (apparently about one half of patients actually do so), and participating in wellness and treatment are the order of the day. Obviously this is easier said than done, especially for stressed out consumers, working two jobs - or none at all. Yet it is critical path. You don't feel like exercising? Instead of going to the refrigerator, get on your sneakers and get going! Even then there are no guarantees - people who "live right" still get sick or have accidents. That is why insurance was invented - get everyone into the pool and spread the risk. Still, do I smell an economic opportunity for gyms, work out shoes and gear, and fruit and vegetable merchants? Obviously there's lots more to be said - but let me acknowledge this article is already way too long for a blog post, and will be republished as a standalone article. However, I see value in launching a "prototype" - just to "get it out there" and get instant feedback - so let me hear from you. The title is intentionally provocative ... meanwhile, I've got to get up out of this chair, follow my own advice, and get some physical exercise.

Posted November 23, 2009 1:56 PM
Permalink | No Comments |