Blog: Dan E. Linstedt

« Averages and Outliers - Where's the REAL business BI? | Main | SOA and Business Rule Changes »

SOA and DW beyond the big picture

I am delighted to see a comment requesting further clarification on SOA, DW and reporting beyond the big picture. I will do my best to dig into additional explanation as to why they fit together, and what business purposes they serve. As always, I'm learning about the SOA world as well, so any feedback or comments (or even corrections) are more than welcome. This blog is basically an opinionated view point on where this technology is heading.

I think there are two blogs to answering this question, so I'll split it into two parts. The first part goes through basic explanations of SOA and operational analysis. The second part will dive into the impact of changes to an SOA, and business metadata.

First, let's clear the air: SOA is not a product, nor is it a technology - it is an architecture. Plain and simple - a set of rules and standards that define service oriented interactions between mechanical solutions. The architecture as it's defined goes deeper than that, and assists with how the tools, technologies, and other items below the surface should react when faced with a request. But basically SOA is an architecture.

If you're familiar with Web-Services and the old CORBA model, you can easily relate it to the SOA definitions, which as a part of the architecture define HOW these "services" play with the outside world. In other words, exposing the services to the internet and B2B, B2C etc... communication.

An example (albeit not so great) might be as follows. Suppose you build a (large) retail store-front, it's important to 1) keep customers coming, 2) provide for large numbers of customers to be serviced quickly, 3) allow them to browse the products without feeling pushed, but at the same time 4) provide quick answers to their questions should they have any.

This is all about service - of course, the less service one receives the more they must know about the products before they walk in the store, or they are most likely to be unsatisfied customers. Along with that, the less service they receive, the less overhead for operating the company, therefore - reduced prices.

Now, back to SOA. An SOA is made operational by defining the electronic services behind the "clerks desk". It's like a menu at a restaurant, or a list of products on a web-site that are categorized and described with specifications. It's also a little more than that, it includes HOW the access from the outside world will come in and request/retrieve/write data to and from the systems behind the systems. But that's where it stops. It does NOT define the systems, methods, or procedures that are used behind the request to meet the need. Just like a menu doesn't tell you how the food is cooked.

With all of that, what roles do data warehousing, metadata, and ETL/ELT/EII/EAI play in an SOA architecture?

Data Warehousing is supposed to provide additional value to the enterprise by containing 60% to 80% of the information that the SOA is requesting, which include history of the information, and a single consistent view of the enterprise. If an existing data warehouse can service a "live request", then there is no need to "rebuild" all the integration, business rules, and cleansing that has already been built. In other words, don't re-invent the wheel.

If on the other hand, the DW doesn't have the data, then the service request must look to additional technologies to get the data, that's when it turns to EII and EAI to find an answer (in most cases). However, when we reach the technology level - we have to rebuild the business logic in EII, or EAI platforms in order to provide the data that they requested.

Let's talk a little about metadata, and security. In regards to metadata, the SOA registered services must be able to describe in business terms, what they offer - again like a menu at a restaurant. Would you order food at a fancy restaurant just by it's title? Probably not, not unless it tells you what's in it, and what comes with it. We need descriptions of the entire offering in the service in order to understand if it's the service we want to use. It relies heavily on metadata.

But it goes deeper than that! In medical communities, and soon in other verticals it will rely more and more on commonly accepted definitions, and commonly accepted metadata. Why? Because these services are interchanging data with many other companies around the world (depending on corporate "service and data exchange" agreements). Metadata is a HOT world inside SOA, and must be the focus of any good SOA implementation, otherwise end-users/customers won't understand how or why to use the service itself.

In regards to security, because these services are exposed (like e-filing your Taxes on the IRS web-site), they must be secured. In other words, users/agents making requests should be certified, logged in, registered - however you term it; and the actions must be monitored and logged - in other words, at a restaurant it would be similar to recording your favorite table, getting to know you, and what you order when you go to the restaurant. This way, if the restaurant has a sale on your favorite item, they might entice you back in for more business.

All I've done is wrap security in a sugar-like blanket. The real notions are to catch fraudsters, and protect confidential data sets, and to provide you or your software agents with only the services they paid for.

So as we see, the SOA is an architecture that defines WHAT can and will be put in place, how it will be seen (viewed by the outside world), and what will interface the requests to other businesses. Now, on to part two of this blog.

Share: