We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

The Importance of Data Quality in Service-Oriented Architectures Are You Being Served?

Originally published March 6, 2007

IT departments can’t escape the buzz and vendor promises stemming from the latest “killer” architecture – service-oriented architecture (SOA). Companies are being warned that not jumping on the SOA bandwagon will put them at a competitive disadvantage. CIOs are being pressured by executive management to immediately implement and deploy SOAs. But what companies are not being told is that successfully implementing and deploying an enterprise-wide SOA is a phased, evolutionary process that requires that a company first implement an enterprise-wide data architecture and, subsequently, all of the associated data quality processes. The purpose of this article is to explore the importance of data quality and data governance practices as a prerequisite for delivering an SOA.

Before we begin, let’s define some terms:

  • Service-Oriented Architecture
    According to Wikipedia: “A service-oriented architecture (SOA [pronounced "es-ô-â"]) expresses a perspective of software architecture that defines the use of loosely coupled software services to support the requirements of the business processes and software users. In an SOA environment, resources on a network are made available as independent services that can be accessed without knowledge of their underlying platform implementation.”
  • Services (in SOA)
    Reusable software components that are “callable” by a software application or other software components to perform a business function. An SOA service may invoke other SOA services as well as other types of services such as Web services, communications services, and I/O services.
  • Data Quality
    A measure of how well the data satisfies the requirements of its consumers. Some typical metrics components include:
    • Accuracy/precision
    • Completeness
    • Reliability
    • Availability
    • Timeliness/freshness
    • Consistency
    • Uniqueness
  • Data Governance
    This describes a company’s functional approach to the management of data by actively linking integrated business and technology teams with corporate and strategic initiatives.

Features, Benefits, and Promises
The primary benefit that an SOA implementation promises to deliver to users is IT’s ability to respond more quickly and cost-effectively in the response to business requirements and business changes. This is accomplished through developing services that are aligned with business functions and designing the services such that they can be reused and/or easily substituted without affecting overall business functions and business goals. This reduces costs (labor resources) and accelerates time to market.

Success Stories
Standard Life Assurance, the largest mutual assurance company in Europe, implemented an SOA that reduced $3.6 million in system/application development costs. Verizon implemented an SOA that started servicing 10,000 transactions per month and has scaled to servicing 10 million transactions per day. Guardian Life Insurance estimates it saved 30% of its application development budget by using an SOA. The state of Massachusetts used an SOA approach to connect independent insurer, hospital, and physician systems. Using this approach reduced the cost per transaction from $5.00 to $.25 – 20 to 1!

Successfully Transitioning to a Service-Oriented Architecture
MIT’s Sloan Center for Information Systems Research (CISR) has recently completed two IT studies. The studies researched 456 enterprises’ projects from 1995 through 2005. The studies concluded that before the full benefits of an SOA can be realized, business units and IT must go through 4 architectural stages:

  1. Silos – Business silos with IT focused on individual departmental needs

  2. Standardized IT – Standardized technology platforms and technology services

  3. Standardized Business Processes – Enterprise-wide standardized business processes and standardized data

  4. Business Modularity – Business processes and supporting technology modularized for “plug-and-play”

The studies concluded that companies must undergo the successful implementation of each of these stages – there are no quick paths to successful SOA deployments.

The architectural components that need to be either transformed or constructed while going through the stages are shown in the figures in this article. The transformation or constructions of architectural components through these four stages are illustrated in Figures 1 and 2.

Figure 1: SOA Foundation
Source: SOA Practitioner’s Guide Part 2


Figure 2: SOA Architecture
Source: Service-Oriented Architecture Compass: Business Value, Planning, and Enterprise Roadmap

As shown in the figures, Data & Information Management is a fundamental architecture component for an SOA. The Data and Information Management services are responsible for retrieving, maintaining, managing, and aggregating data across heterogeneous data sources or repositories. The following examples illustrate the role of this service component.

Transamerica operates in a highly regulated environment governed by Sarbanes-Oxley, the Patriot Act, anti-laundering laws, tax laws, and other types of controls. As both the legislative and competitive environments change, Transamerica needs to react quickly by modifying internal IT systems. Regulatory changes may require changes to tax calculations for services and products. Customized products and services may be developed for different distribution channels, and special niche products may be developed for specific banks or broker dealers.

Accurately calculating the applicable taxes requires the retrieval and execution of the correct set of tax business rules with associated location, customer, and product/service data.

Not all agents are allowed to sell all products. Agents must be validated as licensed and are appointed to sell specific products in specific states. The validation process is complex and requires the aggregation of multiple data elements. The associated commission paid to the agent is also complex, requiring navigation through a hierarchy of business rules and associated data.

Countrywide Servicing Systems Development
The supporting loan division unit of Countrywide Financial, Countrywide Servicing Systems Development (CSSD), began its SOA effort in 2002. Its SOA’s service layer includes reusable load-related services such as Calculate Credit Score, Get Borrower Information, Generate Bill, and Calculate Payments. These services access three separate data stores using two different database technologies.

These two SOA implementations exemplify why data quality is vital to the success of SOA projects. For Transamerica, inaccurate data will lead to inaccurate tax calculations, erroneous agent validations, and inaccurate agent commission calculations. For Countrywide Financial, inaccurate data will result in erroneous borrower qualifications, inaccurate loan pricing, and inaccurate invoicing. In both cases, if the data is of poor quality, non-traceable or non-existent, hefty regulatory fines and/or negative customer experiences are probable outcomes.

SOA and Data Governance
Successfully reaching stage 3 of the CISR study recommendations (enterprise-wide standardized business processes or data) requires a joint business-IT working group that is empowered to address the following:

  • Which business activities can and should be instantiated into reusable services?

  • Which services should be shared and by whom?

  • What are the relative deployment priorities?

  • What is the overall architecture? What infrastructure is required?

  • What are the required service levels for the services? What are the security requirements for the services?

  • How will the services be measured, managed, and controlled?

  • How will IT developers know what services are available? How will changes to services be managed and controlled?

  • How will the service be invoked – what are the service’s interfaces?

  • How will the services be developed? What technology will be used? What development standards and methodology should be used?

  • What data is required by the services? How will data be accessed? What are the data quality requirements for the services? What are the service level requirements for the data?

The last set of data questions forms the overlap between SOA governance and data governance. Figure 3 illustrates a data governance framework and many of the functional pillars in a data governance function. These same functional pillars should be incorporated into an SOA governance function.

Figure 3: Governance Framework

Succeeding with SOA
Service-oriented architectures hold great potential for increasing IT alignment with business processes and, consequently, allowing an enterprise to quickly respond to competitive and regulatory changes. To implement a successful SOA, it is important that a company follows a basic tenet – the underlying shared data must be of high quality, highly available, and consistent across the services.


SOA Practioner’s Guide
Surekha Durvasula, Enterprise Architect, Kohls
Martin Guttmann, Principal Architect, Customer Solutions Group, Intel Corp
Ashok Kumar, Manager, SOA Architecture, Avis/Budget
Jeffery Lamb, Enterprise Architect, Wells Fargo
Tom Mitchell, Lead Technical Architect, Wells Fargo Private Client Services
Burc Oral, Individual Contributor
Yogish Pai, Chief Architect AquaLogic Composer, BEA Systems, Inc.
Tom Sedlack, Enterprise Architecture & Engineering, SunTrust Banks, Inc.
Dr. Harsh Sharma, Senior Information Architect, MetLife
Sankar Ram Sundaresan, Chief Architect e-Business, HP-IT

Service-Oriented Architecture Compass: Business Value, Planning, and Enterprise Roadmap
by Norbert Bieberstein et al.
IBM Press © 2005

SOA ensures Guardian gets it right
by Galen Gruman
May 02, 2005

Transamerica turns silos into services
by Leon Erlanger
May 02, 2005

Massachusetts takes a spoonful of SOA
by Galen Gruman
May 02, 2005

Compiled by Christopher Koch, CIO Magazine June 2006

The Four Stages of Enterprise Architecture
by Galen Gruman
December 01, 2006

  • Sid Frank

    Sid is a senior principal for Financial and Government Services at Knightsbridge Solutions, HP’s new Information Management practice. Sid’s expertise includes practice management and systems development. At Knightsbridge, Sid manages data management assessment and development projects. He is a former senior manager with PricewaterhouseCoopers and Naviant. At both firms, Sid focused on the practice of designing and developing business-critical decision support systems, knowledge management systems, and competitive intelligence systems for the financial, telecommunications, and retail industries. At GE, he was responsible for managing both development and R&D programs. Sid holds an executive MBA from Temple University, a Masters of Systems Engineering from the University of Pennsylvania, and a Bachelor's degree in Electrical Engineering from CUNY.

    Sid has written “An Introduction to Six Sigma Pricing” and “What’s in a Price: Losing Earnings Through Price Confusion.” He coauthored “The Partnership of Six Sigma and Data Certification,” “Six Sigma Data Quality Processes” and "Managing to Yield: A Path to Increased Earnings.” Sid can be reached at sfrank@knightsbridge.com

Recent articles by Sid Frank



Want to post a comment? Login or become a member today!

Be the first to comment!