We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

The Evolving Modern Data Warehouse

Originally published February 18, 2014

As I covered in my previous article, modern enterprise data systems are tasked with responding to evolving business landscape – where customer interactions are extending beyond traditional data collection and sharing mechanisms. With data arriving at a greater speed and in different formats, the role of a data warehouse as a single unified place for enterprise data has to change as well.

Historically, enterprises were using data warehouse solutions in an attempt to merge data arriving from operational systems into a single integrated corporate data set so that they could more accurately report on business events. Today, when digital organizations are using technology to fundamentally evolve their businesses, focus is shifting to enabling capabilities such as data federation and virtualization to help reduce time to value and unlock business insights from different systems.  

Data is coming from more sources than ever before – including both structured and unstructured data types. In the enterprise, that means data silos are here to stay. And, today they go beyond various files and spreadsheets to include NoSQL data stores, data lakes, on-demand feeds, cloud-based file systems, etc.

As a result, architectural solutions such as data polyglot persistence are gaining enterprise traction. This allows companies to take advantage of data from these disparate sources without having to architect and migrate all data to a single data store. Frankly, businesses don’t have any appetite for expensive and time-consuming data engineering efforts. They are expecting fast, agile insights and accelerated time to value with rapid delivery cycles.

The Modern Data Warehouse

So what’s the role of the modern data warehouse and how does it continue to deliver business value? The relational database model is excellent for data that monitors transactions and focuses on data consistency. With corresponding data quality and master reference consolidation capabilities, data warehouses will continue to meet the needs of business domains requiring high data fidelity – as well as facilitating record management regulatory compliance requirements for various industries and disciplines. In fact, data lineage and quality requirements are not going away any time soon. On the contrary, these requirements will be amplified as transactional data volumes continue to grow.  

Most of the massively parallel processing data warehouse appliances of today support capabilities that enable bi-directional data movement between their compute nodes and HDFS data nodes. This allows for movement of data that’s persisted in data lakes into the data warehouse for business domain scenarios requiring elevated data consistency as described earlier. Similarly, high-fidelity master reference data sets can be moved into Hadoop metastores to support agile large-scale analysis scenarios. Understanding the roles of different data collection is essential to delivering connected enterprise insights.

In conclusion, data warehouses are here to stay, but their use and role is evolving to meet changing business realities. They are part of an overarching enterprise strategy to help companies deal with large-scale data challenges by combining various data storage and sharing capabilities to deliver agile, fast and accurate insights across the business.  

  • Timur MehmedbasicTimur Mehmedbasic
    Timur is the Southeast Practice Lead for Data & Analytics services services at Avanade. He specializes in information management and governance, data systems strategy development, data warehouse design, implementation, performance tuning and Microsoft big data adoption. Timur has a broad industry background including media and entertainment, financial services, retail, insurance, consumer goods, transportation, and oil and gas. In his current role, he consults on the implementation of enterprise data and information management programs within Fortune 100 companies.

Recent articles by Timur Mehmedbasic



Want to post a comment? Login or become a member today!

Be the first to comment!