The changing state of data integration - looking at Informatica 9.5 - Blog: Lyndsay Wise

The changing state of data integration - looking at Informatica 9.5

One of the great things about the business intelligence industry is the fact that the market is in a constant state of change with the goal of helping organizations use technology to take advantage of the vast amounts of information available for analysis. The way in which data is handled has changed from traditional ETL jobs and data warehouse storage that required batch processes to deliver results, towards more real-time and ad-hoc interactive capabilities. As we've seen, the market is now able to handle large and complex data sets from a wider variety of sources that include social media and unstructured sources. The addition of cloud also makes it easier to to develop alternate frameworks external to the firewall.

All of these changes, as well as the countless other trends within the marketplace beg the question - what about data integration? After all, it is fine to load data and use ETL to transform data, but what about the complications of gaining valuable insights out of data in a constantly shifting world that demands up to date information that can be acted upon immediately? How can data be securely and successfully placed in the cloud? How will unstructured data sources be supported as social networking becomes a more prominent way of identifying customer sentiment and business opportunities? These questions are a subset of what organizations are asking themselves as they try to adopt technologies that will help them stay ahead of their competitors.

Although many vendors are meeting these newer challenges head on, since Informatica 9.5 was released earlier this week, it makes sense to take a closer look at how they are handling the changing data landscape. Informatica has always provided a wide range of data integration solutions. Their 9.5 release actually ties in the aspects mentioned above - such as cloud, big data, social media, etc.- to provide organizations with all of the pieces they require to develop a data infrastructure that addresses all of the key aspects of BI related trends. For instance, Informatica 9.5 provides:

Embeddable cloud service that sends data where it is required through the use of a hybrid IT platform, leaving control for information in the hands of the organization
Support for big data through broader data virtualization and replication, as well as partition to use resources more effectively
The use of Natural Language Processing to analyze social data from sources that are becoming more relevant to organizations, such as LinkedIn, Facebook, social networks, etc.
The ability to stream data and support real-time events through CEP (Complex Event Processing) to provide alerts and notifications to be able to act upon important issues that arise
Interoperability with Hadoop to support big data processing

All of these areas are becoming increasingly important to organizations. What interests me most aside from the ability to access a large stack of offerings that cover the key aspects required for a strong data integration platform, is the additional focus on data stewardship and ensuring that the full data integration process is governed effectively to ensure that data quality or other issues that arise can be dealt with in an effective manner by data owners.

Most organizations want to focus on dashboards and getting the insights they require. In some cases, this occurs at the expense of strong data management. The reality, though, is that to gain an understanding of what is happening within the realm of Facebook or on Twitter, or to access relevant information through Hadoop, a strong set of data integration processes are required. Consequently, Informatica seems to be focusing their efforts on the areas that will most matter to businesses to maintain competitive edge within the marketplace today.

Posted May 17, 2012 9:25 AM
Permalink | No Comments |

The increasing importance of collaboration

Most BI offerings have some level of collaborative feature sets. What this really means in many cases, is that users can email dashboards or share links, and in some cases add annotations or notes. Essentially, when it comes to capabilities that let people collaborate the way they naturally would by the water cooler or in a meeting, much is lacking in the world of BI. Whether this is because of the limitations that exist due to vendor infrastructure or a lack of imagination, most solutions are far from providing organizations with the ability to interact with their BI solutions in a way that enables broad collaboration.

Despite this lack of innovation in the area of collaboration, there are a few vendors that seem to be getting it right. The ones I think of first are Yellowfin and Lyzasoft - they both build their platform in a way that takes advantage of social media concepts and expectations. While at QlikView Qonnections, I also noticed that their view of collaboration extends to support the way in which business users interact with technology. Aside from being able to share screens and work simultaneously, there is a strong focus on the user experience and how people interact with technology and each other.

In an age where Facebook and Twitter mark the way people interact, annotations within a solution just doesn't cut it. Sending a link to a colleague is no longer enough. Now people need to be able to share their screen, work together, and create real-time conversations to help identify what is happening and how to deal with it. Until BI can do this effectively, solutions will continue to lack in overall effectively.

Obviously, until more recently, this was not the purpose of business intelligence at all. The ability to consolidate information across disparate business sources, and make sure that targets were being met provided a large part of the value add. Now, with market trends such as mobility, social media analysis, and collaboration focusing around better and immediate access to information in an easier to consume way, it makes sense that the importance of collaboration is increasing.

Overall, the fact remains that the majority of the BI market is missing the point of collaboration and what it needs to be. Where QlikView provides the ability to work simultaneously on dashboards and Lyzasoft lets people create real-time conversations around analytics, the same cannot be said for many other vendors in the marketplace. And even though many of these solution providers are working towards providing this functionality, unless these tools are provided in an easy to use and intuitive interface, their value proposition will continue to be low at best.

Posted April 29, 2012 11:31 AM
Permalink | No Comments |

Understanding data virtualization

This week I interviewed Bob Eve at Composite Software to understand the value proposition of data virtualization for mid-sized organizations. The interview itself can be found on the WiseAnalytics BI4SMB Community. Bob and I discussed what data virtualization is, how it differs from traditional data integration approaches, and how it enhances business agility. In addition, Bob provided some examples of use within mid-sized organizations.

One example of data virtualization adoption being Compassion International that provides charitable support services to children in developing countries. Their goal is to quadruple the number of its beneficiaries. To achieve this goal, the IT team needed to modernize the organization's information infrastructure. Bob shared that, "data virtualization has enabled Compassion to both scale their BI infrastructure and meet demanding new information requirements much more easily and quickly than before. Time-to-solution for new information requests has improved by more than 50%. And data quality and integrity have improved as well."

Bob also shared, that "like the larger enterprises, mid-sized organizations share substantive business pressures and face significant technology transformations. In every case, they were willing to look beyond traditional data integration methods and try something new, data virtualization. And as a result, all are seeing significant business decision, time-to-solution and resource agility benefits."

Based on what I've seen in the market and what Bob shared, I definitely feel that the broader BI market is ready for a shake up. Traditional ETL and simple data integration activities are no longer as effective within more complex data environments. Before selecting one style over another, businesses should really look at what options exist. Even for organizations that may need to be more creative with their resources, stepping outside of the box might help when looking at the best way to access information assets.

Posted March 8, 2012 5:57 AM
Permalink | No Comments |

Do solution providers really help SMBs with their expansion towards the mid-market?

Today I read Julie Hunt's blog entitled Selling Software to SMB Markets - Who and How to Target which discusses the fact that many solution providers fall short in how they target their solution offerings because they are applying enterprise viewpoints to small and medium-sized organizations. The implications of this are astronomical for SMBs looking at business intelligence offerings and trying to adapt enterprise targeted applications to their organizations, while being told that solution providers have moved downstream to target their specific needs. Overall, the level of success of BI for SMBs is directly tied to the fact that solutions developed actually reflect the needs of the market.

By creating software offerings that are subsets or smaller versions of enterprise solutions, SMBs are only getting a small percentage of potential benefits. Smaller businesses may not have the same financial or IT resources as their enterprise counterparts, but they are generally more dynamic, collaborative, and quick to act. BI solutions need to reflect this, and so far, only a small number of market solutions do. Requiring heavy IT infrastructures or only being able to capture a specific number of rows or needing technical skills to build analytics. Even many of the self-proclaimed self-service offerings are far from that for many business users tasked with donning their analytical hats consistently and on-the-fly.

The question to ask is how does this lack of focus help or hinder SMBs on the whole? As mentioned, SMBs may have similar business challenges to their enterprise counterparts, but are much more dynamic in their ability to take action. As BI offerings become more mature the hope is that these features will become built within the overall infrastructure. The reality for both small and mid-sized organizations, however, is that solution providers need to understand that although they have great ways to integrate their technology within broader IT infrastructures, integration shouldn't be the main challenge for SMB adoption of BI. SMBs should be able to link and consolidate the information of their choice and interact with that data without weeks or months of development.

However, the reality is that the BI market is not there yet. And, I personally don't think it will ever be, because vendors with robust infrastructures are not willing to put their current offerings on the shelf and develop solutions that are net new. Luckily, there will always be new entrants, and the few strategic vendors that like to take advantage of new technologies. Even though dynamic, quick to deploy, and very easy to use BI is slowly becoming available, SMBs still have a while before solutions really address their business pains and are developed with them specifically in mind. In many ways this can be compared to politicians making decisions that affect other geographic regions without fully understanding the culture or the consequences. Although not as serious within B2B organizations, the fact remains that many smaller companies do not have the resources to develop solutions that best meet their needs and are reliant on what is already available without understanding the complexity of the market around them.

Overall, SMBs require caution when evaluating BI offerings. It is important to make sure that vendor SMB strategy directly ties to what small and mid-sized companies need, versus a stripped away approach to solution functionality.

Posted February 20, 2012 11:17 AM
Permalink | No Comments |

Looking at BI in the Cloud

Last week I attended MicroStrategy World in Miami and just posted a general overview of the conference and takeaways on the BI4SMB community forum. One of their key strategic focuses is cloud computing, with three tiers of solution offerings that have recently been released to the market. As I won't be giving an overview of the solutions, here is a link to more details:http://www.microstrategy.com/cloud/.

Because MicroStrategy is traditionally an enterprise offering in relation to market penetration and overall focus, the increasing focus on cloud offerings provides more support for SMB adoption over time. After all, the promise of no upfront costs coupled with the ability to take advantage of leading BI and data warehousing technologies is valuable when evaluating the costs and benefits associated with BI deployments. Hopefully these solutions will indeed provide this value to SMBs. Currently, the free offerings can at least give organizations an idea of whether broader adoption will benefit them.

Based on security, performance, features, etc. there isn't much that can be negatively attributed to moving to the cloud. But, SMBs should be aware that when looking at the costs over time, cloud may not be as cost effective. Evaluations should include the following:

Upfront hardware costs or allocating specific hardware to a BI project.
Resources required to maintain hardware/software/project over time.
New hardware over time to account for expansions.
Yearly fee structure - how things changed based on number of users, data stored, type of user, etc.
Cost comparison based on initial hardware expenditure and internal resources vs. cloud expansion over time.
What features/modules are available and do these differ based on price points.
What about data access and integration?

In reality, the list is close to endless. In addition to costs, organizations need to look at the effort required from business and technical staff as well as the ease of use, training required, etc.

BI in the cloud is definitely opening up the playing field to many organizations that could not adopt broad BI in the past due to limitations. Offerings from vendors such as MicroStrategy that base their cloud offerings on the same solution platform as their on premise version, can help provide full BI offerings to SMBs providing the above factors make sense.

Posted January 31, 2012 8:32 AM
Permalink | No Comments |

PREV 1 2 NEXT

Blog: Lyndsay Wise