We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Blog: William McKnight Subscribe to this blog's RSS feed!

William McKnight

Hello and welcome to my blog!

I will periodically be sharing my thoughts and observations on information management here in the blog. I am passionate about the effective creation, management and distribution of information for the benefit of company goals, and I'm thrilled to be a part of my clients' growth plans and connect what the industry provides to those goals. I have played many roles, but the perspective I come from is benefit to the end client. I hope the entries can be of some modest benefit to that goal. Please share your thoughts and input to the topics.

About the author >

William is the president of McKnight Consulting Group, a firm focused on delivering business value and solving business challenges utilizing proven, streamlined approaches in data warehousing, master data management and business intelligence, all with a focus on data quality and scalable architectures. William functions as strategist, information architect and program manager for complex, high-volume, full life-cycle implementations worldwide. William is a Southwest Entrepreneur of the Year finalist, a frequent best-practices judge, has authored hundreds of articles and white papers, and given hundreds of international keynotes and public seminars. His team's implementations from both IT and consultant positions have won Best Practices awards. He is a former IT Vice President of a Fortune company, a former software engineer, and holds an MBA. William is author of the book 90 Days to Success in Consulting. Contact William at wmcknight@mcknightcg.com.

Editor's Note: More articles and resources are available in William's BeyeNETWORK Expert Channel. Be sure to visit today!

Open Source.  Check.  Columnar.   Check.  Dallas' answer to a couple of others checking these boxes out there is InfiniDB from Calpont.  I had a chance to catch up with them this week as they announced their latest release, 2.1, about 15 months after their commercial launch.

 

InfiniDB does not have indexes, does late materialization (or "just in time" as they call it) and multi-table hash joins.

 

Some of the items they were stressing were scalability, SQL extension, compression and performance.

 

The modules that perform user functions and performance functions can be scaled individually to accommodate need in either area.  This is part of their plan to provide linear scalability.

 

Partitioning can be vertical or horizontal and comes by default with CREATE TABLE.

 

There was a lot of stressing of the predictable, linear performance.

 

  calpont1.jpg 

Here are some SQL extensions that were added in last few releases that were stressed:

 

Subqueries

Limit keyword

User Defined Functions

STDDEV and related functions

Views

Auto incrementing

Partition drop - to take individual partitions offline

Insert into table select from... where the from can be to/from InfiniDB and MySQL

Vertical and horizontal partitioning

 

Most of these are available in the enterprise edition.  If you want their syntax guide, drop me an email.

 

Compression is also new and improved.   Using the Piwik.org data set, they achieved 3x to 9x compression.  Compression can be set on individual columns.

 

Performance with compression was a key takeaway: 

calpont2.jpg

Several recommendations for effectively using InfiniDB were given:

 

Use tight data type declarations (4 byte instead of 8 byte)

Use fixed rather than long strings

Loading can be slow, which is a byproduct of being columnar

Other advice around using Cpimport, which is their fastest bulk loader for flat files  

 

InfiniDB performance testing with the Star Schema benchmark looks like this: 

calpont3.jpg

The sales model seems streamlined to maximize a product reach in 2011.  Like a lot of open source, it reflects a "try before you buy" approach, happy to work "bottoms up" in organizations, with short-term contracts and simple pricing ($6,000 per CPU core).

  

  

 


Posted May 6, 2011 7:35 AM
Permalink | No Comments |

Leave a comment

    
   VISIT MY EXPERT CHANNEL

Search this blog
Categories ›
Archives ›
Recent Entries ›