Blog: Dan E. Linstedt: Hardware: What do you need for VLDW?

« Commoditization of the EDW/BI Market. | Main | Operating Systems: What you need for VLDW »

Hardware: What do you need for VLDW?

In this entry we'll explore the requirements of VLDW, what you might need to build one, or to grow it going forward. Don't forget that everyone's definition of VLDW is different depending on the starting point. No matter if you're scaling from 30GB to 1 TB, or from 50 TB to 1 PB, these guidelines and simple pieces should help you move forward. I guess you could treat some of this as a "score-card" for your environment. This is a HARDWARE LOOK at what you need. In the future I'll blog on the software requirements, and then finally, I'll address appliances.

What about the basics?
The basics are easy; I'd suggest you look at the following landscape of options:

1) High speed disk should be transferring data to / from disk (any disk) at I/O rates of 300 MB to 400 MB per second. Without this magic number, the whole scalability effort is sunk right out of the gate.

2) High speed VPN server to server networking, and Server to Disk Networking. Gigabit Ethernet is about as fast as you can get today, the faster the better. BUT the mistake that's often made is to put the DISK device on a common shared network, and then wire it to servers, and THEN expect it to perform. WRONG! SORRY! That simply won't work. You cannot achieve scalability to the Petabyte Levels with this approach. You _might_ make it to 20 to 40 TB before you see problems, but even then.... ISOLATE THE NETWORK TRAFFIC, Big data costs big money, if it's not bringing BIG BUSINESS VALUE in it's return, then DUMP some of the data, reduce the data set size until value can be achieved or justified.

3) RAM CACHES on Disk Devices, double buffering the I/O across the network channels provide all the performance you need - again FOCUS on the throughput numbers on an average day with average load on your Disk Devices. 300MB to 400MB per second, no less. MORE is better in this case. But if your disk device does not house a large RAM cache (to buffer you from the impact of writes) then you WILL see performance problems.

4) Network, network, network: VPN everywhere. Ensure dedicated network cards between the servers and the disk arrays, and server to server communication on a separate VPN network connection. The more "segmentation" the higher the degree of parallelism across each VPN networked devices. The more segmentation, the faster the processing, the easier it is to route correct network traffic for directed / specific tasks. If you are using Optical to reach the DISK, then ENSURE that there are enough optical up-links to handle the performance throughput requirements. Again, it's _all_ about the performance in volume situations.

5) CPU's and RAM... Ensure that you have 1.2x the amount of RAM (multiplied by) the number of CPU Core's on the machine. So If I have 2, dual-core CPU's that’s 4 cpu cores, I would need at LEAST 1.2 x 4 = 4.8 GB of RAM JUST to handle the executing code on the CPU's and maximize parallelism and threading on the CPU cores. I like to say, at the maximum: 2x cpu cores is what I run with. If I have 32 cores, I like to have 64GB of RAM just to handle the operations of the CPU's. People often forget that CPU's are constantly swapping threads in & out of RAM, and NEED absolutely NEED RAM to do this. Now, about on-board caches... Level 1 caches are most important, they are closest to the CPU, and THE LARGER THE BETTER!!! Level 1 caches are expensive due to miniaturization. Level 2 Caches are next in line, THE LARGER THE BETTER - this will help the multi-threading, and reduce execution cycles, particularly useful in Database Joins, string matches (where clauses), Transformation operations, etc...

6) RAM & Network Packet Sizes. If POSSIBLE, then increase the network packet sizes, look for network cards with high levels of "RAM CACHE" and feature sets like auto-compression, and double buffering. This means faster transfer rates over the Giga ether cables. Increasing the network packet sizes means that the network card itself will build "larger" packets, and hopefully with compression and double buffering you'll see more data transferred faster with less actual network traffic.

7) WATCH THE BOOT LOGS FOR HARDWARE ERRORS. Often times, administrators don't have time to construct the proper script to look at the boot logs for RAM, CPU, Device, and Network Errors. But this is CRITICAL, it can be a significant and negative impact on the performance. Especially bad checksums in the RAM chips, this can cause erroneous errors, AND performance degradation by a factor of 4x to 20x.

If you have some hardware tips to add to this list, I'd love to hear from you. Let's build some knowledge here.

Cheers,
Dan Linstedt

Share: