Hello world. Waaw, time went by too fast. Happy new year, and here is the long past due update on the new Big Data Appliance and the software updates.
Big Data Appliance X3-2
Both the software as well as the hardware of the Big Data Appliance got a refresher.
Hardware Update
A good place to start is to quickly review the hardware differences (no price changes!). On a per node basis the following is a comparison between old and new (X3-2) hardware:
| Big Data Appliance v1 | Big Data Appliance X3-2 | |
|---|---|---|
CPU | 2 x 6-Core Intel® Xeon® 5675 (3.06 GHz) | 2 x 8-Core Intel® Xeon® E5-2660 (2.2 GHz) |
| Memory | 48GB | 64GB expandable to 512GB |
| Disk |
| 12 x 3TB High Capacity SAS |
| InfiniBand | 40Gb/sec | 40Gb/sec |
| Ethernet | 10Gb/sec | 10Gb/sec |
| KVM | 1 KVM Switch | N/A (removed) |
For all the details on the environmentals and other useful information, review the data sheet for Big Data Appliance X3-2. For those wondering what we did with the 2RU we now have left from the KVM, that is open space, at the top of the rack.
The higher core count gives a BDA X3-2 more parallel compute power while saving some 30% in energy and heat.
Software Update
As we did with Hardware, a good place to start is a quick overview of the software changes in below table:
| Big Data Appliance v1.1.x Software Stack | Big Data Appliance V2.0.1 Software Stack | |
|---|---|---|
| Linux | Oracle Linux 5.6 | Oracle Linux 5.8 with UEK |
| JDK | 1.6 | 1.6u35 |
| Cloudera CDH | CDH 3u4 | CDH 4.1.x |
| Cloudera Manager | CM 3 | CM 4.1 |
| Oracle Enterprise Manager | N/A | Big Data Appliance Plug-In for Enterprise Manager |
| R | Open Source R | Oracle R Distribution 2.x |
| Big Data Connectors * | Big Data Connectors 1.1.x | Big Data Connectors 2.0.x |
| Oracle NoSQL Database CE ** | NoSQL DB 1.x | NoSQL DB 2.x |
* Oracle Big Data Connectors is a separately licensed product which can be pre-installed and pre-configured on BDA
** Oracle NoSQL DB 2.x will be pre-installed in a future update to Mammoth but can be applied manually today
Apart from the versions updates, bug fixes and a great number of performance improvements across the entire system, the biggest updates are the inclusion of CDH 4.1.2 and the default set up of highly available name nodes for Hadoop, the Enterprise Manager management of the BDA, the uptake of the Oracle R Distribution and the updates to Oracle NoSQL Database. In a nutshell these updates deliver the following improvements:
Cloudera CDH 4.1.x
The latest version of CDH and CM deliver:
- Higher overall performance
- Highly available name nodes with the BDA using failover quorum processes instead of an external HA filer solution
- Vastly expanded management capabilities via CM 4
On top of this, BDA now has both Zookeeper and Oozie configured out of the box.
Oracle Enterprise Manager
The new Big Data Appliance Plug-In for Enterprise Manager delivers the first end-to-end management of the Hadoop cluster from hardware metrics to software and Hadoop metrics. To achieve the end-to-end management of the system Enterprise Manager delivers all the system metrics users are used to from the Exadata Plug-In for Enterprise Manager. Enterprise Manager enables a seamless transition between the Hardware and high level software monitoring and the expanded Hadoop monitoring and diagnostics from Cloudera Manager. This combination of functionality makes operations for a BDA simpler and allows operations staff to seamlessly switch between their Exadata, Big Data Appliance and other Oracle Engineered systems.
Oracle R Distribution
The big difference between Oracle R Distribution and the Opensource R distribution is that Oracle R Distribution is enabled to dynamically load the math kernel libraries on the CPUs from both Intel and AMD. This increases performance of basic calculations, which in turn increases the performance of the overall R calculations because more math is off-loaded into the CPUs.
Oracle NoSQL Database 2.x
A great number of great new features are added into NoSQL DB 2.x. Most of these are in both the Community Edition as well in the Enterprise Edition. Charles Lamb has a nice concise post describing what is new here.
Big Data Connectors
To close out, Big Data Connectors got a refresher focused on performance, so download the new products here and give them a go via this download page. More information on news, read the data sheet here.