What comes after “Big Data”? I’d say “Faster Big Data.” And it’s going to be a game changer well beyond what Big Data has done so far.
Fast and efficient Big Data applications will change our lives. Some of them will drive our cars, move packages from warehouses to us, speed drugs from theory to reality, even predict crime. The applications seem limitless, and high performance will be revolutionary.
Intel is helping this trend along with its acceleration library, called DAAL (Data Analytics Acceleration Library), for speeding up Big Data problems.
When I code, I like to use the highest-performance libraries available to give me an edge. That’s why DAAL is really worth knowing. The DAAL open source project started by Intel, and the tightly related Intel DAAL product, include support for Hadoop, Spark, R, and Matlab, with language bindings for C++, Java, and Python.
DAAL handles BIG data much better than in-core libraries can
If you know about Intel’s Math Kernel Library (MKL), you might immediately wonder “why DAAL”? Data scientists have been using MKL for Big Data problems for some time; MKL is well respected by high performance programmers, and it’s the math library gold standard in accuracy and performance for x86.
However, most of Intel MKL was designed assuming that the data to be operated upon fits in memory all at once. Intel DAAL is designed to handle those situations where the data is too big to fit in memory all at once by dealing with it in chunks.