Databricks Inc. said it’s wedding Big Data with deep learning in the latest update to its Apache Spark-based platform.
The new support for deep learning — a variant of machine learning — means data developers and data scientists can use the platform to more easily create deep learning models, leveraging GPU computing power and new integration with various related code libraries.
Deep learning, according to Wikipedia, is: “is a branch of machine learning based on a set of algorithms that attempt to model high level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and non-linear transformations.”
It can be used for projects ranging from recognizing images and supply relevant captions to machine log analysis and risk detection. It’s but one segment of new-age software development related to artificial intelligence, cognitive computing and machine learning.
“Data scientists looking to combine deep learning with Big Data — whether it’s recognizing handwriting, translating speech between languages, or distinguishing between malignant and benign tumors — can now utilize Databricks for every stage of their workflow, from data wrangling to model tuning,” the company said in a news release last week. “Databricks is the first to integrate these diverse workloads in a fast, secure, and easy-to-use Apache Spark platform in the cloud.