Healthcare

AI is being used in various areas of healthcare, including the following:

• Genomics

• Drug discovery

• Cancer research

• Image scanning

• Surgery

• Longevity research

• Resource usage

• Robot carers

Let’s take a look at genomics for some illustrative examples. Deep learning is being used extensively in genomics research and in the development of new biotech products. Few domains exploit so well the essential power of deep learning: its capability to ingest extremely large datasets and find patterns in that data. The human genome has approximately four billion base pairs, so this is a big dataset. Now think of analyzing thousands or millions of genomes, and looking at petabytes of data.

Fortunately, with the advent of deep learning–optimized hardware (GPU and ASIC) and cloud services (machine learning as a service, or MLaaS), deep learning algorithms can analyze these huge datasets in realistic timescales—days rather than the months required just a few years ago. This makes it time and cost effective to use this methodology in the field of genomics to understand the human genome. The results are exploited to fight cancer and other diseases such as Alzheimer’s and Parkinson’s as well as to accelerate drug discovery for myriad diseases (such as ALS) and mental health issues (such as schizophrenia) that afflict humans today.

DeepVariant is a genomics framework recently open sourced by Google Alphabet,¹¹ in conjunction with its healthcare company Verily. The code is on GitHub, with a license allowing anyone to download, use, and contribute to it. You can find genomics datasets on the web, ^{12, 13} or, if you are a healthcare company, you might, of course, use your own.

Microsoft is also using AI to help improve the accuracy of gene editing with CRISPR. Several companies have been set up specifically to use machine learning to accelerate medical research and development. These include BenevolentAI and Deep Genomics.

Founded in 2013, BenevolentAI is based in London and is the largest private AI company in Europe. By applying AI to the mass analysis of vast amounts of scientific information, such as scientific papers, patents, clinical trials, data, and images, it is augmenting the insights of experienced scientists with the analytical tools they need to create usable and deep knowledge that dramatically speeds up scientific discovery. In the biotech industry, a new paper is published every 30 seconds; BenevolentAI applies AI algorithms to read and understand these papers. It then creates intelligent hypotheses about likely cures for various diseases. The company has already made major breakthroughs, including one related to ALS. BenevolentAI also uses all of this data analysis to design and predict new molecules.

Deep Genomics is a Toronto-based company founded in 2015 by Brendan Frey. Frey was a researcher with Geoff Hinton and then professor of engineering and medicine at the University of Toronto before forming his company. Deep Genomics’ founding belief is that the future of medicine will rely on artificial intelligence because biology is too complex for humans to understand. Deep Genomics is building a biologically accurate data- and AI-driven platform that supports geneticists, molecular biologists, and chemists in the development of therapies. For the company’s Project Saturn, for example, researchers will use the platform to search across a vast space of more than 69 billion molecules with the goal of generating a library of 1,000 compounds that can be used to manipulate cell biology and design therapies.

Finally, it is worth noting that both Google and Microsoft cloud services offer genomics as a service (GaaS). Researchers can use these powerful platforms to analyze vast datasets, either public or proprietary. (For further information, read the whitepaper on genomics from the team at Google Cloud Platform [GCP]). Other papers describing machine learning techniques applied to genomics research are listed in the References section;^{14, 15} the interested reader is encouraged to view them.

Finance

Some areas of FinTech that employ AI include the following:

• Trading

• Investment

• Insurance

• Risk management

• Fraud detection

• Blockchain

Let’s take a look at two aspects of finance for which AI is currently being deployed: blockchain and algorithmic trading.

Transportation

AI is being used in the various areas of transportation, including the following:

• Self-driving cars

• Route optimization

• Smart cities

• Flight

• Shipping

Let’s take a look at self-driving cars. Self-driving vehicles are set to disrupt cities and transportation overall. Potentially, they can transform not only the way people and goods move around the world, but also patterns of employment, new transport potential for many populations, and the organization of urban environments.

Every year, 1.25 million people lose their lives on the world’s roads. Causes of death include speeding, alcohol, distractions, and drowsiness. Self-driving vehicles are expected to reduce this number significantly, by at least 99%. Not only could self-driving cars reduce the road toll each year, but time spent commuting could be time spent doing what one wants while the car handles all of the driving.

Driverless cars will enable new ride- and car-sharing services. New types of cars will be invented, resembling offices, living rooms, or hotel rooms on wheels. Travelers will simply order up the type of vehicle they want based on their destination and activities planned along the way.

Ultimately, self-driving vehicles will reshape the future of society. The self-driving car market is expected to rise rapidly to an estimated $20 billion by 2024, with a compound annual growth rate of around 26%.

Machine learning algorithms are used to enable the vehicle to safely and intelligently navigate through the driving environment, predicting movements and avoiding collisions with objects such as people, animals, and other vehicles. Self-driving cars come equipped with various external and internal sensors to track both the environment and the driver, respectively. Sensors include Lidar (light radar), radar, infrared, ultrasound, microphones, and cameras. To elaborate:

• Originating in the early 1960s, Lidar is a surveying method that measures distance to a target by illuminating that target with a pulsed laser light and measuring the reflected pulses with a sensor. Differences in laser return times and wavelengths can then be used to make digital 3D-representations of the target.

• Ultrasonic sensors track and measure positions of objects very close to the car, like curbs and sidewalks, as well as other cars when parking.

Information collected by these sensors is then used to calculate distances, speeds, and types of objects surrounding the vehicle, as well as to predict motion through time and space.

A GPS system is used for navigation and vehicle-to-vehicle communication. Finally, a powerful in-car computer, comprising GPU processors that are usually built into the trunk of the car, runs machine learning algorithms such as CNNs to identify and track objects and to navigate through the environment.²⁵ These algorithms are updated online as new and better software becomes available and, along with improvements in training data and hardware, are expected to take us to Level 5 (fully autonomous) self-driving cars.

NVIDIA presently has the largest market share of the self-driving car on-board processors with its DRIVE PX GPUs. The DRIVE PX Xavier processor, with more than seven billion transistors, is the most complex system on a chip (SoC) ever created, representing the work of more than 2,000 NVIDIA engineers over a 4-year period and an investment of $2 billion in research and development. It is built around a custom 8-core CPU, a new 512-core Volta GPU, a new deep learning accelerator, computer vision accelerators, and new 8K HDR video processors. These on-board computers have 30 trillion operations per second (TOPS) of processing power while consuming just 30 watts of power, so it is like having a supercomputer in your car. The DRIVE PX Xavier is the first AI car supercomputer designed for fully autonomous Level 5 robotaxis and will be available Q1 2018.

The next generation platform from NVIDIA, the DRIVE PX Pegasus, delivers more than 320 TOPS and extends the DRIVE PX AI computing platform to handle Level 5 driverless vehicles. It will be available to NVIDIA automotive partners in the second half of 2018. Like a datacenter on wheels, NVIDIA DRIVE PX Pegasus will help make possible a new class of vehicles that can operate without a driver—fully autonomous vehicles without steering wheels, pedals, or mirrors, and interiors that feel like a living room or office. These vehicles will arrive on demand to whisk passengers safely to their destinations, bringing mobility to everyone, including the elderly and disabled. Millions of hours of lost time will be recaptured by drivers as they work, play, eat, or sleep on their daily commutes. And countless lives will be saved by vehicles that are never fatigued, impaired, or distracted—increasing road safety, reducing congestion, and freeing up valuable land currently used for parking lots.

The AI performance and capabilities of the PX Pegasus platform are expected to ensure the reliability and safety of self-driving cars as well as autonomous trucking fleets. A unified architecture enables the same software algorithms, libraries, and tools that run in the datacenter to also perform inferencing in the car. A cloud-to-car approach enables cars to receive over-the-air updates to add new features and capabilities throughout the life of a vehicle. You can find further details here and software libraries are available here.

Along with the incumbent car manufacturers such as GM, Ford, Mercedes, Volkswagen, and Toyota, a host of new companies have entered this market, including the likes of Waymo (spun out from Google in 2016), Tesla, Uber, Baidu, NuTonomy, Oxbotica, and Aurora. AT CES 2018, NVIDIA and Aurora announced that they are working together to create a new Level 4 and Level 5 self-driving hardware platform.

Waymo currently drives more than 25,000 autonomous miles each week, largely on complex city streets. That’s on top of 2.5 billion simulated miles it drove just in 2016. By driving every day in different types of real-world conditions, Waymo’s cars are taught to navigate safely through all kinds of situations. The company’s vehicles have sensors and software that are designed to detect pedestrians, cyclists, vehicles, road work, and more from a distance of up to two football fields away in all directions. Waymo’s cars are currently undergoing a public trial in Phoenix, Arizona, and as of November 2017, Waymo’s fully self-driving vehicles are test-driving on public roads, without anyone in the driver’s seat. Soon, members of the public will have the opportunity to use these vehicles in their daily lives. GM also says it will launch a robot taxi service in 2019.

Technology

AI is being used in the various areas of the technology industry, including the following:

• DevOps

• Systems-level—compilers, processors, and memory

• Software development

Let’s take a look at software development. Most would claim that the ultimate aim of technology is to make human life easier and more pleasurable by automating the tasks we find mundane and repetitious or that simply keep us away from doing the things we’d really love to be doing. Some types of programming might fit into this category, and automating the mundane aspects of software development would make programmers happier and more productive. Also, businesses would like to reduce costs and improve the speed and accuracy of any workflow process, including programming, so this automation of much of the software development life cycle (SDLC) is inevitable. Finally, there’s a chronic shortage of accomplished programmers, thus automation of the SDLC is strongly needed.

Let’s now look at some of the efforts we have seen toward automating the SDLC. We can separate these into three categories, each of which is sufficiently different so as to have some of its own unique characteristics.

• Web development

• Application programming

• Machine learning development

Energy

AI is being used in the various areas of energy management, such as the following:

• Smart metering

• Distribution

• Energy efficiency

• Datacenter

Let’s take a look at datacenter energy management. AI is being used in various areas of the tech industry, such as inside the hyperscalers (Google, Microsoft, Amazon, IBM, Apple). This includes energy management inside of Google’s datacenters. Back in July 2016, Google announced that by using DeepMind’s machine learning algorithms, the company had reduced the amount of energy used for cooling its datacenters by up to 40%. Using a system of neural networks trained on different operating scenarios and parameters within the datacenters, Google created a more efficient and adaptive framework to understand datacenter dynamics and optimize efficiency.

Then, in March 2017, DeepMind announced it had been in talks with the UK government about making the national grid more efficient by applying the same algorithms to the national power supply. Demis Hassabis, cofounder and CEO of DeepMind, hopes that the UK’s energy usage could be reduced by as much as 10%, just through AI-led optimization. This would reduce the cost of energy for consumers and reduce global warming, especially if scaled to every country in the world.

Science

AI is being used in the various areas of science:

• Artificial scientists

• Physics

• Materials science

• Mathematics

• Theorem provers

With the well-publicized success of deep learning in other fields, including image classification, gaming, and NLP, it was only a matter of time before scientists became interested in seeing whether it could be used to solve problems in their respective fields. This includes analyzing large scientific datasets and coming up with new hypotheses. We discuss several areas in which deep learning models have surpassed other techniques in analyzing and finding patterns in data and coming up with hypotheses of their own.

First up, we take a look at particle physics. Various particle physics groups, including at CERN,^{35, 36, 37, 38} have been using a variety of machine learning algorithms to sift through data and identify particle signatures. A popular Kaggle contest used machine learning techniques to identify the Higgs boson.³⁹

Next, working on the GPU-accelerated Titan supercomputer at Oak Ridge National Laboratory (ORNL), a team developed an algorithm that automatically generates neural networks. MENNDL, which is short for Multi-node Evolutionary Neural Networks for Deep Learning, evaluates, tests, and recommends neural networks for unique datasets like those that scientists collect. In the past, the team would have spent months testing neural networks to find one that would work for a given problem. MENNDL did this in just 24 hours.

Deep learning methods have also been used to measure the various properties of neutrinos. At Fermilab, the NOvA collaboration is using hand-crafted CNNs to analyze neutrino experiment datasets⁴⁰ to enhance its results. The team concludes:

With minimal event reconstruction, we were able to build and train a single algorithm, which achieved excellent separation of signal and background for both of the NOvA electron-neutrino appearance and muon-neutrino disappearance oscillation channels. This algorithm, CVN, is a powerful approach to the problem of event classification and represents a novel proof of concept—that CNNs can work extremely well with non-natural images like the readout of a sampling calorimeter. We expect this approach to be transferable to a wide range of detector technologies and analyses.

Astrophysics is another area in which deep learning algorithms are being used to great effect, aiding in the discovery of various astronomical objects and phenomena including gravitational lensing and exoplanets. Researchers from the Stanford Linear Accelerator Center (SLAC) have for the first time shown that neural networks can accurately analyze the complex distortions in space-time known as gravitational lenses 10 million times faster than traditional methods.⁴¹ The scientists used neural networks to analyze images of strong gravitational lensing, where the image of a faraway galaxy is multiplied and distorted into rings and arcs by the gravity of a massive object such as a galaxy cluster. The distortions provide important clues about how mass is distributed in space and how that distribution changes over time. These properties are linked to dark matter, which comprises up to 80% of all matter in the universe, and to dark energy that’s accelerating the expansion.

To train the neural networks in what to look for, the researchers fed them about half a million simulated images of gravitational lenses, taking about a day. After they were trained, the networks were able to analyze new lenses almost instantaneously with a precision that was comparable to traditional analysis methods. “Analyses that typically take weeks to months to complete, that require the input of experts and that are computationally demanding, can be done by neural nets within a fraction of a second, in a fully automated way and, in principle, on a cell phone’s computer chip,” said postdoctoral fellow Laurence Perreault Levasseur.

Discovery of exoplanets is another area augmented by machine learning in general, and neural nets in particular.^{42, 43} Finding exoplanets isn’t easy. Compared to their host stars, exoplanets are cold, small, and dark, but with the help of machine learning, progress has recently been made. The measured brightness of a star decreases slightly when an orbiting planet blocks some of the light. The Kepler space telescope observed the brightness of 200,000 stars for four years to hunt for these characteristic signals caused by transiting planets. Using a dataset of more than 15,000 labeled Kepler signals, researchers from Google and the University of Texas, Austin, created a TensorFlow model to distinguish planets from non-planets. After they had a working model, the two research teams then looked at data from 670 stars that were already known to host exoplanets, discovering two new exoplanets in the process. Using this technique, further exoplanet discoveries are expected in the future.

Creating an artificial scientist that could explore datasets and then come up with hypotheses of its own is the dream of many a scientist, not least because it would accelerate scientific discovery, not only in basic research, but also in applied areas such as medicine and materials science. Clearly this would be of great benefit to society with the development of cures for diseases and of life-enhancing drugs, along with stronger, lighter materials. These “robot scientists” are already in use, coming up with hypotheses and equations in a wide variety of areas, including dynamical systems,⁴⁴ quantum experiments,⁴⁵ genomics, drug discovery, scanning journals and scientific databases, solving the Schrodinger equation,^{46, 47} investigating the quantum many-body problem, discovering phase transitions,⁴⁸ exploring transport phenomena,⁴⁹ and research in many more areas.⁵⁰ The interested reader can find further areas of science in which machine learning is being used in this article from O’Reilly and in the presentations and accepted papers from the NIPS 2017 conference.

Why Big Data Is the Foundation of Any Machine Learning Initiative

The good news is that, if you started a big data initiative half a decade ago, it’s about to pay off. Through exponentially increasing advances in hardware and data, the time is right to capitalize on this Cambrian explosion of machine learning technologies. We’ve spoken briefly about advances in hardware (CPU, GPU, FPGA, and ASIC), but thanks to the web there is also an explosion in valuable data that individuals and companies can use for innovation and increased profits.

Data can come from anywhere, both internal and external to the company. A lot of companies and organizations are open sourcing datasets with which you can help train your machine learning algorithms. Other companies make a living by collating and selling annotated (labeled) datasets or models trained on various datasets. Figure 1-5 presents some examples of data sources.

Figure 1-5. Where does data come from?

Here are some examples of open source datasets. The interested reader is encouraged to explore these further:

Images

MNIST, CIFAR-10, ImageNet, PASCAL VOC, Mini-Places2, Food 101, flickr8k, flickr30k, COCO

Text

IMDB, Penn Treebank, Shakespeare Text, bAbI, Hutter-prize

Video

UCF101, Kinetics, YouTube-8M, CMU mocap

Speech

AudioSet, wav2letter, openSLR, TIMIT, BABEL

What Is a Data Scientist?

A data scientist is someone who solves business problems through the identification, collection, and analysis of data. Companies hire data scientists to help solve existing business problems and to suggest and help design new products and services. It is now possible to create products and services that were not feasible before, and even to enter entire new fields and markets, by exploiting the explosion of data and exponential hardware improvements. Applying statistical methods, including machine learning, data scientists can wrangle raw data to identify patterns and make predictions hitherto infeasible using traditional compute and datasets. Distributed computing technologies such as Hadoop and Spark can process huge datasets (sometimes at petabyte scale) that would have been impossible a few years ago. Data scientists are sometimes referred to as “unicorns” because they require a rare combination of computer science, statistics and domain knowledge. A typical data science workflow might look like the following:

1. Data collection

   Data sources, both public and private, are identified and the data is collected and stored in the company’s data storage system in varying formats. This could include SQL or NoSQL databases, which are used for storing structured and unstructured data, respectively.

2. Data preparation

   The process whereby the data is examined and cleaned up. This includes normalizing the data, addressing missing or anomalous data, and removing duplicates. Often, much of this process can be automated using open source or proprietary data preparation tools, such as those listed in the References section.^{51, 52, 53}

3. Feature engineering

   Feature engineering attempts to increase the predictive power of learning algorithms by creating features from raw data that help facilitate the learning process. More recently, automation of feature engineering helps data scientists reduce data exploration time, allowing them to try many ideas quickly. On the other hand, it enables people who are not familiar with data science to quickly extract value from their data with little effort, time, and cost. Tools include FeatureTools (MIT), OneBM (IBM), and ExploreKit (Berkeley).

4. Feature selection

   Depending on the actual problem being addressed, certain features of the data will be more useful than others in solving the problem. These are highlighted and extracted using domain knowledge, data analysis tools, and statistical methods. Feature selection can be done manually or automated.

5. Sampling

   In statistical analysis, one can calculate how large a sample of data is needed to yield a given accuracy in the data analysis. The actual sample size depends on the distribution of the data, whether it’s normal, Poisson, binomial, and so on.

6. Algorithm implementation

   At this point, we can decide on the specific algorithms we are going to use in our data analysis. The idea is to try various algorithms to see which gives the highest accuracy and/or the most precision. Often an ensemble of different algorithms yields the best results. This stage entails a certain amount of trial and error, along with data science experience.

7. Hyperparameter tuning

   Machine learning algorithms consist of various hyperparameters, such as learning rates, initialization values, decay rates, regularization, number and types of layers, activation functions—there are many depending on the specific type of algorithm we are using. Tuning hyperparameters can be a time-consuming process and often more of an art than a science. Fortunately, this process has recently become more automated through the use of various tools such as AutoML, as discussed earlier.

8. Model selection

   We need to decide which model we are going to use on a particular dataset. For example, if we decide to use deep learning, we would need to select from the various available neural network models such as VGGnet, ResNet, Inception, and AlexNet.

9. Model serving (scoring)

   Now it’s time to test and compare our machine learning models. By assigning a score to the results of each model, possibly depending on a multitude of factors like accuracy, time to run, and so forth, we can rank the models in order of efficiency and select the top-performing model(s).

10. Prediction insights

    By running the models on some data of interest, we are now in the position to use our models to make predictions. This can be classification, regression (fitting a curve through the data), or the clustering of unsupervised data.

11. Metrics

    Again, we score the capability of our models by comparing their performance against various metrics that we have determined to be important for our tests. From these tests we can choose what we consider to be the one best model with which to move forward.

12. Proof of Concept

    We need to test the chosen model; that is, deploy it on our hardware, whether that is locally on our laptop, or in a large-scale distributed test environment.

13. Productize and scale

    Finally, we productize our machine learning model by deploying it into our production environment. This will typically be a multinode distributed environment running in a datacenter, or perhaps geographically dispersed for resiliency. TensorFlow Serving is an example of a machine learning framework that productizes machine learning models.

Automation of the Data Science Life Cycle

Another strong trend we see is the automation of the data science pipeline with tools and projects such as Automatic Statistician, along with those discussed and referenced earlier. By the end of 2018, most data science toolkits will include tools for data preparation, exploratory data analysis (EDA), automated feature engineering, hyperparameter tuning, model deployment, and other pipeline tasks.^{54, 55} The field of data science has progressed rapidly in the past five years and will continue to do so.

Buying a Commercial-Off-the-Shelf Solution

Buying a commercial-off-the-shelf (COTS) solution might be best for those companies that lack the personnel to create AI on their own or have non-unique business data. Although buying a proprietary COTS solution is never inexpensive, it can save a lot of time and labor if data science capability is scarce or difficult to hire. Proprietary COTS machine learning solutions include the following:

SAS

Founded in 1976, SAS grew out of the need for a computerized statistics program to analyze vast amounts of agricultural data collected through United States Department of Agriculture (USDA) grants. An abbreviation for Statistical Analysis System, today SAS has more than 14,000 employees in 148 countries.

RapidMiner

RapidMiner is a data science platform that supports all steps of the machine learning process including data preparation, results visualization, model validation, and optimization. It was developed starting in 2001 by a team at the Artificial Intelligence Unit of the Technical University of Dortmund, Germany. Formerly known as YALE (Yet Another Learning Environment), the name of the software was changed to RapidMiner in 2007.

Ayasdi

Ayasdi’s machine intelligence platform combines scalable computing and big data infrastructure with the latest machine learning, statistical, and geometric algorithms and topological data analysis (TDA) to enable data scientists, domain experts, and business people to be more productive. TDA is based on topology, the mathematical discipline that studies the notion of shape. It draws on the philosophy that all data has an underlying shape and that shape has meaning. As a framework, TDA uses dozens of machine learning algorithms to automatically determine the best shape of the data. In doing so, it uncovers subtle patterns that can be missed by other approaches that rely on a single algorithmic dimension. It can determine what in the data is important in an unsupervised fashion—meaning users often find answers to question they didn’t know to ask. Finally, topological data analysis provides justification for its findings and recommendations. Ayasdi was founded by three researchers at Stanford University in 2008.

BigML

Founded in 2011, BigML’s mission is to make machine learning simple and beautiful for everyone. BigML is a consumable, programmable, and scalable machine learning platform designed to solve and automate classification, regression, time-series forecasting, cluster analysis, anomaly detection, association discovery, and topic modeling tasks. With a user-friendly interactive interface, BigML makes it easy to fully automate end-to-end machine learning tasks.

H2O

H2O.ai makes machine learning accessible and allows business users to extract insights from data without needing expertise in deploying or tuning machine learning models. Founded in 2011, H2O provides an open source machine learning platform. Products include Sparkling Water (machine learning on Spark) and Driverless AI, which automates machine learning. Solutions include claims processing, credit scoring, customer churn, fraud detection, ICU monitoring, operational intelligence, and predictive maintenance.

Wolfram Alpha

Wolfram Alpha is the computational knowledge engine from the company founded by Stephen Wolfram, and a leader in applying computing to the sciences. Wolfram Alpha’s long-term goal is to make all systematic knowledge immediately computable and accessible to everyone. It works by using its vast store of expert-level knowledge and algorithms to automatically answer questions, do analysis, and generate reports. It contains over 10-trillion pieces of data from primary sources with continuous updating and more than 50,000 types of algorithms and equations.

Matlab

MathWorks, the maker of Matlab, is a leading developer of mathematical computing software for engineers and scientists. Matlab’s Neural Network Toolbox (NNT) available in the 2017b release, uses the frameworks Caffe and TensorFlow-Keras and models including AlexNet, GoogLeNet, VGG-16, VGG-19, and ResNet-50 to deploy deep learning models. It provides algorithms, pretrained models, and apps to create, train, visualize, and simulate both shallow and deep NNs. You can perform classification, regression, clustering, dimensionality reduction, time-series forecasting, and dynamic system modeling and control. For time-series classification and prediction, the toolbox provides LSTM networks. With NNT, you can visualize intermediate layers and activations, modify network architecture, and monitor training progress.

IBM Watson

Watson is IBM’s machine learning platform. It covers all machine learning analysis such as NLP, image classification, and time-series analysis, in all business domains, including healthcare, FinTech, transportation, and retail. Watson on the IBM Cloud allows you to integrate AI into applications and store, train, and manage data securely and at scale.

Dataiku

Dataiku’s collaborative Data Science Software (DSS) platform helps to automate the data science work flow. It acts as a central hub for technical and nontechnical users to prototype, build, scale, deploy, and manage advanced data science products. The DSS platform contains a collaborative and team-based user interface for data scientists and beginner analysts with plug-ins to common machine learning platforms, such as scikit-learn, MLlib, and H2O.

Bonsai

Bonsai abstracts away the complexity of machine learning libraries like TensorFlow so that developers, data scientists, and subject matter experts can more effectively program and manage AI models. As machine learning and deep learning algorithms evolve, you can recompile and retrain Bonsai’s code to take advantage of low-level technology advances. Applications include increasing automation of control systems such as robotics, warehouse operations, smart manufacturing, and smart sensors. It also optimizes real-time decision support for business systems including energy efficiency, supply chain, network optimization, and planned maintenance.

DataRobot

Founded in 2012, with more than 270 employees, DataRobot offers an automated machine learning platform. It automatically builds and evaluates hundreds of models, enabling users to rapidly build and deploy highly accurate machine learning models. Incorporating a library of hundreds of the most powerful open source machine learning algorithms, the DataRobot platform automates, trains, and evaluates predictive models in parallel. DataRobot brings data science to the masses—what were previously costly initiatives that only a data scientist could design or interpret are now, with automated machine learning, viable for the mainstream market.

Languages

The common languages of data science are Python, R, and Julia. All have their pros and cons. Python is a popular multipurpose language with a large database of users, many coming from the scientific community. R is particularly suited to statistical analysis, growing as it did from the statistics community. Julia combines the user friendliness of Python with the speed of compiled languages such as C/C++. It thus solves the “two-language problem”: the tendency to prototype your data science models in a high-level scripting language and then port them to a faster language like C++ for production. For this reason, Julia is proving to be a very popular and well-received new language among the data science community. Languages are made up of packages, modules, and frameworks. Let’s take a look at some of the ML frameworks and packages that R, Python, and Julia offer.

Open Source Machine Learning Solutions

Building your own solution (e.g., through Python) is appropriate for those organizations that can hire a data scientist or that have unique data. Following is a list of some open source machine learning and deep learning packages:

KNIME

Zurich-based KNIME, with more than 1,500 modules, hundreds of ready-to-run examples, a comprehensive range of integrated tools, and a wide choice of advanced algorithms, provides a comprehensive data scientist toolbox. Through Keras, users have access to a variety of different state-of-the-art deep learning frameworks, such as TensorFlow and CNTK.

NuPIC

NuPIC (Numenta Platform for Intelligent Computing) is an implementation of hierarchical temporal memory (HTM), a theory of intelligence based on the neuroscience of the neocortex. NuPIC is suited to a variety of problems, including anomaly detection and prediction of streaming data sources.

scikit-learn

scikit-learn was started in 2007 as a Google Summer of Code project. A popular open source machine learning toolkit, it contains algorithms for supervised and unsupervised learning, including Gaussian processes and neural networks. It has been a consistent entrant in the Google Summer of Code program, held each summer. It includes dataset loading and transformation, along with model selection and evaluation. Before the release of TensorFlow in November 2015, scikit-learn was the most popular open source machine learning framework. It is built on the Python packages NumPy, SciPy, and matplotlib.

MLlib

MLlib is Apache Spark’s scalable machine learning library consisting of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, and dimensionality reduction. It uses the linear algebra package Breeze.

Stan

Stan is a framework used for statistical modeling, data analysis, and prediction in the social, biological, and physical sciences, engineering, and business. Users specify log density functions in Stan’s probabilistic programming language and get full Bayesian statistical inference with Markov Chain Monte Carlo (MCMC) sampling, approximate Bayesian inference with variational inference, and penalized maximum likelihood estimation with optimization. Stan’s math library provides differentiable probability functions and linear algebra. It is a part on the NumFocus open source software organization.

Weka

Weka (Waikato Environment for Knowledge Analysis) is data mining software written in Java. It is a collection of machine learning algorithms that can either be applied directly to a dataset or called from the users’ Java code. Weka contains tools for data preprocessing, classification, regression, clustering, association rules, and visualization.

Core ML

Core ML is an open source machine learning framework used across Apple products, including Siri. Core ML enables users to build apps with intelligent new features using just a few lines of code. In addition to supporting extensive deep learning with more than 30 layer types, it also supports standard ML models such as tree ensembles, support vector machines (SVMs), and generalized linear models. Features include face tracking, face detection, object tracking, language identification, tokenization, lemmatization, part of speech, and named entity recognition.

Prediction.io

Apache PredictionIO is an open source machine learning server built on top of an open source stack for developers and data scientists to create predictive engines for machine learning tasks. Built on Spark, it supports machine learning and data processing libraries such as Spark MLlib and OpenNLP. Following a successful launch in 2012, PredictionIO was acquired by Salesforce and open sourced to the Apache Foundation in July 2016.

Additional Machine Learning Frameworks

You can find additional machine learning frameworks, both proprietary and open source, at the following locations:

• Collection on GitHub

• Machine learning software

• Comparison of statistical packages

• List of statistical packages

Open Source Deep Learning Frameworks

As we saw earlier, deep learning is a subset of machine learning that involves neural networks as the underlying data analysis technology. Let’s look at a number of those frameworks here.

Commercial Open Source

Commercial open source languages are ones whereby two types of product can be offered: a free open source edition and a commercial edition (which might include more than one type of license). Open source software with commercial support combines the customizability and community of open source with the dedicated support of a commercial partner. These hybrid options are appropriate for teams that want the flexibility of open source packages but also need a support safety net for mission-critical applications. Here’s what commercial support generally means:

• SLA-backed support so that companies can meet their deadlines, rather than relying on community message boards.

• Regular maintenance releases so that upgrades can be planned.

• License indemnification, so if a developer makes changes to an open source library in a way that violates that library’s licensing terms, the commercial support vendor rather than the company is held liable.

Examples of commercial open source vendors include the following:

Anaconda

Anaconda, previously Continuum Analytics, offer a data science package manager with major open source data science libraries prebundled. It offers Anaconda Distribution (free edition) and Anaconda Enterprise, its commercial edition. Prebundled packages include Jupyter, NumPy, pandas, pip, scipy, dask, spyder, and scikit-learn. Both editions include over 1,000 popular data science packages along with the Conda package and virtual environment manager for Windows, Linux, and macOS. The commercial product allows organizations to collaborate, govern, scale, and secure Python and R in enterprise environments.

RStudio

RStudio is an integrated development environment (IDE) for the R programming language. It is available in open source and commercial editions and runs on the desktop (Windows, macOS, and Linux) or in a browser connected to RStudio Server. It includes a code editor, debugging, and visualization tools. RStudio develops free and open tools for R and enterprise-ready professional products for teams to scale and share work.

ActiveState

ActiveState’s Python (ActivePython) for machine learning includes the packages TensorFlow, Keras, scikit-learn, pandas, NumPy, scipy, and matplotlib. ActivePython comes in a community edition and various commercial editions (Business, Enterprise, and OEM). ActiveState Enterprise Edition provides guaranteed technical support, security, legal indemnification, and quality assurance, yielding the advantages of open source while minimizing the risks. Language distributions are available on all platforms (Window/Linux/macOS) as well as Big Iron (AIX, Solaris, and HP-UX).

AI as a Service (Cloud Machine Learning)

Of course, there are always cloud services available, which offer a range of data science services and machine learning frameworks including most of those already mentioned. The most common services are provided by the big four:

• Amazon Web Services (AWS)

• Google Cloud Platform (GCP)

• Microsoft Azure

• IBM Cloud

The following subsections provide login info for the various cloud providers and descriptions of some of their machine learning related service offerings.

AWS

Amazon Machine Learning (Amazon ML; see Figure 1-6) is a cloud-based service that allows developers of all skill levels to use machine learning technology. AI services include Rekognition image and video (vision), Lex (conversational chatbots), and Comprehend, Translate, Transcribe, and Polly (language). AWS uses Kinesis for real-time streaming data, which you can use to ingest data into your machine learning platform.

Figure 1-6. Login screen for AWS

To ease the difficulty of the data science process, Amazon SageMaker enables data scientists and developers to build, train, and deploy machine learning models with high-performance machine learning algorithms, broad framework support, and training, tuning, and inference. SageMaker has a modular architecture so that data scientists can use any or all of its capabilities in existing machine learning workflows.

AWS Deep Learning AMIs are preconfigured environments with which you can quickly build deep learning applications. Built for Amazon Linux and Ubuntu, the AMIs come preconfigured with popular deep learning frameworks such as MXnet with Gluon, TensorFlow, CNTK, Caffe2, PyTorch, and Keras to train custom AI models and experiment with new algorithms.

The AWS Deep Learning AMIs run on Amazon EC2 P2 instances as well as P3 instances that take advantage of NVIDIA’s Volta architecture. The AMIs come integrated with the Intel Math Kernel Library (MKL) and installed with Jupyter notebooks. To simplify package management and deployment, the AMIs install the Anaconda Data Science Platform for large-scale data processing, predictive analytics, and scientific computing. You can find more information at https://aws.amazon.com/machine-learning/amis/ or in the AWS documentation for AMIs.

You can find additional details for all of the above services at https://aws.amazon.com/aml/.

Finally, you can access Amazon machine learning documentation here.

GCP

Google Cloud’s AI (Figure 1-7) provides modern machine learning services with pretrained models and a service to generate tailored models. Google Cloud ML Engine makes it easy for users to build large-scale machine learning models that cover a broad set of scenarios from regression models to image classification. It is integrated with other GCP products such as Cloud Storage, Dataflow, and Datalab. GCP uses Dataflow or Dataproc for real-time streaming data, which you can use to ingest data into a machine learning platform. Cloud Datalab is an interactive tool created to explore, analyze, transform, and visualize data and build machine learning models on GCP.

Figure 1-7. Login screen for GCP

Google Cloud Vision API classifies images into thousands of categories, detects individual objects and faces within images, and finds printed words contained within images. Video analysis, provided by the Google Cloud Video Intelligence API, makes videos searchable and discoverable by extracting metadata, identifying key nouns, and annotating the content of the video.

Google Natural Language API reveals the structure and meaning of text through machine learning models. It extracts information about people, places, and events mentioned in text documents as well as understanding sentiment. Google Cloud Translation API provides a simple interface for translating an arbitrary string into any supported language (e.g., French to English).

Google Cloud Speech API converts audio to text by applying neural network models in an API that recognizes more than 110 languages. DialogFlow is an end-to-end development suite for building conversational interfaces (chatbots) for websites and mobile applications that are capable of natural and rich interactions between your users and your business.

Finally, a strong differentiator that GCP has is the integration of TensorFlow, Kubernetes (the distributed container service), and TPUs into the GCP stack, as they were all developed by Google. As mentioned earlier, AutoML is a service that automatically trains models on various datasets. You can find additional information in the Cloud Machine Learning Engine Documentation.

Azure

Microsoft Azure (Figure 1-8) offers a range of cognitive services on its cloud platform.

Figure 1-8. Login screen for Microsoft Azure

Infrastructure services include Spark and Azure Container Services (AKS), based on Kubernetes. Azure uses Stream Analytics for real-time streaming data, which you can use to ingest data into a machine learning platform.

You can find additional information on the Azure AI Platform here. Documentation for Azure machine learning is available at https://docs.microsoft.com/en-gb/azure/machine-learning/preview/.

Data Science Notebooks

Data science notebooks are proving very popular in the data science community due to their ease of use and capabilities. One can code in them as well as display visualizations and text for the data science analysis. They are also very convenient to share work among collaborative data science efforts. Notebooks come as standalone as well as being offered by the major cloud providers. Jupyter notebooks (the name being a clever concatenation of JUlia, PYThon and R) is the most popular notebook, and several of the following are in fact built on top of the Jupyter kernel:

• Jupyter

• JupyterLab

• Zeppelin

• CoLaboratory

• JunoLab (Julia)

Cloud notebooks:

• Azure

• AWS

• GCP

• IBM

Pros and Cons of Machine Learning Open Source Tools

Open source software is free, which is perfect for those who are budget constrained or just wanting to prototype and try the source code for themselves. Open source is also attractive because the tools are not likely to shrivel up or take an unexpected direction, as proprietary offerings sometimes do. The open source communities around the more popular frameworks such as TensorFlow and MXnet are in the tens of thousands, so there is a great deal of contribution and support in general. The flexibility you have in running open source code is something a lot of developers appreciate because they can fork the code repositories and have full access to the codebase to extend and change the source as they see fit.