What Is an ML Model?

Business analytics, often referred to as simply “analytics,” are business rules encoded into software by an analyst or developer that subsequently converts data into useful insights to inform decision making. By comparison, ML models use learning algorithms to generate business rules using prepared training data.

For supervised and reinforcement learning techniques, these learned rules are encoded in a special file known as a model. Because they are designed to be written and interpreted by machines, these models are much harder to read (explain) than business analytics.

Unlike conventional analytics, the act of feeding user data into the analytic and producing insights is known as inference. Updating ML analytics is accomplished with some combination of the original training data, new/updated data, and the trained model.

Traditionally, organizations treat ML models as black boxes where only their inputs and outputs required explanation. For all the reasons described in Chapter 3, mature organizations are abandoning this black box approach in favor of explainable outcomes.¹ From a leadership standpoint, this shift toward explainability will continue to change how results are documented in the short term and overall design processes in the long term.

ML models are also unique in that similar models and data can produce radically different results. Identical source code is compiled into identical applications, but identical training data produces a variety of models, depending on training parameters. Since it is difficult to predict the best configuration in advance, it is often best practice to train with many different configurations and then compare performance for each of the results during training.

This is especially true for more sophisticated ML approaches, such as neural networks, and accounts for part of these techniques’ reputation for large computational and data storage overhead. Additionally, even the smallest changes to training data can produce meaningful changes to the end result.

These trends result in the potential for hundreds or even thousands of individual ML model versions. Only a small number will be operational, but many (or all, depending on your goals) are worth preserving to provide additional insight into the training process. You should choose how much data and the number of models you’d like to retain based on how explainable the analysis must be and how tightly you can control the operational and training data inputs.

As with any other form of software, it is important for ML models to be documented, tested, and version-controlled to help the team maintain knowledge of its structure and functionality. Unlike conventional analytics, it’s not possible to break up, document, or independently test a model’s internal structure. Instead, we must rely on surrounding documentation, testing, and metadata. While not entirely unique to ML analytics, metadata plays a much larger role in describing a model’s behavior than it does with normal (or classic) software. MLOps (discussed in “Machine Learning Operations (MLOps)”) can help with this documentation, testing, and version control.

DataOps provides the data pipelines to develop and train ML models. Then, MLOps provides an integrated and automated process with a documented CI/CD process to integrate these models into larger software applications that leverage said models using DevSecOps. This approach ensures resulting software applications that utilize integrated models are secure and that data scientists or software engineers can rapidly update the software and models as required. These combined frameworks allow data science practitioners to continuously refine models and their overall approach to analytics.

ML Methodologies

Much like selecting the right tool for a given job, it’s important to understand the algorithms supporting our analytic efforts. Particularly in the case of ML, we use these underlying processes to characterize and define each unique ML approach. By understanding these techniques, we can address any assumptions and identify appropriate performance evaluation metrics. Understanding these techniques also enables us to consider a model’s ability to generalize (i.e., ability to apply each model to sets of unfamiliar data) when putting models into a production environment, and to better monitor and improve our overall design choices.

Supervised learning is perhaps the most often discussed example of ML. With this technique, algorithms are used to analyze numeric or labelled data (e.g., input-output pairs, tagged images, or other closely associated information). Common applications include regression models used for predicting continuous numeric values (e.g., stock prices) and classification models used for predicting categorical (e.g., file is/not malicious) class labels from operational data features.

Supervised ML breaks data into input (called “features”) and outputs (known as “labels” for classification and “targets” for regression) and attempts to predict the latter from the former. Supervised learning’s key benefit is that it enables us to focus the analytic model on a specific task for high accuracy, precision, and recall.² This makes supervised learning the most practical technique for most operational use cases.

Supervised learning’s operational success has resulted in additional research and development into explainability and sustainability, currently far in advance of other ML techniques. The key distinction between supervised learning and other ML approaches centers on the optimization of the model through the process of training its weights (i.e., learned parameters), as depicted in Figure 5-2.

Figure 5-2. Supervised ML process

With supervised learning, labeled input data is fed into the algorithm to train the model, where it refines the algorithm’s “weighting” by testing against specified criteria. The process used to refine a model is rooted in statistical cross-validation, where data is divided into two random subsets—one to train the model, the other to test its performance. Although the specifics get incredibly technical very quickly, the bottom line is this: once sufficiently trained, we can refine and improve deployed models by reengineering training data features and retraining the model as necessary while monitoring its behavior throughout the ML life cycle.

A second ML technique, unsupervised learning, trains a model directly from patterns present in data rather than the input-output approach found in other types of ML. The key advantage with unsupervised algorithms is its capacity for finding patterns not known to the developer beforehand, presenting new avenues of exploration. This approach’s downside is that these algorithms generally find a high volume of patterns in production data, with no focused way of knowing which patterns are valuable to a particular use case apart from abstract data-fit metadata. Additionally, unsupervised learning approaches are inherently less explainable than supervised models, as they usually leverage training data lacking a specified classification or input-output relationship between features. This high-level process is depicted in Figure 5-3.

Figure 5-3. Unsupervised ML process

While unsupervised learning techniques are certainly used as stand-alone approaches to modeling (e.g., anomaly detection and data clustering), we regularly leverage these techniques in tandem with supervised processes to simplify the problem space or gain additional insight into our data through exploratory analysis.

A third technique is reinforcement learning, which provides a novel approach to training models within a specified environment. While there are similarities with supervised learning (e.g., optimization around a relevant penalty or reward metric), reinforcement learning takes a distinct approach to model refinement. Approaches such as Q-learning allow so-called “agents” (i.e., software bots) to interact directly with their environment (physically, virtually, or both) to refine its behavior with a goal of iterating toward a specified objective. The underlying algorithms, including support vector machines (SVM) and neural networks (defined later in this section), generate a model to dictate the agent’s behavior and continue to update based on increased experience within the environment.

Additionally, this technique is considered to be more explainable than a purely unsupervised approach. During a reinforcement learning process, we incrementally feed training data into a model, producing a result evaluated by a reward function, and then provide the model with feedback in the form of the results of the reward function that is combined with subsequent training data to continually refine the model parameters.

As shown in Figure 5-4, the reinforcement learning process begins with users establishing goals, which become the primary objectives for the ML algorithm. When we initialize this process, the agent is completely ignorant of the rules of the experiment, necessitating multiple training runs before it can improve its performance. As the agent’s model continues to update based on reward feedback from the environment, it continually refines its performance, learning from previous successful approaches while deemphasizing less useful tactics. This approach initially gained notoriety through the application to games like chess, AlphaGo, and Starcraft, where, after execution of significant training, the models were able to outperform expert human players at these games.

Figure 5-4. Reinforcement ML process

The primary issue with reinforcement learning is that not all use cases allow for good reward functions, and reward functions may be more difficult to explain than training data for a given use case. Despite its more obscure processing, reinforcement learning provides spectacular results when employed in use cases that provide appropriate and measurable reward functions, such as with robotics and video games. Another benefit of reinforcement learning is that it can produce structured training data to support model development for applications where training data does not currently exist.

Neural networks are an ML implementation that uses layered network architectures (i.e., neurons and weights) to perform a specific function, passing inputs through the network’s layers to calculate an output of interest. While neural networks also leverage statistical methods to evaluate a given input, they are unique in that their inner processes are largely nonlinear abstractions of conventional calculations. A neural network contains a collection of parameters, known as weights and biases, trained to optimize its outputs based on a given criteria. While neural networks continue to grow in complexity and layers of abstraction, in every instance the network’s architecture determines its function.

On their own, neural networks do not specify the method used to train them. They are capable of following the supervised, unsupervised, and reinforcement learning paradigms interchangeably.

For example, the convolutional neural network in Figure 5-5 can ingest machine-readable images, training the algorithm to assess latent patterns and relationships among encoded pixels, and then classify the image with a given degree of certainty.

Figure 5-5. Neural network example

Developing a neural network’s structure is a difficult and expensive design process. To minimize this expense in time and effort, organizations often tailor existing neural networks for a specific application by starting with the structural information and regenerating weighting information from different training data. This process, called transfer learning, is the most approachable way for many organizations to use pretrained neural networks and leverage the extensive training previously executed for similar applications.

As previously mentioned, models are updated in one of several ways, depending on the learning technique. In some use cases, it is not desirable to reload all of the training data that were initially used to create a model to update a model. In this situation, an online learning technique can be used to allow training data to be incrementally added to the model without having to retrain from scratch.

An even more extreme version of this is distributed/federated learning. Recent developments in so-called “edge” technologies, such as widely used mobile phones and other IoT devices, enable ML development using distributed—or remote—architectures.

Distributed learning with edge devices occurs in parallel: training occurs in multiple online locations at the same time, before merging results to combine their overall knowledge. Distributed learning’s key advantage is the ability to enable the model to be configured for inference and training on low-powered edge devices while combining significant amounts of information globally in a single model. The disadvantage of edge learning is that the learning process is increasingly dynamic, complex, and difficult to curate. This environment can create complex scenarios that are extremely difficult to predict, detect, and correct in real time to combine that distributed learning into a valid global model.

Examining the architecture outlined in Figure 5-6, distributed learning’s first step involves pushing a pretrained “pseudo model” to edge devices from a central hub. Once on the edge device, this initial model is refined by available data at the edge, performing inference and training updates on the local device. After these edge-based models are sufficiently trained and updated, we aggregate them at a central hub, versioning the models for eventual redeployment to edge devices. The central hub orchestrates these iterative improvements by using an MLOps approach to monitor model performance.

Figure 5-6. Distributed learning example

ML Advanced Topics

Having discussed conventional ML modeling approaches and their considerations, we should pause a moment to examine emerging frontiers. These specialized applications leverage complex algorithms that combine and further abstract ML approaches. Each of the following approaches enables more mature AI capability. However, as with all techniques, we recommend you choose cautiously when considering the wider implications of interpretability and explainability. Due to the complex architecture within these techniques, we are generally unable to “open the black box” of these models, often forcing us to verify their reliability in development and production environments.³

One of the most impactful developments in ML over the last several decades is deep learning, a collection of methods leveraging multilayered “deep” neural networks to perform highly specialized tasks. Use cases include image classification, machine translation, and speech recognition. In practice, deep learning takes advantage of multiple ML methodologies (e.g., applying unsupervised approaches to transform data as the input for neural networks) to engineer features and improve model performance. These complex models enable AI by leveraging large quantities of data to train and refine parameters of deep neural networks, whose specific architecture enables performance able to exceed human capabilities. As with any ML approach, the underlying structure of the algorithm dictates the functional benefit of its specialized task, underscoring the need to consider responsible implementation, resource requirements, and interpretability of complex deep learning models.

Natural language processing is a broad field containing both manual approaches and AI applications to analyze natural language data. Enabling machines to parse textual data is a critical component of natural language processing, without which we could not ingest data necessary to train the model. Traditionally, challenges are rooted in the subjectivity of human language and difficulty achieving semantic generalization, particularly when working with jargon or translating between structurally dissimilar languages.⁴

Fortunately, recent research⁵ provides novel approaches to encapsulating language, necessitating more robust benchmarking metrics as performance improves. Current natural language processing applications span document classification, sentiment analysis, topic modeling, neural machine translation, speech recognition, and more. As this field continues to mature, you can anticipate increasingly novel applications to promote automated language understanding.

ML Life Cycle

Operationalizing ML consists of four primary phases: business analysis, model development, model vetting, and model operation (e.g., monitoring your model in production). Executing these components with common approaches, practices, and understanding allows an organization to scale ML applications while reducing time-to-market business value of AI systems. To achieve this common understanding, it helps to decompose each component into its respective activities and discuss the methodologies, approaches, and opportunities presented by each, as illustrated in Figure 5-7.

Figure 5-7. ML life cycle

Model Development

Developing ML for business use requires incremental steps (outline in Figure 5-8). Because this development process involves scoping the problem, conducting experiments, reviewing results, and refining the approach, it’s difficult to depict in a straightforward fashion. While each of the five steps may occur at various times, their contribution to the overall process remains consistent.

Figure 5-8. Model development steps

Once the project objectives are set and the business problem is understood, an exploratory analysis is necessary to identify and collect all relevant data and assess their ability to meet the intended business purpose. This process is designed to create a comprehensive view of the available data and use analytic techniques to explore important relationships in the data. Exploratory analysis also serves to identify deficiencies in the data that may prevent the model from meeting the intended business purpose. Data exploration analyzes the data we have available and also informs the type of data needed to develop a useful model.

The data preparation stage transforms data to be used with ML algorithms. You’ll want to check for issues such as missing information, duplicate/invalid data, or imbalanced data.

This stage also includes techniques to help prepare data, including: removing duplicate/invalid records from the dataset; addressing class imbalance (when the total number of a class of data [positive] is far less than the total number of another class of data [negative]); enrichment using additional data sources; and selecting data features and executing feature engineering (to include combining multiple features for additional training benefits). This process is designed to help determine the required input variables when developing a predictive model. Your subject matter experts are invaluable during this phase of model development.

Model training begins once data is prepared. In this step, training and validation data is used to compute the model parameters that optimize performance. Prior to initializing the model training process, it is necessary to define the model to be trained and its corresponding configurations. A general process for model training can typically be used across different algorithms.

For most types of ML, the general training procedure begins by randomly initializing the model and its associated parameters. This model is then trained using data to fine-tune it, generating and comparing predictions for each entry in the training dataset. Using these predictions, the model training computes error and other performance metrics between model prediction and ground truth for each entry (e.g., case, record, image). The computed error, often referred to as “loss,” is then used to adjust model parameters to optimize for desired metrics. The training script will continue to loop through the aforementioned steps until it achieves the desired performance or until the maximum number of training steps has been reached.

The next step, model versioning, is crucial to ensure a workflow is explainable and can be replicated. In this step, models are versioned based on the data used for training, selected specifications (e.g., learning rate, batch size, number of epochs),⁶ and the resulting output. An identifier should be assigned to each model to provide context. Versioning enables your team to track model lineage and pedigree, which will improve results as your project progresses.

Here are some items we recommend tracking during your development efforts: software used to train and run the model; development environment; model architecture; training data used; model parameters; and the artifacts collected during training, such as reports and log files. This process should be standardized and conducted for each model training event with parameters stored in a common repository.

The final step in development, model validation, is vetting a model’s training by using a validation dataset to determine if any additional performance adjustments are necessary. It’s important that this validation dataset not include any data used in the model training stage to ensure validation metrics are fully independent from the training procedure.

In the validation stage, the model you’ve created is used to make predictions for every entry in the test dataset. Prediction error and other model performance metrics are calculated to ensure observed model performance meets user-defined requirements before deploying the model.

Model Vetting

The next step in the ML life cycle is to vet your model, as shown in Figure 5-9. Vetting a trained ML model is a crucial, yet underappreciated, step in the path to operationalizing machine learning. Rigorous model vetting enables the local data science team to discover costly bugs and correct them while still in a low-risk environment. Once vetting is executed locally, it is then replicated in the production environment. Deploying properly working models will build organizational trust, not only in the deployed system but in AI projects as a whole.

Figure 5-9. Model vetting steps

Trained models that meet initial performance requirements are deployed for testing to a production-like environment for a rigorous test and evaluation stage. This step involves conducting computational and performance testing to ensure the model meets requirements. For the sake of consistency, deploying a model to the test environment should parallel the steps taken to deploy a model to an actual production environment. Your development team should work closely with subject matter experts and functional users to ensure the test environment mirrors actual production.

Similar to model development stages, it is important to verify results. This step ensures model performance is in line with expectations and has not suffered from performance degradation. It’s important to note that, based on our experience, most preproduction model issues are discovered during this step. As noted previously, it’s crucial that the data sent to the model during this stage be representative of the data the model will analyze while in a production environment.

Part of the results verification process is exploring how the model handles malformed or otherwise improper data. The model should be able to handle improper data and ensure that important data is not lost, an appropriate error notification is sent to the software development team, and the model remains functional after encountering the error.

Result tracking is critical to ensure your model is effectively reviewed and audited. The ability to store and review historical model output allows subject matter experts and functional users to conduct periodic reviews, ensuring the model is making acceptable and appropriate predictions.

The final step, staging for deployment, prepares the model to be moved to the production environment. Once the model is deployed to the production environment, the model is considered “live” and will be available to all authorized end users and systems. Data scientists, software engineers, and all appropriate experts should be consulted to ensure the model is deployed in a manner that matches expectations in the production environment.

Model Operation

The operationalizing ML workflow’s ultimate goal is the successful deployment and smooth operation of a well-functioning model. The ideal model provides end users with predictions when requested in a timely manner, performs automatic monitoring of model performance and usage, and notifies people of relevant model activity through reporting and alert mechanisms. In short, a successful deployment means that your ML application performs as expected and alerts you when your attention is needed. Model operations (shown in Figure 5-10) will allow you to meet these needs.

Figure 5-10. Model operation steps

The first step in operationalizing a model is deploying the model to the production environment and verifying that the model has been successfully transferred and no issues exist with running the model in the intended environment. The deployment verification is typically done by closely monitoring the model immediately after deployment to ensure it’s receiving, analyzing, and returning appropriate data. This step also involves verifying input data formatting is correct and analyzing output for correctness to ensure that no model performance degradation has occurred. These tests are best performed on the production hosting environment by the software engineering team immediately after deployment.

The operation step involves executing the model when required. During this step, the software engineering team must ensure all aspects of the deployed model are working as intended. Any system issues should be fixed based on their order of importance to ensure the model continues to operate as intended. This process typically requires using regularly scheduled automated tests of the system, validating model responsiveness, ensuring models are being retrained when necessary, and verifying that models are resilient against ingesting malformed or improper data. These events result in system alerts to request human inspection when necessary and a support request system that allows end users to submit help tickets.

Smooth operation of a deployed model requires robust monitoring capabilities. A deployed monitoring system should be able to track model input and output, capture any warning or error message produced by the software environment, compute anomalous model activity (e.g., drift detection analysis, adversarial AI attacks detection), and forward all relevant information to the system’s reporting and alerting interfaces.

Effective monitoring of a deployed system requires several component systems to operate together seamlessly. This is best done by automating as much of the system as possible. Given the speed and scale at which many AI systems operate, it is insufficient to rely solely on human monitoring and analysis to capture all necessary details in a timely fashion. As such, automated monitoring tasks are required for desired monitoring activity and set to run as needed. All results of these automated monitoring tasks are reported and sent as an alert, when deemed necessary.

Reporting allows for human inspection of the deployed model. The speed and scale at which many AI systems operate may make it impractical for a human to review model performance in real time. However, automated reporting at regular time intervals will enable users to verify the system is working as intended. It is also possible for these reports to be used by other automated systems to monitor the deployed model. These reports are examined using visually engaging formats such as a dashboard, spreadsheet, or document to provide user insights.

Alerting modules are responsible for creating real-time notifications about the deployed system’s status. These notifications are meant to either signal people for immediate system review or to create micro-reports to be included in automated system logs. Events that require alert notifications are initially flagged by the model monitoring module. These flagged events are received by the alerting module, where the alerts are drafted based on prewritten notification templates. These notifications are then sent out via a messaging service to the recipient. Based on requirements, notifications can be sent as text messages, emails, Slack messages, app push notifications, or automated phone calls.

Machine Learning Operations (MLOps)

Early in the history of software development, innovations in tooling, automation, and process optimization were focused almost exclusively on efficient delivery of new functionality. DevOps, and later DevSecOps, established frameworks that introduced similar innovations to balance functionality against priorities like automation, reliability, distributed development, and security.

As reviewed in this ML section, there are many steps to the design, training, testing, deployment, and monitoring of ML models. In many cases, these steps are executed manually. However, this greatly reduces the ability to scale the model development process. MLOps⁷ provides a potential solution to this problem through integration and automation to allow for increased efficiencies in the development and training of ML models.

MLOps is similar to DevSecOps, with some unique differences (e.g., use of production data for model training, rapid iterations of models, accounting for bias, need for continuous monitoring) that you must consider. Organizations today are primarily prioritizing the development of AI models and not the process used to develop those models. Through the integration of these manual processes using the rapidly expanding open source and commercial solutions, MLOps will allow an organization to scale their AI development pipelines to meet future demand and operate at the enterprise level.