We observe that only a small fraction of real-world ML systems are composed of ML code (the small black box in the figure shown here). However, the infrastructure surrounding this ML code is vast and complex. There are many services offered by cloud providers to provide and manage the infrastructure required for modern machine learning applications:

The following figure illustrates a typical machine learning pipeline at a conceptual level. However, real-life ML pipelines are a lot more complicated with several models being trained, tuned, combined, and so on.

The following figure shows core elements of a typical machine learning application split into two parts—the modeling including model training and deployed model (used on streaming data to output the results):

Typically, the data scientists experiment or do their modeling work in Python and/or R. Their work is then re-implemented in Java/Scala before deployment in a production environment. The enterprise production environments often consist of web servers, application servers, databases, middleware, and so on. The conversion of prototypical models to production-ready models, typically results in additional design and development efforts that lead to delays in rolling out updated models. However, with cloud services and platforms now available, this effort is reduced substantially as we will see in Chapter 8, Designing a Big Data Application.