Practical AI and how it is done
Artificial Intelligence in business is practical. When you think about neural networks, don’t think about abstract mathematical structures, but rather computer systems that need data to learn business processes and how to operate within them.
Data Science is not a real science, it’s an experimentation domain, where you need to constantly adjust, test, build prototypes from scratch, and rebuild what you have. It’s a framework for approaching problems rather than a specific set of tools. This paradigm of using neural networks, statistics on steroids, is what makes AI both practically and theoretically complex, with such a broad range of applications, which we’re going to cover in the next chapter.
So how Data Science or Artificial Intelligence is currently done? You could split the actual work into two parts, connected strongly with each other:
- implementation,
- research.
Implementation phase is focused on delivering practical solutions to a specific business problem. By using data from within your organisation, data scientists implement machine learning models to learn on this data. This phase is heavily focused on engineering aspects of data science:
- cleaning data
- feature extraction
- statistical analysis
- training neural networks
- setting up virtual machines and a general framework.
Research phase is about looking for possible tweaks, ameliorations, or totally new approaches to existing problems. It may consist of reading scientific papers, white papers from other organisations, browsing open-source code on Github, talking with fellow machine learning engineers, attending conferences. The goal is to broaden perspective and find new strategies to implement.
It’s in general impossible to say what comes first, implementation or research, as the first steps of data scientists are often building the very first naive model, seeing how it works on given data, and then looking for other approaches and enhancements. For harder and more engaging projects, machine learning engineers might start with research, reading what’s possible to find on a subject on the web, and only then choosing a couple of models to implement and try.
Nevertheless, data scientists spend most of the time in front of the computer, whether reading, writing code, or training machine learning models. What’s often misunderstood in corporations is that usual sprints done in classical software development (lean startup method) are not always beneficial to finding solutions to more involved problems that require deep thinking. That’s why the 20% rule of Google, allowing for 20% time off to work on software engineers’ own projects, is so fruitful. Data scientists need to tinker and play around with ideas to boost their creativity.
Research in Artificial Intelligence
The research community in Artificial Intelligence can be split into three divisions:
- machine learning community
- ethics and social community
- business community
Machine Learning community is concerned primarily with research questions related to building machine learning models: from architecture through data to implementations. PhD in computer science or STEM field is necessary to participate actively in it.
Ethics and social community focuses on social ramifications of doing AI research and applying it in practice: from legislations to important questions or limits on what should be the goal of AI research. People in this community often work in social departments of universities, think tanks, or public institutions.
Business community focuses on applying cutting-edge research to business problems. Those may include manufacturing, drug design, cybersecurity, video games, and others. Researchers here work mostly at research labs of large organisations. PhD is not necessary, but often an additional advantage when it comes to looking for a job in those.
If one wants to become a researcher in AI, the standard road is via university, doing a PhD in computer science, and then becoming an assistant professor or a research fellow. Thanks to recent changes in how research works, for example how Amazon, Facebook, Google and similar large tech companies are participating in doing research, it often happens that freshly mint PhDs go directly to one of tech giants’ research laboratories. It’s also possible that they do simultaneously PhD and work at one of those companies which is beneficial to each party: a company sponsors a PhD, the university is relieved from costs, a PhD student has a job and does something relevant to the industry.
PhD thesis itself is a monograph discussing and solving an open problem or some case of it, using novel methods in an already established problem or inventing new problems related to existing knowledge. Some topics are more in fashion at a particular time than others and this relates to interests of particular professors or interest of the market (where the money is). Often during the time of doing a PhD a student publishes a couple of papers, which then consists of the main body of a PhD thesis.
For a PhD student, the most important is finding a good advisor with access to interesting problems, funds, and a research group. Interesting problems will allow him to do meaningful research, funds will allow him to travel to conferences and spend money on infrastructure, the research group will be invaluable for research discussions.
Going to conferences is a great way to connect with fellow scientists. The most popular and most prestigious machine learning conference is NIPS, Neural Information Processing Systems’ annual meeting. The number of scientists applying with their papers to NIPS is growing by 30% each year, which also shows how lively is the machine learning community currently.
From the point of view of business that competes with academia for talent, the crucial aspect is creating a vibrant environment to do research in. Assigning free time to do any research is a good solution, but crucial is building a research group around a senior figure in the field. It was often the case that large organisations hired a professor from the machine learning department together with his PhD students as a way to start up a research community quickly. For example, this is what Uber did with poaching people from Carnegie Mellon’s robotics department.5
A big problem for established institutions like banks or insurers is presenting themselves in an appealing way to potential machine learning employees. Crucial here is understanding that what’s appealing to researches is being able to innovate, have freedom of thought, an atmosphere of openness, and hard problems at hand to solve. No one wants to be stuck with linear regressions all the time. It’s often better to pose too hard problems than too easy problems to attract talent (think Tesla or SpaceX).
Excellent examples of good problems are on Kaggle (www.kaggle.com), where companies run data science challenges for their business problems setting a prize for top entries. Often these competitions are attended by thousands of teams. One of the most famous ones was a competition run by Netflix6 to make better their recommendation algorithms. By putting the prize at $1M the competition attracted a lot of data scientists, put Netflix on a map of great tech companies to work at, and gave Netflix a lot of new research input relevant to their business operation.
Open-source community
Important from a business perspective and still largely underused by the business is the open-source community within machine learning. Much of research is available for free on GitHub, a repository of code, and can be picked up and used jointly with other pieces to build something unique you need. Never making a prototype was so fast and cheap as now. The open-source community is also an excellent source for potential hires as it accurately shows what a given person is capable of by just looking at his or her code repository.
Business-wise supporting the open-source community has many advantages: access to the talent pool, staying informed about current research. Moreover, it can bring business leads. Recall the model of Red Hat which was responsible for maintaining Linux and then earning money via support and customisations. In the end, Red Hat was acquired by IBM in one of the largest tech acquisitions to date at massive $34 billion closed in 2019.
GitHub itself was acquired by Microsoft in 2018 for $7.5 billion,7 and Kaggle was acquired by Google in 2017.8 This not only shows how important open-source community is for business but actually that you can make a business out of open source efforts if you’re able to deliver a great product and build a community of engaged users around it.
From research to applications
Having discussed how research is done in AI, it’s now time to focus on applications. Assuming you already have a data science team in place and preliminary research on a problem you want to solve done, the next step is to gather and clean data. This process can be short if most of your business is digital with easy access to data, or long and painful if you have many sources to look at and data is far from clean (say, surveys of customers done in various formats). If that’s the case, preprocessing is a task that would need a separate team to complete. It’s especially essential for all the later work, so don’t ignore cleaning data.
Applying research to business applications means using machine learning models on data coming from your business and measuring how well they behave compared to how you usually solve the problem at hand (e.g. time spent on a business process, marketing/sales, number of relevant leads). After receiving data, your team of machine learning engineers will put metrics in place to measure progress and start implementing (coding) machine learning algorithms, filling them with data you have provided. The end result will be either a metric of the accuracy of prediction, automated business process, or optimized computations.
What’s crucial in the implementation stage is having good metrics to compare models (architecture), machines (infrastructure), and data used in training and evaluation. Understanding why the results are what they are is as important as arriving at the model yielding the best results.
Scaling the solution to more data and using models in production often requires more engineering than machine learning talent and thus requires hiring a different talent pool than in the research phase. However, as implementation and research are tightly knit, the best is having on board both researchers and engineers to work closely together.
Summing up, practical AI in business is done through a cycle of:
- researching possible machine learning models;
- gathering data;
- using models on collected data;
- improving upon infrastructure, data, and scale.
After step 4, we come again to step 1, having feedback on how models work in practice and what can be improved, researchers can look for new algorithms and methods to tackle the problem at hand. The whole cycle is repeated until metrics defined at the start of the process are satisfied. That’s why to finish the project successfully it’s crucial to build a framework for machine learning experiments at the very beginning, even if it is to change later on with new experiments.
Cons of using AI
Using Artificial Intelligence solutions can create three risks.
Firstly, the machines may have hidden biases due to the data provided for training. For instance, if a system learns which job applicants to accept for an interview by using a data set of decisions made by human recruiters in the past, it may inadvertently learn to perpetuate their racial, gender, ethnic, or other biases. These biases are hard to detect as they won’t appear explicitly, but rather be embedded in the solution where other factors are considered.
The second risk is that, unlike traditional software engineered systems built on explicit logic rules, neural network systems deal with statistical truths rather than literal truths. Thus it much harder or sometimes impossible to prove that the system will work in all cases — especially in situations that weren’t represented in the training data. Lack of verifiability can be a concern in critical applications, such as controlling a nuclear power plant, or when life-death decisions are involved (healthcare, autonomous vehicles).
The third risk is explainability. When a machine learning system makes errors, as it almost inevitably will, diagnosing and correcting precisely what went wrong can be difficult. The underlying algorithmic structure is convoluted and depends on many factors, hard to unwind.
Having discussed risks, let us now discuss the limitations of AI. As every other technology AI has a couple of shortcomings, starting with data requirements and going beyond it, we can list four challenges:
The first challenge appears when we want to apply supervised learning and need labeled data for training. Labeling is now often done by hired annotators, and it can take time to prepare an adequate dataset. New algorithms emerge that require less data or use other tools to create labels by itself.
The second difficulty lies in obtaining data sets that are sufficiently large and comprehensive for training. This is relative to the algorithms we want to use, but for many business use cases, creating or obtaining such massive data sets can be difficult—for example, limited clinical-trial data to predict healthcare treatment outcomes more accurately.
The third challenge is explaining the results of complex machine learning models. Why is this decision suggested? Explainability is vital in certified systems like in healthcare or finances, where regulations play a significant role.
The fourth limitation is generalizability. Transferring knowledge from one set of circumstances to another is still one of the hardest machine learning tasks - and it is studied under the name of transfer learning. Lack of transferability means that companies need to retrain certain models and commit more resources to go outside of known cases.
Summing up, AI is a great tool for any organisation that can harness its power. However it’s not a magical box that solves every type of problem, and often it requires rich resources to do properly, both in talent and in infrastructure.