C++ Software Interoperability for Windows Programmers
Connecting to C#, R, and Python Clients
This Apress imprint is published by the registered company APress Media, LLC part of Springer Nature.
The registered company address is: 1 New York Plaza, New York, NY 10004, U.S.A.
What This Book Is About
At a general level, software interoperability deals with how software components written in one language may be connected to other components written in other languages. More concretely, in C++ Software Interoperability for Windows Programmers we have a specific software ecosystem in mind. We want to connect a C++ codebase (our starting point) to client software written in three different languages: C#, R, and Python. This book has been written for those interested in learning how to build components that connect C++ to these languages and environments.
C++ is the language of choice for developing robust, scalable, and high-performance software. It offers both high-level software design capabilities through object-oriented, generic, and functional features as well as facilities for low-level memory manipulation. However, C++ is often awkward to connect to other languages. In fact, it doesn't communicate particularly well with clients even if these are written in C++. Consider by contrast how easily components written in different .NET languages can be put together and interoperate in different hosting environments. This book deals in a practical way with how C++ software components, specifically static and dynamic-link libraries, can be connected to components written in other languages. The overall aim is to demonstrate how to make the functionality offered by C++ code available to other clients and thereby leverage the power of C++.
This book is about developing the components that connect C++ libraries to client software written in other languages. It is not a book about the most performant or most up-to-date C++ code. These components form a translation or bridging layer and therefore will always have a performance cost. This is not a book about developing new applications. This book is more about how to connect possibly legacy systems with new(er) codebases. It deals with the in-between pieces of software that are not widely covered directly elsewhere. In this book, we are less interested in the specifics of the code itself and more interested in how the code is put together. We use examples from statistics, but these could equally well be from game engine development or quantitative finance. Our focus is on the wrapper components and how they connect to the different client languages.
Broadly speaking, this book looks at two main areas: firstly, the practicalities of building C++ components that target different languages, specifically C#, R, and Python. The main focus here is on the setup and structure of these middle layer components. We cover project types, project setup, and configuration. Secondly, this book is about the usage of frameworks that support interfacing to other languages. We make use of several such frameworks: C++/CLI (C++ adapted for the Common Language Infrastructure) for connecting to C# and the .NET universe, Rcpp to connect to the R language and environment. And lastly, for connecting to Python, we use CPython, Boost.Python, and PyBind. In each case, we will see how the frameworks help in the translation of types between C++ and the target language. We look at the specifics of how types from the C++ universe are converted to and from C#, R, and Python. We also touch on other areas that are important when developing wrapper components. We look at exception handling in the wrapper layer, unit testing, and also debugging. The focus here is on demonstrating in a realistic way some of the fundamental facilities offered by the frameworks.
It is worth emphasizing that there are advantages to developing separate wrapper components that interface with other client languages. From one point of view, legacy systems can be made more available and more open. This in turn improves the lifetime of those systems. And it allows us to retain the original code without having a costly rewrite. From another point of view, we can continue to use high-performance C++ code while at the same time reaping the benefits of perhaps more specialized languages (like R for data analysis) or perhaps languages with a possibly gentler learning curve (like Python). Apart from the benefits of the languages themselves, both R and Python have extensive libraries that support many different types of software development. By writing wrapper components, we can leverage C++ while at the same time taking advantage of what these languages offer.
What This Book Covers
Part 1 Foundations
Chapter 1 introduces the Software Interoperability project. Here, we cover some of the prerequisites. We describe the main features of the project and we give a high-level view of how the different components fit together.
Chapter 2 lays the foundations for the chapters to come. We introduce a simple C++ library of statistical functions that serves as the point of departure for the functionality that we want to expose to other clients. The library is simple enough to be readily understandable. The main focus is not on the (limited) functionality it offers but rather, in a general way, on what we want to expose to clients: built-in types, Standard Library types, and user-defined types. The statistical library is built as both a static library and a dynamic-link library for convenience. Following this, we take a brief look at two C++ clients: a simple console application that uses the static library and a Windows GUI application that uses the dynamic-link library. We see how simple it is to make the functionality available to C++ clients, but at the same time we see that if we want to expose the functionality outside the C++ world, it requires another layer and some different approaches.
Part 2 C++/CLI and .NET
Chapter 3 builds on these foundations. We'd like to make use of the statistics functionality offered by our C++ library. But what can we do if the client application is not written in C++ as in the previous chapter? One answer is to use C++/CLI to build a managed wrapper component for .NET.1 .NET components are extremely versatile: we can easily drop them into a Windows WPF application or an ASP.NET web application, or have them available via Windows PowerShell. We see how the component we build in this chapter can be used anywhere within the extensive .NET framework.
Chapter 4 focuses on consuming the managed wrapper component. We illustrate some .NET features using a simple C# console application. After this, we take a short excursion into Excel as a client. Now that we have the statistical library available as a .NET component, we might like to be able to use it from Excel. Using Excel-DNA, we can easily connect our .NET wrapper component to Excel and take advantage of everything Excel offers as a hosting environment.
Part 3 R and Rcpp
Chapter 5 takes a similar approach to before. This time the question is what if we want to use the statistical functions in our C++ library from R? Again, we need to write a wrapper layer. This is somewhat more involved than previously. We need to set up R/RStudio to compile a library into a package. R has quite specific requirements in terms of how packages should be built and managed, including what compiler should be used. We deal with the details here.
Chapter 6 takes the R package that we built in the previous chapter and exposes the functions via the Rcpp framework. Rcpp eases considerably the task of connecting C++ code to R. We investigate the Rcpp framework and write some R scripts to exercise our statistical functions. We also test out the statistical functions exercised from R.
Part 4 Python
Chapter 7 turns our attention to Python as the client language. In this chapter, we explore a basic approach to building a Python module. This low-level approach is instructive since we are dealing directly with Python objects and have to take care of all the low-level type conversion details ourselves. Here, we create a simple Python script that exercises the functions and classes in the statistical library. But it could just as easily be a micro-service built with Flask.
Chapter 8 extends the previous chapter. First, we use Boost.Python and then PyBind to illustrate different approaches to the wrapper layer. Boost.Python alleviates the burden of writing some translation code and allows us to expose C++ classes and functions to Python, at the cost of some complexity. We do the same using PyBind, a header-only library that simplifies exposing functions and classes in a wrapper component. In fact, we will see that the slightly thick wrapper we started with is slimmed down to almost nothing. PyBind does almost all the work. We just tell it what we want to expose. Now that we have a Python C++ extension module, we develop a Python script to measure the relative performance of using Python vs. C++.
Chapter 9 is the final chapter. We look back at what we've built: we now have working wrapper components that connect C++ to C#, R, and Python. And these should provide starting points for real-world development. The end goal is to broaden the architectural choices available when developing software systems. With these components you should be able to take a C++ codebase and make it available to client software written in C#, R, and Python.
Who This Book Is For
This book is aimed at intermediate-level software developers with some programming experience, particularly in C++ and particularly on Windows. We assume some familiarity with Windows static libraries, dynamic-link libraries, executables, and so on. You should be comfortable building software on Windows. No great expertise in C#, R, or Python is necessary, but a basic knowledge of, and some experience with, the languages and environments is useful. This book is not a primer for any of the languages we deal with and is not exhaustive or systematic in its treatment of either the languages or the frameworks. We let the C++ codebase drive what we need to use in a pragmatic way.
Throughout the book, we use a variety of tools, and you should be comfortable with these (or be able to use suitable alternatives). We use Visual Studio Community Edition 2019 for C++ related development. We use R/RStudio for R development (and Rtools for building the wrapper). We use CodeBlocks as a cross-platform development environment for building using gcc and related tools. We use Visual Studio Code for Python development.
This book is aimed at two slightly different audiences. On the one hand, the book is useful to C++ developers wanting to expand the types of software clients that can use their code. On the other hand, the book is also conceived for data scientists looking to leverage C++ code, either building their own codebases or making use of existing libraries. If you have ever wanted to connect some C++ code to .NET, perform some data analysis using C++ code from R, or build a Python extension module, this book is for you.
In both cases, the aim of this book is to broaden the architectural choices available when developing a loosely coupled software system. This book aims to make you comfortable developing and maintaining those in-between layers. If nothing else, I hope this book helps you get started if you are building components that make use of a C++ codebase and helps avoid some of the difficulties and traps you may find along the way.
First, I'd like to thank my wife for her patience with me during the writing of this book.
I'd also like to thank the technical reviewer for his attention to detail and suggestions while reviewing this book. Thanks as well go to the team at Apress for helping to make the writing process a smooth experience.
