MATLAB is amazing—but you already know this. With a few lines of code, you can load data, manipulate it countless ways, view it, and extract its deeper meaning. MATLAB is also a terrific sandbox for prototyping algorithms and exploring numerical experiments, for simulating phenomena, for churning through test results, and for generating reports.
Additionally, MATLAB’s documentation and tutorials—either built-in, hosted at The MathWorks’ website, or available on YouTube—are thorough and high quality.
The underdocumented and underappreciated MATLAB py module, however, motivated the creation of this book. The py module provides a binary interface between MATLAB and Python. That’s huge—it opens a door from MATLAB to hundreds of thousands of open source Python packages for machine learning, artificial intelligence, data mining, database access, astronometric computation, geographic information services, web services, high performance computing, and countless others.
Better still, the power of Python via MATLAB’s py is available for free to individuals and at low cost to corporations (see Section 1.2). There is a caveat though: to use py , you must also know Python. This book aims to be your guide to learning Python and taking full advantage of MATLAB’s ability to interact with it. When viewed as a MATLAB extension, it can be said without exaggeration that
Python is the most powerful of all MATLAB toolboxes.
1.1 Learn Python Through MATLAB Equivalents
MATLAB: | Python: |
|---|---|
>> format long e >> x = [2.3 3.3]; >> y = [-4 5.0]; >> x*y' 7.300000000000001e+00 >> whos Name Size Bytes Class x 1x2 16 double y 1x2 16 double | In :import numpy as np In : x = np.array([2.3, 3.3]) In : y = np.array([-4, 5.0]) In : x.dot(y) Out: 7.300000000000001 In : whos Var Type Data/Info x ndarray 2: float64, 16 bytes y ndarray 2: float64, 16 bytes |
To keep the examples tight, I’ve edited them for brevity; the actual result from both whos commands has more columns than are shown. MATLAB’s ans = line is also stripped throughout.
Some function names common to MATLAB and Python
Category | MATLAB | Python | Notes |
|---|---|---|---|
Creation | meshgrid() | np.meshgrid() | Create a pair of coordinate arrays |
logspace() | np.logspace() | Exponentially distributed terms | |
linspace() | np.linspace() | Evenly distributed terms | |
ones() | np.ones() | Create a ones-filled array | |
zeros() | np.zeros() | Create a zero-filled array | |
rand() | npr.rand() | Create a random array | |
Operations | all() | np.all() | True if all terms evaluate as true |
angle() | np.angle() | Angle of complex terms | |
any() | np.any() | True if any term evaluates as true | |
ceil() | np.ceil() | Ceiling function | |
cross() | np.cross() | Cross product | |
cumprod() | np.cumprod() | Cumulative product | |
cumsum() | np.cumsum() | Cumulative sum | |
diag() | np.diag() | Create a diagonal matrix or extract diagonal terms | |
tril() | np.tril() | Extract lower triangular terms | |
triu() | np.triu() | Extract upper triangular terms | |
floor() | np.floor() | Floor function | |
histogram() | np.histogram() | Create histogram matrices from coordinate vectors | |
Interpolation | polyfit() | np.polyfit() | Fit a polynomial to data |
griddata() | sci.griddata() | 2D interpolation | |
Plotting | plot() | plt.plot() | Create line plots |
imshow() | plt.imshow() | Display an array graphically | |
contourf() | plt.contourf() | Display filled contours | |
spy() | plt.spy() | Display sparsity pattern |
While the similarities are encouraging, the two languages have fundamental differences that will be most noticeable in Chapter 4.
1.2 Is Python Really Free?
The source code for Python and its most important modules is freely available under open source licenses. However, there’s a world of difference between millions of lines of free source code and a robust Python installation with secure package management on your organization’s Windows, Linux, and macOS computers.
At least two companies, Anaconda and ActiveState, offer licensed Python distributions which solve complex installation, bundling, deployment, package management, and security-vetting issues. Both companies permit free use of their distributions for individuals, but charge for commercial deployments.
Large organizations can bypass licensed distributions from Anaconda or ActiveState by building their own distributions—but this requires considerable expertise and adds commensurate labor costs. In addition to the challenges of compiling Python and the hundreds of library dependencies needed by popular modules on multiple architectures, a skilled team would be needed to maintain the organization’s private module repository and continuously update and scan modules for security.
In this sense, a Python deployment for anything beyond a small organization is decidedly not free. This situation is not unique to Python. Any popular open source software package deployed widely in an organization should be managed with installation tracking, security vetting, version management, and so on. This applies to node and its package manager, npm, Ruby and its Gems, Rust and crates, Perl and CPAN, R and CRAN, and so on.
If Python has a substantial cost, why not just stick with MATLAB? There are four main reasons: (1) Python is so popular that large organizations may have Anaconda or ActiveState licenses anyway, independently of MATLAB and its users’ needs; (2) Python licenses from Anaconda and ActiveState are far less expensive than MATLAB licenses; (3) unlike MATLAB toolboxes, extra Python modules do not add costs; and (4) Python instances are not bound to license servers or license files—once a corporation purchases licenses from Anaconda or ActiveState, subsequent Python use by the covered number of developers is unrestricted. This last point greatly reduces logistical hassles for deployment on stand-alone networks and use at home or on the road.
1.3 What About Toolboxes?
MATLAB’s full power is realized through a large collection of domain-specific—and separately sold—toolboxes. The examples in this book, however, use only the core MATLAB product.
Nonetheless, many solutions that are otherwise only available from many toolboxes, namely, Mapping, Curve Fitting, Database, Parallel Computing, Symbolic, Statistics, Optimization, Controls, and Signal Processing, are available to MATLAB by calling Python functions. As a simple example, a weighted least squares function exists in MATLAB’s Curve Fitting Toolbox. If you don’t have the Curve Fitting Toolbox, though, you can compute weighted least squares in MATLAB by calling Python’s WLS() function in the statsmodels module (demonstrated in Section 11.7.1).
1.4 Why Python Won’t Replace MATLAB
I’m occasionally asked if Python will replace MATLAB. For certain tasks, yes, it absolutely can. As problems become more specialized, however, you’ll find that MATLAB toolboxes made to solve those problems have no credible competition. Controls engineers will rightly insist on one or more of the Controls Toolboxes; engineers that work with embedded hardware will need Coder and the Fixed-Point Designer; aircraft designers will be able to explore large tradespaces with the Aerospace Blockset.
Even in cases where Python alternatives exist, it is much easier and cleaner to call a toolbox function than maintain a hybrid Python/MATLAB code base, keep Python installations synchronized with MATLAB updates, and write harder-to-read code with awkward-looking function calls and type conversions. In the long run, it may even cost less to use toolboxes by avoiding the complexity of adding Python to your MATLAB work flow. The balance of benefits against drawbacks will vary between use cases, individuals, projects, and organizations.
No further mention of Simulink will appear in this book because it stands in a league of its own; there are no Python projects with remotely similar capabilites.1
The MathWorks products are invaluable when you’re using them to solve the most niche and technically specialized problems conveniently, thoroughly, and quickly. Python can play a supporting role alongside MATLAB and Simulink but won’t be their replacement.
Having said that, once you’re proficient with Python you’ll find you need MATLAB less. That’s not so much an indictment against MATLAB as it is an endorsement of Python. Python, being a general-purpose language, excels at many things MATLAB is less suited for. Let Python take care of the “low-hanging fruit” tasks such as aggregating data from a collection of NetCDF4 files, scanning a directory for Excel files having tabs named “2017 inventory,” and reformatting plain text data into a set of JSON records.
Oddly, Python may also be the better choice at the other end of the spectrum for the biggest and most computationally challenging jobs. Unlike MATLAB, Python’s parallel processing modules such as Dask are not restricted by licenses, so you can run on as many processors as you can access.
Knowing Python alongside MATLAB expands your problem-solving options. Whether that means solving new problems with just Python or with MATLAB/Python hybrids, you’ll have considerably more power at your command than you have with MATLAB alone.
1.5 Contents at a Glance
This book aims to be several things: a Python tutorial; a collection of MATLAB/Python equivalent expressions, functions, and programs; a reference manual for solving computational problems in Python; and a cookbook of MATLAB-calling-Python examples. Where should you start?
All readers, even Python experts, should begin with Chapter 2, “Installation.” It describes how to create a MATLAB-friendly virtual environment needed to run the examples in this book.
MATLAB developers unfamiliar with Python can get up to speed by reading—and practicing examples in—Chapters 3 and 4, “Language Basics” and “Data Containers.” Once you’re comfortable with the basics of Python, skip around to sections that interest you.
Chapter 5, “Dates and Times,” covers the details of measuring, reading, writing, and shifting date and time objects.
Chapter 6, “Call Python Functions from MATLAB,” begins the heart of this book. It covers the basics of MATLAB’s py module, shows how to extend the Python search path within MATLAB, describes the nuances of Python-native variables in MATLAB and how to convert them to MATLAB-native variables, and shows other mechanisms for the two languages to interact.
“Input and Output,” Chapter 7, covers reading and writing text and binary files common to numeric work: plain text, JSON, YAML, XML, CSV, Excel, raw binary, HDF5, netCDF. The section on network I/O shows how to write TCP/IP clients and servers in both Python and MATLAB. The first simple MATLAB-calling-Python recipes (to let MATLAB read YAML and .ini files) appear in this chapter.
In addition to reading and writing files, programs must sometimes read directory contents, search for files, get file metadata like size and modification time, and remove directories. These actions are covered in Chapter 8, “Interacting with the File System.”
Python and MATLAB occasionally need to run external executables, check environment variables, and monitor the computer’s load. Chapter 9, “Interacting with the Operating System and External Executables,” shows how this is done in the two languages.
Chapter 10, “Object-Oriented Programming,” shows how classes are defined and instantiated in MATLAB and Python.
Chapter 11, “NumPy and SciPy,” spans nearly a quarter of this book. It covers NumPy’s ndarray, the Python object closest to MATLAB’s matrix, as well as the many functions that perform array operations, linear algebra, and interpolation. SciPy, a scientific computing package built on NumPy, brings additional capability such as sparse matrices, optimization, curve fitting, and differential equation solvers.
Data visualization is covered in Chapter 12, “Plotting.” In addition to the basics of point, line, and contour plots, overlaying data on maps is also described.
The Pandas module for Python appeared five years before The MathWorks added tables to MATLAB. Chapter 13, “Tables and Dataframes,” shows the basics of Pandas dataframes and how equivalent operations are done with MATLAB tables.
Chapter 14, “High Performance Computing,” explores ways to make Python run faster—and faster Python can mean faster MATLAB programs if you’re willing to implement slow MATLAB functions in Python. The chapter has examples of using Cython, Pythran, f2py, and Numba to make Python run faster on a single computer, and dask to run Python on clusters. You’ll see that wherever a Python program is accelerated, the analogous MATLAB program can also be accelerated the same amount just by having MATLAB call faster Python functions.
MATLAB and Python both have sharp edges that can bring grief to new developers. Chapter 15, “Language Pitfalls,” covers aspects of the two languages to watch out for.
Appendix A lists all MATLAB-calling-Python recipes in this book while appendices B-E contain source listings that are too long for the body of the book. All source code from the book can be found at the book’s Github repository at https://github.com/Apress/python-for-matlab-development.
1.6 I Already Know Python. How Do I Call Python Functions in MATLAB?
First, create a MATLAB-friendly virtual environment, matpy , as described in Section 2.5.1. Then jump straight to Chapter 6, “Call Python Functions from MATLAB.” Refer to Appendix A for the list of MATLAB-calling-Python recipes; these examples can help guide your own code development.
1.7 The Recipes Don’t Work! MATLAB Crashes! (and What to Do About It)
MATLAB and Python, specifically the Anaconda distribution, occupy a large footprint on your computer. Among other files, each comes with more than a thousand shared libraries. Some libraries are common to both (e.g., ARPACK, BLAS, HDF5, Intel Math Kernel Library, JPEG, LAPACK, Qt, Xerces-C), but these common libraries are often at different—and incompatible—versions.
This is a problem. If you’re in MATLAB and you use its py module to access a Python function which loads a shared library common to both Python and MATLAB, which version is loaded, MATLAB’s or Python’s? The answer depends on several factors including your environment variables. If your environment is not set up properly, the function call may invoke an incompatible library routine causing the function call to fail, or MATLAB to crash.
The best MATLAB-friendly environment I’ve been able to devise is the matpy virtual environment described in Section 2.5.1. matpy cannot prevent all MATLAB crashes. An example is loading Python’s matplotlib module into MATLAB, then making a plot using matplotlib's default Qt backend. MATLAB crashes as soon as the plot command is called:
The workaround is to configure a different matplotlib backend: TkAgg on Linux and macOS and WXAgg on Windows; details are in Section 12.4.5.
Software on a computer is in constant flux. Operating system updates, security patches, MATLAB version changes, and Python module updates can all affect how hybrid MATLAB/Python programs work from a matpy environment.
If an example from this book doesn’t work or causes MATLAB to crash, describe your setup in an issue at the book’s GitHub page, https://github.com/Apress/python-for-matlab-development.
Ideally, I’ll be able to post a working solution. That won’t always be possible since I might not be able to duplicate your problem or match your combination of OS and MATLAB versions. If nothing else, the problem you hit will get wider exposure—and possibly a crowd-sourced solution.