4.1.1 Matrices

By default, all variables in MATLAB are double-precision matrices. You do not need to declare a type for these variables. Matrices can be multidimensional and are accessed using 1-based indices via parentheses. You can address elements of a matrix using a single index, taken column-wise, or one index per dimension. To create a matrix variable, simply assign a value to it, like this 2 × 2 matrix a:

>> a = [1 2; 3 4];

>> a(1,1)

     1

>> a(3)

     2

You can simply add, subtract, multiply, and divide matrices with no special syntax. The matrices must be the correct size for the linear algebra operation requested. A transpose is indicated using a single quote suffix, A’, and the matrix power uses the operator ̂.

>> b = a’*a;

>> c = a^2;

>> d = b + c;

By default, every variable is a numerical variable. You can initialize matrices to a given size using the zeros, ones, eye, or rand functions, which produce zeros, ones, identity matrices (ones on the diagonal), and random numbers, respectively. Use isnumeric to identify numeric variables.

Table 4.1 summarizes some key functions for interacting with matrices.

4.1.2 Cell Arrays

One variable type unique to MATLAB is cell arrays. This is really a list container, and you can store variables of any type in elements of a cell array. Cell arrays can be multidimensional, just like matrices, and are useful in many contexts.

Cell arrays are indicated by curly braces, {}. They can be of any dimension and contain any data, including string, structures, and objects. You can initialize them using the cell function, recursively display the contents using celldisp, and access subsets using parentheses just like for a matrix. A short example is below.

>> c = cell(3,1);

>> c{1} = ’string’;

>> c{2} = false;

>> c{3} = [1 2; 3 4];

>> b = c(1:2);

>> celldisp(b)

b{1} =

string

b{2} =

     0

Using curly braces for access gives you the element data as the underlying type. When you access elements of a cell array using parentheses, the contents are returned as another cell array, rather than the cell contents. MATLAB help has a special section called Comma-Separated Lists that highlights the use of cell arrays as lists. The code analyzer will also suggest more efficient ways to use cell arrays. For instance,

Replace

a = {b{:} c};

with

a = [b {c}];

Cell arrays are especially useful for sets of strings, with many of MATLAB’s string search functions optimized for cell arrays, such as strcmp.

Use iscell to identify cell array variables. Use deal to manipulate structure array and cell array contents.

Table 4.2 summarizes some key functions for interacting with cell arrays.

4.1.3 Data Structures

Data structures in MATLAB are highly flexible, leaving it up to the user to enforce consistency in fields and types. You are not required to initialize a data structure before assigning fields to it, but it is a good idea to do so, especially in scripts, to avoid variable conflicts.

Replace

d.fieldName = 0;

with

d = struct;

d.fieldName = 0;

In fact, we have found it generally a good idea to create a special function to initialize larger structures that are used throughout a set of functions. This is similar to creating a class definition. Generating your data structure from a function, instead of typing out the fields in a script, means you always start with the correct fields. Having an initialization function also allows you to specify the types of variables and provide sample or default data. Remember, since MATLAB does not require you to declare variable types, doing so yourself with default data makes your code that much clearer.

■ TIP Create an initialization function for data structures.

You make a data structure into an array simply by assigning an additional copy. The fields must be in the same order, which is yet another reason to use a function to initialize your structure. You can nest data structures with no limit on depth.

d = MyStruct;

d(2) = MyStruct;

function d = MyStruct

d = struct;

d.a = 1.0;

d.b = ’string’;

MATLAB now allows for dynamic field names using variables, that is, structName.(dynamic Expression). This provides improved performance over getfield, where the field name is passed as a string. This allows for all sorts of inventive structure programming. Take our data structure array in the previous code snippet, and let’s get the values of field a using a dynamic field name; the values are returned in a cell array.

>> field = ’a’;

>> values = {d.(field)}

values =

    [1]    [1]

Use isstruct to identify structure variables and isfield to check for the existence of fields. Note that isempty will return false for a struct initialized with struct, even if it has no fields.

Table 4.3 provides key functions for structs.

4.1.4 Numerics

While MATLAB defaults to doubles for any data entered at the command line or in a script, you can specify a variety of other numeric types, including single, uint8, uint16, uint32, uint64, logical (i.e., an array of booleans). Use of the integer types is especially relevant to using large data sets such as images. Use the minimum data type you need, especially when your data sets are large.

4.1.5 Images

MATLAB supports a variety of formats, including GIF, JPG, TIFF, PNG, HDF, FITS, and BMP. You can read in an image directly using imread, which can determine the type automatically from the extension, or fitsread. (FITS stands for Flexible Image Transport System and the interface is provided by the CFITSIO library.) imread has special syntaxes for some image types, such as handling alpha channels for PNG, so you should review the options for your specific images. imformats manages the file format registry and allows you to specify handling of new user-defined types if you can provide read and write functions.

You can display an image using either imshow, image, or imagesc, which scales the colormap for the range of data in the image.

For example, we use a set of images of cats in Chapter 7, Face Recognition. If we look at the image info for one of these sample images using imfinfo,

>> imfinfo(’IMG_4901.JPG’)

ans =

            Filename: ’MATLAB/Cats/IMG_4901.JPG’

         FileModDate: ’28-Sep-2016␣12:48:15’

            FileSize: 1963302

              Format: ’jpg’

       FormatVersion: ’’

               Width: 3264

              Height: 2448

            BitDepth: 24

           ColorType: ’truecolor’

     FormatSignature: ’’

     NumberOfSamples: 3

        CodingMethod: ’Huffman’

       CodingProcess: ’Sequential’

             Comment: {}

                Make: ’Apple’

               Model: ’iPhone␣6’

         Orientation: 1

         XResolution: 72

         YResolution: 72

      ResolutionUnit: ’Inch’

            Software: ’9.3.5’

            DateTime: ’2016:09:17␣22:05:08’

    YCbCrPositioning: ’Centered’

       DigitalCamera: [1x1 struct]

             GPSInfo: [1x1 struct]

       ExifThumbnail: [1x1 struct]

and we view this image using imshow, it will publish a warning that the image is too big to fit on the screen and that it is displayed at 33%. If we view it using image, there will be a visible set of axes. image is useful for displaying other two-dimensional matrix data as individual elements per pixel. Both functions return a handle to an image object; only the axes’ properties are different. Figure 4.1 shows the resulting figures. Note the labeled axes on the right figure.

Figure 4.1 Image display options. A figure created using imshow is on the left and a figure using image is on the right.

Table 4.4 provides key images for interacting with images.

4.1.6 Datastore

Datastores allow you to interact with files containing data that are too large to fit in memory. There are different types of datastores for tabular data, images, spreadsheets, databases, and custom files. Each datastore provides functions to extract smaller amounts of data that do fit in memory for analysis. For example, you can search a collection of images for those with the brightest pixels or maximum saturation values. We will use our directory of cat images as an example.

>> location = pwd

location =

/Users/Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats

>> ds = datastore(location)

ds =

  ImageDatastore with properties:

      Files: {

             ’␣.../Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats/IMG_0191.png’;

             ’␣.../Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats/IMG_1603.png’;

             ’␣.../Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats/IMG_1625.png’

              ... and 19 more

             }

     Labels: {}

    ReadFcn: @readDatastoreImage

Once the datastore is created, you use the applicable class functions to interact with it. Datastores have standard container-style functions like read, partition, and reset. Each type of datastore has different properties. The DatabaseDatastore requires the Database Toolbox and allows you to use SQL queries.

MATLAB provides the MapReduce framework for working with out-of-memory data in datastores. The input data can be any of the datastore types, and the output is a key-value datastore. The map function processes the datastore input in chunks and the reduce function calculates the output values for each key. mapreduce can be sped up by using it with the MATLAB Parallel Computing Toolbox, Distributed Computer Server, or Compiler. Table 4.5 gives key functions for using datastores.

4.1.7 Tall Arrays

Tall arrays are new to release R2016b of MATLAB. They are allowed to have more rows than will fit in memory. You can use them to work with datastores that might have millions of rows. Tall arrays can use almost any MATLAB type as a column variable, including numeric data, cell arrays, strings, datetimes, and categoricals. The MATLAB documentation provides a list of functions that support tall arrays. Results for operations on the array are only evaluated when they are explicitly requested using the gather function. The histogram function can be used with tall arrays and will execute immediately.

The MATLAB Statistic and Machine Learning Toolbox^TM, Database Toolbox, Parallel Computing Toolbox, Distributed Computing Server, and Compiler all provide additional extensions for working with tall arrays. For more information about this new feature, use the following topics in the documentation:

• Tall Arrays

• Analysis of Big Data with Tall Arrays

• Functions That Support Tall Arrays (AZ)

• Index and View Tall Array Elements

• Visualization of Tall Arrays

• Extend Tall Arrays with Other Products

• Tall Array Support, Usage Notes, and Limitations

Table 4.6 gives key functions for using Tall Arrays.

4.1.8 Sparse Matrices

Sparse matrices are a special category of matrix in which most of the elements are zero. They appear commonly in large optimization problems and are used by many such packages. The zeros are “squeezed” out and MATLAB stores only the nonzero elements along with index data such that the full matrix can be recreated. Many regular MATLAB functions, such as chol or diag, preserve the sparseness of an input matrix. Table 4.7 gives key functions for sparse matrices.

4.1.9 Tables and Categoricals

Tables were introduced in release R2013 of MATLAB and allow tabular data to be stored with metadata in one workspace variable. It is an effective way to store and interact with data that one might put in, or import from, a spreadsheet. The table columns can be named, assigned units and descriptions, and accessed as one would fields in a data structure, that is, T.DataName. See readtable on creating a table from a file, or try out the Import Data button from the command window.

Categorical arrays allow for storage of discrete nonnumeric data, and they are often used within a table to define groups of rows. For example, time data may have the day of the week, or geographic data may be organized by state or county. They can be leveraged to rearrange data in a table using unstack.

You can also combine multiple data sets into single tables using join, innerjoin, and outerjoin, which will be familiar to you if you have worked with databases.

Table 4.8 lists key functions for using tables.

4.1.10 Large MAT-Files

You can access parts of a large MAT-file without loading the entire file into memory by using the matfile function. This creates an object that is connected to the requested MAT-file without loading it. Data are only loaded when you request a particular variable, or part of a variable. You can also dynamically add new data to the MAT-file.

For example, we can load a MAT-file of neural net weights generated in a later chapter.

>> m = matfile(’PitchNNWeights’,’Writable’,true)

m =

  matlab.io.MatFile

  Properties:

      Properties.Source: ’/Users/Shared/svn/Manuals/MATLABMachineLearning/MATLAB/PitchNNWeights.mat’

    Properties.Writable: true

                      w: [1x8 double]

We can access a portion of the previously unloaded w variable, or add a new variable name, all using this object m.

>> y = m.w(1:4)

y =

     1     1     1     1

>> m.name = ’Pitch␣Weights’

m =

  matlab.io.MatFile

  Properties:

      Properties.Source: ’/Users/Shared/svn/Manuals/MATLABMachineLearning/MATLAB/PitchNNWeights.mat’

    Properties.Writable: true

                   name: [1x13 char]

                      w: [1x8  double]

>> d = load(’PitchNNWeights’)

d =

       w: [1 1 1 1 1 1 1 1]

    name: ’Pitch␣Weights’

There are some limits to the indexing into unloaded data, such as struct arrays and sparse arrays. Also, matfile requires MAT-files using version 7.3, which is not the default for a generic save operation as of R2016b. You must either create the MAT-file using matfile to take advantage of these features or use the -v7.3’ flag when saving the file.

4.2.1 Problem

It’s always a good idea to use a special function to define a data structure you are using as a type in your codebase, similar to writing a class but with less overhead. Users can then overload individual fields in their code, but there is an alternative way to set many fields at once: an initialization function, which can handle a parameter-pair input list. This allows you to do additional processing in your initialization function. Also, your parameter string names can be more descriptive than you would choose to make your field names.

4.2.2 Solution

The simplest way to implement the parameter pairs is using varargin and a switch statement. Alternatively, you could write an inputParser, which allows you to specify required and optional inputs as well as named parameters. In that case, you have to write separate or anonymous functions for validation that can be passed to the inputParser, rather than just write out the validation in your code.

4.2.3 How It Works

We will use the data structure developed for the automobile simulation in Chapter 12 as an example. The header lists the input parameters along with the input dimensions and units, if applicable.

 %% AUTOMOBILEINITIALIZE Initialize the automobile data structure.

 %

 %% Form:

 %  d = AutomobileInitialize( varargin )

 %

 %% Description

 % Initializes the data structure using parameter pairs.

 %

 %% Inputs

% varargin:  (’parameter’,value,...)

%

% ’mass’                                   (1,1) (kg)

% ’steering angle’                         (1,1) (rad)

% ’position tires’                         (2,4) (m)

% ’frontal drag coefficient’               (1,1)

% ’side drag coefficient’                  (1,1)

% ’tire friction coefficient’              (1,1)

% ’tire radius’                            (1,1) (m)

% ’engine torque’                          (1,1) (Nm)

% ’rotational inertia’                     (1,1) (kg-m^2)

% ’state’                                  (6,1) [m;m;m/s;m/s;rad;rad/s]

The function first creates the data structure using a set of defaults and then handles the parameter pairs entered by a user. After the parameters have been processed, two areas are calculated using the dimensions and the height.

function d = AutomobileInitialize( varargin )

% Defaults

d.mass        = 1513;

d.delta       = 0;

d.r           = [  1.17 1.17 -1.68 -1.68;...

                  -0.77 0.77 -0.77  0.77];

d.cDF         = 0.25;

d.cDS         = 0.5;

d.cF           = 0.01; % Ordinary car tires on concrete

d.radiusTire  = 0.4572; % m

d.torque      = d.radiusTire*200.0; % N

d.inr         = 2443.26;

d.x           = [0;0;0;0;0;0];

d.fRR         = [0.013 6.5e-6];

d.dim         = [1.17+1.68 2*0.77];

d.h           = 2/0.77;

d.errOld      = 0;

d.passState   = 0;

n = length(varargin);

for k = 1:2:length(varargin)

  switch lower(varargin{k})

    case ’mass’

      d.mass        = varargin{k+1};

    case ’steering␣angle’

      d.delta       = varargin{k+1};

    case ’position␣tires’

      d.r           = varargin{k+1};

    case ’frontal␣drag␣coefficient’

      d.cDF         = varargin{k+1};

    case ’side␣drag␣coefficient’

      d.cDS         = varargin{k+1};

    case ’tire␣friction␣coefficient’

      d.cF          = varargin{k+1};

    case ’tire␣radius’

      d.radiusTire    = varargin{k+1};

    case ’engine␣torque’

      d.torque      = varargin{k+1};

    case ’rotational␣inertia’

      d.inertia     = varargin{k+1};

    case ’state’

      d.x           = varargin{k+1};

    case ’rolling␣resistance␣coefficients’

      d.fRR         = varargin{k+1};

    case ’height␣automobile’

      d.h           = varargin{k+1};

    case ’side␣and␣frontal␣automobile␣dimensions’

      d.dim         = varargin{k+1};

  end

end

% Processing

d.areaF      = d.dim(2)*d.h;

d.areaS      = d.dim(1)*d.h;

To perform the same tasks with inputParser, you add a addRequired, addOptional, or addParameter call for every item in the switch statement. The named parameters require default values. You can optionally specify a validation function; in the example below we use isNumeric to limit the values to numeric data.

p = inputParser(’FunctionName’,’AutomobileInitialize’,... % throw errors as from AutomobileInitialize

                ’PartialMatching’,false);  % disallow partial matches

cDF_Default         = 0.25;

mass_Default         = 1513;

addParameter(p,’mass’,mass_Default,@isnumeric);

addParameter(p,’cDF’,cDF_Default,@isnumeric);

parse(p,varargin{:});

d = p.Results;

In this case, the results of the parsed parameters are stored in a Results substructure.

4.3.1 Problem

We discussed the datastore class in the introduction to the chapter. Now let’s use it to perform analysis on the full set of cat images using mapreduce, which is scalable to very large numbers of images.

4.3.2 Solution

We create the datastore by passing in the path to the folder of cat images. We also need to create a map function and a reduce function, to pass into mapreduce. If you are using additional toolboxes like the Parallel Computing Toolbox, you would specify the reduce environment using mapreducer.

4.3.3 How It Works

First, create the datastore using the path to the images.

>> imds = imageDatastore(’MATLAB/Cats’);

imds =

  ImageDatastore with properties:

      Files: {

             ’␣.../Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats/IMG_0191.png’;

             ’␣.../Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats/IMG_1603.png’;

             ’␣.../Shared/svn/Manuals/MATLABMachineLearning/MATLAB/Cats/IMG_1625.png’

              ... and 19 more

             }

     Labels: {}

    ReadFcn: @readDatastoreImage

Second, we write the map function. This must generate and store the intermediate values that will be processed by the reduce function. Each intermediate value must be stored as a key in the intermediate key-value datastore using add. In this case, the map function will receive one image each time it is called.

function catColorMapper(data, info, intermediateStore)

add(intermediateStore, ’Avg␣Red’, struct(’Filename’, info.Filename, ’Val’, mean(mean(data(:,:,1)))) );

add(intermediateStore, ’Avg␣Blue’, struct(’Filename’, info.Filename, ’Val’, mean(mean(data(:,:,2)))) );

add(intermediateStore, ’Avg␣Green’, struct(’Filename’, info.Filename, ’Val’, mean(mean(data(:,:,3)))) );

The reduce function will then receive the list of the image files from the datastore once for each key in the intermediate data. It receives an iterator to the intermediate datastore as well as an output datastore. Again, each output must be a key-value pair. The hasnext and getnext functions used are part of the mapreduce ValueIterator class.

function catColorReducer(key, intermediateIter, outputStore)

% Iterate over values for each key

minVal = 255;

minImageFilename = ’’;

while hasnext(intermediateIter)

  value = getnext(intermediateIter);

  % Compare values to find the minimum

  if value.Val < minVal

      minVal = value.Val;

      minImageFilename = value.Filename;

  end

end

% Add final key-value pair

add(outputStore, [’Maximum␣’ key], minImageFilename);

Finally, we call mapreduce using function handles to our two helper functions.

maxRGB = mapreduce(imds, @catColorMapper, @hueSaturationValueReducer);

********************************

*      MAPREDUCE PROGRESS      *

********************************

Map   0% Reduce   0%

Map  13% Reduce   0%

Map  27% Reduce   0%

Map  40% Reduce   0%

Map  50% Reduce   0%

Map  63% Reduce   0%

Map  77% Reduce   0%

Map  90% Reduce   0%

Map 100% Reduce   0%

Map 100% Reduce  33%

Map 100% Reduce  67%

Map 100% Reduce 100%

The results are stored in a MAT-file, for example, results_1_28-Sep-2016_16-28-38_347. The store returned is a key-value store to this MAT-file, which in turn contains the store with the final key-value results.

>> output = readall(maxRGB)

output =

          Key                                Value

    _______________    __________________________________________

    ’Maximum␣Avg␣Red’      ’/MATLAB/Cats/IMG_1625.png’

    ’Maximum␣Avg␣Blue’     ’/MATLAB/Cats/IMG_4866.JPG’

    ’Maximum␣Avg␣Green’    ’/MATLAB/Cats/IMG_4866.JPG’