When performing ML on the edge, you lose some of the luxuries you tend to have when running on a more powerful device (albeit this is shifting all the time). Here is a list of considerations to keep in mind:

• Model size: Previously, we walked through building a simple linear regression model. The model itself consists of two floats (bias and weight coefficients), which of course are negligible in terms of memory requirements. But, as you dive into the world of deep learning, it's common to find models hundreds of megabytes in size. For example, the VGG16 model is a 16-layer conventional neural network architecture trained on the ImageNet dataset used for image classification, available on Apple's site. It is just over 500 megabytes. Currently, Apple allows apps 2 gigabytes in size, but asking your user to download such a large file may well put them off. 
• Memory: It's not just the executable size that you need to be mindful of, but also the amount of working memory available. It's common for desktop machines to have memory in the range of 16-32 gigabytes, but the memory for the latest iPhone (iPhone 8) is just 2 gigabytes—impressive for a mobile device, but quite a difference from its counterpart. This constraint is likely to dictate what model you choose, more so than how much memory it takes on disk. It is also worth mentioning that it's not just the model weights you'll need to load into the memory; you will also need to load in any label data and, of course, the input data you are performing inference on.
• Speed: This, of course, is correlated to the model size (in normal circumstances) and relevant to your specific use case. Just keep in mind that performing inference is only one part of the workflow. You have pre-processing and post-processing tasks that also need to be taken into account, such as loading and pre-processing the input data. In some cases, you may have to trade off accuracy with performance and size.
• Supported algorithms and data types: In the previous section, we presented the current algorithms that Core ML supports. Along with these, Core ML supports a subset of data types, summarized in the following table for convenience:

Here, we have presented just a few of the considerations at a high level when performing ML on a mobile device. The specifics will be dependent on your use case and models available, but it's worth keeping these in the back of your mind and reminding yourself that these, albeit very powerful devices, are still mobile devices. They run on a battery and are therefore subject to the typical considerations and optimizations normally required for a mobile project. These considerations are even more applicable to those who plan to create their own model, which should be most of you if you plan to take advantage of ML.