We proceed with the optimization as follows:

1. Install Android Studio from https://developer.android.com/studio/install.html.

2. Install Bazel by following the instructions at https://docs.bazel.build/versions/master/install.html. For macOS, we are going to use Homebrew:

/usr/bin/ruby -e "$(curl -fsSL 
 https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew install bazel
bazel version
brew upgrade bazel

3. Clone the TensorFlow distribution from GitHub:

git clone https://github.com/TensorFlow/TensorFlow.git

4. Build the graph transformer that summarizes the graph itself:

cd ~/TensorFlow/
bazel build TensorFlow/tools/graph_transforms:summarize_graph
[2,326 / 2,531] Compiling TensorFlow/core/kernels/cwise_op_greater.cc
INFO: From Linking TensorFlow/tools/graph_transforms/summarize_graph:
clang: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
Target //TensorFlow/tools/graph_transforms:summarize_graph up-to-date:
bazel-bin/TensorFlow/tools/graph_transforms/summarize_graph
INFO: Elapsed time: 1521.260s, Critical Path: 103.87s

7. Download a TensorFlow graph to be used as an example. In this case, we will use Inception v1 TensorFlow graph:

mkdir -p ~/graphs
 curl -o ~/graphs/inception5h.zip 
 https://storage.googleapis.com/download.TensorFlow.org/models/inception5h.zip 
 && unzip ~/graphs/inception5h.zip -d ~/graphs/inception5h

8. Summarize the Inception graph and notice the number of const parameters: 13.46Million. Each of them is stored with 32-bit floats, and this is quite expensive:

bazel-bin/TensorFlow/tools/graph_transforms/summarize_graph --in_graph=/Users/gulli/graphs/TensorFlow_inception_graph.pb
Found 1 possible inputs: (name=input, type=float(1), shape=[])
No variables spotted.
Found 3 possible outputs: (name=output, op=Identity) (name=output1, op=Identity) (name=output2, op=Identity)
Found 13462015 (13.46M) const parameters, 0 (0) variable parameters, and 0 control_edges
370 nodes assigned to device '/cpu:0'Op types used: 142 Const, 64 BiasAdd, 61 Relu, 59 Conv2D, 13 MaxPool, 9 Concat, 5 Reshape, 5 MatMul, 3 Softmax, 3 Identity, 3 AvgPool, 2 LRN, 1 Placeholder
To use with TensorFlow/tools/benchmark:benchmark_model try these arguments:
bazel run TensorFlow/tools/benchmark:benchmark_model -- --graph=/Users/gulli/graphs/TensorFlow_inception_graph.pb --show_flops --input_layer=input --input_layer_type=float --input_layer_shape= --output_layer=output,output1,output2

9. Compile the tool for quantizing the const operations down to 8 bits:

bazel build TensorFlow/tools/graph_transforms:transform_graph
INFO: From Linking TensorFlow/tools/graph_transforms/transform_graph:
clang: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
Target //TensorFlow/tools/graph_transforms:transform_graph up-to-date:
bazel-bin/TensorFlow/tools/graph_transforms/transform_graph
INFO: Elapsed time: 294.421s, Critical Path: 28.83s

10. Run the tool for quantizing the Inception V1 graph:

bazel-bin/TensorFlow/tools/graph_transforms/transform_graph --in_graph=/Users/gulli/graphs/inception5h/TensorFlow_inception_graph.pb --out_graph=/tmp/TensorFlow_inception_quantized.pb --inputs='Mul:0' --outputs='softmax:0' --transforms='quantize_weights'
2017-10-15 18:56:01.192498: I TensorFlow/tools/graph_transforms/transform_graph.cc:264] Applying quantize_weights

11. Compare the two models:

ls -lah /Users/gulli/graphs/inception5h/TensorFlow_inception_graph.pb
-rw-r----- 1 gulli 5001 51M Nov 19 2015 /Users/gulli/graphs/inception5h/TensorFlow_inception_graph.pb
ls -lah /tmp/TensorFlow_inception_quantized.pb
-rw-r--r-- 1 gulli wheel 13M Oct 15 18:56 /tmp/TensorFlow_inception_quantized.pb