Completing our pose detection component

Getting back to our Pose.vue component, we now have to fill in the script section. We are going to need the following import statements and class definition for our component (remember that I promised we would build this class up from scratch). Again, we can see the use of @Component to give us a component registration. We see this time and time again with Vue components:

import { Component, Vue } from 'vue-property-decorator';
import {PoseClassifier} from '@/Models/PoseClassifier';
import {Keypoint} from '@tensorflow-models/posenet';
@Component
export default class Pose extends Vue {
}

We have reached the point where we can write our Classify method, which will retrieve the image and canvas when they have been created and pass this through to the PoseClassifier class. We need a couple of private fields to hold the PoseClassifier instance and the returned Keypoint array:

private readonly classifier: PoseClassifier = new PoseClassifier();
private keypoints: Keypoint[] | null;

Inside our Classify code, we are going to employ the same life cycle trick of waiting for nextTick before we retrieve the image referenced as poseId, and the canvas referenced as posecanvas:

public Classify(): void {
this.$nextTick().then(async () => {
/* tslint:disable:no-string-literal */
const pose = this.$refs['poseId'];
const poseCanvas = this.$refs['posecanvas'];
/* tslint:enable:no-string-literal */
});
}

Once we have the image reference, we cast them to the appropriate HTMLImageElement and HTMLCanvasElement types, before we call the Pose method and populate our keypoints member with the resulting values:

if (pose !== null) {
const image: HTMLImageElement = pose as HTMLImageElement;
const canvas: HTMLCanvasElement = poseCanvas as HTMLCanvasElement
this.keypoints = await this.classifier.Pose(image, canvas);
}

At this point, we can run the application. It's very satisfying seeing the keypoints results being overlaid onto the image, but we can go further. With just a little bit of extra effort, we can display the keypoints results in a Bootstrap table. Go back to our template and add the following div statements to add a Bootstrap row and column below the image:

<div class="row">
<div class="col">
</div>
</div>

Since we have already exposed the keypoints results, we can simply create a Vue Bootstrap table using b-table. We set the binding to the items using :items, setting it to the keypoints results that we defined in our class. This means that, whenever the keypoints entry gets new values, the table will be updated to display these values:

<b-table striped hover :items="keypoints"></b-table>

Refreshing our application adds the table below the image, with the table looking like this:

While this is a reasonable start, it would be good if we could take a bit more control of the table. Right now, the fields are picked up and automatically formatted by b-table. With a small change, we can separate the Position instance out into two separate entries and make the Score and Part fields sortable.

In our Pose class, we will create a fields entry. The fields entry maps the score entry to use the Confidence label and sets it to be sortable. The part field maps to a label value of Part and is also set to be sortable. We break position into two separate mapped entries labeled X and Y, respectively:

private fields =
{'score':
{ label: 'Confidence', sortable: true},
'part':
{ label: 'Part', sortable: true},
'position.x':
{label:'X'},
'position.y': {label: 'Y'}};

The last thing we need to do is hook the fields entry into b-table. We do this using the :fields property, like this:

<b-table striped hover :items="keypoints" :fields="fields"></b-table>

Refreshing our application shows us the effect of such little changes. This is a much more attractive screen, and the fact that the user can sort the Confidence (originally called score) and Part fields with such little effort shows just how powerful Vue really is:

That's it—we have reached the end of introducing TensorFlow and Vue. We have steered clear of the mathematical aspects behind CNNs because although they can appear intimidating at first glance, they aren't really as bad as all of that, but there are a lot of parts to a typical CNN. There is also a lot more that we can do with Vue; for such a small library, it is incredibly powerful, and this combination of small size and power is one of the reasons that it is becoming ever more popular. 

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.224.214.215