Activating the warning

Let's delay our discussion of training a bit further and put together a simple example to see how this works. Open up Unity again and perform the following steps:

  1. Create a new C# script called EnvironmentScanner in the Assets/ARCoreML/Scripts folder. Then, open the script in your editor.
  2. Add the code, as shown, to the class definition:
[RequireComponent(typeof(AudioSource))]
public class EnvironmentalScanner : MonoBehaviour //before me
  1. RequireComponent is a custom Unity attribute that forces a GameObject to require a specific class before this component can be added. In this example, we require an AudioSource component.
  2. Enter the following new properties/fields and method to the class; don't delete anything:
public NeuralNet net;
public List<DataSet> dataSets;

private float min = float.MaxValue;
private float maxRange = float.MinValue;
private float[] inputs;
private double[] output;
private double temp;
private bool warning;
private AudioSource audioSource;
private double lastTimestamp;

public void Awake()
{
int numInputs, numHiddenLayers, numOutputs;
numInputs = 1; numHiddenLayers = 4; numOutputs = 1;
net = new NeuralNet(numInputs, numHiddenLayers, numOutputs);
dataSets = new List<DataSet>();
}
  1. The Awake method is special in Unity in that it gets called when the object first wakes up or becomes active. Awake varies from Start in that it is called upon initialization of the object, whereas Start is called before the first frame an object is rendered. The difference is subtle and is typically only relevant when you are worried about object load time.
    Next, we create a number of temporary input variables for setting the number of input, hidden, and output neurons. For this example, we will use one input, four hidden, and one output. These inputs are used to create NeuralNet in the next line, which is followed by the initialization of the dataSets list.
  2. Next, let's modify the Start method to resemble the following:
void Start()
{
dataSets.Add(new DataSet(new double[]{ 1,.1,0.0}, new double[] { 0.0,1.0,1.0 } ));
net.Train(dataSets, .001);
audioSource = GetComponent<AudioSource>();
}
  1. The first line inside Start creates a very simple DataSet with inputs and outputs. Since we are using a single input and output neuron, these inputs and outputs map 1 to 1 and thus produce the following chart:
Chart of training inputs
  1. Then, net.Train trains the neural network with a minimum error of .001. After that, it gets the required AudioSource, remembers the RequireComponent attribute, and sets it to a private audioSource field. We will use sound in order to warn the user when they get too close. Think about what it is that those points are describing as a function.
  2. Finally, modify the Update method to include the following:
void Update()
{
if (warning)
{
audioSource.Play();
}
else
{
audioSource.Stop();
}
// Do not update if ARCore is not tracking.
if (Frame.TrackingState != FrameTrackingState.Tracking)
{
return;
}

min = float.MaxValue;
PointCloud pointCloud = Frame.PointCloud;
if (pointCloud.PointCount > 0 && pointCloud.Timestamp > lastTimestamp)
{
lastTimestamp = pointCloud.Timestamp;
//find min
for (int i = 0; i < pointCloud.PointCount; i++)
{
var rng = Mathf.Clamp01((pointCloud.GetPoint(i)- transform.parent.parent.transform.position).magnitude);
min = Mathf.Min(rng, min);
}

//compute output
output = net.Compute(new double[] { (double)min });
if(output.Length > 0)
{
warning = output[0] > .001;
}
else
{
warning = false;
}
}
}
  1. There is a lot going on here, so let's break it down. We first check whether the warning is true. If it is, we play a sound, otherwise we stop playing; warning will be our flag to indicate when our NN is signalling. Next, we ensure that the Frame is tracking, with the same code as we saw earlier. Then, we reset min and get the current point cloud from the Frame.
    After that, we ensure that pointCloud has points, and it is the most recent. This is checked by testing the timestamp. Then, inside the if block, we calculate the current min by looping through all points. We then push this through our NN with net.Compute, the value of min (minimum point); this returns our signal or neuron output. In this particular case, we are testing for .001 to determine whether the neuron is signalling an activation. This sets the warning to true or false.
  2. Save the code and return to Unity; ensure that you see no compiler errors.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.142.98.108