In this section, we will implement the supporting methods that will be used to construct a path to show the user where a specific item is when requested. All of this will reside in the AssistantItemFinderMain class, so open this up in Visual Studio, if not already open, and let's get started.
As usual, we will start by declaring the necessary variables. Add the following to the AssistantItemFinderMain class:
private string requestedSightingTerm = string.Empty;
private Node targetNode = null;
private Sighting targetSighting = null;
private Renderer targetNodeRenderer;
The variable requestedSightingTerm stores the item we are currently searching for, while targetNode and targetSighting will be the associated node and sighting related to this search term. targetNodeRenderer will be used to differentiate the last node from the others. Let's start by instantiating targetNodeRenderer. Make the following amendments to the AssistantItemFinderMain class:
public void SetHolographicSpace(HolographicSpace holographicSpace)
{
// ...
nodeRenderer = new NodeRenderer(deviceResources, new Vector3(0.3f, 0.3f, 1.0f), 0.05f);
nodeRenderer.CreateDeviceDependentResourcesAsync();
targetNodeRenderer = new NodeRenderer(deviceResources, new Vector3(0.3f, 1.0f, 0.3f), 0.09f);
targetNodeRenderer.CreateDeviceDependentResourcesAsync();
// ...
}
public void Dispose()
{
// ...
if (nodeRenderer != null)
{
nodeRenderer.Dispose();
nodeRenderer = null;
}
if (targetNodeRenderer != null)
{
targetNodeRenderer.Dispose();
targetNodeRenderer = null;
}
// ...
}
public void OnDeviceLost(Object sender, EventArgs e)
{
// ...
nodeRenderer.ReleaseDeviceDependentResources();
targetNodeRenderer.ReleaseDeviceDependentResources();
// ...
}
public void OnDeviceRestored(Object sender, EventArgs e)
{
// ...
nodeRenderer.CreateDeviceDependentResourcesAsync();
targetNodeRenderer.CreateDeviceDependentResourcesAsync();
// ...
}
With the management of targetNodeRenderer now complete, let's turn our attention to the Update method. Currently, we are creating an entity for each node. We want to change this, so that we only create entities (visual representations of the nodes) when we are constructing a path to targetNode, that is, when we are assisting the user to a specified item.
Jump into the Update method where we start making these changes, starting with removing the statement CreateEntitiesForAllNodes(referenceFrameCoordinateSystem).
Our first addition here has to do with setting targetNode and targetSighting. This will be performed when the variable requestedSightingTerm is set (setting this will be the topic of the next section). Add the following block to the Update method:
if (!string.IsNullOrEmpty(requestedSightingTerm))
{
var candidates = FindClosestNodesWithSightedItem(referenceFrameCoordinateSystem, pose, requestedSightingTerm);
if (candidates != null && candidates.Count > 0)
{
targetNode = candidates[0];
targetSighting = candidates[0].Sightings.Where(sighting => sighting.Tokens.Any(token => token.Equals(requestedSightingTerm, StringComparison.OrdinalIgnoreCase))).First();
}
requestedSightingTerm = string.Empty;
}
When requestedSightingTerm is not null, we search for the closest node (relative to the user) for Sighting that has a tag equal to requestedSightingTerm. If found, we set targetNode and targetSighting. Searching is delegated to the method FindClosestNodesWithSightedItem, which we will return to very soon.
Our next block of code is concerned with dismissing a target. This occurs when the user is in proximity (same node) of targetNode and has resided there longer than a set period. Add the following code just under the preceding snippet:
if (CurrentNode == targetNode)
{
if (dwellTimeAtCurrentNode >= 5)
{
targetNode = null;
targetSighting = null;
entities.Clear();
}
}
The final addition to the Update method is the construction of the path if targetNode is set. Add the following snippet:
if (targetNode != null)
{
RebuildTrailToTarget(referenceFrameCoordinateSystem, prediction.Timestamp, CurrentNode, targetNode);
}
Here, we simply delegate the construction of the path to the method RebuildTrailToTarget. Our Update method is now complete, ignoring the errors. Let's now move on to implementing some of the dependencies, namely, FindClosestNodesWithSightedItem and RebuildTrailToTarget.
Let's start with FindClosestNodesWithSightedItem. This method simply constructs and returns a list of nodes, filtering out those that don't have a sighting with the specified tag (sightingItem) and then orders them based on their distance from the user:
public IList<Node> FindClosestNodesWithSightedItem(SpatialCoordinateSystem referenceFrameCoordinateSystem, SpatialPointerPose pose, string sightingItem)
{
var filteredNodes = nodes.Where(node =>
{
return node.Sightings.Any(sighting =>
{
return sighting.Tokens.Any(token => token.Equals(sightingItem, StringComparison.OrdinalIgnoreCase));
});
});
if (filteredNodes != null)
{
return filteredNodes.OrderBy(node =>
{
return node.TryGetDistance(referenceFrameCoordinateSystem, pose.Head.Position);
});
}
return null;
}
Our next method, RebuildTrailToTarget, replaces our CreateEntitiesForAllNodes--instead of creating an entity for all nodes, it finds the shortest path, based on the edges connecting the nodes, from the given start and end nodes, and then creates entities for each of the nodes along this path. Add the following method to the AssistantItemFinderMain class:
int RebuildTrailToTarget(SpatialCoordinateSystem referenceFrameCoordinateSystem, PerceptionTimestamp perceptionTimestamp, Node startNode, Node endNode,
int lookAhead = 100)
{
entities.Clear();
Stack<Node> trail = new Stack<Node>();
BuildPath(endNode, startNode, trail);
if (trail.Count == 0) return -1;
var rootEntity = GetRootEntity(referenceFrameCoordinateSystem);
int i = 0;
while (i < lookAhead && trail.Count > 0)
{
var node = trail.Pop();
var entity = new Entity($"node_{i}");
entity.Node = node;
entity.Renderer = entity.Node == targetNode ? targetNodeRenderer : nodeRenderer;
entity.UpdateTransform(referenceFrameCoordinateSystem);
var targetPosition = rootEntity.Transform.Translation;
entity.Position = new Vector3(0, (targetPosition - entity.Transform.Translation).Y, 0f);
entities.Add(entity);
i = i + 1;
}
return i;
}
We first clear the entities collection and then delegate the path planning to the method BuildPath, passing it through the start and end nodes along with a collection to store the path--here, to construct the path from the node where the item is located back to the user. This reduces our search space and, therefore, improves the performance.
Once the path has been constructed, we create a root entity (rootEntity), as we did before, to use as a reference for the y position for all the entities. Then, we iterate through the collection of nodes and create an associated entity, almost the same as we did in CreateEntitiesForAllNodes with the only difference, that we assign targetNodeRenderer to the end node and nodeRenderer to all the others. This is used to help communicate to the user that this is the location of the item they are searching for.
Let's finish off the construction of the path by implementing the last method, BuildPath. BuildPath is a recursive function that traverses the connected nodes (using the edges) until it finds the node where the user resides (remembering that we are searching from the Node where the item is to the user), or until a dead end, that is, there are no other nodes to traverse. When this node is found, it builds up the path while it unwinds up the stack from the recursive calls. Let's see what this looks like in code. Add the following snippet to the AssitantItemFinderMain class:
bool BuildPath(Node currentNode, Node endNode, Stack<Node> trail)
{
List<Node> connectedNodes = edges.Where(edge => edge.NodeA == currentNode || edge.NodeB == currentNode).Select(edge => (edge.NodeA != currentNode ? edge.NodeA : edge.NodeB)).ToList();
if (connectedNodes.Contains(endNode))
{
trail.Push(currentNode);
return true;
}
else{
foreach (var node in connectedNodes)
{
if (BuildPath(node, endNode, trail))
{
trail.Push(currentNode);
return true;
}
}
}
return false;
}
To elaborate on the preceding explanation, we first find all the edges associated with currentNode. We then check for our termination condition, which is if one of the edges connects to our endNode. If so, then we add currentNode to the trail (our path) and return true. Otherwise, we iterate through each of the connected nodes and recursively call BuildPath, which will return true if the path leads to endNode, in which case, we append currentNode to the trail. Otherwise, we ignore.
This now completes the code required for constructing a path. We are left with setting targetNode. As seen before, this node is set in the Update method when requestedSightingTerm. When this is set, we enter a block that searches through each node's (starting with the closest to the user) sighting for a matching tag. If found, then we set the node to targetNode which, in turn, constructs our path. The next, and final, section will cover how we set the variable requestedSightingTerm.