Augmented reality

The augmented reality (AR) is an interesting solution in games and applications that combines the real world with elements generated by computers.

In the Space Aim 3D game, you will use the augmented reality to present locations of other players into the image received from the camera. The size of the player indicator depends on the distance. If the player is close to your location, the rectangle is big. Similarly, while you are moving further, its size is smaller. The indicator contains both, the player name and the Navigate button that launches navigation from your current location to GPS coordinates of the place where the player was while playing the game. The following sketch presents indicators of three players on the World screen:

Augmented reality

Geo Augmented Reality Toolkit

The augmented reality feature can be implemented from the scratch or with the usage of external libraries, like GART (Geo Augmented Reality Toolkit) available at Microsoft Limited Permissive License (MS-LPL). To keep simplicity, the second approach is chosen in this chapter.

GART is a library that can be applied to applications which need to mark various places in the vicinity, for example, shops or historic buildings. The exemplary game also needs to indicate particular places, that is, locations of other players. Using the GART, you can implement the augmented reality feature in a significantly easier and faster way, because it automatically manages all required sensors, calculates distance to various places, updates results while moving, and puts indicators on the screen. If you choose to implement all of these mechanisms on your own, creation of the augmented reality feature will not be such an easy task.

It is interesting that GART can be used to present places both in 2D (on the map) and in 3D (in the world visible in the image obtained from the camera). It also supports the presentation of a special element that indicates the current heading direction, thus you can see in which direction you are looking.

A process of adding the GART library into the project is different than in case of the Windows Phone Toolkit. At the beginning, you download the .zip file from the project website, available at http://gart.codeplex.com. In this book, the GART in 1.2.0 version is described. Then, you extract the archive with library files and samples. Next, in the IDE, you should choose the Add Reference option from the context menu of the References node in the SpaceAim3D project. In the newly opened window, you select the Browse group (on the left), then click on the Browse button (in bottom-right corner), choose the GART.WP8.dll file (from the GART_1_2_0LibWP8ARM directory), and click on OK. Then, GART.WP8 should be also located in the project references list. Now, you can rebuild the solution.

Locations of other players

In this part, you will learn how to implement the augmented reality feature with the GART library. At the end, the World screen of the game will show player indicators.

Tip

To obtain image from the camera, you should not forget to enable the ID_CAP_ISV_CAMERA capability in the manifest file.

ARItemExtended.cs

Each indicator used by the GART library is represented by an instance of the ARItem class or any class that derives from it. With the usage of ARItem you can store, for example, the player name and GPS coordinates. However, you also should be able to get data for navigation that is composed of both, the player name and the current location. Thus, you implement a new class, called ARItemExtended (in the .cs file inside the Models directory) that derives from ARItem:

public class ARItemExtended : ARItem
{
  public PlayerData Player
  {
    get
    {
      return new PlayerData() { Name = (string)this.Content, 
        Location = this.GeoLocation };
    }
  }
}

The class contains the Player property with the get accessor, which returns the PlayerData instance with suitable values of the Name and Location properties, content of ARItem (as string) and location, respectively.

WorldViewModel.cs

A few modifications are necessary in the view model of the World screen.

At the beginning, you add the Players property, which will store instances of the ARItemExtended class representing the player data. By default, the m_players field contains an instance of ObservableCollection<ARItem> with zero items:

private ObservableCollection<ARItem> m_players = 
  new ObservableCollection<ARItem>();
public ObservableCollection<ARItem> Players { (...) }

Many operations regarding the GART library (like starting necessary services) require access to the ARDisplay object, which you will create declaratively in the XAML code. Thus, you also add the Display property to the WorldViewModel class, to be able to perform some operations directly from the view model. You will set a reference to the ARDisplay instance from code-behind.

public ARDisplay Display { get; set; }

When the user navigates to the World screen, the OnNavigatedTo method will be called. Its code is as follows:

public void OnNavigatedTo(NavigationEventArgs e)
{
  this.Players.Clear();
  this.Players.Add(new ARItemExtended() { Content = "Marcin", 
    GeoLocation = new GeoCoordinate(50.0401, 22.0070) }); (...)
  this.Display.ServiceErrors += this.Display_ServiceErrors;
  this.Display.StartServices();
  this.Display.Orientation = ControlOrientation.Clockwise270Degrees;
}

The previous code requires addition of a few using directives, including for GART.BaseControls, GART.Controls, and GART.Data namespaces.

In the code, you remove data of all player indicators (items inside the Players collection) and add some for testing purposes. Then, you configure and start the augmented reality mechanism by performing the following steps:

  • Specifying a method called in case of errors
  • Starting services required by the GART library
  • Setting a device orientation (rotated 270 degrees clockwise in your case)

With the usage of GART, the process of managing sensors is really easy, because all required operations are performed automatically. You just need to call StartServices and StopServices in proper methods.

In the OnNavigatedFrom method, you unsubscribe the ServiceErrors event, and then stop services used by GART:

public void OnNavigatedFrom(NavigationEventArgs e)
{
  this.Display.ServiceErrors -= this.Display_ServiceErrors;
  if (this.Display.Motion != null)
    { this.Display.StopServices(); }
}

You need to ensure that the last operation is performed only if the motion has been initialized correctly. Without this check, an exception will be thrown while closing the World page in the emulator.

In case of errors regarding the GART library, the Display_ServiceErrors method is called. Here, you remove all indicators so as not to show them on the screen:

private void Display_ServiceErrors(object sender, 
  ServiceErrorsEventArgs e)
{
  this.Players.Clear();
}

The NavigateToPlayer method will start navigation to a location of the particular player, but currently it does not contain any content, as shown in the following line:

public void NavigateToPlayer(PlayerData player) { }

The current version of the augmented reality feature requires that the compass is already calibrated. Otherwise, the mechanism will not work correctly. You will learn how to present a way of compass calibration as a video clip, when necessary, in Chapter 10, Social networks, Feeds, Settings, and Local Rank.

WorldPage.xaml

The World screen differs significantly from others, including Map, because it will use the image from the camera as the background of the whole screen.

Here, the control from the GART library is used. Thus, in the root element, you should add information about the namespace with GART controls. You need to specify a mapped prefix (GART in your case), as shown in the following line:

xmlns:GART="clr-namespace:GART.Controls;assembly=GART.WP8"

Resources for the World page are defined as follows:

<phone:PhoneApplicationPage.Resources>
  <DataTemplate x:Key="PlayerIndicator">
    <Border Background="White" BorderBrush="Black" 
      BorderThickness="3" Padding="5">
      <StackPanel>
        <TextBlock Text="{Binding Content}" 
          Foreground="Black" FontSize="24" 
          HorizontalAlignment="Center" />
        <Button Content="[ localized string ]"
          Tag="{Binding Player}" Click="BtnNavigate_Click" 
          Foreground="Black" BorderBrush="Black" />
      </StackPanel>
    </Border>
  </DataTemplate>
  <Style TargetType="TextBlock" x:Key="LoadingMessage"> (...)
  </Style>
</phone:PhoneApplicationPage.Resources>

The data template (with the PlayerIndicator key) represents an indicator of a single player. Here, the player name and the button are placed inside the Border and StackPanel controls. It is worth mentioning that the text presented in the TextBlock control comes from the Content property of the ARItem class. Similarly, the Tag property of Button is bound to the Player property of the ARItemExtended instance. Thus, you are able to pass the player name and the GPS coordinates to the method starting the navigation.

You also specify a style (with the LoadingMessage key) for the TextBlock control which presents a message about loading the augmented reality feature. It will be displayed as the centered white text with size 26. It is worth mentioning that this text will automatically disappear when the image from the camera is available.

The main Grid contains just one column and two rows, as for other screens, for example, the Map one. However, in case of the World page, you do not need to specify the background image, because it will not be visible. Instead of it, you add the StackPanel layout control with the two TextBlock elements, which display information about loading the augmented reality feature, as shown in the following code:

<Grid>
  <Grid.ColumnDefinitions> (...) </Grid.ColumnDefinitions>
  <Grid.RowDefinitions> (...) </Grid.RowDefinitions>
  <StackPanel Grid.Row="1" Grid.Column="0" 
    HorizontalAlignment="Center" VerticalAlignment="Center">
    <TextBlock FontWeight="Bold" Text="[ localized string ]"
      Style="{StaticResource LoadingMessage}" />
    <TextBlock Text="[ localized string ]" 
      Style="{StaticResource LoadingMessage}" />
  </StackPanel>
  <GART:ARDisplay x:Name="Display" ARItems="{Binding Players}"
    Grid.Row="0" Grid.Column="0" Grid.RowSpan="2">
    <GART:VideoPreview />
    <GART:WorldView FarClippingPlane="10000" 
      ItemTemplate="{StaticResource PlayerIndicator}" />
  </GART:ARDisplay>
</Grid>

The most important part of the preceding code is the ARDisplay control, which is defined in the GART library. That makes it possible to show four layers represented by the following child elements:

  • VideoPreview - Presenting an image from the device camera
  • OverheadMap - Showing 2D map
  • HeadingIndicator - Displaying the current heading direction
  • WorldView - Presenting indicators in 3D world

In the XAML code of the World screen, you specify a name of the ARDisplay control (to Display), as well as bind the ARItems property to the Players collection. You use the VideoPreview and WorldView controls added as child elements to ARDisplay. In case of WorldView, you specify the ItemTemplate property (as the PlayerIndicator data template), which makes it possible to present indicators in other way than as a text. Apart from that, you set a value of the FarClippingPlane property that specifies the maximum distance (in meters) to items to present them on the screen. If a distance is longer for a particular indicator, it is not shown.

WorldPage.xaml.cs

Apart from the XAML code, you need to make a few minor changes in code-behind. First of all, you modify the constructor, as follows:

public WorldPage()
{ (...)
  this.m_viewModel.Display = this.Display;
}

In the constructor, you assign a reference to the ARDisplay control to the Display property, defined in the view model class. Thus, you will be able to perform some operations regarding the GART library directly from the view model, as described earlier.

You should also override the OnNavigatedTo and OnNavigatedFrom methods. Inside them you call the suitable methods (OnNavigatedTo and OnNavigatedFrom, respectively) from the view model class. Such an approach allows you to move the logic from view to view model.

The BtnNavigate_Click method should be also implemented inside the code-behind file. It works in almost the same way as in case of the Map page.

Tip

The author honestly encourages you to learn more about the GART library and the augmented reality possibilities. For instance, you can modify various properties of ARDisplay (like MovementThreshold) and WorldView (for example, NearClippingPlane or MaxItemScale) and observe differences. It is worth remembering that the augmented reality can be a really interesting and impressive way of presenting data to users.

Error message

Currently, the World screen can present an image from the camera as well as indicators representing particular players. However, in this part you will make some minor modifications which allow you to present an additional message in case of an error.

VisibilityConverter.cs

At the beginning, you create the VisibilityConverter class (defined in the file inside the Models directory) which implements the IValueConverter interface. This class makes it possible to convert string into the Visibility enumeration value that indicates whether the control is shown (Visible) or hidden (Collapsed). You will use this class to show or hide the message depending on the content. If it is empty, the visibility is set to Collapse, otherwise set to Visible, as shown in the following code:

public class VisibilityConverter : IValueConverter
{
  public object Convert(object value, Type targetType, 
    object parameter, CultureInfo culture)
  {
    string text = (string)value;
    return string.IsNullOrEmpty(text) ?
      Visibility.Collapsed : Visibility.Visible;
  } (...)
}

WorldViewModel.cs

Some minor changes are necessary in the view model of the World screen.

First of all, you create a new property named ErrorMessage, which represents additional information which should be presented to the user:

private string m_errorMessage = string.Empty;
public string ErrorMessage { (...) }

Another change involves the Display_ServiceErrors method. Here, you assign a message (from the localized strings) to the ErrorMessage property:

private void Display_ServiceErrors(object sender, 
  ServiceErrorsEventArgs e)
{ (...)
  this.ErrorMessage = AppResources.WorldCannotStartAR;
}

App.xaml

In the App.xaml file, you define a unified style for messages that can be presented to the user on various screens. For this reason, two styles related to messages are specified:

<Style x:Key="SA3DMessageBox" TargetType="Border" 
  BasedOn="{StaticResource SA3DBox}"> (...) </Style>
<Style x:Key="SA3DMessage" TargetType="TextBlock"> (...) </Style>

The first style (with SA3DMessageBox key) represents a rectangular area where the message is shown. It has a specified border thickness and brush, and a linear gradient as a background. The other (SA3DMessage key) defines an appearance of the text presented in the area. It should be displayed in white font with size 26. The text should be wrapped (TextWrapping) and centered (the TextAlignment and VerticalAlignment properties set to Center).

WorldPage.xaml

The last modifications are required in the XAML code of the World screen. At the beginning, you map the Models prefix and add VisibilityConverter to the resources, as shown in the following line:

<Models:VisibilityConverter x:Key="VisibilityConverter" />

Then, you add the Border control (just before the closing Grid tag) which can present information about an error, for example, that the augmented reality feature cannot be started. A suitable part of the XAML code is as follows:

<Grid> (...)
  <Border Visibility="{Binding ErrorMessage, 
    Converter={StaticResource VisibilityConverter}}"
    Style="{StaticResource SA3DMessageBox}" Grid.Row="1" 
    Grid.Column="0" Width="600" Height="100">
    <TextBlock Text="{Binding ErrorMessage}"
      Style="{StaticResource SA3DMessage}" />
  </Border>
</Grid>

Let's launch the game and open the World screen. If you run the application in the emulator, you will see just a gray rectangle with a smaller moving one, as well as the information that the feature cannot be started. To display the real image from the camera, you need to run the application on the phone. Then, you will see the result:

WorldPage.xaml
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.146.178.165