Chapter 8. Server-side integration

This chapter covers

• Creating a fast and efficient server
• Choosing a database for your server
• Performance tips
• Third-party GIS platforms

A mobile application is only as good as the server backbone behind it. The server that communicates with a mobile app provides it with all the shared and reference data, processes all the user requests that come from the mobile app, and updates the backend database as necessary. The server makes sure the mobile app is synchronized with the latest data and information, as well as making sure users are authorized and have permissions to perform certain actions. The servers usually sit at a single location; however, cloud solutions are becoming more and more popular. Especially for an LBS application with a rich point of interest (POI) database and powerful search functionality, the server becomes even more crucial. Users of your LBS app will switch to another app instantly if your server can’t respond to user requests correctly and in a timely fashion. In the age of hyper-competition with low-cost startups, you can’t afford the risk of not implementing your backend in the best way possible.

In this chapter, we’ll look at what an LBS server does, how to build one with some code samples, and also some tips and pointers on optimizing the whole infrastructure. First, let’s look at what a server does in an LBS application.

8.1. Server functionality

An LBS server provides most of the functionality for the mobile clients. An LBS server can be a single server running over a simple database or it can be a scalable cloud solution that’s spread over multiple databases. No matter how complicated or simple, all LBS servers share some common characteristics, and they provide a common set of functionality. Now let’s look at what a typical LBS server does to understand the scope and responsibilities of an LBS server:

• Manage end users— Log in and log out end users and handle their permissions to access and update data. In most applications, a user has to log in to the system to be able to access the appropriate resources, for example, the content that they have created previously in the application. For location-based social networks, users have to log in to update their location and other data.
• Serve map tiles— Render and serve map tiles to the clients. The LBS server sends the appropriate map tiles rendered with the preferred styles (terrain, satellite, hybrid, and so on) to the client. Please see section 3.3 and section 8.6 for more on this.
• Manage the locations and states of dynamic entities— Insert, update, and remove locations of dynamic entities, such as users and vehicles that are being tracked. Especially for location-based social networks, each user is a dynamic entity that can move around the map. The LBS server has to keep track of the location of each dynamic entity in the system.
• Manage user-generated content— Insert, update, and remove user-generated content (UGC), such as reviews for businesses or favorite locations. In most LBS applications, users can create location content such as their favorite spot in the park or where they’re having their birthday party. The LBS server saves and indexes all UGC.
• Manage POIs— Import and manage POIs (as shown in figure 8.1) from multiple sources. The LBS server imports POI data from third parties and indexes it so that it can serve relevant POI data to the clients.

  Figure 8.1. Various different types of POIs seen in an LBS application

  

• Dynamic search— Search and return dynamic entities, such as users, around a given location. This is the same as static search except the entities returned are dynamic, such as users and vehicles.
• Routing— Find and return the route between two given locations. The LBS server finds the best route between two given locations and returns the turn-by-turn navigation information to the mobile client, as shown in figure 8.2.

  Figure 8.2. Results of a routing request displaying turn-by-turn directions

  

• Alerts— Alert for certain events, such as fencing. Fencing allows an alert to be sent to a client when a user enters a region (static fencing) or when two dynamic entities are close to each other (dynamic fencing).

Now that you’ve learned about different server functions, let’s look at how you can structure your server so that it can talk to a wide variety of clients. Although there are many ways a server can talk to a mobile client, standardizing on common methods is always the best way to be able to reuse existing software components. For example, you can create a proprietary communication mechanism between your server and your mobile app, but if you do this, you won’t be able to use any open source servers or client libraries that are widely available. Also, in the future, if you want others to communicate with your server too, you’ll be in a tough spot, requiring them to learn your proprietary way of communicating with your server. Now let’s look at the most commonly accepted ways of communicating between a server and a mobile app.

8.2. Server APIs

Server functionality is usually accessed through an HTTP API call, such as the example shown here:

http://api.example.com/poi/get/?lat=45&lng=-72

In this example, the API call returns all the POIs near latitude 45 and longitude 72. The way the API is structured is called RESTful, and the results of this API call can be in many different formats, but the most common formats are XML and JSON, which are simple text formats that can be easily parsed using a common open source library. First let’s look at what a discussion of a RESTful API call.

8.2.1. REST

Representational state transfer (REST) is a style of software architecture for distributed hypermedia systems such as the World Wide Web.

REST-style architectures consist of clients and servers, as shown in figure 8.3. Clients initiate requests to servers; servers process requests and return appropriate responses. Requests and responses are built around the transfer of representations of resources. A resource can be any coherent and meaningful concept that may be addressed. A representation of a resource is typically a document that captures the current or intended state of a resource, such as poi in our example, as shown in listing 8.1. Resource poi is then applied with the action get in our example.

Figure 8.3. Applications and servers exchange data using REST. The application makes REST requests from the server, and the server sends the application REST responses and notifications (source: mng.bx/272K).

Servers mostly return data as text for cross-platform compatibility, in the form of XML or JSON.

8.2.2. XML data exchange format

XML (Extensible Markup Language) is a simple text-based data exchange format. XML is a set of rules for encoding documents as text. HTML is also a form of XML, and hundreds of XML-based languages have been developed, including RSS, Atom, SOAP, and XHTML. XML is a common data exchange format because you can easily represent any kind of data in XML in a structured way. The following listing is a simple XML example that contains some POIs.

Listing 8.1. An XML example

This XML example shows that XML objects can contain attributes , other objects , and values . For example, the poi tag contains name and rating tags.

XML is a comprehensive text format that can represent any data, but sometimes it may be too heavy a representation when the transmission time between the server and the client is of concern. And because of this, even though XML is a common data exchange format, its popularity is rapidly decreasing with the advent of the much lighter JSON format.

8.2.3. JSON data exchange format

JSON, short for JavaScript Object Notation, is a lightweight computer data interchange format. It’s a text-based, human-readable format for representing simple data structures and associative arrays (called objects).

LBS apps that retrieve extensive location, map, and POI information from servers definitely need to pay attention to transmission time, because using heavy formats can unnecessarily increase wait times for end users, which can spell doom for your application.

You can see the same data example as in in JSON format in listing 8.2. As demonstrated by this example, which contains some POIs, JSON is a lighter format than XML and should be preferred whenever large amounts of data have to be exchanged between a client and a server.

Listing 8.2. A JSON example

In this JSON example, a key can contain a number of key-value pairs and a sample key-value pair . For example, the poi tag contains two objects that have name and rating tags. As you’ve seen, overall an LBS app can save enormous amounts of time by sending and receiving data in the light JSON format.

Now let’s look at how to store and manipulate your LBS data on the server.

8.3. Spatial databases

Spatial data is a key part of LBS applications. Also known as geospatial data or geographic information, spatial data is the data or information that identifies the geographic location of features and boundaries on Earth, such as natural or constructed features, oceans, and more. Spatial data is usually stored as coordinates and topology and is data that can be mapped. LBS servers need to store and access spatial data efficiently, because this is a large part of their functionality.

Databases that are capable of storing and manipulating spatial data efficiently are important for LBS servers, because each data structure and all functionality require spatial operations, such as storing latitude and longitude of objects, and operating with respect to this location data, such as searching for nearby objects.

Spatial databases usually support the data types shown in figure 8.4. As you can see, storing, retrieving, and manipulating these spatial data types are compute-intensive and require efficient databases to deal with them. Now let’s start looking at available spatial databases that you can use in your LBS servers.

Figure 8.4. Common spatial data types

8.3.1. PostgreSQL and PostGIS

PostgreSQL and PostGIS are commonly used spatial databases in LBS applications. Post-GIS especially provides unprecedented speed and functionality that’s invaluable for LBS applications. PostgreSQL^[1] is an object-relational database management system (ORDBMS). It’s released under a BSD-style license and is thus free and open source software.

¹http://www.postgresql.org/

PostGIS^[2] is an open source software program that adds support for geographic objects to PostgreSQL. In effect, PostGIS spatially enables the PostgreSQL server, allowing it to be used as a backend spatial database. PostGIS follows the “Simple Features for SQL” specification^[3] from the Open Geospatial Consortium.^[4] As such, PostGIS includes the following:

²http://postgis.refractions.net/

³http://www.opengis.org/docs/99-049.pdf

⁴http://www.opengeospatial.org/

• Geometry types for Points, LineStrings, Polygons, MultiPoints, MultiLineStrings, MultiPolygons, and Geometry Collections, as displayed in with examples
• Spatial predicates for determining the interactions of geometries
• Spatial operators for determining geospatial measurements like area, distance, length, and perimeter
• Spatial operators for determining geospatial set operations, like union, difference, symmetric difference, and buffers
• R-tree-over-GiST (Generalized Search Tree) spatial indexes for high-speed spatial querying
• Index selectivity support, to provide high-performance query plans for mixed spatial/nonspatial queries

The following listing shows examples of various different geometry types.

Listing 8.3. Examples of geometry types supported in PostGIS

Point
Example: POINT (10 10)
LineString
Example: LINESTRING( 10 10, 20 20, 30 40)
Polygon
Example: POLYGON ((10 10, 10 20, 20 20, 20 15, 10 10))
Multipoint
Example: MULTIPOINT(10 10, 20 20)
Multipolygon
Example: MULTIPOLYGON(((10 10, 10 20, 20 20, 20 15, 10 10)), ((60 60, 70
70, 80 60, 60 60)))
GeomCollection
Example: GEOMETRYCOLLECTION(POINT (10 10), POINT(30 30), LINESTRING(15
15, 20 20))

This code example shows how to declare various types of geometry in PostGIS, such as Point, LineString, Polygon, MultiPoint, MultiPolygon, and GeometryCollection. For example, a neighborhood boundary in an LBS app can be efficiently represented by a Polygon or MultiPolygon.

The PostGIS implementation is based on lightweight geometries and indexes optimized to reduce disk and memory footprint. Using lightweight geometries helps servers increase the amount of data migrated up from physical disk storage into RAM, improving query performance substantially. Now let’s look at another popular database, MySQL, that has some added-on spatial support.

8.3.2. MySQL spatial support

MySQL (now part of Oracle, Inc.) is a relational database management system (RDBMS) that has more than 6 million installations. MySQL is often used in free software projects that require a full-featured database management system.

MySQL supports spatial extensions to allow the generation, storage, and analysis of geographic features. Spatial indexes also optimize search operations. With the help of a great variety of multidimensional indexing methods that have previously been designed, it’s possible to optimize spatial searches. The most typical spatial searches are the following:

• Point queries that search for all objects that contain a given point
• Region queries that search for all objects that overlap a given region

Spatial extensions to MySQL support the following geometric classes:

• Geometry (non-instantiable)
• Point (instantiable)
• Curve (non-instantiable)
• LineString (instantiable)
• Line
• LinearRing
• Surface (non-instantiable)
• Polygon (instantiable)
• GeometryCollection (instantiable)
• MultiPoint (instantiable)
• MultiCurve (non-instantiable)
• MultiLineString (instantiable)
• MultiSurface (non-instantiable)
• MultiPolygon (instantiable)

For example, a bus route in an LBS app can be represented by a MultiLineString, which can be handled by MySQL efficiently. Still, spatial support for MySQL is not as efficient and well established as that for PostGIS. Now let’s look at the spatial support in the database from Microsoft.

8.3.3. Microsoft SQL Server spatial support

Microsoft SQL Server is a relational model database server produced by Microsoft. LBS applications that run on Microsoft platforms need to understand Microsoft SQL Server to see if it suits their needs.

SQL Server 2008 adds geospatial support to the SQL Server product suite. This allows the storage of spatial data in SQL tables (in the form of points, lines, and polygons) and a set of functions to allow the manipulation of this data. Also included are new spatial indexes to support the execution of these functions.

SQL Server 2008 supports two different spatial data types:

• GEOMETRY— This data type stores data in projected planar surfaces.
• GEOGRAPHY— This data type stores data in an ellipsoidal model.

The geometry types include the following:

• Point— A point is an object representing a single location. It always has an X coordinate and a Y coordinate and may additionally have an elevation Z and a measure M.
• MultiPoint— A MultiPoint object is a collection of points. It differs from a LineString and a Polygon because there are no implied connections between the points in the collection. Because of this, the boundary of a MultiPoint object is empty.
• LineString— A LineString is again a collection of points. This differs from the MultiPoint object because the points are in sequence and the LineString object also represents the line segments connecting the points.
• MultiLineString— A MultiLineString is a collection of LineStrings.
• Polygon— A Polygon is a collection of points representing a two-dimensional surface. A Polygon may consist of an exterior ring and a number of interior rings. For a Polygon object to be a valid instance, the interior rings can’t cross one another.
• MultiPolygon— A MultiPolygon is a collection of Polygons.
• GeometryCollection— A GeometryCollection is a collection of geometry (or geography) objects.

The next listing shows how to create a spatial table and import data into it. The code first creates tables with geometry fields. Then it inserts data into these tables.

Listing 8.4. Example code for creating a spatial table and importing data into it

In this example, you see how to create new tables and insert values (rows) into them . We create the tables Districts and Streets. Then we insert three rows into Districts and two rows into Streets.

Now that you’ve seen the offering from Microsoft, let’s look at the offering from the biggest database company in the world, Oracle.

8.3.4. Oracle Spatial

Oracle Spatial^[5] forms a separately licensed option component of the Oracle database. If you need an industrial-strength LBS database or if you’re building on top of an existing Oracle database, Oracle Spatial is another solution for your LBS app. Oracle Spatial aids users in managing geographic and location data in a native type within an Oracle database, potentially supporting a wide range of applications from automated mapping/facilities-management and geographic information systems to wireless location services and location-enabled e-business.

⁵http://www.oracle.com/technology/products/spatial/index.html

Oracle Spatial consists of the following:

• A schema that prescribes the storage, syntax, and semantics of supported geometric data types
• A spatial indexing system
• Operators, functions, and procedures for performing area-of-interest queries, spatial join queries, and other spatial analysis operations
• Functions and procedures for utility and tuning operations
• A topology data model for working with data about nodes, edges, and faces in a topology
• A network data model for representing capabilities or objects (modeled as nodes and links) in a network
• A GeoRaster feature to store, index, query, analyze, and deliver GeoRaster data (raster image and gridded data and its associated metadata)

Oracle is an industrial-strength solution, just like the DB2 database offering from IBM.

8.3.5. IBM DB2 Spatial Extender

DB2 Spatial Extender allows you to store, manage, and analyze spatial data (information about the location of geographic features) in DB2 Universal Database along with traditional data for text and numbers.

With this capability, you can generate, analyze, and exploit spatial information about geographic features, such as the locations of office buildings or the size of a flood zone. DB2 Spatial Extender extends the function of DB2 Universal Database with a set of advanced spatial data types that represent geometries such as points, lines, and polygons and many functions and features that interoperate with those new data types. These capabilities allow you to integrate spatial information with your business data, adding another element of intelligence to your database.

Now that you’ve learned about all the database and API options, let’s look at how you can have your server running at maximum performance no matter how many millions of users you get.

8.4. Performance

It’s imperative that your server be optimized for the most time-consuming operations such as search and updating of a large number of entities. As you can expect, as data size grows, the time to complete complex operations grows exponentially, so it’s important to employ optimization techniques to implement a responsive server. Caching is one of the most efficient techniques to increase server performance.

8.4.1. Caching

LBS servers are often hit with similar queries, and they access their databases to read the same objects over and over again. Caching the results of the queries and caching the objects in memory save a tremendous amount of time, in some instances up to 90% improvement over noncaching servers.

The most common mechanism of caching objects in memory is Memcached. Memcached^[6] is a general-purpose distributed memory caching system developed by Danga Interactive for LiveJournal but is now used by many other sites. It’s often used to speed up dynamic database-driven websites by caching data and objects in memory to reduce the number of times an external data source (such as a database or API) must be read. Memcached is distributed under a permissive free software license.

⁶http://memcached.org/

Memcached is very easy to use, as you can see in figure 8.5. Instead of dynamically querying your database for every request, you cache the results of the previous queries in Memcached and use the cached values whenever possible to avoid hitting the database. Each database hit takes multiple orders of magnitude longer than using cached values.

Figure 8.5. The basic operation of Memcached (source: mng.bz/001q)

The following sample pseudo code accesses a database to read the contents of a user object:

function get_user_from_id(int id) {

  result = db_select("SELECT * FROM user WHERE id = ?", id);
  return result;

}

This code interacts with the database directly every time, and it will be slow because of this.

The following listing shows the same code after Memcached is integrated into it to cache the user object, so that the second time this function is called, it will return the user object from memory instead of reading from the database.

Listing 8.5. Example code with Memcached

This code checks Memcached before requesting from the database and gets the user object from the database only if it isn’t already in Memcached. Because of this, the code is fast compared to the code in listing 8.6.

Now that you’ve learned about some of the issues related to writing an efficient and fast LBS server, let’s look at an example.

8.5. Returning POIs example

Let’s look at how we can write a simple LBS server request that can return POIs near a given location.

First we need to create a database table and import our data into it, as shown here. This code first creates a table and then inserts data into this table.

Listing 8.6. Creating a PostGIS table and importing POIs into it

This code sample creates a table and inserts many rows that represent the POIs.

Now let’s see how to access this table and search data in it using PHP, as shown in listing 8.7. PHP is one of the most common server languages. It’s widely used and hence there’s extensive support for it in terms of developer communities and available open source libraries. Ruby, Python, and Java are other popular alternatives.

Listing 8.7. PHP code that searches the database for nearby POIs

This code example first gets the lat and long from the request , then connects to the database , and gets all the POIs around that lat/long . It returns each of them as JSON and then closes the connection . This is a common example of how most LBS servers work. As you move around, the LBS server continually sends you updated POI information using this code.

Sometimes, instead of writing an LBS server from scratch, you may be able to use a third-party LBS server. Now let’s look at your options when it comes to efficient third-party LBS servers.

8.6. Third-party LBS servers

There are many LBS servers available for licensing or use. If your application doesn’t require customization, you can use these GIS systems as your backend. Some of these GIS systems even allow for minor customizations through built-in scripting languages.

8.6.1. MapServer

MapServer,^[7] shown in figure 8.6, is an open source development environment for building spatially enabled internet applications. It was developed by the University of Minnesota. MapServer was originally developed with support from NASA, which needed a way to make its satellite imagery available to the public. MapServer is now a project of OSGeo^[8] and is maintained by a growing number of developers from around the world.

⁷http://www.mapserver.org/

⁸http://www.osgeo.org/

Figure 8.6. MapServer client view

MapServer is not a fully featured GIS server, but it has been in development for a long time and can be used in many applications easily. It supports advanced cartographic output with the following features:

• Scale-dependent feature drawing and application execution
• Feature labeling including label collision mediation
• Fully customizable, template-driven output
• TrueType fonts
• Map element automation (scalebar, reference map, and legend)
• Thematic mapping using logical- or regular expression–based classes
• Support for popular scripting and development environments (PHP, Python, Perl, Ruby, Java, and .NET)
• Cross-platform support (Linux, Windows, Mac OS X, Solaris, and more)
• Support of numerous Open Geospatial Consortium (OGC) standards, such as WMS (client/server), nontransactional WFS (client/server), WMC, WCS, Filter Encoding, SLD, GML, SOS, and OM

MapServer’s advantages are its stability and the wide support that comes with being open source. If you need more solid commercial support, then ESRI’s ArcGIS server is a good alternative.

8.6.2. ESRI ArcGIS Server

ArcGIS Server^[9] is a GIS software package made by ESRI to provide web-oriented spatial data services. Since version 9.2 ArcGIS Server also includes the spatial data-management software (formerly known as ArcSDE). If you’re looking for professional support and assistance for your server on an ongoing basis, ArcGIS is a good solution.

⁹http://www.esri.com/software/arcgis/

ArcGIS Server supports software development on the .NET Framework and the Java programming language. ArcGIS Server services can be consumed by web browsers, mobile devices, and desktop systems. ArcGIS Server supports interoperability standards such as OGC and W3C. Several services, including mapping services, geocoding services, geodata management services, geoprocessing services, virtual globe services, and network analysis services, are available via a SOAP API and a REST API.

Now let’s look at another commercial LBS server, Maptitude.

8.6.3. Maptitude

Maptitude^[10] is a mapping software program created by Caliper Corporation that allows users to view, edit, and integrate maps. The software and technology are designed to facilitate the geographical visualization and analysis of either included data or custom external data. This commercial application for Microsoft Windows includes the following abilities:

¹⁰http://www.caliper.com/maptovu.htm

• Creating map displays
• Enhancing reports and presentations with maps
• Finding geographic patterns that can’t be seen in database tables and spreadsheets
• Answering geographic questions that impact business operations
• Sharing geographic data with a workgroup, department, or organization

Maptitude is mainly targeted at business users but competes at all levels of the GIS market in many different sectors. It integrates with Microsoft Office, works with data mapping from various sources including Microsoft Excel, and includes a proprietary BASIC-like programming language (Caliper Script) within a development interface (GISDK) that allows automation of the Maptitude environment.

8.6.4. GeoMedia

GeoMedia^[11] is the technology suite of software components in Intergraph Corporation’s GIS family of software products. It was developed as client or server software specifically for the Microsoft Windows environment, as shown in figure 8.7.

¹¹ mng.bz/B8w8

Figure 8.7. GeoMedia client view

The core technology of GeoMedia makes it possible to simultaneously read data directly from multiple GIS data sources, which include

• Shapefiles (ESRI)
• ESRI Coverage (ESRI)
• ESRI SDE via FME plug-in for GeoMedia (ESRI)
• AutoCAD DWG, AutoCAD DXF (Autodesk)
• MicroStation DGN (Bentley Systems)
• Oracle Spatial (Oracle Corporation))
• MapInfo (MapInfo)
• FRAMME (Intergraph)
• Modular GIS Environment (MGE) (Intergraph)
• GeoMedia warehouses on the base of Microsoft Access or Microsoft SQL Server (Intergraph)

Next let’s look at another commercial offering, MapInfo Professional.

8.6.5. MapInfo Professional

MapInfo Professional^[12] is a desktop mapping system software product produced by MapInfo Corporation. MapInfo Professional has the ability to combine and display, on a single map, data from a variety of sources in different formats and projections. The software is capable of overlaying raster and vector layers on the same map; the former can be made semitransparent, so that they can serve as more than mere backdrops. MapInfo is typically used for analyzing prebuilt map data layers.

¹² mng.bz/3Ii3

Next we’ll look at a very advanced LBS server from Microsoft, the MapPoint server.

8.6.6. Microsoft MapPoint

MapPoint Web Service^[13] is an XML-based web service that enables developers to integrate location-based services, such as maps, driving directions, and proximity searches, into their applications and business processes, as shown in figure 8.8.

¹³http://www.microsoft.com/mappoint/

Figure 8.8. Microsoft MapPoint client view

MapPoint Location Server is a component of MapPoint Web Service that allows the integration of real-time location into business applications. It provides access to location providers that integrate with mobile operator networks and acts as a proxy between the real-time location data and services provided by those networks and the MapPoint Web Service.

MapPoint Location Server is available to customers with a valid license agreement for the MapPoint Web Service. If you’re deploying an enterprise-grade LBS application and need scalability and solid support and have the budget, MapPoint is a good alternative.

We’ve now analyzed all the viable third-party options for LBS servers. It’s important to be very careful when picking a server because it’s a big investment in terms of time and money.

8.7. Summary

The server is a crucial part of your mobile LBS application. If you don’t pick the right approach, technologies, and databases, your mobile application might end up slow and unresponsive and users will switch to another application quickly.

Performance is very important. As database size grows, performance degrades exponentially, so cache anything you can. Then start analyzing and profiling your server requests, find bottlenecks, and cache more. Most of the time, profiling service requests exposes problems in unexpected places. Also, performance test all your target platforms with less-than-ideal network connections. You can also stress test your server to see what will happen if your application is highly successful.

Explore third-party GIS system solutions first. If none fit your application, then implement your own custom server. But most of the time, most applications need their own custom server if you’re doing anything other than getting and viewing data.

Load balance your server, both for processing and for data access. Set up your database with replication so that if something bad happens, your service won’t be interrupted.

In the next chapter, we’ll start looking at the privacy issues surrounding LBS applications.