Chapter 11. Using PostGIS in web applications

 

 

In a short span of 15 years, the World Wide Web has emerged as the leading method of information delivery, threatening to replace printed media altogether. For GIS, this has been a godsend; not only did the web introduce GIS to the popular imagination, it also affords a delivery mechanism for GIS data that wouldn’t have been possible via traditional printed media. Only 20 years ago, a GIS practitioner wishing to share his data would have had to print out large maps on oversized printers. And then came the web.

To deliver textual data and image data, conventional web technologies suffice, but for the ultimate GIS web-surfing experience, we need additional tools, both on the delivery end (the server) and on the receiving end (the client).

In this chapter we’ll cover web tools that work with PostGIS. We’ll start with two server tools, MapServer and GeoServer, which can read data from PostGIS and serve images or data according to OGC standards. We’ll then move on to the client side of the equation, where we look at OpenLayers, a JavaScript-based tool that greatly enriches the viewing experience for the user. Along with OpenLayers, we’ll check out the new GeoExt extension to OpenLayers based on the new ExtJS JavaScript framework.

11.1. GIS and the web

The first question many readers might ask would be why we need anything above and beyond the technologies widely available to produce web pages. After all, we can render our textual GIS data with HTML and our maps using many of the supported image formats and send them off to the browsers upon request. This section starts by pointing out the limitations of conventional web technologies in serving up dynamically generated maps, both from the server and from the client perspective. We then introduce the current de facto standard for serving up GIS data: OGC web services.

11.1.1. Limitations of conventional web technologies

Conventional web technologies work well for static data and images, but suppose that we need a website where users can extract our map at various zoom levels. Using conventional web server technology, we’d have to limit the user to a fixed set of zoom levels, generate the images beforehand, and serve them as requested. Now consider what would happen if the user would like to see only subsections of the map: We would have to slice up our maps beforehand and restrict users to picking from one of our prepared slices. There are two big problems here: First, we can’t possibly predict what portions the user would like to see. Second, even if we were to generate thousands of subsections for the user to pick from, our server will most likely run out of storage space after just a few maps. Add back zoom levels, and the problem becomes intractable.

The client side of the picture isn’t much rosier. For zoom-level selectors we could use standard HTML combo boxes, but the drop-down list would have to be changed from map to map. If a map has three zoom levels, we’d have to prepopulate our combo box with three values. If the next map has 30 zoom levels, we’d have to have 30 rows in the combo box for the user to pick from. Using various programming technologies now available, such as Python, PHP, and ASP.Net, we can dynamically generate our HTML combo box, but this requires that the mapping person also be a web programmer—and not in just one language. The demands become even more challenging if users have to be able to draw rectangles around subsections to be blown up, add their own markers, or have pop-up description balloons when hovering over certain points of interest. These interface features would all require extensive programming on the client side. If server-side programming didn’t already discourage the GIS specialist, the client-side programming surely will. What we need is a suite of client tools with useful controls for map viewing and editing already built. Sure, the suite will dictate the overall appearance and functionality, but this still is preferable to building our own solution from the ground up. After all, our goal is to disseminate our maps, not to program web servers.

11.1.2. Mapping servers

Mapping servers have one central purpose: to render images for delivery to a client on the fly. As mentioned previously, conventional web servers can’t serve up images unless they already exist, but generating and storing all possible subsections and zoom levels associated with a map is impractical. Mapping servers solve this problem by quickly generating the static images only when requested by the client.

At the time of this writing, four major open source server products dominate the market: MapServer, GeoServer, FeatureServer, and SharpMap.NET. Because mapping servers are rarely the starting point of a GIS project, people generally start from a need to spatially extend existing web applications or to disseminate existing data via the web.

To decide which server products to use, we recommend that you judge how easily each fits into your current infrastructure and data landscape. You should consider the following:

• Will the selected product require a major change in existing platform?
• Which OGC web services, if any, do you need to provide?
• How well will it connect to the data sources you already have, be they PostGIS, Oracle Spatial/Locator, SQL Server 2008, SpatiaLite, MySQL, shapefiles, raster, or something else?

 

 

Platform Considerations

One of the most important deciding factors for choosing a tool is the platform requirements. If you’re on a shared web host, you may not be able to use anything that requires installation. Even if you have complete control over your server, you may shy away from technologies that require additional installation. Table 11.1 outlines the prerequisites for each mapping server.

Table 11.1. Mapping server prerequisites

Service	MapServer	GeoServer	FeatureServer	SharpMap.Net
Java SDK	No	Yes	No	No
Python	No	No	Yes	No
.NET or Mono.Net	No	No	No	Yes
CGI/Fast-CGI	Yes	No	No	No

MapServer is perhaps the most popular of these tools because it contains a lot of functionality and can run under practically any web server without requiring installation. Just drop the compiled .so/.dlls/.exe into the cgi or some other web server executable folder, and you have a completely functional web mapping service. MapServer also offers an API called MapScript in many flavors, with PHP MapScript, Python Map-Script, and C# MapScript being the most common. This allows for more granular control by allowing you to create layers and other map objects from PHP, C#, and Python server-side code. The downside of the MapScript interface is that it also requires writing more code in general than using the Common Gateway Interface (CGI) executable interface.

GeoServer is built on Java servlets. Some binary distributions of GeoServer come packaged with their own mini web server called Jetty. GeoServer requires an existing installation of Java 1.5+ SDK. If you need to run GeoServer under the context of an existing web server service, you’ll need to get a servlet plug-in for your web server such as Tomcat and install the Java Web Archive (WAR) version. Unlike the other tools, it comes packaged with a user-friendly web-based administrative interface. This makes GeoServer a popular option for those who prefer GUIs and wizards over configuration scripts.

SharpMap.Net is a popular option for .NET programmers. It comes packaged as a .NET .dll. All you have to do is drop it into the bin folder of your .NET application. You can therefore run it on a shared host environment where you don’t have your own web server. On the downside, SharpMap.Net does a lot less out of the box than MapS-erver or GeoServer, and you’ll need to make up for any shortcomings by adding additional coding yourself.

FeatureServer is a REST-based web feature server written in Python and was designed to work with OpenLayers. In order to serve PostGIS layers in the various formats, you need to have psychopg or psychopg2 installed. It can run as a standalone Python server, as a CGI, or in Apache via mod_python. It can also be run under IIS if you have Python bindings enabled. It’s trickier to set up than MapServer or Geo-Server, so we won’t be covering it. The main features that make it stand out from other web-mapping servers are that it’s written purely in Python, it supports both the OGC WFS as well as a much simpler feature service REST interface for both querying and editing feature data, and it has some kinds of data sources that some of the other mapping servers don’t support. FeatureServer supported formats include data sources such as Twitter, SQLite, DBM, OSM, and all OGR (if you have the OGR Python plug-in configured). It can output in KML, GeoRSS, GeoJSON, GML, HTML, and OSM. However, it’s a feature server and doesn’t handle web image map requests.

OGC Web Service Support

You may recall from earlier chapters that OGC is short for the Open Geospatial Consortium, the accepted standards organization in the world of GIS. OGC has outlined a series of web services that mapping servers should provide. By adhering to these standard OGC web services, mapping servers won’t limit end users to their particular web or desktop client. All the open source web-mapping clients and the desktop tools that we’ll cover in chapter 12 consume OGC web services. Even proprietary desktop applications such as Manifold, Cadcorp, and MapInfo nowadays offer decent support for OGC web-mapping services. The most common of the web services defined by OGC are the following:

• Web Mapping Service (WMS)— For rendering vector and raster data as map images in JPEG, PNG, TIFF, or some other raster format. This is suitable if you want to show a map of an area, but downloading and rendering the data would be too processor intensive. For example, if you want to display maps on a mobile device with limited processing power, retrieving ready-made images from a WMS server makes more sense than pulling the raw vector data and then provide visual rendering on the fly.
• Web Feature Service (WFS)— For outputting vector data generally using some XML standard such as GML or KML. Geography JavaScript Object Notation (Geo-JSON) is another option and is more processor friendly for consumption by JavaScript because it’s a native JavaScript format. This includes both the geometry represented as JSON encoded as well as the standard database column attributes like dates, numbers, and strings encoded as JSON. This service is most suitable if users need to highlight regions of a map and display attribute info or styling options, without making roundtrips to the server. It’s often used in conjunction with WMS, where WMS would be used to show aerials or large zoomedout regions of a map, and WFS for overlaid key features or more granular control when zoomed in.
• Web Feature Service Transactional (WFS-T)— For editing vector data in transactional mode. This is necessary if you expect end users such as web users or desktop applications to edit geometry data in the database without giving them direct access to the database.

There are other web services as well, such as Web Tiling Services (WTS) and Web Coverage Services (WCS). Table 11.2 is a brief summary of the key OGC web services and which tools support them.

Table 11.2. Web services support

Service	MapServer	GeoServer	FeatureServer	SharpMap.Net
WMS	Yes	Yes	No	Yes
WFS	Yes	Yes	Yes	No
WFS-T	No	Yes	No	No
Custom REST	Yes	Yes	Yes	Yes[*]

^* This means support via an extra downloadable plug-in or library.

The REST architecture is a lighter weight interface than WFS and relies on concepts of GETs, PUTs, and DELETEs to update data and output XML streams. A WFS that supports GET requests can be considered for all intents and purposes as a REST service.

Supported Data Sources

All maps are derived from data. The WMS/WFS/WFS-T protocols allow various data sources to be accessed via one web interface. They provide an abstract interface for GIS data similar to ODBC and JDBC drivers for databases. All web-mapping server tools support various data formats. Table 11.3 describes which tool supports which format, so you can make an informed choice. They all support PostGIS geometries and ESRI shapefiles out of the box, so we left those out.

Table 11.3. Data source formats supported

Service	MapServer	GeoServer	FeatureServer	SharpMap.Net
Oracle Spatial/Locator	Yes[*]	Yes[*]	No	Yes
SQL Server 2008	Yes[*]	Yes[*]	No	Yes
DB2	No	Yes[*]	No	No
PostGIS geography	Yes[*]	No	No	No
PostGIS WKT Raster	Yes	No	No	No
Basic Raster	Yes	Yes	No	Yes
MrSID	Yes	Yes	No	No
SpatiaLite	Yes[*]	No	No	Yes
MySQL	Yes[*]	Yes[*]	No	Yes[*]

^* This means support via an extra downloadable plug-in or library.

11.1.3. Mapping clients

Once the web-mapping services have been set up, you need client applications to consume them. Client applications come in two flavors: desktop and web. Web applications are often implemented using Ajax and a mix of web-scripting languages.

Many desktop mapping toolkits are also capable of consuming standard OGC web-mapping services. A desktop client can either be an open source desktop tool such as Quantum GIS, uDig, gvSIG, OpenJUMP, and countless others or a proprietary desktop tool such as Manifold, MapInfo, Cadcorp SIS, and ArcGIS desktop, to name a few. We’ll cover the open source desktop tools in the next chapter.

As far as web-mapping clients go, OpenLayers tends to be the most popular, particularly in the open source GIS arena. The main reason for this is that it gives you the ability to overlay proprietary non-OGC-compliant mapping layers with OGC WMS, WFS, and WFS-T layers.

OpenLayers is often extended to create more advanced or specific toolkits. Two common ones that build on top of OpenLayers are GeoExt, which is used by OpenGeo’s GeoExplorer, and MapFish. GeoExt is a web-mapping JavaScript framework that combines OpenLayers with ExtJS to provide a web client interface with more of a desktop feel. MapFish combines OpenLayers, GeoExt/ExtJS for the client side, and Python/ Pylons on the server side to create a complete solution for client and server. It offers printing to PDF and a user-authentication service among other advanced features.

11.1.4. Proprietary services

We’d be remiss if we failed to mention that the most popular web-mapping services around are still proprietary, such as Google Maps, Bing, and MapQuest. These services package server, client, and data together in a slick, easy-to-use interface and make mapping accessible to the general public. Though these packages are easy to use, each has its own proprietary JavaScript API with limited control over overlaying data. You won’t be able to write SQL queries let alone represent anything more complex than points and line segments, at least without extensive effort.

One serious drawback is their proprietary and inflexible nature, even on the data level. You can’t remove one core feature. For example, if you wanted to display foliage density over a region instead of the usual streets and places, you can’t do so with these popular packages. You also can’t suppress the commercial licensing clause of these packages. For recreational use, these packages are in most cases free, but once you start to use them for profit or for non-public websites, you’ll find yourself needing to cough up a rather exorbitant licensing fee. Because each has its own custom API that’s incompatible with any other one, you’ll have to rewrite much of your custom data overlay logic when deciding to swap services.

Despite their commercial bent, we must pay homage to these popular services for planting the seeds of GIS into the popular imagination. They were first to show the world the power of dynamic mapping on the internet and continue to lead the way in the development of display technologies. Because this book is devoted to open source solutions, we won’t cover these proprietary JavaScript APIs, but we advise you to not lose sight of the important role they play on the World Wide Web today.

Each of these web GIS tools provides a lot of functionality out of the box. They do so by limiting you to certain protocols when you interact with your database and other spatial data. For many solutions that need only light support for maps but heavier support for data, you may want to forgo web-mapping services altogether and build the logic to display PostGIS data right in your application.

In the sections that follow we go into detail on the basics of setting up and using MapServer and GeoServer as well as creating solutions that don’t require you to host your own web-mapping services.

11.2. Using MapServer

If we wanted to do some heavy lifting by showing thousands of hefty features, then outputting vector features would be slow and cumbersome. In this case, it’s better to output image tiles using a web-mapping service or tile service. As a user zooms in, we might want to complement this with either a vector output we rolled our own or with a WFS. For this next example, we’ll demonstrate using MapServer’s WMS features.

11.2.1. Installing MapServer

MapServer is a mature product, and as such there’s little need to compile from scratch unless you want to. There are already precompiled binaries for most any operating system. (See http://mapserver.org/download.html#binaries.)

Windows Install

For MS Windows installs several options are available. The OSGeo4W and MS4W are bulky installations because they include an Apache server and various other GIS open source packages.

We like using the FWTools package because it’s much lighter weight, tends to be up to date, and also often contains the latest developer version. It also includes the C# Interop extensions to allow the use of MapScript from an ASP.NET (VB.Net or C#) environment. To deploy on a Windows IIS server as CGI, we usually do the following:

1. Extract the FWTools executable installer file (yes, you can treat it as if it were a zip file).
2. Copy the contents of the $HWNPARENT/bin folder to somewhere on the web server that’s marked as allowing executables (this can be a cgi-bin or some folder you create that you mark as allowing executables).
3. Copy the proj_lib folder onto the web server. You’ll need to reference the path in your MapServer map file later, but it doesn’t need to be web accessible.
4. If you want to use .NET MapScript in VB.NET or C# or some other .NET language, then copy the csharp folder files into the bin folder of your .NET application. There are thread issues, especially in .NET MapScript, so many people prefer to use SharpMap.NET for mapping if tight integration with a .NET application is needed.

Security Considerations

If you’re going to have PostGIS layers, you may need to put the password in the map file or in a file included in the map file. You don’t want this information readable, and may not want your map files readable at all for copyright reasons.

There are a couple of safeguards against this. Please do at least one of these:

• Don’t put your map file in a folder that’s web accessible. Admittedly, we tend to break this rule out of convenience of having everything related together.
• Use the msencrypt executable packaged with MapServer to encrypt the password, and use only the encrypted password.
• Use an INCLUDE clause in your map file and make sure the INCLUDE file is of an extension type that isn’t served by a web server. For example, we use the .config extension in IIS because ASP.NET will never serve a file with this extension. Using an INCLUDE for the PostGIS connection string is also convenient, at least if all your PostGIS layers use the same database. This saves you from having to repeat the same information over and over again.
• If you have control of your own web server, you can block dishing out map files by editing your httpd.conf or in IIS mapping the files to 404.dll or some other IIS ISAPI processor.

11.2.2. Creating WMS and WFS services

MapServer supports its own non-OGC API as well as WMS, WFS, WCS, and other web service interfaces. We’re going to focus on its OGC WMS and WFS functionality. For the OGC WMS/WFS features, you don’t need template files. A correctly configured map file with WFS/WMS metadata sections, a set of fonts, a symbol set, and proj_lib will do.

For our map files, we like to use INCLUDEs for sections that we reuse repeatedly within the map or reuse across several maps, such as for the PostGIS connection string, or for general configurations like the location of the projection library.

The following listing shows what such a map file looks like.

Listing 11.1. Map with INCLUDEs

We include a file called config.inc.map that contains the paths to our projection library, symbolset, and fontset. All INCLUDEs are relative to the location of the file they’re included in. This defines the default output projection of the map if none is given. Each layer can be in a different projection, but they’ll be reprojected to the map projection when the map is called. This projection is often overridden in WMS calls with the SRS parameter. The metadata section is particularly important, because this makes the map file behave like a true WMS/WFS. The ows_* elements are shorthand for WFS and WMS so properties that are the same for both don’t have to be specified twice. WFS version 1.0.0 (supported by MapServer 5.6) can have only one SRS. The WMS standard allows many SRSes, and the ones listed are the ones the WMS service will allow as parameters passed in SRS. The online resource gets displayed in the WMS capabilities as the URL to call to access the service.

The config.inc.map defines the location of the symbolset, proj library, and fonts. It’s shown in the following snippet:

CONFIG PROJ_LIB "c:/mapserv/proj_lib/"
SYMBOLSET "symbols/postgis_in_action.sym"
FONTSET "c:/mapserv/fonts/fonts.list"

The proj library is always an absolute physical path, but the symbolset and fontset can be absolute or relative to the location of the map file. If you’re on Windows, you can copy the fonts you’ll use from your Windows/fonts folder into your mapserv fonts folder and then list them in the fonts.list file (as shown in http://mapserver.org/mapfile/fontset.html).

For the symbol set you can use map symbolset codes or images. A sample of both is packaged in the MapServer source download file.

In the next example, we show one of the layers in our layers.inc.map file. Note that you can include layers directly in the main map file.

Listing 11.2. Sample layer from layers.inc.map

Every map layer starts with LAYER and has a NAME and TYPE. TYPE for PostGIS layers is usually LINE, POINT, POLYGON, or ANNOTATION. We include a file called postgis.config and will include this for each of our PostGIS layers to define the connection string to our PostGIS database. MapServer supports angled text, which is useful for labeling streets. Using ANGLE AUTO, the labels will wrap along the line segments.

The postgis.config file looks something like this:

CONNECTIONTYPE POSTGIS
CONNECTION "host=localhost dbname=somedb user=someuser port=5432 password=something"
PROCESSING "CLOSE_CONNECTION=DEFER"

The CLOSE_CONNECTION=DEFER ensures that if multiple PostGIS layers are asked for, the connection will be reused instead of creating a new connection. This results in faster performance.

So we have a map file now, but how do we turn this map file into a WMS/WFS service? We call the MapServer CGI with the map file as argument. The following code snippet calls the GetCapabilities request to show what layers and functionality are provided:

http://yourserver/cgi-bin/mapserv.exe?map=c:/mapserv/maps/
     postgis_in_action.map&REQUEST=GetCapabilities&SERVICE=WMS&VERSION=1.1.1

11.2.3. Calling a mapping service using a reverse proxy

Specifying a map file for each call is often unwanted. Many people prefer to set up either a CGI script or a reverse proxy so that the map file doesn’t have to be explicitly named. You can do more with a reverse proxy than with a CGI script.

 

 

If we use a reverse proxy or a cgi-bin script, our long map URL example can be reduced to

http://yourserver/GetPAMap.ashx?REQUEST=GetCapabilities&SERVICE=WMS &VERSION=1.1.1

In listing 11.3, we demonstrate what a simple reverse proxy written in C# looks like. This is just a snippet. We have equivalent code in the source download packaged for VB.NET. If you’re using PHP, you can implement similar logic using curl. It can also be used to set up GeoServer web-mapping services, for example, if you want GeoServer to run on its own Apache or Jetty web server on a local port or even on a separate server in your internal network, while keeping the regular port 80 for a regular Apache or IIS server. The next example deals only with GET requests, which is generally what most WMS servers use. For POST you can do a check on the Request method by looping through the REQUEST and POST variables.

Listing 11.3. Snippet of a reverse proxy in C#

We first loop through all the arguments received via the client query string. We then check to see if the OGC request is a GetCapabilities or GetFeatureInfo, and if it is, we assume that the result returned by our internal server is XML. If it isn’t, we’ll assume it’s an image and process it as such.

In order to overlap our PostGIS MapServer layers using our reverse proxy, we’d use code similar to that in the following listing.

Listing 11.4. PostGIS MapServer layers using proxy

var postgiswmsurl = "http://www.postgis.us/demos/chapter_11/GetPAMap.ashx?"
map.addLayer(new OpenLayers.Layer.WMS("My PostGIS Layers", postgiswmsurl,
{ 'layers': "hospitals,major_roads,openspace",
'transparent': "true", 'FORMAT': "image/gif"},
{ 'isBaseLayer': false, 'visibility': true, 'buffer': 1, 'singleTile':false,
     'tileSize': new OpenLayers.Size(200,200),
'attribution': 'Data downloaded from <a href="http://www.mass.gov/mgis/
     ">MassGIS</a>'})
);

In the next section, we discuss setting up GeoServer and configuring it for WMS and WFS services. GeoServer (as mentioned earlier) is another map-serving program similar to MapServer.

11.3. Using GeoServer

GeoServer is similar in flavor to MapServer except that it’s a bit heftier and comes with an administrative user interface, so there’s not as much need for manually configuring files with a text editor, and it supports WFS-T.

11.3.1. Installing GeoServer

GeoServer has several installation packages that can be downloaded from http://geoserver.org/display/GEOS/Stable.

• Setup installers for Windows and Mac guide you through the setup. They come with the mini web server Jetty.
• Java binaries are available for all operating systems. You need only extract them to a folder and manually set the environment variables. Jetty is included too.
• A web application archive (WAR) is available for those who already have a servlet engine installed on their server and just want to run GeoServer as another servlet application. This one doesn’t come with Jetty.

We chose the Java binary geoserver-2.0.1 version. To set it up, do the following:

1. Make sure you have Java JDK 1.5+ installed.
2. Extract the folder into the root, for example, C:geoserver or /usr/local/ geoserver.
3. On Windows, set the appropriate system environment variables. JAVA_HOME would be something like C:Program FilesJavajdk1.6.0_16 (or whatever JDK you have).
4. cd into the geoserverin folder and from the command line run startup.bat (for Windows) or startup.sh for Linux/Unix.
5. You then should be able to get to the administrative panel by navigating to the following link on your web browser: http://localhost:8080/geoserver.

11.3.2. Setting up PostGIS workspaces

In this section we’ll cover setting up a GeoServer workspace to house our tables and registering PostGIS tables with GeoServer. Follow these steps:

1.  From the Admin menu > Data, choose Workspaces and click to add a new workspace. Your New Workspace screen should look something like figure 11.1.

Figure 11.1. Setting up a GeoServer workspace

2.  From the Admin left navigation menu choose Data >Stores.

3.  Click Add New Store and then choose PostGIS from the list of options, as shown in figure 11.2.

Figure 11.2. Adding a GeoServer PostGIS data source

4.  Give the data source a name—ch11—and fill in all the credentials asked for. By default GeoServer uses the public schema, which means it will list only layers from that schema. If you want it to list a different schema, like in our case ch11, then replace public with ch11.

5.  Select Layers from the Admin menu, click Add a New Resource, and choose the postgis_in_action store you created previously. Your screen should look something like figure 11.3.

Figure 11.3. Selecting PostGIS layers

GeoServer data stores from other schemas

It’s possible to leave the schema setting blank in GeoServer for PostGIS, and the layer chooser will list them all. However, we’ve found that in that case publishing layers in non-public will throw an error. So be sure to create a different data store for each schema you want to publish.

6.  Publish the layer you want. Make sure to choose Compute Bounding Boxes from Data.

7.  Click Add New Resource.

8.  Repeat steps 6 and 7 for each layer you want to publish.

11.3.3. Accessing PostGIS Layers via GeoServer WMS/WFS

Once you’ve published your PostGIS layers, you can quickly see them via the Layer Preview menu link. Figure 11.4 shows what that screen looks like. Note that it also shows the OpenLayers code to be used to call the layer. It also shows GeoJSON as a direct WFS output format.

Figure 11.4. Layer preview screen of GeoServer

OpenLayers is a popular web-mapping client companion for GeoServer and UMN MapServer. As you saw in the layer preview, GeoServer even autogenerates Open-Layers sample JavaScript code to display each of your layers.

In the next section, we’ll introduce you to using OpenLayers and GeoExt. Open-Layers and GeoExt are web-mapping JavaScript frameworks that are designed to work together. OpenLayers provides basic mapping functionality for loading layers, editing widgets, and so forth. GeoExt builds on top of OpenLayers by providing data grids and other controls that give the web application more of a desktop feel. These two frameworks are useful for commercial web services, WMS services offered by Geo-Server/MapServer/SharpMap.NET, and for scripted applications with languages such as ASP.NET or PHP.

11.4. Basics of OpenLayers and GeoExt

In the beginning, mapping services like Google Maps, Virtual Earth, MapQuest, and Yahoo had their own proprietary JavaScript APIs to access their data. This was a Bad Thing, because if you decided you liked the maps of service A better than the maps of service B, or if usage and pricing became too cumbersome, then you had to rewrite everything. Worst yet, if you wanted to feed your own data via an OGC WMS or ArcGIS/ IMS server for your area of interest, it was hard to integrate the base layers provided by these services with your custom study area layers.

OpenLayers changed the landscape quite a bit by allowing layers provided by different vendors with vastly different APIs to be accessed using the same API, or, better yet, to be used together in the same map. OpenLayers started life as an incubation project of MetaCarta (now a part of Nokia), because it needed to create an easy-to-use toolkit for customers to digest its map product offerings. OpenLayers is now an incubation project of OSGeo.

What does OpenLayers give you that you can’t easily get elsewhere?

• Layer classes to access most of the proprietary non-OGC-compliant tile map offerings, such as Google Maps, Virtual Earth (Bing), MapQuest, Yahoo, and ArcGIS Rest, using the same interface for all
• Layer classes to access OGC-compliant map servers WMS, WFS, and WFS-T, again using the same fairly consistent map layer creation call
• The ability to overlay all these competing proprietary services in one map
• Various controls to build custom menus, toolbars, and widgets to enable map editing

All these things are wonderful, and that’s why OpenLayers has become so popular. Most great things aren’t without their tradeoffs though. So what are these tradeoffs?

• It’s hard to get to the deep features of a proprietary service, such as the 3D street views provided by Google Maps and Bing. This may change as new OpenLayers layer classes are added to support these features. Note that GeoExt does have controls to get to Google Maps Street View and to synchronize it with your map.
• Yet another API to learn with the hope that you don’t have to learn any other API.

11.4.1. Using OpenLayers

The official site for OpenLayers is http://www.openlayers.org. Class documentation is available, although you’ll probably find the numerous code samples to be much more useful for getting started. Because OpenLayers is nothing more than a glorified JavaScript file, you can download the file and use it directly from your web server. Alternatively, you can link your code directly with the version on OpenLayers, ensuring that you always have the latest version.

One thing that OpenLayers is particularly good at is allowing you to integrate various map sources from disparate services. It has classes for accessing ArcGIS Rest, ArcIMS, Google, Bing (Virtual Earth), OpenStreetMap (OSM), MapServer-specific API, as well as standard OGC-compliant WMS and WFS services produced with tools like MapServer, GeoServer, Degree, and TinyOWS.

For our first example, we’re going to use a base layer offered by OpenStreetMap and add a WKT layer with points marking New York City, Los Angeles, Chicago, Houston, and Philadelphia (the five largest cities in the United States at the time of writing). We want you to observe from the code that an OpenLayers HTML file almost always comprises the following sections:

• OpenLayers and other relevant JS includes. For custom layers such as OSM, Google, or Bing, you’d include the script source that points to those sites. In this case we include the custom script source from the OpenStreetMap site that includes the OSM class. Because OpenStreetMap builds its sites on OpenLayers as well, it extends the OpenLayers base classes. For other layers such as Google, you’ll find in the OpenLayers kits classes that wrap the Google API in a stub OpenLayers.Layer.Google class and so forth.
• The map object that’s defined in JavaScript and is created and initialized in an initialization method.
• The call to the initialization method, either in the body onload event or at the end of the JavaScript section. We don’t like the first method, because certain languages like ASP.NET get messy when calls are put in the body load events.
• The full map is available at http://www.postgis.us/demos/chapter_11/osm_newengland1.htm and can be downloaded as part of the chapter 11 download. The following listing shows the OpenLayers general setup.

Listing 11.5. OpenLayers general setup

We first include the source to OpenLayers.js. If you want to customize or fully control this script, you should download and reference it locally, as shown in this code. You can also link directly to the version on the OpenLayers website, http://openlayers.org/api/2.8/OpenLayers.js, if you want to get going quickly. We include a link to additional OpenStreepMap classes. In the case of Google Maps or Bing, you’d need to include scripts from those providers to use as map layers. We declare our global variables. The zoom denotes the default zoom level for our PanZoomBar. The init function is central to the setup. It instantiates the OpenLayers map and loads it into a div called “map” (you can call the div anything you want). We load only a few basic controls. Note that because we set the displayProjection to EPSG:4326, the mouse position is shown in lon lat instead of map units. We declare the OSM tile class. There are several to choose from: Mapnik, CycleMap, and Osmarender, each of which has a slightly different look and feel and data. We center the map at the lon lat point we declared in . Note the transformation from lon lat to map units.

In our next part, we’ll add the body of our HTML page:

    <div style="width:100%; height:100%" id="map"></div>
    <script type="text/javascript" defer="true">
       init();
    </script>

We create the div for the map at the position where we want to place the map on the screen. In this case we set its width and height to 100% by 100%, so the map will expand dynamically to fit the page. In many cases you’ll set this to a fixed size, expressed in pixels, for example, 500px. We then include the JavaScript with the defer options so it isn’t called until the rest of the page has loaded.

After this our map will look like the one in figure 11.5.

Figure 11.5. Result of our map in listing 11.5

In many cases, the user should be able to be able to pick the layers to display from a menu or toggle between several base layers. In order to allow this, we’ll add the OpenLayers control called LayerSwitcher as well as another third-party layer. Some layers are always base layers, but to make an included WMS layer a base layer, you need to set the isBaseLayer property of the layer explicitly to true.

In the following listing, we’ll add Yahoo! Maps and the layer switcher control. The resulting map can be seen at http://www.postgis.us/demos/chapter_11/osm_newengland2.htm.

Listing 11.6. Revising the map to have Yahoo as an option

In we add the script source for Yahoo! Maps, which is needed for the OpenLayers Yahoo layer class. This class only acts as a proxy and translates from the OpenLayers API to the Yahoo API. We then revise our init function to create the Yahoo layer. Note the sphericalMercator setting. Without this, the Yahoo layer can’t be overlaid with the OSM layer, which is in a Mercator projection. We add a layer switcher control so that the user can toggle back and forth between the two layers.

You can see in figure 11.6 the layer switch control showing the available layers and with the Yahoo layer currently selected.

Figure 11.6. Map after adding change in listing 11.6

We’ve just experimented with base layers. For any map, there can be only one base layer selected. The other type of layer is an overlay layer that can be combined with the base layer. You can select as many overlays as you want to be shown together.

Adding WMS Layers to A Map

All open source web-mapping servers support the OGC web-mapping service standard. This means you can add your own layers as well as layers from third parties.

In this next example we’ll demonstrate how to add MassGIS layers to our Open-Layers map. MassGIS has a useful page describing how to use its web services as well as detailing the fundamentals of the WMS and WFS standards: http://lyceum.massgis.state.ma.us/wiki/doku.php.

For this example, we’re going to use the OpenStreetMap Mapnik map style as a base layer and add MassGIS WMS layers. Note that MassGIS, the primary provider of GIS data for Massachusetts, uses GeoServer. So when you set up GeoServer, the way you overlay your services will be pretty much the same as you see here. This particular example can be viewed at http://www.postgis.us/demos/chapter_11/olmapmassfish.htm.

Listing 11.7. Adding WMS layers to OpenLayers

In this code snippet we add three seafood layers from MassGIS web services. These are overlays by default. Because MassGIS doesn’t have a default attribution, we add an attribution text that will appear in the attribution section whenever this layer is selected. We also use another feature of WMS, the GetLegendGraphic, to get the graphics for each of the layers and put it in a div element called “legend.”

The result of this snippet is shown in figure 11.7.

Figure 11.7. Example of overlaying WMS layers on an OpenStreetMap base using the code in listing 11.7

For our next example, we’re going to enhance our OpenLayers experience with GeoExt.

11.4.2. Enhancing OpenLayers with GeoExt

In the beginning, all neogeographers were happy with OpenLayers. With happiness came the realization that we can become happier still. People wanted grids to show attribute data. We wanted to be able to drag and drop things. We wanted to sort tables and have tree menus. We wanted selections on a grid and to use them to reposition the map. We wanted sliding controls, date pickers, and charts. In essence, we wanted to build web-mapping applications that felt more like desktop applications without the need of Flash and Silverlight plug-ins. All these things could be done with OpenLayers if we were willing to do the additional custom JavaScript programming, but a lot of this functionality already existed in ExtJS. ExtJS is a popular JavaScript API for making rich web applications that look like desktop applications. What the GeoExt project did was to combine the OpenLayers mapping system with the ExtJS web application–building interface to create something that would be the best of both toolkits. Haiti Crisis Map, http://haiticrisismap.org/, shown in figure 11.8, is an example of an application built on GeoExt that uses ExtJS accordion panes to enable different features. Many of the GeoServer administrative web interfaces are now also built with GeoExt.

Figure 11.8. Example of a GeoExt application using ExtJS collapsible panels and OpenLayers

In short, you can use OpenLayers by itself, and most people still do, or you can enrich it with GeoExt. We can’t talk about GeoExt without first demonstrating OpenLayers. In the next section, we’ll demonstrate a common activity with Open-Layers and then enhance it with GeoExt.

Both OpenLayers and GeoExt have fairly liberal and commercial-friendly licenses. OpenLayers is under an MIT License, and GeoExt is under a BSD license. But GeoExt also relies on ExtJS, which is under a dual GPL v3 and commercial license. This means that if you need to extend or modify any of the classes in ExtJS without making your source code available to the public, then you need to use the commercial ExtJS license. Details are available here: http://www.extjs.com/products/license-faq.php.

In order to use GeoExt, you need OpenLayers, GeoExt, and ExtJS. You can download the additional files. GeoExt.js is part of the download file at http://www.geoext.org/, and you can download ExtJS from http://www.extjs.com/. For our examples, we’ll be using ExtJS 3.3.1 and GeoExt 0.6.

For this first example, we create a page that loads an OpenLayers map into an ExtJS window using GeoExt. We divide the code into two parts: the .htm file, shown in the following listing, and the .js file, shown in listing 11.9.

Listing 11.8. geoextnewenglandwin.htm: basic structure of page

In we include all the JS dependency files we need, in this case the APIs for Yahoo!, OpenLayers, ExtJS, GeoExt, and OpenStreetMap. We then include the JavaScript file for our custom app. Note that for we could have put all the JavaScript on the page itself, but it’s standard practice to put it in a separate file, especially if there’s a fair amount of JavaScript code.

The meat of the application is in the JS, which is shown in the following listing.

Listing 11.9. geoextnewenglandwin.js: using GeoExt to display OL map in Ext window

This leads to the page shown in figure 11.9. The benefit of putting an OpenLayers map in a separate window is that you can move the window around on the browser screen, resize it, and minimize it. There are other controls you can use in GeoExt, such as view ports and panels. View ports allow for autostretching within a div. They allow you to position the map alongside grids, collapsible panels, and other form controls and to let the map interact with them.

Figure 11.9. OpenLayers map in an ExtJS movable, stretchable, collapsible window using listing 11.9

For our next example, we’re going to create a PHP app that queries our database and outputs a GeoJSON layer using the PostGIS ST_AsJSON function.

11.5. Displaying data with server-side web scripting

In this section, we’ll demonstrate examples of plain server-side web scripting without support of web-mapping services. Although none of these use any of the OGC web-mapping services, we hope it will be clear that you can mix and match these with standard web-mapping services.

We’ll demonstrate the following concepts:

• Outputting layers with PostGIS ST_As* output functions
• Consuming the output layers with GeoExt and Google Earth
• Proximity queries with PostGIS geography

11.5.1. Using PostGIS output functions with PHP

For this exercise, we’re going to use PHP and the PHP helper libraries Smarty and PHP ADOdb to build a datafeeder.php script file. We’ll later use this PHP script file to feed our web mapping and Google Earth client frontends. ADOdb is a database abstraction layer used by many PHP applications to provide a generic interface to all databases. Smarty is a templating engine for PHP that allows the separation of presentation logic from application logic. Both are free and open source with LGPL/BSD–style licenses. You can obtain them from the following sites:

• PHP ADOdb—http://adodb.sourceforge.net/
• Smarty—http://www.smarty.net/download.php

When we build applications with these, our general convention is to create a file called app.inc.php that includes all the includes we’ll need. Such a file looks something like this:

<?php
include_once("config.inc.php");
include_once("libs/adodb5/adodb.inc.php");
include_once("libs/smarty/Smarty.class.php");
?>

We create a separate file to contain connection strings and so forth. Our config.inc.php for this app looks like the following, which is a standard ADOdb data URL format:

<?php
define("DSN", 'postgres://userhere:passwordhere@localhost:5432/
     dbhere?persist'),
?>

Creating A Datafeeder in PHP for Our Map

The datafeeder.php will accept an argument called format, which will be used to determine the type of output format. In our code we’ve defined a KML and a JSON output format that we output using Smarty templates. Using Smarty allows us to extend the number of layouts we support without cluttering our request control and data query logic.

• We use PHP ADOdb for data abstraction. Note that you can just as easily use PHP PEAR. This keeps our data load logic short and also has the benefit of making it more generic. As can be clearly seen, you wouldn’t be able to tell this was a PostgreSQL database unless you saw the connection string and PostGIS function calls.
• The datafeeder.php file acts as the controller—reading the request and figuring out which layout and, if applicable, which query to use to satisfy the request.
• We define one Smarty template for each format. Note that formats like KML include styling options, which make using a generic KML handler such as OGR2OGR not ideal in some cases. Using a template allows us to customize this and also to inject database-stored styles in the file.
• We’re using PHP classes instead of standard PHP procedures. Because our class inherits from Smarty, we can call all built-in Smarty functions as if they were native.

The core pieces of the page are shown in the following listing.

Listing 11.10. Core pieces of datafeeder.php

When a class is instantiated in PHP the first function that gets called is the __construct function, which in our case sets up the database connection and also changes the Smarty markup tag. In we also redefine the plug-ins to look in our extraplugins folder for additional plug-ins. Then we call the page_load() function to handle a web request. You can put all that logic in the __construct function, but we break it out for clarity. In our page load we look up the passed-in format in our associative array to pull the name of PostGIS convert functions. We then build our SQL using that output function to output the geom field, load this data into a PHP array using the ADOdb GetRows() function, assign it in our template, and then display the merged data and template.

Our JSON Smarty template looks like the following listing.

Listing 11.11. Smarty data_json.tpl file

We use a Smarty section tag to loop through our PHP array. We then use a foreach to loop through all the fields of each record and output the ones that aren’t id or geom, and then we output the geometry as a separate field. Note the json_encode: This is a Smarty modifier that’s just a wrapper around the PHP 5.2+ built in Json_encode function that will convert our value into safe JSON text. We need to separate each feature by a comma, except if it’s the last feature in our result set.

We put the json_encode Smarty modifier in a file: libs/smarty/extraplugins/ modifier.json_encode.php. It looks like the following:

<?php
function smarty_modifier_json_encode($string){
    return json_encode($string);
}
?>

The generated output of our JSON can be seen on the book site; use this URL: http://www.postgis.us/demos/chapter_11/datafeeder.php?format=json.

If we wanted our data to also be accessible from a KML viewer such as Google Earth, we’d offer the KML format by adding in a KML template. The data_kml.tpl template file is shown in the following listing.

Listing 11.12. KML template to format in KML format

<?xml version='1.0' encoding='UTF-8'?>
<kml xmlns='http://earth.google.com/kml/2.1'>
<Document>
<Style id='defaultStyle'>
   <LineStyle><color>ff00ff00</color><width>3</width></LineStyle>
   <PolyStyle><color>5f00ff00</color><outline>1</outline></PolyStyle>
</Style>
<!--(section name=sec loop=$rs)-->
<Placemark>
  <name><!--($rs[sec].id|escape:html)--></name>
<description>
<!--(foreach from=$rs[sec] key=prop item=val)-->
 <!--(if $prop != 'geom' && $prop != 'id')-->
 <b><!--($prop|escape:html)--></b> <!--($val|escape:html)--><br />
 <!--(/if)-->
<!--(/foreach)-->
</description>
   <styleUrl>#defaultStyle</styleUrl>
  <!--($rs[sec].geom)-->
</Placemark>
<!--(/section)-->
</Document>
</kml>

In this listing, we do more or less the same as we did in our JSON format template, except that we use the escape:html Smarty modifier to make the fields KML friendly. The escape modifier is included in the Smarty download.

11.5.2. Displaying data in Google Earth

We can use Google Earth to display data output by the datafeeder script using the KML format. In Google Earth we create a network link, http://www.postgis.us/demos/chapter_11/datafeeder.php?format=kml, as shown in figure 11.10.

Figure 11.10. Displaying KML layer in Google Earth using template in listing 11.12

In the next exercise, we’ll use the GeoJSON output of our datafeeder.php in a simple GeoExt application.

11.5.3. Loading custom layers with GeoExt

For this exercise, we’ll display a sortable grid using GeoExt where the grid data comes from a GeoJSON datastream output by datafeeder.php. This datastream can be created with any web-scripting language such as Python or ASP.NET. When you’re finished, your page will look like the one shown in figure 11.11.

Figure 11.11. Application with feature grid in sync with map using table column layout from the code in listing 11.13

For this application we created a new .js file similar to the one shown earlier, but with the following parts added:

Listing 11.13. Adding a feature grid window

In we define a new OpenLayers layer that will store the features. We create a GeoExt feature store object that automatically retrieves data from our custom-defined PHP script. We then define a grid panel to display the attribute portion of the features; the select model determines what happens when an item is selected. In this case, the feature gets highlighted on the map. Instead of a movable window we use a two-column table layout, with the first column holding our map (note the use of gx_mappanel) and the second column holding our feature grid panel. This panel is the only panel that gets displayed because of the renderTo: Ext.getBody(), and it contains the mapPanel and gridPanel features as child items.

11.5.4. Proximity queries with PostGIS geography

For many use cases, you don’t care about maps. You just care about how far away the data elements are from where you are or where you want to go. In this case, the PostGIS geography data type introduced in PostGIS 1.5 is probably the simplest way to achieve that goal.

The next listing is a simple PHP snippet of code that demonstrates how to use ADOdb (or some other database abstraction layer) and Smarty.

Listing 11.14. PHP find all roads within one mile of a requested lon lat

We verify that the input is passed in via a GET or POST request. Note that if you wanted to limit to just POST variables, you’d use $_POST[..]. To prevent SQL injection, we verify that the inputs are numeric. We could also validate for range to make sure the values fit in the range -180 to 180. Next, we convert our lon lat to a PostGIS geography POINT expression , and we build the SQL statement. Because geography is always in meters, we multiply by 1609*$range to convert miles to meters for ST_DWithin. We also use ST_Distance to sort the results by proximity. We execute the statement, dump the results into a PHP array, and then assign a Smarty variable that will be used in a section loop similar to what we did in earlier examples.

11.6. Summary

There’s nothing wrong with using proprietary packages such as Google Maps or Microsoft Bing, at least if the data you need to deliver doesn’t exceed their limitations and your usage complies with their licensing terms. Face it: You’ll never be able to recreate the jaw-dropping, uber-cool features that proprietary packages now include, and as a GIS expert, you shouldn’t have to.

If you can’t or choose not to take advantage of proprietary packages, you need to worry about the web setup at both the server level and the client level. On the server side, you need to inject a mapping server between your data source and your existing web server. We covered MapServer and GeoServer in detail. MapServer has a lot of power and is more portable than GeoServer, but you must learn how to create map files in order to work with it. GeoServer comes with a nice interface, but as with most screen-driven software, you face some limitations once you start to outgrow what’s offered in the UI.

Though GIS is a data-centered pursuit, you can’t ignore the clients’ browser experience when it comes to delivery of GIS data. Web consumers are no longer satisfied with the mere display of a map on the web browser. They want zooming, panning, and the ability to tag, edit, layer, and so on. To give users what they want, we recommend that you use OpenLayers. Its JavaScript base makes it more portable than anything else, and it’s the dominant client tool in the market today. We also suggest that you enhance OpenLayers with the new GeoExt framework for a web-browsing experience that could rival that of many proprietary packages available today.

If sharing data via the World Wide Web isn’t for you, in the next chapter we cover desktop tools that connect easily with PostGIS. Many of the desktop tools can consume standard OGC web services that we described in this chapter, such as WMS or WFS. This means that even if you don’t intend to share data via the web, setting up a mapping server may still be a good idea because many desktop tools can also take advantage of the data it serves up even if they don’t have direct support for PostGIS.