Appendix A. Additional resources

This appendix includes links to resources useful for PostGIS users of all walks of life. Some of these links we may have already covered in previous chapters, but they’re also listed here so you can have all of these resources in one place.

PostGIS-focused tutorials and sites

These lists present good tutorials as well as sites focused on PostGIS content.

Getting-started tutorials

Below is a compendium of tutorials accumulated over the years. We tried to list the most up-to-date ones first.

FOSS4G 2009 Sydney Australia— Introduction to PostGIS, by Mark Leslie, complete with sample data and instructions for viewing in uDig. http://revenant.ca/www/postgis/workshop/

BostonGIS— Part 1: Getting Started With PostGIS: An Almost Idiot’s Guide. Mostly geared to the Windows user, this covers how to install PostGIS and load data and offers quick-use examples. It’s still useful to Mac and Linux users because it covers a few basics about loading and using, which are pretty much the same across all OSes. http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

OpenGeo’s Open Source Geostack tutorial— Includes setting up PostGIS, MapServer, GeoServer, and QuantumGIS and creating an application. Also includes a link to download the stack and tutorial data. http://workshops.opengeo.org/stack-intro/. For a PostGIS-specific tutorial, see http://workshops.opengeo.org/postgis-spatialdbtips/

FOSS4G 2007— Introduction to PostGIS. These are PowerPoint and data documents for the intro workshop given by Paul Ramsey at the FOSS4G 2007 conference. This is complete with using UMN MapServer and Canadian data examples. http://www.foss4g2007.org/workshops/W-04/

Paolo Corti, Installing PostGIS on Ubuntu— Paolo goes through not only how to install PostGIS and PostgreSQL on Ubuntu but also how to install QGIS, uDig, and gvSig. He also covers a little about creating databases, users, and roles in PostgreSQL, as well as gives a quickie tour of QGIS, uDig, and gvSig. It’s worth a read even if you aren’t using Ubuntu. http://www.paolocorti.net/public/wordpress/index.php/2008/01/20/installing-postgison-ubuntu

Webb Sprague’s PostgreSQL 2007 talk on PostGIS complete with slides, audio and code— http://www.postgresqlconference.org/2007/talks

Lincoln Ritter, Installing PostgreSQL, PostGIS and More on OS-X Leopard— http://www.lincolnritter.com/blog/2007/12/04/installing-postgresql-postgis-and-more-on-os-x-leopard/

Important GIS sites

OSGeo— OSGeo is the foundation that spearheads and cradles many open source GIS projects. http://www.osgeo.org/

Open Geospatial Consortium (OGC)— This is the body that defines standards for interoperability between GIS products that are both open source and commercial. It defines data portability, web service standards, and spatial SQL standards. http://www.opengeospatial.org/

PostGIS main site— http://www.postgis.org

PostGIS User Wiki and Bug Tracker— http://trac.osgeo.org/postgis

PostgreSQL main site— http://www.postgresql.org

Free GIS— This is a site put together by folks at Intevation GmbH (http://intevation.net/) and is a directory listing of free and open source GIS software, data, documents, and projects. http://www.freegis.org

Spatial Reference Org— This is an invaluable site for looking up spatial reference systems. It’s so important we’re listing it twice. http://spatialreference.org

Noteworthy PostGIS blogs and sites

The blogs and sites in this list are high in PostGIS material and helpful tips and tricks or are just too good on their overall breath of GIS not to be mentioned.

PostGIS in Action book site— We set up this site where you can download data and code discussed in this book. We’ll also be posting our presentations, chapter summaries, other chapter-related information, links, and demos. http://www.postgis.us

Paul Ramsey— One of the original co-developers of PostGIS; many think of him as the face of PostGIS. He does a fair amount of blogging about what’s going on in PostGIS land. More specifically, he focuses on open source GIS and how it fits into the overall GIS ecosystem. He’s a member of the PostGIS steering committee and the 2008 recipient of the Sol Katz Award for Geospatial Free and Open Source leadership. He’s also a contributor to MapServer and the founder of the uDig desktop kit. http://blog.cleverelephant.ca

Martin Davis aka Dr. JTS— Martin is the lead architect behind the Java Topology Suite (JTS), which GEOS (Geometry Engine Open Source) is a C++ port of. A lot of the great geometry-manipulation algorithms you’ll find in PostGIS and other commercial and open source packages that rely on GEOS and JTS are due in large part to his efforts. He blogs about some of the algorithms behind these processes as well as general GIS and sometimes just interesting random technology topics. http://lin-ear-th-inking.blogspot.com/

Simon Greener, Spatial DB Advisor— If you want to see how it’s done in other spatial databases such as Oracle and SQL Server and other tools such as Manifold as well as find some extra tips for getting the most out of PostGIS, then Simon’s your man.

His blog is full of freely available functions he’s written to explore spatial SQL in all its beautiful forms. He has functions for PostGIS, Oracle Locator/Spatial, SQL Server 2008 Spatial, and Manifold. http://www.spatialdbadvisor.com

Dylan Beaudette, California Soil Resource Lab— This site largely authored by Dylan is full of PostGIS/R/GRASS/GDAL tutorials. It’s a must read for anyone doing analytical work with these tools. http://casoilresource.lawr.ucdavis.edu/drupal/blog/2

Bill Dollins, GeoMusings— Bill’s blog is one of the best in terms of its breath of examples of interoperability between commercial and open source GIS. He blogs about SQL Server 2008 Spatial, PostGIS, SpatiaLite, and integration of other open source GIS with other commercial GIS, primarily ESRI ArcGIS. He’s also one of the developers of zigGIS (a plug-in for ArcGIS Desktop 9.1 and above for editing and displaying PostGIS data) along with Abe Gillespie and Paolo Corti. http://geobabble.wordpress.com/

Paolo Corti, Thinking in GIS— I don’t think there is any web GIS, particularly of an open source nature, that Paolo hasn’t tried and oftentimes blogged about with helpful tutorials. His site contains everything from ArcGIS, using PostGIS in ArcGIS, to open source topics such as using GeoDJango, KML Overlays, UMN MapServer, Ruby on Rails, TileCache, OpenLayers, and even NoSQL databases. It also has lots of useful tutorials on installing these packages and getting up and running on Ubuntu. http://www.paolocorti.net/

Postgres OnLine Journal— Check out our more or less monthly journal (available as individual articles online, full month in HTML, and full month in PDF format). It covers general PostgreSQL tips and tricks such as doing automated backups, the TSearch integrated full-text search engine, as well as PostgreSQL integration with other tools such as MS Access, Open Office, and web programming (PHP, ASP.NET, Flex), and comparisons of different databases and administration tools. http://www.postgresonline.com/

BostonGIS— This is our other satellite site focused on OpenGIS standards and open source GIS. We try to pack a lot of PostGIS tutorials in here but also provide tips, tricks, and tutorials on other open source GIS and OpenGIS concepts. You’ll find not only PostGIS here but also tutorials on SpatiaLite, SQL Server 2008, SharpMap.NET, and PL/R and various cheat sheets we’ve developed over the years. The theme of the site centers on using Boston data to demonstrate spatial concepts. http://www.bostongis.com

Nicklas Avén— Nicklas is a member of the core PostGIS developer team. He made major contributions to the distance functions in PostGIS 1.5. He improved the efficiency of the existing functions on large geometries and also introduced some new ones such as ST_MaxDistance, ST_ClosestPoint, ST_LongestLine, ST_ShortestLine, and various others. In PostGIS 2.0 he’s working on 3D measurement functions. In his blog, he chronicles some of his thought processes in adding to the PostGIS code base. http://blog.jordogskog.no/

Mateusz Loskot— Mateusz is a core GEOS developer and PostGIS Raster developer. He blogs a lot about various geo processing kits, packaging, and also PostGIS and PostGIS Raster. http://mateusz.loskot.net/

Sandro Santilli aka strk— Sandro is a long-time PostGIS and GEOS core developer. He’s responsible for integrating much of the GEOS functionality you find in PostGIS and has done work on PostGIS Raster. His blog content is both technical—PostGIS, OpenStreetMap—as well as familial—things like bats. http://strk.keybit.net/blog/

James Fee— No GIS site list would be complete without James, the king of GIS blogging. James is all over the map from commercial GIS to open source GIS to GIS data services and combining them all. He still manages to throw in a bit of PostGIS as well. He’s not afraid to give a candid view of what he thinks is hot and what is not. He’s probably the most-read GIS blogger around. He’s also the maintainer for Planet Geo-spatial. http://www.spatiallyadjusted.com/

Planet OSGeo— This is a blog aggregator of OSGeo community bloggers. Many key OSGeo movers and shakers can be found in this list. Lots of Project community blogs like Quantum GIS and OpenLayers team. Good for staying abreast of OSGeo projects. http://planet.osgeo.org/

Planet Geospatial— This is an aggregator of the more popular and up and coming GIS focused blogs and news sites. http://www.planetgs.com/

JASPA (Java Spatial)— This is an open source spatial extender patterned after PostGIS but written in Java instead of C. It currently has two implementations: PostgreSQL and HSQL. The PostgreSQL implementation has more or less the same functions as PostGIS 1.5 plus some additional ones (minus the geography support). The other database is HSQLDB—a Java-built relational database. The core of its logic is built using JTS, GeoTools, and PL/Java. http://www.osor.eu/projects/jaspa

Noteworthy R, PL/R sites, and newsgroups

These are sites rich in R and PL/R content:

PL/R official site— This is where you can download the source and binaries for PL/R PostgreSQL language handler. You can also subscribe to the PL/R mailing list from here. http://www.joeconway.com/plr/

PL/R Wiki— As of this writing, this is a work in progress that already contains useful install manuals and snippets of PL/R code. The main PL/R page will eventually be merged in here. http://www.joeconway.com/web/guest/pl/r

R— This is where you can download the R software package and basic tutorials on R. http://www.r-project.org/

California Soil Resource Lab— These pages are chock full of GIS-related R scripts as well as PL/PgSQL scripts. Keep in mind that although many of the R scripts are raw R, they can be easily flipped into PL/R scripts with minor changes. http://casoilresource.lawr.ucdavis.edu/drupal/blog

Quick-R— Lots of snippet R recipes you can cut and paste from. This is authored by Robert Kabacoff, the author of the Manning book R in Action. http://www.statmethods.net/

Boston GIS: Getting started with PL/R— These are three quick tutorials we’ve written on PL/R:

http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01

http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut02

http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut03

Connecting PostGIS with R— This demonstrates using output of PostGIS queries in R. This particular example doesn’t use PL/R but instead uses R directly. http://wiki.intamap.org/index.php/PostGIS#Connecting_PostGIS_with_R

R-sig-Geo— This is a newsgroup focused on using R in geoinfomatics and geographical mapping. It’s a good group to join if you want to learn tricks of the trade and what R packages are available for doing geospatial analysis with R. https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Analysis of Spatial Data— This is a descriptive listing of R packages commonly used in spatial analysis in R. http://cran.r-project.org/web/views/Spatial.html

Applied Spatial Data Analysis with R— This is the book site for the 2008 Springer publication Applied Spatial Data Analysis with R. The book is written by Roger S. Bivand (manager of the R-sig-Geo newsgroup) and others. The book site contains downloadable code and data sets from the book. It demonstrates various exercises using the R geospatial packages. http://www.asdar-book.org/

A Practical Guide to Geostatistical Mapping— This is a 2009 publication by Tomislav Hengl. It comes as free e-book or $13 hard print course workbook. It has many examples of using R geostatistical packages as well as using GRASS and is licensed under Creative Commons Attribution-Noncommercial-No Derivative Works 3.0. http://spatial-analyst.net/book/order

pgRouting installation and examples

These are sites specific to pgRouting where you can download source and binaries or learn how to compile and use it:

pgRouting official site— Here you’ll find links to download the source and various binaries available for different OSes. http://pgrouting.postlbs.org

pgRouting on Ubuntu Netbook Remix 9.10— Details how to compile and install PgRouting on Ubuntu. http://www.mkgeomatics.com/wordpress/?p=312

FOSS4G 2009 Tokyo PgRouting Workshop— Workshop tutorial slides on getting up and running with pgRouting by Daniel Kastl. http://www.osgeo.jp/wordpress/wp-content/uploads/2009/11/workshop_manual.pdf

Complete documents and sample data for workshop that details using pgRouting with Open-Layers and MapFish— http://pgrouting.postlbs.org/wiki/WorkshopFOSS4G2008

PL/Python installation and examples

Our Postgres OnLine Journal— We have a cheatsheet and collection of intro articles on PL/Python detailing installing, samples, and basic flow. http://www.postgresonline.com/journal/archives/106-PL-Python.html

Official docs for PostgreSQL 8.4 on PL/Python— Similar docs exist for 8.2 and 8.3. http://www.postgresql.org/docs/8.4/interactive/plpython.html

GDAL— Geographic Data Abstraction Library has both a C and a Python interface. It’s probably the most common Python library used by the GIS open source Python crowd. http://www.gdal.org

Enabling GDAL in Python and GDAL Python helper packages— http://pypi.python.org/pypi/GDAL/. As of this writing, there are no precompiled Windows binaries for 1.7 and above. If you want precompiled, use the 1.6 http://pypi.python.org/pypi/GDAL/1.6.1, which is available for Python 2.5/2.6. This will allow you to access the OGR2OGR and GDAL objects in PL/Python similar to what we demonstrated in PL/R.

NumPy— This is an open source numerical processing library for Python that’s similar in functionality to things like MatLab. It’s used for dealing with complex matrices. It’s a common favorite among scientific professionals. It also has some useful GIS bindings and is commonly combined with GDAL. http://numpy.scipy.org/

Gnuplot.py— This is a Python library for interfacing with Gnuplot that allows generating attractive graphical plots in Python. http://gnuplot-py.sourceforge.net/

Windows 32-bit and 64-bit nightly build binaries by Tamas Szekeres— These contain the latest Python PyGDAL. http://vbkto.dyndns.org/sdk/

Raster-related information

PostGIS Raster Home Page— This page will give you links to other PostGIS Raster resources, provide you with status of the project, show where you can download source or binaries, and link to the road map. http://trac.osgeo.org/postgis/wiki/WKTRaster

The GDAL 1.6+ libraries— these have a PostGIS Raster driver that will allow you to export data out of PostGIS raster format. This page will also give you more extensive details about using the gdal2raster.py script we described. http://trac.osgeo.org/gdal/wiki/frmts_wtkraster.html

GDAL Raster Formats— This page lists raster formats GDAL supports; you can load in any of the formats supported by your version of GDAL into the PostGIS raster data type using gdal2raster.py. http://www.gdal.org/formats_list.html

RasterLite— This is a raster extender for SQLite similar to how SpatiaLite is a vector extender for SQLite. Its focus is more on storing raster in a database for rendering rather than PostGIS Raster for analysis. GDAL 1.7+ has drivers that support it. http://www.gaia-gis.it/spatialite/rasterlite-man.pdf.

Precompiled binaries of GDAL and Python needed for PostGIS Raster loading— For Windows the aforementioned Tamas Szekeres binaries are the latest and greatest and are built nightly. http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries.

rasdaman— This is a raster server implemented using PostgreSQL with its own matrix-like query language called rasql. Its focus is high performance and raster support for both rendering and matrix-like analysis, but it currently lacks support for georeferencing, integration with the core SQL base of PostgreSQL and PostGIS functions. The Rasdaman Group is currently working on providing integration with GDAL and the ability to use it directly in PostgreSQL SQL queries and georeferencing capabilities. http://www.rasdaman.com/

Open source tools and offerings

The following listings offer prepackaged open source tools that include PostGIS as a core component of their mix. They include one-click installers, fully contained application stacks, and single-download virtual machines.

Installers and self-contained suites that include/work with PostGIS

The following GIS suites contain PostGIS as part of an integrated GIS desktop and/or web mapping tool:

For Mac users— There are binaries for Mac OS X graciously supplied by KyngChaos. The offerings include PostgreSQL, PostGIS, and some other useful open source GIS toolkits. http://www.kyngchaos.com/wiki/software:postgres

PostgreSQL Yum Repository— For Red Hat Linux (Enterprise and Fedora) and CentOS, there is the recently released PostgreSQL Yum repository that has packages for PostgreSQL, PostGIS, and several other PostgreSQL accessories. http://www.pgrpms.org/

OpenGeo Stack— This stack contains GeoServer, GeoExt, GeoEditor, GeoWebCache, PostGIS, PostGIS GUI shapefile loader, and optional extensions for ArcSDE and Oracle Spatial. It has one-click installers available for Windows, Mac OS X, and soon Linux. The GeoEditor is a web-based GIS editor that allows you to edit PostGIS data via the web interface. It comes in both Enterprise and Community Editions. The main differences are that Enterprise includes support, training, and Service Level Assurances (SLA), as well as hand-holding help with upgrades for those new to GIS or who need more predictable professional support. The stack also comes prepackaged with sample data to get you started. The Community Edition is a free open source and binary download for the more experienced user, student user, or consultant looking for an easy-to-configure stack for a client and to extend with their own web product. http://opengeo.org/community/suite/download/ Comparison between the community and enterprise editions of OpenGeo Stack can be found at http://opengeo.org/products/suite/compare/

Portable GIS— If you’re a Windows user, check out Portable GIS managed by Jo Cook. It’s really cool and comes packaged with PostgreSQL, PostGIS, MySQL, Quantum, GRASS, FWTools, MapServer, GeoServer, FeatureServer, and OpenLayers, all of which can be run from a thumb drive. http://www.archaeogeek.com/blog/portable-gis/

GISVM— If you want a fully contained GIS Virtual Machine that you can play with a VMPlayer such as VMWare’s freely available VMWare VM Player and that contains best-of-the-breed open source GIS tools, check out GISVM. This is an Ubuntu VM that comes in three flavors: GIS VM Basic English, GIS VM Geostatistics English, and a Portuguese version. http://www.gisvm.com/

The basic International English version comes packaged with PostgreSQL/PostGIS, GeoServer, Mapserver, FWTools, QGIS/Grass, gvSIG, uDig, OpenJump, and Kosmo.

The Geostatistical Version contains all the above plus PL/R and R Statistical Environment (similar to SAS and S-Plus), SAGA, and MySQL 5.

DebianGis— This provides binary packages for MapServer, PostgreSQL/PostGIS, GDAL, QGIS, and GEOS for Debian Linux. http://wiki.debian.org/DebianGis

EnterpriseDB One-Click PostgreSQL/PostGIS installer— If you’re on a Windows system or a desktop Linux or Mac OS X, the easiest way to get started is to use the respective One-Click installers provided by EnterpriseDB. These we cover in the installation guide appendix. http://www.enterprisedb.com/products/pgdownload.do

Free open source desktop GIS

The following desktop tools have integration features with PostGIS to allow viewing and editing PostGIS data:

OpenJUMP— This is one of our favorite desktop GIS tools and what we used to render many of the ad hoc spatial queries you see in this book. It is a Java-based GIS desktop toolkit based on a plug-in architecture and has many user-contributed plug-ins. It runs on Linux/Windows/Mac OS X. http://www.openjump.org/

QuantumGIS (QGIS)— This is perhaps the most popular of the free open source desktop tools. It also has a plug-in architecture. QGIS is written in C++ but offers a rich Python scripting environment and various GRASS integration options. It also includes drivers for connecting to PostGIS data as well as various other GIS data sources. QGIS is GNU GPL licensed. http://www.qgis.org/

gvSIG— This Java-based desktop platform offers lots of integration features to ArcIMS and other ESRI services. http://www.gvsig.gva.es/

uDig— This is an Eclipse-based Java desktop application and SDK. It has lots of integration features with OGC-compliant web services and more advanced cartography. http://udig.refractions.net

OSGeo4W Installer— This is an online installer for MS Windows that packages QuantumGIS, GDAL/OGR, Python bindings for MapServer, GDAL, and Apache WebServer with web apps and sample data in a single install that allows you to pick and choose what you want. If you want to use the osgeo/gdal package under Python 2.5+ and don’t want to compile it yourself, this is currently the easiest package to use. http://trac.osgeo.org/osgeo4w/

Geographic Resources Analysis Support System (GRASS)— GRASS is probably the oldest and one of the most advanced free and open source tools for analyzing vector, raster, and other GIS data. It’s designed more for the advanced GIS analyst rather than a new GIS or pure spatial database user. Although it’s not set up specifically for PostGIS, there are many avenues of integration such as the PostGRASS driver, JGRASS, and QGIS GRASS integration tools. http://grass.osgeo.org/

Extract Transform Load (ETL)

GDAL/OGR— This is the most popular of all-purpose open source free ETL tools. It’s licensed under the MIT license, which is similar to BSD. http://gdal.org/ogr2ogr.html. Binaries can be downloaded from http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries.

shp2pgsql, pgsql2shp, shp2pgsql-gui— Packaged with PostGIS for dumping and loading data from ESRI shapefile format. These are downloadable as part of the source tar ball. http://www.postgis.org/download/ These are also available in binary form for Windows from http://www.postgis.org/download/windows/.

osm2pgsql— This is a command-line tool specifically designed for converting Open-StreetMap XML (OSM) format to PostgreSQL/PostGIS. Binaries for most OSes including Windows and Mac OS X can be downloaded from http://wiki.openstreetmap.org/wiki/Osm2pgsql.

Spatial Data Integrator— This is a GPL v2 licensed open source ETL tool with geospatial capabilities spearheaded by CamptoCamp and Talend. It’s based on Talend Open Studio, Talend’s generic ETL solution, and extends it with geospatial components. http://www.spatialdataintegrator.com/

Some Talend Geospatial use case examples can be found at http://www.talendforge.org/wiki/doku.php?id=sdi:examples and http://www.talendforge.org/wiki/doku.php?id=sdi:geocomponentslist.

As of this writing current formats supported are ESRI shapefile, MapInfo, WKT, WFS, GPX, OSM, and PostGIS.

GeoKettle— This LGPL-released open source ETL loader is based on Pentaho Kettle ETL. It currently has built-in support for PostGIS, Oracle Spatial, MySQL, and ESRI shapefiles. http://sourceforge.net/projects/geokettle/

Proprietary tools that support PostGIS

Cadcorp SIS— This suite of products includes desktop GIS and web-mapping OGC-compliant WMS, WFS, great raster, and CAD support. http://www.cadcorp.com/

Safe FME (ETL)—This is the most recognized name in the industry for spatial ETL and automating spatial ETL workflows. http://www.safe.com/

ESRI ArcGIS 9.3— This requires an ArcSDE license to work with PostGIS. http://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?TopicName=The%20PostGIS%20geometry%20type

zigGIS for ArcGIS— This plug-in for ArcGIS is useful if you just need to do desktop work and don’t want to shell out money for an ArcSDE license. The code is technically commercial open source. http://pub.obtusesoft.com/

Manifold— Pretty nice all in one package, it also has some neat SQL functions for dealing with Raster. It has its own dialect of Spatial SQL very similar in style to Microsoft Access Jet (for example, it supports cross tabs using Transform PIVOT and TOP and has lots of spatial functions). It works with all the popular spatial databases without additional cost: PostGIS, Oracle, DB2, and SQL Server 2008. http://www.manifold.net

Pitney Bowes MapInfo 10— It’s a favorite among data analysts and casual GIS users because of the ease with which you can link to data sources, import data, and run basic SQL queries. http://www.pbinsight.com/products/location-intelligence/applications/mapping-analytical/mapinfo-professional/

MapDotNet— This web-mapping toolkit for ASP.NET similar is in style to UMN Map-Server and its mapfile format follows a similar scheme. It includes wizards to build maps. http://www.mapdotnet.com

Places to get free vector data

Following are some useful places to find data. In chapter 6 we grab data from some of these places to demonstrate how to load up on spatial data.

All geographic regions

OpenStreetMap— This community-driven spatial database and map repository has contributions from people all over the world. You can think of it as a free and open source Google map that has both web services and data you can download. It has base map information you can access via tile services as well as other crowd-sourced information such as biking trails and other GPS traces and waypoints in GPX format. You can use it as an overlay directly with your maps using something like OpenLayers. http://www.openstreetmap.org

In addition, you can load some of this data right into your PostGIS-enabled PostgreSQL database using the osm2pgsql command-line tool. http://wiki.openstreetmap.org/wiki/Osm2pgsql.

Natural Earth— This offers public domain map datasets that contain both raster and vector data. Most data can be used in any manner for private or commercial consumption to build upon. Data currently offered includes world administrative boundaries, city and town points with population, and various natural land and water geometries. http://www.naturalearthdata.com/

Centers for Disease Control administrative boundary files— The United States Centers for Disease Control maintains boundary files for all the continents and countries in ESRI shapefile format. These are circa 2000 and are all in the WGS 84 long lat spatial reference system (SRID = 4326). http://www.cdc.gov/epiinfo/shape.htm

Some of notable interest:

By Country— This contains the multipolygon boundaries for each country as well as information such as name of country, currency, population, and iso-code. We’ll be using this layer in some of our future exercises—Cntry00.zip.

By State— This is more granular than the country boundaries and breaks the various countries into states and provinces. It lists the administrative name, country, population, and type of boundary—Admin00.zip.

Some of the files that used to be located at http://biogeo.berkeley.edu are now redirected here: http://www.gadm.org/.

Global Administration Areas is a fairly new site that tries to maintain an up-to-date version of administration boundaries at various resolutions. Data is licensed for free use for noncommercial and educational purposes. Data is stored in ESRI shapefiles, ESRI geodatabase, and Google KMZ, and some is in RData format for R statistical packages. You can download files by country at http://www.gadm.org/country or download the whole set at http://www.gadm.org/world.

GeoCommons— GeoCommons is a directory of both free and non-free GIS data sources in ESRI shapefiles, KML, and GeoRss. http://finder.geocommons.com/

It contains spatial data containing statistical information on a wide variety of topics including health, demographics, and boundaries. Many of these are user contributed.

Infochimps— This is a search engine specifically for finding datasets. Many of the data sets are of a geospatial nature. http://infochimps.org

GIS Data Depot— This is a source of both free and premium downloads of both vector and raster data. All require free registration. Premium downloads require paid subscription. Much of the data provided is public domain. http://data.geocomm.com/

North America

U.S. Census Bureau data and TIGER data— By far the most popular, complete, and free source of data for the United States is the U.S. Census Bureau’s Topologically Integrated Geographic Encoding and Referencing (TIGER) system. The most recent distribution of this data is the 2010 version, which was released on a rolling basis starting November 3, 2010. This can be downloaded from http://www.census.gov/geo/www/tiger/.

The latest release of TIGER data is distributed in ESRI shapefile and dBase DBF format, so it can be loaded easily with the shp2pgsql tools provided with PostGIS. Versions prior to 2007 are released in a TIGER proprietary format, which can be loaded with OGR2OGR.

The TIGER data set includes the following items in EPSG:4269 (US NAD 83 long lat):

National Layers— U.S. state boundaries

Census block groups, blocks, and tracts broken out by state-county—A census tract is the smallest demarcation for population. The U.S. census tries to maintain the same population across all census tracts or tracts of a particular type. A lot of statistics such as employment and disease are calculated against census blocks and tracts. Census block polygons are recalculated every 10 years or so, and populations in them may change drastically.

2002 5-Digit and 3-Digit Zip Code Tabulation Areas— These are polygons that approximate Zip Code area routes for the entire United States as single files. Keep in mind that U.S. Postal Zone Improvement Plan (Zip) routes are really street segments, so the ZCTA is a simplification of these into polygons by aggregating census blocks that intersect these street segments.

Other state county files: Streets, roads, water lines, and water polygons, address ranges, points of interest, voting districts (aka wards and precincts), and congressional and senatorial districts.

New England cities and towns

Data.gov— This is a new site launched by the Obama administration as part of an open data government initiative. It contains both geographic data in ESRI shapefile and KML formats as well as various statistical data in CSV tabular format. All of these are fairly easy to import and analyze in PostgreSQL. You’ll find all sorts of interesting data such as spending, toxic waste zones, and air emissions. We encourage you to explore it. http://www.data.gov/

U.S. National Atlas— The National Atlas offers numerous geographic layers for the United States such as railroads, airports, and political boundaries. Most are in ESRI shapefile format. http://nationalatlas.gov/atlasftp.html

U.S. National Weather Service— The National Weather Service has a catalog listing of ESRI shapefile boundaries and weather-related data mostly of the United States, but some of it covers the globe. http://www.nws.noaa.gov/geodata/

NatureServe— If you are into the study of animals and ecological effects, Nature-Serve has an extensive assortment of ecological data for the United States and Canada and some other regions of North America, all in ESRI shapefile format. These are mostly point and polygon regions where these species are naturally found. http://www.natureserve.org/getData/animalData.jsp

GeoBase.ca— GeoBase is a portal that provides free spatial data for Canada. http://www.geobase.ca

It provides fairly up-to-date data for following types of features:

• Administrative boundaries
• Digital elevation data
• Hydrology
• Satellite imagery
• Roads for each province
• Statistics Canada, the National Statistical Agency—Statistics Canada has data for free as well as for cost download, including Canadian boundary and road network files from 2005 through 2007. http://www.statcan.gc.ca/mgeo/boundary-limite-eng.htm
• GeoGratis—Natural Resources Canada maintains mostly boundary files for Canada as well as raster data in ESRI shapefiles, raster tiffs, and tabular data. Data is free for download for both commercial and noncommercial. http://geogratis.cgdi.gc.ca/

Probably the one for most general use is the framework data files: http://geogratis.cgdi.gc.ca/geogratis/en/download/framework.html

Other countries and continents

UK Ordnance Survey— Ordnance Survey recently launched its open data site that offers both free (with very unrestrictive licenses) as well as for-purchase data. You can find both vector and raster data here. http://www.ordnancesurvey.co.uk/oswebsite/opendata/

Regional

These locations cover a small region such as a state or city but have a lot of spatial data for that region:

MassGIS— We couldn’t talk about data without talking about our favorite state and the state we live in. MassGIS has an extensive inventory of both vector data and high-resolution aerial imagery for most of Massachusetts, all free for download. All the vector data comes in ESRI format for easy loading into PostGIS and other spatial databases. In addition, it has various web services you can use to consume its spatial data if you just need to use it in your mapping applications. We demonstrate this in chapter 11.

Raw Data http://www.mass.gov/mgis/laylist.htm

Web Mapping Services http://lyceum.massgis.state.ma.us/wiki/doku.php?id=history:home

DataSF.org— If you live in San Francisco, California, or are just looking for data to play with, you’ll want to check out DataSF.org. It’s part of a pilot project called CivicDB, which hopes to provide a reference implementation for other government agencies. Details can be found here: http://it.toolbox.com/blogs/database-soup/datasforg-is-now-up-33563?rss=1.

At DataSF, you can find lots of spatial vector and aerial data for San Francisco, including data such as streets, shorelines, bridges, Zip Codes, city projects such as renewal and revitalization, city parcels, and zones. http://www.datasf.org

Sample data for training

OpenGeo Introduction to PostGIS— This includes both data and a workshop mostly licensed under Creative Commons. http://workshops.opengeo.org/postgis-intro/

North Carolina Educational Dataset— North Carolina has provided a free data set for training purposes that contains both vector and raster data. This can be downloaded from http://www.grassbook.org/ncexternal/index.html.

This is from our PGCon 2009 presentation and is more of an exercise on how not to build a town. The data is made up and free to use and improve on. http://www.bostongis.com/PrinterFriendly.aspx?content_name=pgcon2009_postgis_spatial

Spatial reference systems resources

Next, we list some resources on spatial reference systems that we’ve found useful:

Enchanted Learning— A good primer on map projections. http://www.enchantedlearning.com/geography/glossary/projections.shtml

Spatial Reference— This is an invaluable site for looking up spatial reference systems and adding them to PostGIS, especially when you have a somewhat obscure one. This site contains both EPSG defined (many of the US State Plane Feet that don’t come packaged with the default PostGIS spatial_ref_sys table) and many user-contributed ones from around the globe. The nice thing about this site is it will provide you an insert statement for PostGIS. You can submit an SRS text of an obscure projection, and it will calculate the PostGIS/Proj4text equivalent. You can also search for user-submitted ones. http://spatialreference.org

Summary by Morten Nielsen— http://www.sharpgis.net/post/2007/05/Spatial-references2c-coordinate-systems2c-projections2c-datums2c-ellipsoids-e28093-confusing.aspx

A good description of conical/cylindrical and oblique, equatorial, transverse—http://geology.isu.edu/geostac/Field_Exercise/topomaps/map_proj.htm

Gory details of the mathematical definition of an ellipsoid—http://en.wikipedia.org/wiki/Ellipsoid

Figure of the earth and various ellipsoids used over the years—http://en.wikipedia.org/wiki/Figure_of_the_Earth

SQL Server 2008 documentation by Isaac Kunen explaining spatial coordinate systems and the difference between flat and round earth models—It’s a surprisingly good description complete with pictures. http://msdn.microsoft.com/en-us/library/cc749633.aspx

PROJ.4 Wiki— PROJ.4 is the Cartographic Projections Library used by PostGIS. This website includes documentation on how to use the raw API and will be of value to those wanting to create their own custom spatial reference systems. http://trac.osgeo.org/proj/wiki

Projections Transform List— This article is a quick primer on common spatial reference systems and the PROJ.4 equivalents. Again, this is of use for those who want to look at an example of how to define a custom spatial reference system with PROJ.4 syntax. http://www.remotesensing.org/geotiff/proj_list/