Quality of the World Shapefiles

file TM_WORLD_BORDERS_SIMPL-0.3.shpBarend Gehrels from the Boost.Geometry project has posted an interesting analysis comparing available versions of the same vector data. It is Shapefile with borders of the World countries. The data is the same but different and sometimes the differences are very surprising.

Here is part 1 and part 2 of the excellent comparison made by Barend.

Linux.com about GeoServer

Linux.com published an interesting article – a tutorial – about famous components of Free and Open Source Software stack for Web Mapping. Justin Palk, the author, gives a very accessible introduction to building Web Mapping solution using GeoServer, PostGIS and OpenLayers. I think it’s a great writing for anyone who takes first steps in Web Mapping with FOSS4G.

Serving and styling maps with GeoServer by Justin Palk on September 24, 2008

FWTools 2.2.4 for Windows

Frank has just released and announced new version of FWTools 2.2.4. It brings new versions of:

  • GDAL 1.6.0dev (2008-08-05, current SVN trunk)
  • PROJ.4 4.6.1 (2008-07-21)
  • MapServer 5.3-dev (2008-08-05, current SVN trunk)

Also, one bug related to HDF5 format has been fixed.

The new installer FWTools224.exe is available from the primary site or mirror site

SHPTree depth vs index file size

This is a simple test I run to check how tree based spatial index for Shapefile influences size of index file (.qix). I used OGR and spatial index implementation available in Shapelib, in file shptree.c.

Tested Shapefile has following characteristics:

$ ogrinfo -so test.shp test
Layer name: test
Geometry: 3D Point
Feature Count: 932870
Extent: (440001.000000, 5652001.000000) - (441999.000000, 5653999.000000)
Layer SRS WKT:
(unknown)

Also, file size of test.dbf is 15M, test.shp is 33M and test.shx is 7.2M.

Next, I run a sequence of commands generating spatial index tree, using higher depth on each turn, from 2 to 16. This is the first iteration for depth of 2 levels:

$ ogrinfo -sql "CREATE SPATIAL INDEX ON test DEPTH 2" test.shp
SHAPE: Creating index file test.qix

I collected size of index files (.qix) and collated with corresponding depth value. Here is the result:
Shapefile index tree depth vs index file size

Using DEPTH of 16, produced index file is of size 213 MB, six time bigger than test.shp (33MB) file with geometries. Using that huge index files for shapefile will likely make the index exploitation inefficient due to memory use and heavy hard disk I/O operations. I’d use index depth of 8 or 10 for this shapefile.

In fact, due to some problems with memory allocation while generating huge indexes, default depth value estimated by SHPTree implementation has been limited to 12 levels. Certainly, users still have the liberty to explicitly request depth higher than 12 levels, for example:

$ ogrinfo -sql "CREATE SPATIAL INDEX ON test DEPTH 100" test.shp

Certainly, it’s always a good idea to make a few simple tests and observe performance of querying shapefile features using indexes of different depth.