Preparing for Subversion to Git migration

GEOS is slowly moving to Git, so I decided to clear the Subversion specific settings, namely the svn:keywords. The procedure is quick and based on the commands I used to perform bulk svn:keywords property update.

First, svn:keywords property is removed:

find . -path '*/.svn' -prune -o -type f -print  | xargs svn propdel -q svn:keywords

Next, line consisting of $Id$ keyword is stripped from every plain text file using a tiny script coded in Python:


#! /usr/bin/env python
import fileinput
import re
import sys

def strip_line(filename, rx):
    sys.stderr.write(filename + '\n')
    for line in fileinput.input(filename, inplace=1):
        m = re.match(rx, line)
        if m is None:
            sys.stdout.write(line)
        else:
            sys.stderr.write(line)

if len(sys.argv) < 2:
    sys.exit("Missing filename")

pattern = '^.*\$Id.*$'
rx = re.compile(pattern, re.DOTALL)
strip_line(sys.argv[1], rx)

The script is executed for every file, excluding the working copy admin area in .svn:

for f in `find . -path '*/.svn' -prune -o -type f -print`; do ~/bin/strip_line_regex.py $f; done;

Voila!

Update: Torvalds answers Does GIT has vc keywords like CVS/Subversion?

Quick CMake of GEOS

It’s an example of b***dy quick attempt to configure CMake-based build of a fairly large library written in C++ programming language. It is GEOS.

Seven. A lucky number. Arnulf‘s nickname. Total number of lines (commands) of CMake script to configure build of GEOS C++ core as a static library using sources form SVN trunk:

Create trunk/CMakeLists.txt file:

project(geos)
cmake_minimum_required(VERSION 2.6)
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
add_subdirectory(src)

Create trunk/src/CMakeLists.txt file:

include_directories(${CMAKE_SOURCE_DIR}/include)
file(GLOB_RECURSE geos_SRC ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
add_library(geos STATIC ${geos_SRC})

In fact, only six lines are really required. The third line in file trunk/CMakeLists.txt is optional.

mloskot@dog:~/dev/geos/_svn/trunk$ svn status
?       CMakeLists.txt
?       src/CMakeLists.txt
mloskot@dog:~/dev/geos/_svn/trunk$ mkdir ../build
mloskot@dog:~/dev/geos/_svn/trunk$ cd ../build/
mloskot@dog:~/dev/geos/_svn/build$ cmake ../trunk
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mloskot/dev/geos/_svn/build
mloskot@dog:~/dev/geos/_svn/build$
mloskot@dog:~/dev/geos/_svn/build$ make -j5
Scanning dependencies of target geos
[  0%] Building CXX object src/CMakeFiles/geos_lib.dir/operation/valid/RepeatedPointTester.cpp.o
...

and 53 seconds later my Intel Core 2 Duo 2.4GHz box announces:

[100%] Building CXX object src/CMakeFiles/geos.dir/linearref/LengthIndexOfPoint.cpp.o
Linking CXX static library libgeos.a
[100%] Built target geos

This build configuration is universal and cross platform, thus should work on all platforms supported by CMake.

Simply, you can not afford not use the best cross-platform build system that ever existed :-)

PostGIS 1.5.0 will require GEOS 3.1.0+

PostGIS spatial database extension for PostgreSQLPostGirls and PostBoys, be prepared!

The upcoming release of PostGIS 1.5.0 (available as beta1 beta2) will require GEOS in version 3.1.0 or later.

The beta1 testing results suggested to bump the minimum GEOS version to 3.1.0. The GEOS requirement issue was one of the heavily discussed topics on the postgis-devel this month.

The PostGIS team is going to give GEOS a nice present for its 10th round month birthday.

New GEOS source structure

Following quick announce a few days ago, I started shifting GEOS source code directories to flatten the structure a bit and to separate header files (.h) from implementation files (.cpp).

The whole process is documented as ticket #315 in the GEOS Trac. The transition has been remarkable smooth. The mission accomplished.

I’ve managed to build and successfully test GEOS with the following configurations:

  • GCC 4.4.1 on Ubuntu 9.10 64-bit with 64-bit build target
  • Visual C++ 8.0 on Windows XP Professional 32-bit with build target 32-bit
  • Visual C++ 9.0 on Windows Vista 64-bit (target 32-bit)
  • Visual C++ 10.0 on Windows 7 Professional 64-bit (target 32-bit)

All build configuration should work well. Let me know if any doesn’t.

Spatial Relations for Dummies

My friend Jo asked on GEOS mailing list for approachable explanation of spatial relations and Dale Lutz from Safe Software suggested something I’ve not came across myself before and what (in generalised form) I’d consider as a great idea for…a book really :-)

Spatial Relation for Dummies

Be strict or…

…you may have to eat the C++ programming language standard (may be with the C one together, brrr) half-baked or even raw.

GEOS 3.0.0 does not compile using GCC 4.3.3. The compiler complains, surprisingly, about missing functions like std::memcpy (GEOS core is written in C++, not C):

DoubleBits.cpp: In static member function 'static double geos::index::quadtree::DoubleBits::powerOf2(int)':
DoubleBits.cpp:51: error: 'memcpy' was not declared in this scope
DoubleBits.cpp: In constructor 'geos::index::quadtree::DoubleBits::DoubleBits(double)':
DoubleBits.cpp:94: error: 'memcpy' was not declared in this scope

StringTokenizer.cpp: In member function 'int geos::io::StringTokenizer::nextToken()':
StringTokenizer.cpp:75: error: 'strtod' was not declared in this scope
StringTokenizer.cpp: In member function 'int geos::io::StringTokenizer::peekNextToken()':
StringTokenizer.cpp:123: error: 'strtod' was not declared in this scope

These functions come from the C/C++ Standard Library, so how they could be missing? They are because the files reported above do not include standard header <cstring>. GEOS 3.0.0 had used to compiled using GCC without problems, what happened? GCC is just being aligned to the standards closer and closer, so it becomes strict and unforgiving.

There is a lot of C/C++ code of FOSS written in sloppy way being compiled in relaxed mode without use of strict compilation flags. Having in mind number of bugs reported after GCC 4.3 was released, I presume dark clouds are coming again with just released GCC 4.4.0 and upcoming 4.5.0. It’s always a good idea to not to trust chain or implicit inclusion of standard headers – wherever std::malloc is used <cstdlib> must be included, same about std::vector and <vector> and so on.

By the way, this particular bug in GEOS was spotted using GCC 4.3, reported, patched and fix was released as GEOS 3.0.3.

Detecting Visual C++ version in NMAKE makefile

Traditionally when building GDAL/OGR, GEOS or libLAS using NMAKE users specify version of Visual C++ compiler being used as value of MSVC_VER macro. This macros is not required but it’s recommended, so NMAKE can compose best possible set of compilation and linking flags for particular version of Visual C++. For instance, command specifying Visual C++ 8.0 (Visual Studio 2005) looks like this:

nmake -f makefile.vc MSVC_VER=1400

Recently, I hacked libLAS makefiles (and GEOS makefiles too), so Visual C++ version can be determined (semi-)automatically. NMAKE 1.62 or higher reports its version as value of _NMAKE_VER macro. The solution is to defined compiler version based on version of NMAKE tool:

Update 2009-04-03: Added _NMAKE_VER number 9.00.21022.08

!IF "$(_NMAKE_VER)" == ""
MSVC_VER = 4.0
!ERROR *** Prehistoric version of Visual C++
!ELSEIF "$(_NMAKE_VER)" == "162"
MSVC_VER = 5.0
!ERROR *** Prehistoric version of Visual C++
!ELSEIF "$(_NMAKE_VER)" == "6.00.8168.0"
MSVC_VER = 6.0
MSC_VER = 1200
!ERROR *** Prehistoric version of Visual C++
!ELSEIF "$(_NMAKE_VER)" == "7.00.9466"
MSVC_VER = 7.0
MSC_VER = 1300
!ELSEIF "$(_NMAKE_VER)" == "7.10.3077"
MSVC_VER = 7.1
MSC_VER = 1310
!ELSEIF "$(_NMAKE_VER)" == "8.00.50727.42"
MSVC_VER = 8.0
MSC_VER = 1400
!ELSEIF "$(_NMAKE_VER)" == "8.00.50727.762"
MSVC_VER = 8.0
MSC_VER = 1400
!ELSEIF "$(_NMAKE_VER)" == "9.00.21022.08"
MSVC_VER = 9.0
MSC_VER = 1500
!ELSEIF "$(_NMAKE_VER)" == "9.00.30729.01"
MSVC_VER = 9.0
MSC_VER = 1500
!ELSE
MSVC_VER = 0.0
MSC_VER = 0
!ENDIF

Later macros MSVC_VER and MSC_VER can be used to report Visual C++ version:

!IF "$(MSVC_VER)" == "0.0" && "$(MSC_VER)" == "0"
!MESSAGE *** Cannot determined Visual C++ version
!ERROR *** Aborting make job
!ELSE
!MESSAGE *** Using Microsoft NMAKE version $(_NMAKE_VER)
!MESSAGE *** Using Microsoft Visual C++ version $(MSVC_VER)
!MESSAGE *** Using Microsoft C/C++ version $(MSC_VER)
!ENDIF

This is not a rocket science, but it seems to work well and it frees users from manual specification of Visual C++ version. I’m not sure how the NMAKE version numbers vary across Visual Studio versions and builds. It would be good collect these version numbers to make the solution reliable. So, I’d be thankful if readers using Visual Studio 2002, 2003, 2005 and 2008 or newer :-) could report their NMAKE versions directly to me or post them as comments below.