Be strict or…

…you may have to eat the C++ programming language standard (may be with the C one together, brrr) half-baked or even raw.

GEOS 3.0.0 does not compile using GCC 4.3.3. The compiler complains, surprisingly, about missing functions like std::memcpy (GEOS core is written in C++, not C):

DoubleBits.cpp: In static member function 'static double geos::index::quadtree::DoubleBits::powerOf2(int)':
DoubleBits.cpp:51: error: 'memcpy' was not declared in this scope
DoubleBits.cpp: In constructor 'geos::index::quadtree::DoubleBits::DoubleBits(double)':
DoubleBits.cpp:94: error: 'memcpy' was not declared in this scope

StringTokenizer.cpp: In member function 'int geos::io::StringTokenizer::nextToken()':
StringTokenizer.cpp:75: error: 'strtod' was not declared in this scope
StringTokenizer.cpp: In member function 'int geos::io::StringTokenizer::peekNextToken()':
StringTokenizer.cpp:123: error: 'strtod' was not declared in this scope

These functions come from the C/C++ Standard Library, so how they could be missing? They are because the files reported above do not include standard header <cstring>. GEOS 3.0.0 had used to compiled using GCC without problems, what happened? GCC is just being aligned to the standards closer and closer, so it becomes strict and unforgiving.

There is a lot of C/C++ code of FOSS written in sloppy way being compiled in relaxed mode without use of strict compilation flags. Having in mind number of bugs reported after GCC 4.3 was released, I presume dark clouds are coming again with just released GCC 4.4.0 and upcoming 4.5.0. It’s always a good idea to not to trust chain or implicit inclusion of standard headers – wherever std::malloc is used <cstdlib> must be included, same about std::vector and <vector> and so on.

By the way, this particular bug in GEOS was spotted using GCC 4.3, reported, patched and fix was released as GEOS 3.0.3.

bjam output in colours

Recently, I’ve been heavily using Boost Build and bjam utility for building GGL library tests on Linux, so I developed a small tool that makes it easier to visually recognize compilation output by printing bjam messages in colours.

The script is called colorbb and was inspired by colormake written by Bjarni R. Einarsson. Actually, I grabbed Bjarni’s two scripts written in Perl and Shell and adopted it to parse bjam output.

colorbb-ggl-build

colorbb is a very simple utility and it likely needs to be improved to properly recognize all bjam messages. It’s available from my SVN repository at http://svn.devjavu.com/mloskot/scripts/colorbb/ personal Git repository at http://github.com/mloskot/.

WKT Raster crash course #1

Recently, a good friend of mine Sandro has started spreading the story of WKT Raster project. Here, I’d like to join him and post a bunch of technical notes about how to start using tandem of PostGIS and WKT Raster. This is the first post that hopefully will start a series discussing various aspects of use and development of WKT Raster extension.

What

From the overview of WKT Raster project:

WKT Raster is an ongoing project aiming at developing raster support in PostGIS. (…) WKT Raster’s goal is to implement the RASTER type as much as possible like the GEOMETRY type is implemented in PostGIS and to offer a single set of overlay SQL functions (like ST_Intersects) operating seamlessly on vector and raster coverages.

The idea of WKT Raster extension was presented (PDF 1.1 MB) in December 2008 by Pierre Racine from University Laval. Pierre’s presentation made foundation of the WKT Raster Specification.

Hacker #1: Why WKTRaster? sounds a bit silly :D
Hacker #2: Apparently the initial thoughts on it were expressed using a textual representation for the rasters and the thinking was that this would be core to the effort, though it has proven not so important. I also find the name unhelpful.

Actually, this is not as trivial problem as it may seem. I am not sure myself that the name is WKT Raster or WKTRaster, nor I’m confident if the name is fixed and will be valid in future.

Update 2009-03-08 14:05: See Pierre’s comment below.

Where

At the moment, WKT Raster project does not have a single home, but different parts of it live in different places:

Who

Currently, there is a small team of people who contribute their time, skills and money to move the project forward. Here is a summary of contributors everyone can collect from the scattered homepage(s) of WKT Raster:

  • Pierre Racine – originator and author of the WKT Raster idea and specification and implementation developer.
  • Sandro Santilli – well-known member of the PostGIS core team who is the architect of RASTER type definition, canonical and Well-Known-Binary format of stored raster, developer of input/output operations, and more.

There are also prime financial contributors who established development of the WKT Raster project: Steve Cumming, Martin Daly (from Cadcorp, where I work as member of core development team) and Tyler Erickson.

Recently, I’ve also joined the WKT Raster team and being led by Martin Daly I’m focused on programming new features and core testing.

News

Today and during next 3 days, a few people interested in the PostGIS / WKT Raster project are meeting together for the Toronto Code Sprint 2009. I’m also present there, though in spirit by joining the event on #tosprint IRC channel. I have a hope that during this meeting, we will discuss and agree about a couple of outstanding issues:

  • meta data tables: Do we need them? How to define schema of meta data?
  • review of specification of RT functions
  • improvements in the solution architecture and design to handle common raster use cases, for both categories, visualizations and analysis
  • how to answer needs of pyramids/overviews
  • does the project need an infrastructure, a new home?
  • and last but not least, how to achieve sustainability and rise more funds.

By the way, isn’t it good time to create WKT Raster article on Wikipedia? Writers are welcome!

libgeotiff lesson for today

Doing final tests of libLAS on Linux (Ubuntu 7.04), before Hobu will release 1.0.0, I attempted to build it with latest libgeotiff 1.2.5. I moved my fingers on keyboard and did ./configure && make, shortly then I saw strange looking error message:

/usr/bin/ld: makegeo: hidden symbol `__stack_chk_fail_local' in /usr/lib/libc_nonshared.a(stack_chk_fail_local.oS) is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status

Google is your friend, so in a few seconds I knew what’s going wrong. ./configure sets Makefile to call ld -shared. However, it does not work with GCC 4, because ld linker does not include all required references to other shared libraries. Simply, GCC linker driver seems to handle shared references better, than bare ld.

The conclusion is, that if you use GCC 4.x and you want to build libgeotiff, configure it this way:

./configure --with-ld-shared="gcc -shared"

Update July 31, 2008: Another solution is to add -fno-stack-protector option to CFLAGS and CXXFLAGS. Regarding Ubuntu Linux, this issue seems to be related to enabled SSP support (see also Ubuntu Wiki) available in GCC 4.1.

Yet another argument for -Wall always on

Yes, I’m religious about writing correct and standard C/C++ code and the decalogue of my religion tells do not ignore compiler warnings. Yet another pinch to the vast jar of examples is about getting detailed warning on mixed types in printf format specifiers:

#include <stdio .h>
int main (void)
{
printf ("%d", 0.0);
printf ("%lu", 0u);
printf ("%lu", sizeof 0);
return 0;
}

Building the test above with following three commands will answer why it’s a good idea to help the compiler to help us:

colormake on Mac OS X

I have just discovered colormake utility – a simple wrapper around make to colorize its output.

colormake is really helpful for someone who heavily works in Unix shell environments to visually analyse messages generated during programs compilation. Simply, colors are used as another language for communication with a user, complementing letters printed out to the console. The colormake wrapper in combination with clean console fonts like Terminus serve as a great tandem in hacker daily job.

First, I started to use colormake 0.2 on Ubuntu Linux:

$ sudo apt-get install colormake

Next, I tried it on Mac OS X 10.5, though with no luck. Basically, because of subtle differences in Bash and format of GCC output messages. So, simple patch (colormake-0.2-mloskot.patch) is required to get colormake 0.2 working on Mac OS X. I’ve sent this patch to Bjarni – the author of colormake, so perhaps he will like to include it in next (0.3?) version. BTW, thanks to Bjarni for the colormake tool!