Archive for the ‘projects’ Category

Open Source, Decoupled and Accelerated

Wednesday, March 17th, 2010

All right, I confess, I’ve lied a bit in the topic. I’m not going to discuss any of the Free and Open Source Software philosophies here. What I’m going to do is to smuggle an interesting discussion that I believe it may be interesting to other Open Source Software hackers, especially projects from the C/C++ camp of the OSGeo Foundation. I would also say that subject of this discussion is quite idiomatic to the universe of FOSS production. It is about a software project. A project that has grown and it has grown in many dimensions, also in parallel dimensions.

“See the turtle of enormous girth!
On his shell he holds the earth.” — Stephen King

Infrastructure supporting a project becomes insufficient, maintenance is difficult, release process is a full-time job and situation has taken a lot of the fun out of participating. Population of users and developers has grown. As the Community gathers appreciable portfolio of masters of the software development craft, it is in constant state of snowball war exchanging fire of ideas, new projects and discussions. It’s truly a pleasure to learn about them but, well, it pours oil on the fire of entropy. Here we come to the crux.

Gain of entropy eventually is nothing more nor less than loss of information — Gilbert N. Lewis

Today, David Abrahams posted, somewhat provocative, e-mail to the Boost project mailing list. It is titled Boost, Decoupled and Accelerated and delivers the following message: It’s time to make Boost development fun again.

It may sound like yet another internal discussion within an Open Source project. There are zillions of similar debates archived around. Yes, indeed, but not exactly. In fact, David announced something that may be of wider interested. It is

a system called Ryppl to decentralize development, testing, release, and installation of interdependent projects

followed by yet more interesting comment

I believe this project has the potential to change the face not only of Boost, but of open-source software in general.

One may think, well, it seems related to the issue of the current trends or we’re suffering redundancy in IT prophets. Perhaps, perhaps, perhaps… but what I know for sure is that David Abrahams is one of my favourite and highly regarded software developer and author. I can hardly recall any of David’s comments, observations or suggestions that would be lacking of point, I mean a very rational point.

I have licked a bit of experience myself of working with or maintaining complex projects or projects that feel complex. I think I wouldn’t risk anything saying David has a point. I’m looking forward learning more about the whole idea.

I hope I’ll be able to confirm it myself while listening to David’s presentation at BoostCon’10.

Boost.Geometry blog

Sunday, March 14th, 2010

Boost Geometry (aka Generic Geometry Library, GGL)It’s been a month since Barend Gehrels launched blog dedicated to development of Boost.Geometry library which is also known of its former name as Generic Geometry Library or shortly GGL.

Here I go we a bit delayed announcement: http://barendgehrels.blogspot.com

Boost.Geometry and macros made by Apple

Wednesday, March 10th, 2010

Boost Geometry (aka Generic Geometry Library, GGL)I have no pleasure continuing my macros are evil tales but the life of C++ programmer eagerly wants to writes add another chapter to the story. Today, it’s time to rant on Apple and its XCode.

One of Boost Geometry (aka GGL) users, Mark, reported that he can not compile his program using the library with GNU C++ compiler from XCode. The compiler throws mysterious complain of a very low-level nature of C++ programming language:

Expected unqualified-id before 'do' in
/usr/local/include/boost_1_42_0/boost/geometry/geometries/concepts/check.hpp

Thanks to follow-up by Stjepan we quickly know who to blame for that. It is XCode header AssertMacros.h. It even might be one of public headers from development package of XNU, the Mac OS X kernel, what’s even more fun.

What actually happens that causes the problem?

Boost Geometry defines function template for concept checking:

template <typename Geometry>
inline void check()
{
    detail::checker<Geometry, boost::is_const<Geometry>::type::value> c;
    boost::ignore_unused_variable_warning(c);
}

Apple XCode defines macro using exactly the same name as the function check. The C++ preprocessor, which operates before compiler, substitutes the name check with content of the macro. For the Boost Geometry function check it means that a pile of garbage is injected in place were the function name is expected:

template <typename Geometry>
inline void do { if ( __builtin_expect(!(), 0) ) { DebugAssert('?*?*',
0, "Third Party Client" ": " "", 0, 0, "/usr/local/include/boost/
geometry/geometries/concepts/check.hpp", 181, (void*)0); } } while ( 0 )
{
     detail::checker::type::value> c;
     boost::ignore_unused_variable_warning(c);
}

Obviously, it makes compiler to give up to instantiate the check function from the template and to compile it properly.

C/C++ macros are evil, however, not by design but by insanity of programmers. Every macro defined in a public C/C++ header, should be defined using as unique as possible, but still usable name. I wish Apple folks designed their C/C++ macros as unique as they make their hardware products, even if made in China eventually. This particular macro that caused the problems discussed here, could be named to APPLE_XNU_CHECK and life would be easier. Or, given the fact that almost 3000 files using these identifiers live in Boost C++ Libraries only, I probably should say: life would be more productive, efficient and cheaper.

By the way, it’s a known problem @ Boost and it looks Boost Folks are trying to figure out best solution. See ticket #2115 – Avoid bad Apple macros.

…to be continued

AX_LIB_LIBKML macro

Thursday, March 4th, 2010

Recently, I was playing for a while with Brian’s new OGR LIBKML driver and I integrated it with GDAL/OGR build system, so it’s more convenient to build, test and use it. The complete tree is available in GDAL sandbox in mloskot/winkey-libkml. (It is just a give it a try-like prototype and I don’t actively maintain this branch myself. Hopefully, Brian will take it over.)

By the way, I crafted AX_LIB_LIBKML macro for Autoconf. This macros checks for headers and libraries of specified version (or newer) of Google libkml library and defines compilation and linking flags.

I submitted the macro to GNU Autoconf Archive. It is the new incarnation of well-known autoconf-archive.cryp.to. Peter Simons announced not long time ago that

The archive has moved to Savannah: http://www.nongnu.org/autoconf-archive/. Version 2009-04-26 was the last to be released at autoconf-archive.cryp.to.

Happy detecting libkml!

Const-correctness schizophrenia in GDAL

Thursday, March 4th, 2010

Const-correctness rants are quite common topic of chats on #gdal IRC channel. Some of the pearls I’ve got printed in to my mind:

A: The lesson is I ought to get things right the first time.
B: The issue with const method is that if you want to add lazy loading later, it can cause problems
C: GDAL is rather painful to use with const correct code, unfortunately :(
B: The solution is obvious: don’t write const correct code

Who’s right then, A or B?

I recall another motto from #gdal channel that sounds like “when unsure, do nothing” which has the following rationale:

especially when I realize afterwards that I’ve f**cked things because I neglected to follow the motto

Remembering these recommendations, it’s pretty clear why the const-mess in GDAL has happened. I’d conclude paraphrasing the motto this way:

I’ve f**cked things because I neglected to make a decision.

Now, poor GDAL beginner deadpickle, trying to find out (it’s me the evil) why compiler complains about his not-that-bad-written code, wandered to find and ask C gurus @ comp.lang.c and got the problem explained by Malcolm who wrote:

The problem is that, when C was first developed, there was no const keyword. So strings literal, which are constant, had to be non-const for backwards compatibility. This means that lots of programmers get lazy and omit the const, even from functions which don’t modify their string arguments. (There are also some subtle problems with const which means that this isn’t always a case of pure laziness). So a sort of solution is to discard the const qualifiers. However this is perpetuating the problem in your own code.

The motto stays in contradiction to a well-known best practice of const correct sooner than later. It’s way easier and cheaper to remove const-correctness once it turns out it does not express properly the actual design and contract than to apply it to existing codebase. Sometimes, the latter is even not possible making things f**cked up twice, in existing code base and in client’s code.

SqlGeometry and POINT EMPTY in WKB

Friday, February 26th, 2010

Inspired by question Paul Ramsey asked today morning on IRC, I’ve inspected what kind of Well-Known-Binary output gives SqlGeometry for EMPTY geometries of all the seven geometry types as specified in OGC SFS. The SqlGeometry class is available from SQL Server System CLR Types for .NET Framework. Here we go.

I checked Well-Known-Binary output as returned by the SqlGeometry method STAsBinary(). Here is a small test program written in C#:

using System;
using System.Linq;
using Microsoft.SqlServer.Types;
namespace SqlGeometryEmpty
{
  class Test
  {
    static void Main(string[] args)
    {
      foreach (string type in
         Enum.GetNames(typeof(OpenGisGeometryType)))
      {
        string wkt = type.ToUpper() + " EMPTY";
        SqlGeometry geom = SqlGeometry.Parse(wkt);
        byte[] wkb = geom.STAsBinary().Buffer;
        string wkbhex = string.Join("",
          wkb.Select(
            b => b.ToString("X2")).ToArray());

        Console.WriteLine("{0}\n{1} ({2} bytes)\n",
          wkt, wkbhex, wkb.Length);
      }
    }
  }
}

The first observation is that WKB of EMPTY geometry for all types is returned as a a slightly different binary. All the binary forms are truncated to nine bytes. The first byte indicates endianness as expected. The second chunk of four bytes indicate geometry type. It is exactly as defined in OGC specifications. The third chunk of remaining four bytes are set to Zero and seem to play a role of size specifier: number of points in LINESTRING or number of rings in POLYGON, number of points in MULTIPOINT, and so on. This makes another observation that WKB for EMPTY is reported as a collection of primitive components.

The difference in binary of WKB of EMPTY geometry I mentioned is in that the actual type of input geometry is preserved, so there seems to be no implicit translation to geometry of some other type.

So far so good but not for too long. In fact, SqlGeometry implicitly casts POINT EMPTY to MULTIPOINT EMPTY geometry with the WKB of the following form (in hex):

010400000000000000

Here is complete output of the test program above:

POINT EMPTY
010400000000000000 (9 bytes)

LINESTRING EMPTY
010200000000000000 (9 bytes)

POLYGON EMPTY
010300000000000000 (9 bytes)

MULTIPOINT EMPTY
010400000000000000 (9 bytes)

MULTILINESTRING EMPTY
010500000000000000 (9 bytes)

MULTIPOLYGON EMPTY
010600000000000000 (9 bytes)

GEOMETRYCOLLECTION EMPTY
010700000000000000 (9 bytes)

A word about how PostGIS behaves. PostGIS reports GEOMETRYCOLLECTION EMPTY, regardless of actual type of input EMPTY geometry. It is in hex form:

010700000000000000

Generally, there is not many choices of how to report EMPTY geometry in clear and usable way and a form of collection with size equal to Zero seems to be the most appropriate choice. POINT EMPTY reported with type set to POINT (010100000000000000) would be ambiguous as feels like truncated or invalid form of POINT(0 0), especially in programming languages like C where native dynamic allocated arrays do not carry information about their size. IOW, geometry type is not enough information to process binary form of POINT EMPTY properly.

Reporting EMPTY geometries as a collection is a useful convention that seems to work well. PostGIS behaves about it in the very consistent manner reporting one type for all empties. SqlGeometry, so SQL Server, forces programmers to write a few more lines of code to handle all the possible cases. Yet another original exotic solution from Microsoft.

Consistent API is a bless!

Update: consistent specification of interface is even better.

Boost Geometry list eNabbled

Saturday, February 13th, 2010

For those who are interested in the subject and prefer Web-based discussion forums, I added the Boost Geometry (aka GGL) mailing list to the Nabble as Boost Geometry forum.

Update: Thanks to Hugo from Nabble for importing all archives of the ggl mailing list.