Setting EnableNatvisDiagnostics in Visual Studio 11

Previously, I posted about the new XML-style debugger visualizers I discovered in Visual Studio 11. I mentioned that one of reasons the feature has been redesigned is to enable debugging of the visualizers templates defined in default or user-defined .natvis files. So, it looks there is something more to discover: how to actually make use of the diagnostics capability.

Unfortunately, it seems the new debugger visualizers have not been documented anywhere. At the time of writing this blog, Google reports one link about defaultvis.natvis, it is the brief post on Visual C++ forum.

Luckily, there is a short manual included as comment in header of the %ProgramFiles%\Microsoft Visual Studio 11.0\Common7\Packages\Debugger\Visualizers\defaultvis.natvis file. It says:

1) give better diagnostic reporting.
a) Set the name-value entry under the Debugger registry key: EnableNatvisDiagnostics which is a REG_DWORD to a value of 1 this will output each expression string that is evaluated under the native visualizer into the output window and is good for debugging type definition typos in .natvis files.

The first thing is to figure out where to set the EnableNatvisDiagnostics flag. The Visual Studio 11 installer does not put it into the registry. Also, there are several Debugger keys. However, a few experiments proved the flag should be set under HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\11.0\Debugger:

vs11-visualizers-enable-natvis-diagnostics

Once it is set, every time Visual Studio 11 debugger is launched and use of visualized types is determined, the .natvis files are parsed and evaluated templates validated. The process is reported in the Output window:

vs11-evaluating-visualizer-1

If anything goes wrong, a type definition template is incorrect and its evaluation fails, it is also reported in the Output window:

vs11-evaluating-visualizer-2-validation

It will be a very useful feature especially when defining visualizers of complex types. Previous versions of Visual Studio used old format of templates defined in autoexp.dat file which stated its own DSL based on regular expressions, so the syntax was complex and fragile, and as Andy Pennel has confessed, not documented. (Here kudos to Avery Lee who did great job reverse-engineering the autoexp.dat syntax.)

At least, this time folks get the XML Schema for the Visual Studio debugger visualizers.

Debugger visualizers in Visual Sudio 11 Developer Preview

Visual Studio 11 Developer Preview introduces new style debugger visualizers. The autoexp.dat definition of templates in previous versions of the Visual Studio debugger is being replaced with new style definitions based on XML and stored in defaultvis.natvis file.

Microsoft explains the motivation behind redesigning the native type visualization mechanism is to give better diagnostic reporting (visualizer expressions can be now evaluated and displayed in the output window to help debugging .natvis file), provide XSD schema defining type definition, support of multiple files for type definition: system-wide in %ProgramFiles%\Microsoft Visual Studio 11.0\Common7\Packages\Debugger\Visualizers\defaultvis.natvis and user-defined in %USERPROFILE%\My Documents\Visual Studio Dev11\Visualizers\xxx.natvis.

It sounds really good and has a great usability potential for programmers who uses C++ generative programming capabilities, defines complex C++ types, heavily uses C++ Standard Library and Boost C++ Libraries.

Here is example of new style visualizer defined in XML copied from the defaultvis.natvis:

  <!--std::unique_ptr from <memory>-->
  <Type Name="std::unique_ptr<*>">
    <DisplayString Condition="_Myptr == 0">empty</DisplayString>
    <DisplayString>unique_ptr {*_Myptr}</DisplayString>
    <Expand>
      <Item Name="[ptr]" Condition="_Myptr != 0">_Myptr</Item>
    </Expand>
  </Type>

For most types from C++ Standard Library and Windows SDK, the new visualizers work very well. However, the currently available Visual Studio 11 Developer Preview (Version 11.0.40825.2 PREREL) is shipped with incorrect definitions in the defaultvis.natvis. The C++TR1 definitions introduced in Visual Studio 2008 are only forwarded (aliased) from std:tr1:: to std:: in Visual Studio 2010. In Visual Studio 11, they have completely moved from std::tr1:: to std:: to align with C++11 standard. But, the new visualizers in defaultvis.natvis define templates are still referring to the C++11 types in std::tr1:: namespace.

This prevents the correct visualization for types like std::shared_ptr:

vs11-visualizers-broken-2

There is an easy workaround possible:

  1. Make private copy of defaultvis.natvis in %USERPROFILE%\My Documents\Visual Studio Dev11\Visualizers\xxx.natvis where xxx is your preferred name.
  2. Open xxx.natvis in a text editor.
  3. Find std::tr1:: and replace with std::.
  4. Save the changes and relaunch the Visual Studio 11.
  5. You should see objects of the C++11 types correctly visualized.

vs11-visualizers-fixed-3

I have reported this issue to Microsoft Connect in Incorrect Type Name references in defaultvis.natvis visualizers for standard library ticket.

File version is not supported by the launcher

UPDATE: This bug (see below) has been fixed a few days ago. The fix is available in CMake git repo and should be release in the upcoming CMake 2.8.7 release.

What is the difference between these two Visual Studio solution (.sln) files?

vs11-cmake-sln-version-compare

They both have been generated using CMake 2.8.6. However, having Visual Studio 11 Developer Preview installed, every time I try to launch the Hello2011.sln I’m getting mysterious error message: File version is not supported by the launcher.

Quick look at what CMake actually outputs in to .sln confirms there is a bug. CMake generates .sln file signature which does not match release version/name of the Visual Studio 11. In Hello2011.sln CMake generated:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 2011

but correct version of the output, fixed in Hello11.sln, is this:

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 11

Bug report submitted to CMake developers.

By the way, I have to admit different icons displayed for correct and incorrect .sln file are quite useful.

Building Boost with Visual Studio 11 Developer Preview

Yesterday, I installed Visual Studio 11 Developer Preview. The new release is available for free and is usable for free until June 2012, so it’s a great opportunity to try and learn what Microsoft has to offer for C++ programmers. And, it looks it offers a lot, including SCARY features.

The first test I usually do when it comes to test a new C++ toolset is to build Boost C++ Libraries using mainstream development sources.

The Boost simply builds well. I have even configured build of Boost.Python using Python version 3.2.

There are no hacks required to make the build happen. Simply, follow Boost Getting Started on Windows, section 5.3. Or, Build Binaries From Source. Once the Boost.Build tools bootstrapped, I issued the following command:

b2 --toolset=msvc variant=debug runtime-link=shared threading=multi define=_CRT_NONSTDC_NO_DEPRECATE define=_CRT_SECURE_NO_DEPRECATE define=_SCL_SECURE_NO_DEPRECATE stage

Fifteen minutes later, the Boost build was ready and staged.

By the way, some time ago Christian Henning posted a quick cheat sheet of bjam commands for Visual Studio. It’s very handy.

shared_ptr almost like intrusive_ptr

The current C++11 Standard consists of a very important remark in section 20.9.11.2.6. It is about creation of shared_ptr object. The remarks state:

Implementations are encouraged, but not required, to perform no more than one memory allocation.
[Note: this provides efficiency equivalent to an intrusive smart pointer. end note]

At work [1], I use Visual C++ 2010+ implementation of C++. I couldn’t resist myself to check if the standard managed to encourage Stephan T. Lavavej (STL) and his team at Microsoft to go for this optimisation.

I compiled and run a quick test:


#include <memory>
int main()
{
    std::shared_ptr<int> p0(
       new int(3)); // #1

    std::shared_ptr<int> p1
        = std::make_shared<int>(3); // #2
    return 0;
}

Quick check using breakpoints confirmed that Visual C++ 2010 indeed optimises construction of the shared_ptr. The difference between #1 and #2 is an important one: the second version causes 50% less memory allocations than the first one. Namely, one allocation. std::make_shared packs bookkeeping data and the user-defined data into a single block of memory.

Know your tools or suffer fragmented.

GCC At Home

I couldn’t leave a blog post alone without some kind of unrelated off-topic and pointless digression. So, here is one: Stephan T. Lavavej on his homepage provides custom-built distribution of MinGW toolset with What MinGW Is section starting as follows:

I recommend that anyone who is learning Standard C++ and who uses Windows for a primary development environment should use two compilers: the most modern version of Microsoft Visual C++ (currently 2010 SP1) and the most modern version of GCC, the GNU Compiler Collection.

Stephan works at Microsoft. I admire this kind of professionalism free from strategically competitive marketing b******t. GCC should definitely be included in the list of New Seven Wonders of the World. Recently, a new wonder has emerged: LLVM/Clang. I’m a user of all the three compilers (and related toolsets). It is easy for me to dream about Visual C++Visual Studio IDE based on LLVM/Clang as C++ compiler and shipped with C/C++ standard libraries provided by GCC. That would be a real C++ Renaissance

SqlGeometry and POINT EMPTY in WKB

Inspired by question Paul Ramsey asked today morning on IRC, I’ve inspected what kind of Well-Known-Binary output gives SqlGeometry for EMPTY geometries of all the seven geometry types as specified in OGC SFS. The SqlGeometry class is available from SQL Server System CLR Types for .NET Framework. Here we go.

I checked Well-Known-Binary output as returned by the SqlGeometry method STAsBinary(). Here is a small test program written in C#:

using System;
using System.Linq;
using Microsoft.SqlServer.Types;
namespace SqlGeometryEmpty
{
  class Test
  {
    static void Main(string[] args)
    {
      foreach (string type in
         Enum.GetNames(typeof(OpenGisGeometryType)))
      {
        string wkt = type.ToUpper() + " EMPTY";
        SqlGeometry geom = SqlGeometry.Parse(wkt);
        byte[] wkb = geom.STAsBinary().Buffer;
        string wkbhex = string.Join("",
          wkb.Select(
            b => b.ToString("X2")).ToArray());

        Console.WriteLine("{0}\n{1} ({2} bytes)\n",
          wkt, wkbhex, wkb.Length);
      }
    }
  }
}

The first observation is that WKB of EMPTY geometry for all types is returned as a a slightly different binary. All the binary forms are truncated to nine bytes. The first byte indicates endianness as expected. The second chunk of four bytes indicate geometry type. It is exactly as defined in OGC specifications. The third chunk of remaining four bytes are set to Zero and seem to play a role of size specifier: number of points in LINESTRING or number of rings in POLYGON, number of points in MULTIPOINT, and so on. This makes another observation that WKB for EMPTY is reported as a collection of primitive components.

The difference in binary of WKB of EMPTY geometry I mentioned is in that the actual type of input geometry is preserved, so there seems to be no implicit translation to geometry of some other type.

So far so good but not for too long. In fact, SqlGeometry implicitly casts POINT EMPTY to MULTIPOINT EMPTY geometry with the WKB of the following form (in hex):

010400000000000000

Here is complete output of the test program above:

POINT EMPTY
010400000000000000 (9 bytes)

LINESTRING EMPTY
010200000000000000 (9 bytes)

POLYGON EMPTY
010300000000000000 (9 bytes)

MULTIPOINT EMPTY
010400000000000000 (9 bytes)

MULTILINESTRING EMPTY
010500000000000000 (9 bytes)

MULTIPOLYGON EMPTY
010600000000000000 (9 bytes)

GEOMETRYCOLLECTION EMPTY
010700000000000000 (9 bytes)

A word about how PostGIS behaves. PostGIS reports GEOMETRYCOLLECTION EMPTY, regardless of actual type of input EMPTY geometry. It is in hex form:

010700000000000000

Generally, there is not many choices of how to report EMPTY geometry in clear and usable way and a form of collection with size equal to Zero seems to be the most appropriate choice. POINT EMPTY reported with type set to POINT (010100000000000000) would be ambiguous as feels like truncated or invalid form of POINT(0 0), especially in programming languages like C where native dynamic allocated arrays do not carry information about their size. IOW, geometry type is not enough information to process binary form of POINT EMPTY properly.

Reporting EMPTY geometries as a collection is a useful convention that seems to work well. PostGIS behaves about it in the very consistent manner reporting one type for all empties. SqlGeometry, so SQL Server, forces programmers to write a few more lines of code to handle all the possible cases. Yet another original exotic solution from Microsoft.

Consistent API is a bless!

Update: consistent specification of interface is even better.