The Goanna VS extensions for VS2005/2008/2010 are written in C#, because there’s a wizard that generates a simple extension in C#. From that starting point, we (meaning I) built the current extensions. If there hadn’t been the wizard, I would have written the extensions in F#, because I prefer the functional style of programming. So I decided to write the command-line tool, that was an opportunity to try out F# in earnest. I’d written a wee bit of F# before; here was a chance to try it out on production code.

When writing the command-line tool, my main concern was, how easy would it be to pull in the C# code that does the project-to-command-line translation. It was very easy: just add an F# project reference to the .DLL containing the code, open the namespace, and I was good to go. I had to make some C# classes explicitly public for visibility, but that was the only change I needed to make.

Programming in F# is very much like programming in OCaml, a language I’ve used off and on for maybe 15 years. Nice thing: instead of the clumsy “delegate” syntax of C#, you can just pass a function argument to another function. Not so nice: in VS2010, the editor does not seem to auto-format F# code, the way it does with C# code (the Ctrl-E F magic). And the editor’s Intellisense feature does not appear to suggest variable names that are in scope. Also: although there are surely good reasons for it, the F# list type is distinct from the C#/.Net System.Collections list type, which is slightly maddening. Finally: I have to build the tool for the various VS versions in slightly different ways, and the conditional compilation facility works for that — but why are there no boolean operations allowed, as you have in C#?

Here’s an example of code that uses .Net lists instead of F# lists:

  let expandedProjs = new System.Collections.Generic.List() in
  while projsIter.MoveNext() do
    expandedProjs.AddRange(ProjectUtil.expandProject(projsIter.Current :?> EnvDTE.Project))
  done;

Ooof.

When the command-line tool starts, it fires up an instance of Visual Studio, no GUI. That way it can get information about solutions and projects from VS, like default include paths and configuration information, using code originally written for the extensions. Sometimes the calls to VS fail with COM retry errors, so those calls are done in a loop containing a try-with block. When the tool finishes, or the user hits Ctrl-C, it gracefully shuts down VS. I often run the tool from a Cygwin shell, and I haven’t yet found a way to trap Cygwin SIGKILL signals, so that VS is still running afterwards.

There’s still some work to do on the command-line tool, like deciding what kind of output it should produce, but it’s basically there. Let me know if you’d like to try it out before we make it generally available. The tool is tentatively called “GoRun”, and its syntax is:

  GoRun sln-file [projName ...] ...

That is, you supply one or more solution files, and for each solution file, zero or more project names. If you don’t supply project names, GoRun invokes Goanna on all the projects in the solution, otherwise only those specified.

A complaint: Soon after VS2010 was released, the MSDN site was updated with all-new documentation for the .Net libraries. But the only language there’s documentation for is C# (OK, sometimes J#) . In the type signatures, there are no hyperlinks for keywords (like public, final, etc.) and types. You can’t tell when a type is really a forall-quantified type variable. It wouldn’t be much harder to do these pages right.

One last comment: programming in F# doesn’t feel all that different than programming in C#, though the code is more concise. You definitely feel the presence of .Net every step of the way, and there’s statefulness lurking everywhere. Functional programming for the masses … sort of!

A few weeks ago, we wrote some time- and heap-profiling CamlP4 macros, which add entries to a centralized table as our code runs. When the program finishes, it spits out statistics that we can examine for time and space sinks. The profiling hints — macros, that is — wrap the code to be profiled. So why not use the same approach for our tests?

I wrote a CamlP4 macro which is used like this:

LET_TEST some_fun : test1 = TestCase (fun () -> .... some_fun ...)


Such a test can be put in the code right next to the function some_fun. In fact, there are two versions of the macro. One version registers the test in a centralized table. The other version expands to (essentially) nothing, so the test is compiled away for release versions of our code.

Writing the macro was tricky — mostly because I don’t have a lot of experience with CamlP4. The subtlest part was converting identifiers (some_fun and test1 above) to strings which are passed to the registration function. I could’ve required the user to use strings directly in the code, but that’s ugly.

The macro calls the following function at expansion time:

let string_of_patt p errMsg =
match Ast.ident_of_patt p with
| Ast.IdLid (_loc,x) -> <:expr<$str:x$>>
| _ -> raise (Failure errMsg)


This function extracts an identifier from the pattern, extracts the identifier text, then builds a string, which is syntactically an expression. The <:expr< >> syntax is sugar for an Ast node representing an expression. The _loc in the pattern clause is required, because it binds an identifier in the desugared syntax (boo — variable capture!).

Writing this macro seemed to harder than writing, say, the equivalent Scheme macro. The CamlP4 syntax is kind of baroque, and some of its conventions aren’t intuitive. But it lets us do some very useful things.

The obvious one in OCaml is the lack of type-classes, specifically the lack of Haskell’s Show. There is no more useful debugging tool than printf, and it is a pain trying to debug complex types when you can’t see them.

This is where s-expressions come in. Specifically, Jane Street’s s-expression library.

It provides a camlp4 pre-processor that lets you take a datatype:

type nat
  = Zero
  | Succ of nat

and tell the pre-processor to generate s-expression support functions:

open Sexplib
open Sexp

type nat
  = Zero
  | Succ of nat
  with sexp

Now in your code you have access to two new functions, with the types:

sexp_of_nat : nat -> Sexp.t
nat_of_sexp : Sexp.t -> nat

We can throw these together with some useful sexplib library functions:

string_of_sexp : Sexp.t -> string
sexp_of_string : string -> Sexp.t

And we have a string representation of our data-type. Easy!

Pretty printing s-expressions

We can use the built-in human-format pretty printer, that does nice line indenting. Here’s a full example file, Tree.ml:

TYPE_CONV_PATH "Succ"
open Sexplib
open Sexp

type tree = Leaf of int | Branch of tree * tree with sexp

let a = Branch ((Leaf 1), (Leaf 2)) in
let b = Branch (a, a) in
let c = Branch (b, a) in
let d = Branch (c, Leaf 3) in
let sexp = sexp_of_tree d in
Format.fprintf Format.std_formatter "%a@n" Sexp.pp_hum sexp

We compile using the sexplib pre-processor and linking against the sexp library:

ocamlfind ocamlc -linkpkg -package sexplib
  -pp "camlp4o -I `ocamlfind query type-conv` -I `ocamlfind query sexplib`
    pa_type_conv.cmo pa_sexp_conv.cmo"
  Tree.ml

Then we run and the output is:

$ ./a.out
(Branch
 (Branch (Branch (Branch (Leaf 1) (Leaf 2)) (Branch (Leaf 1) (Leaf 2)))
  (Branch (Leaf 1) (Leaf 2)))
 (Leaf 3))

Reading in s-expressions

Now let’s read in the s-expression for our natural numbers, add one, and then pretty-print the result:

TYPE_CONV_PATH "Succ"
open Sexplib
open Sexp

type nat = Zero | Succ of nat with sexp

let () =
  let sexp = input_sexp stdin in
  let number = Succ (nat_of_sexp sexp) in
  let new_sexp = sexp_of_nat number in
  Format.fprintf Format.std_formatter "%a@n" Sexp.pp_hum new_sexp

Note the use of a new function, input_sexp : in_channel -> Sexp.t. This reads an s-expression from an input channel. We can compile and run this program as before:

$ echo '(Succ Zero)' | ./a.out
(Succ (Succ Zero))

So now we not only have an equivalent to Haskell’s Show, but also to Read. And all we had to do is add with sexp to our data type!

   'a option list

My first thought when processing this is to use the function:

    cat_options (l: 'a option list): 'a list

which behaves just like Data.Maybe.catMaybes in Haskell. But I often find myself writing things like:

   List.hd (cat_options someList)

…which would be efficient in Haskell, because catMaybes is lazy, and would only process the list until it found a value. But in OCaml, the entire list is processed by cat_options, then the first item is returned. This is wasteful, and could become a noticeable problem for long enough lists.

An efficient function would be:

let first_some (l : 'a option list) : 'a option =
  let rec aux xs =
    match xs with
    | []           -> None
    | None   :: xs -> aux xs
    | Some x :: xs -> Some x
  in
  aux l

Thanks for the kind wishes Pascal!

Here is one of those tests. It was producing a strange result.

void example(void) {
  int *x;
  x = malloc(sizeof(int));
  free(x);
  if (rand()) {
    x = malloc(sizeof(int));
  }

  *x++;
}

For this code, Goanna produced two warnings:

ex.c:12: warning: Variable `x' may have been freed
ex.c:12: warning: Pointer `x' reassigned, possible memory leak

The first was expected. Depending on the output of rand(), sometimes we de-reference after a free(), a segfault waiting to happen. But what is this second warning about memory being reassigned?

I assumed it was a bug in Goanna, and immediately Paul and I started generating debugging output and messing with the example code.

After a full ten minutes of searching, Paul and I suddenly realised what *x++ does: First it de-references the pointer, returning the value pointed to by x, then it post-increments the pointer x. That is, after the statement *x++, x is no longer pointing to the same piece of memory. Calling free(x) after *x++ would lead to disaster.

So the second warning is no bug in Goanna, it is a bug in our C example. Only 13 lines long and I managed to get confused, because when I saw *x++, I read it as (*x)++.