Goanna Static Analysis by Red Lizard Software

Goanna 2.6 is now available from the download page. In this release we have focused on the usability of our Goanna Studio for Eclipse offering which at the same time increases stability and flexibility in the face of the many different configurations that are possible within the Eclipse CDT environment. Here is a summary of what has changed in this release:

- All versions

• Bounds checking for arrays of arbitrary dimension
• Bounds checking for arrays within classes, structs and unions
• Arrays of unspecified size are no longer considered to have size 0
• Constant global variables are now modelled with a value that does not change
• The constant “-1U” and others will now be modelled with an appropriately large value instead of -1
• Check FPT-misuse no longer warns about function pointers that are the result of the ternary operator (?:)
• Check ITR-uninit now works correctly for iterators that are initialized using operator=
• Check RED-unused-param no longer warns for parameters that have the GNU attribute (unused)
• Checks RED-cond-const-assign and EXP-cond-assign no longer consider “+=” and similar operators to be constant assignments
• Non-system #include files are now included in the analysis of a file that includes them.

- Goanna Studio for Visual Studio

• preprocessor macros within parentheses are expanded
• macros in comments are not expanded

- Goanna Central

• Cygwin support for windows, use –compiler-sort=cygwin to create a cygwin configuration
• Other compiler sort added, use –compiler-sort=other for an empty configuration
• Remove the dependencies on the hard to manage predefined_macro.txt files
• Predefined macros are now stored in the Goanna resource files, which are generated during configuration

- Goanna Studio for Eclipse

• Completely re-organised configuration
  • Per project configuration
  • File based (under a goanna directory in the project file system)
  • User editable, or use the Goanna Project Properties dialogs. (two way synchronisation)
• Menu item “Run Goanna on Selected File(s)” now appears when right-clicking on folders, and will analyse all files found in the selected folder.
• Cygwin toolchain support on windows

Goanna 2.5 is now available from the download page. Here’s a summary of what’s changed in this release.

- All versions

• stored filenames are now specified as relative paths, allowing the Goanna database to be used in multiple locations
• better support for Unicode source code
• the ARR-inv-index check now allows for unbounded subscripts

- Goanna Central

• license server options can be specified on the command line
• the language choice, C or C++, can be given on the command line

- Visual Studio

• clicking on a warning in a project summary navigates to the relevant code
• better support for Win64 targets
• partial compatibility with Intel Parallel Studio (on VS2005 and VS2008, Goanna does not work with projects converted to use the Intel C++ compiler)

As usual, feedback is welcome.

You don’t have to download Goanna to try out Goanna.

Once you have an account on redlizards.com, you can try out Goanna via the online demo at http://redlizards.com/products/demo.html. Just log in, paste in your code in the text box and click the “Analyze” button. The results will show up on the Web page.

We’ve limited the amount of code you can analyze in the demo to 250 lines (so you still might want to download Goanna for serious work).

Static analysis of the past has been like an episode of CSI. Some guys in specially marked uniforms enter the scene and photograph memory leaks, establish attack vectors and scrape some dangling pointers off the ceiling. Then they go back to the lab and try to figure out what happened. It’s a post-mortem investigation of something that went horribly wrong.

This could be the tale of a real mission critical bug in the wild or just something that slipped through until final system testing. By any means it probably indicates that someone or something got burned. While many of our customers use our Goanna static analysis tools to establish final code quality or to do some forensic analysis, there are also many opportunities for prevention available that are often overlooked.

We carefully designed Goanna Central and Goanna Studio for two different purposes: Goanna Central can be quickly integrated into the build process and invoked either nightly or at the finish of a project. In any case it gives you an accurate snapshot of your code confidence. This is a must. But ideally, you want your code as clean as possible before it moves upstream in the SDLC. Just imagine how much could be saved by only committing “non-lethal” code to your project; If developers are empowered and can fix code right when they produce it. No more blame games.

We at Red Lizard Software really believe that the maximum value of static analysis tools comes from their usage by developers on an everyday and even every minute base. We try to make deep static analysis ridiculously easy. Checks that still required dedicated severs a few years ago will run on simple laptops with Goanna Studio today.

The advantages are clear: Developers can fix bugs before they are committed, development-testing periods are drastically reduced and products can hit the market sooner with higher quality. Maybe one day we will not need the CSI guys at all.

Goanna now supports Unicode filenames in a principled way. Until now, we allowed Latin-1 names in the Visual Studio versions of Goanna by using an ad hoc encoding. While that approach was effective for many languages, it left us unable to support languages with non-Latin-1 character sets.

Thanks to the authors of the Camomile library for the OCaml language, which allows us to work with Unicode directly.

For Eclipse and command-line users of Goanna, you get Unicode support if your terminal program is set to use UTF-8. That seems to be the default in recent Linux distros.

As part of our efforts to expand the scope of Goanna’s C++ checks, we decided to look into copy control, since this backbone of class architecture can also cause plenty of problems. The most common bugs relating to copy control are memory leaks, which are hard to identify and track down, as they will generally not cause the program to crash. Therefore, they are a priority for us to find.

In addition to finding and warning about the most common flaws in copy control functions, we decided to take the opportunity to cover some of the rarer problems, too. Our ultimate aim was to give some kind of useful warning in any case of potential misuse of a class, since copy control is something that a lot of people can have trouble grasping.

While many of our copy control checks warn for convention violations rather than definite bugs, they all combine to ensure that classes follow widely-accepted best practices, to improve the overall readability and robustness of code. As a case study, I’ll demonstrate the construction of a simple class, showing the bugs and suggestions that Goanna points out along the way.

We’ll start with a class with an int pointer. Of course, if we want the pointer to point somewhere, we need to allocate some memory for it, so we’ll do that in a constructor.

1   class MyClass {
2     public:
3       MyClass();  //constructor
4     private:
5       int* xp;
6   };
7
8   //default constructor
9   MyClass::MyClass(){
10     xp = new int[10];
11  }

We all know what the problem with this is:

cop.cc:9: warning: Goanna[COP-dtor] Missing destructor for class `MyClass' whose
function `MyClass::MyClass()' allocates memory

cop.cc:9: warning: Goanna[COP-member-uninit] Not all members initialized in this
constructor

If there is no explicit destructor, the compiler will only call the synthesized destructor, which is a problem for our allocated memory. The synthesized destructor will release the pointer xp, but not the memory allocated to it - this is our responsibility.

The second warning is there because even though we have allocated memory to ‘xp’, we have failed to initialize the values in the array, which should really be done in the constructor.

Because we’re lazy, let’s define an empty destructor to make the warnings go away (because that’s what half of programming is all about). Here’s the updated class:

1   class MyClass {
2     public:
3       MyClass();    //default constructor
4       ~MyClass(){}  //destructor
5     private:
6       int* xp;
7   };
8  
9   //default constructor
10  MyClass::MyClass(){
11    xp = new int[10];
12    for (int i=0; i!=10; ++i){
13      xp[i] = 0;
14    }
15  }

Alas, Goanna still isn’t happy:

cop.cc:4: warning: Goanna[COP-dealloc-dtor] Class field `xp' has memory allocated
in a constructor that is not freed in the destructor

Let’s release the memory in the destructor.

17  //destructor
18  MyClass::~MyClass(){
19    delete[] xp;
20  }

Now our class is free from bugs. So when we run Goanna, we should get the all clear. Right?

cop.cc:18: warning: Goanna[COP-assign-op] Missing assignment operator for class
`MyClass' which uses dynamic memory allocation

cop.cc:18: warning: Goanna[COP-copy-ctor] Missing copy constructor for class
`MyClass' which uses dynamic memory allocation

Wrong! It seems we’re just digging ourselves deeper. The destructor is not the only thing the compiler will synthesize. In fact, for any class, it will synthesize up to 3 functions if they are not explicitly defined:

1. Destructor - there will always be one synthesized to release stack memory
2. Copy constructor - a constructor taking in a reference of the class type
3. Assignment operator - operator=() defined for the class

Even though we have defined a default constructor (with no parameters), the compiler will still synthesize a copy constructor if we have not provided one. The copy constructor is used when instances of a class are copied, for example, passed as parameters or used in containers. Surprisingly, it is also invoked when an instance of the class is initialized at its declaration. For example:

MyClass a;       //default constructor
MyClass b(a);    //copy constructor
MyClass c = a;   //copy constructor
MyClass d; d=a;  //default constructor; assignment operator

It can sometimes be confusing which operators or functions are being invoked, and for such reasons it is important to ensure that all functions are explicitly defined when appropriate. Basically, if a user-defined destructor is required, then an assignment operator and copy constructor will also be required. This is sometimes called the ‘Rule of 3′, suggesting that if one of these three is required, all probably are.

So let’s add our copy constructor and assignment operator to the class, and to avoid any additional warnings, we’d better make sure we allocate the memory required for xp. And while we’re at it, we should get rid of the magic number used for the array size:

1   class MyClass {
2     public:
3       MyClass();                             //default constructor
4       MyClass(const MyClass& other);         //copy constructor
5       void operator=(const MyClass& other);  //assignment operator
6       ~MyClass();                            //destructor
7     private:
8       static const int ARR_INIT_SIZE = 10;
9       int* xp;
10  };
...
18  //copy constructor
19  MyClass::MyClass(const MyClass& other){
20    xp = new int[ARR_INIT_SIZE];
21    for (int i=0; i!=ARR_INIT_SIZE; ++i){
22      xp[i] = other.xp[i];
23    }
24  }
25
26  //assignment operator
27  void MyClass::operator=(const MyClass& other){
28    delete[] xp;  //reallocate the memory in case the size has changed
29    xp = new int[ARR_INIT_SIZE];
30    for (int i=0; i!=ARR_INIT_SIZE; ++i){
31      xp[i] = other.xp[i];
32    }
33  }

The class is starting to get bulky, but at least we can be sure we’re helping to make it safe for any creating, copying and destroying that we might be doing. However, Goanna isn’t quite happy with the class yet:

cop.cc:27: warning: Goanna[COP-assign-op-ret] Assignment operator `MyClass::operator='
does not return a non-const reference to `this'

cop.cc:29: warning: Goanna[COP-assign-op-self] Assignment operator `MyClass::operator='
does not check for self-assignment before allocating memory to a class member

The first warning is there because it is the convention that an assignment operator will return a reference to the target of the assignment. Such conventions exist so that all objects can be treated like primitive types, and to give the programmer more freedom to write intuitive code. For example:

MyClass a,b,c;
(a = b) = c;
(a = c).f();

The problem causing the second warning, is that self-assignment is generally perfectly legal code. For example:

MyClass c;
c = c;

However, calling a class’ assignment operator on itself will cause problems if dynamic memory allocation takes place. In our assignment operator, we free the memory allocated to ‘xp’, before allocating a fresh store. If the class instance called the assignment operator on itself, then the memory that being copied from in the 4th line of the operator will also be fresh. This means that self-assignment will basically lead to the object being populated with uninitialized data, which is almost certainly not the intention of the programmer. To handle this, we simply need to ensure memory manipulation only takes place if ‘this’ and the parameter refer to different instances of the class.

Our assignment operator must be altered to fix these two problems:

25  //assignment operator
26  MyClass& MyClass::operator=(const MyClass& other){
27    if (this != &other){  //check for self-assignment
28      delete[] xp;  //reallocate the memory in case the size has changed
29      xp = new int[ARR_INIT_SIZE];
30      for (int i=0; i!=ARR_INIT_SIZE; ++i){
31        xp[i] = other.xp[i];
32      }
33    }
34    return *this;  //return a reference to 'this'
35  }

After these changes, Goanna will not give any more warnings, and the class will be very robust. Hopefully this has provided you with some insight to some of our copy control checks, and given you a quick revision on some of the important things to keep in mind when creating classes. We’re still trying to expand our range of C++ checks further, to include checks on the proper use of iterators, containers and exception handling, and many other constructs.

Here is our completed class:

1   class MyClass {
2     public:
3       MyClass();                                 //constructor
4       MyClass(const MyClass& other);             //copy constructor
5       MyClass& operator=(const MyClass& other);  //assignment operator
6       ~MyClass();                                //destructor
7     private:
8       static const int ARR_INIT_SIZE = 10;
9       int* xp;
10  };
11 
12  //default constructor
13  MyClass::MyClass(){
14    xp = new int[ARR_INIT_SIZE];
15    for (int i=0; i!=ARR_INIT_SIZE; ++i){
16      xp[i] = 0;
17    }
18  }
19 
20  //copy constructor
21  MyClass::MyClass(const MyClass& other){
22    xp = new int[ARR_INIT_SIZE];
23    for (int i=0; i!=ARR_INIT_SIZE; ++i){
24      xp[i] = other.xp[i];
25    }
26  }
27 
28  //assignment operator
29  MyClass& MyClass::operator=(const MyClass& other){
30    if (this != &other){  //check for self-assignment
31      delete[] xp;  //reallocate the memory in case the size has changed
32      xp = new int[ARR_INIT_SIZE];
33      for (int i=0; i!=ARR_INIT_SIZE; ++i){
34        xp[i] = other.xp[i];
35      }
36    }
37    return *this;  //return a reference to 'this'
38  }
39 
40  //destructor
41  MyClass::~MyClass(){
42    delete[] xp;
43  }

At Red Lizard Software, we care about providing the most accurate static analysis for your cpu cycle. Therefore, we spend a lot of our time thinking about the nature of false positives (when Goanna gives a warning about completely reasonable code) and how to avoid them.

One class of false positives we have noticed recently happens when you want to warn about an action that must occur on all execution paths. These properties might be expressed as “you must initialise all variables on all paths before accessing their values” for some definitions of initialise and access. A problem with these kinds of requirements appears when the initialisation of a variable is performed within a looping construct, and then access after the loop. This loop is usually designed to execute at least once (thus initialising the variable at least once) and so the programmer knows that the access after the loop is perfectly valid. Goanna has historically not been very good at identifying this false positive and will often warn anyway because there is an execution path that might not initialise the variable, the path where the condition evaluates to false. This is probably a case where the programmer should have used a do .. while loop to convey the desired semantics of the loop, but given that do .. while loops are not as popular as for loops, Goanna needs to be able to deal with this scenario.

There are two steps to making Goanna more intelligent about loops. The first step is identifying when a for or while loop should be represented as a do .. while, and the second is presenting this information to Goannas internal analysis engine.

In order to determine that a loop will execute at least once, it may be simpler to ask the inverse question. When will a loop not execute at least once? A sub question of this is when will we not know if a loop can execute at least once? This is actually much easier to answer because it can be boiled down to a structural condition. If the condition of the loop contains global variable references or function calls, then it is almost impossible to determine if a loop will execute only once. So what is left? Loops that contain only literals and local variable references. Parameters are a trickier issue since each call to the function is potentially different. With additional interprocedural analysis it may be possible to determine the boundaries of function parameters accurately but at present these loops can be ignored as well. The only thing left to do is to determine the state of the variables used in the loop condition right before it is evaluated and then evaluate the condition.

The analysis engine of Goanna works upon what is known as a control flow graph. This graph is created by looking at the source tree and determining which operations happen in which order. So the best way to present this modification of a for loop is through modifications to the control flow graph. Specifically we would like to create a copy of the control flow graph of the loops condition and wire up the rest of the graph such that there is a direct path through this path to the body of the for loop. The graph must also go into this new condition instead of into the old condition in order for the modification to be complete.

After implementing this change we have noticed that there is a small drop in the number of certain types of false positives, specifically in the SPC-uninit-var-some, with no impact on the runtime performance of the Goanna analysis engine. We hope to roll this improvement into the next release of the Goanna static analysis product line.

It is out! We just released a major upgrade to Goanna Studio version 2.0. There has been a lot of work going into the new version and some of the new key features include:

• Full (whole program) interprocedural analysis to track effects across functions and files
• Incremental analysis to minimize time for reanalyzing files/projects
• Around 100 classes of checks, up almost 70% compared to the previous release
• Much improved precision and elimination of some existing false positives
• Improved Path Simulator to display error traces
• New project reporting mechanism and export facilities

For existing customers:

• We are also happy to announce that all existing customers have the possibility to upgrade to 2.0 free of charge!
• If you were a trial user in the past and need a trial extension visit: http://redlizards.com/trial-extension

Overall, the new version is another leap forward and enables to detect more and deeper critical issues early in the development cycle.

Goanna version 1.2 has been released. Download it now.

The major change is More Checks, in fact 40% more than were previously available in v1.1. Over the next few months we will continue to add new checks with each release. You can expect to see up to 100 additional high quality checks within the coming 6 months, which as usual will be free for all existing customers. Additionally, should you require a 30 day Trial Extension for your version 1.2 update please complete this trial extension request form.

We are also very pleased to announce the Beta release of Goanna for Command Line. This new command line version enables more flexibility and freedom for those wishing to integrate our powerful C/C++ static analyzer into their own development process. The Beta is currently available for Linux users and a version for Windows users is scheduled to be available in May. Linux users can now access a fully gcc-compatible solution integrated with over 60 classes of flow-sensitive quality checks to detect critical bugs as early as possible in the development cycle.

Inter-procedural analysis is also well under way, so stay tuned for a public Beta release soon!

We’re proud to have been selected for simultaneous shipment of our Goanna static analysis extension with Microsoft Visual Studio 2010. Here is a short introductory video demonstrating our Visual Studio 2010 integration, and we’re on schedule for April release:

We have some further news regarding recent developments (more high quality checks being one) and we’ll be posting more information next week.