Goanna 2.6 Released

Goanna 2.6 is now available from the download page. In this release we have focused on the usability of our Goanna Studio for Eclipse offering which at the same time increases stability and flexibility in the face of the many different configurations that are possible within the Eclipse CDT environment. Here is a summary of what has changed in this release:

- All versions

  • Bounds checking for arrays of arbitrary dimension
  • Bounds checking for arrays within classes, structs and unions
  • Arrays of unspecified size are no longer considered to have size 0
  • Constant global variables are now modelled with a value that does not change
  • The constant “-1U” and others will now be modelled with an appropriately large value instead of -1
  • Check FPT-misuse no longer warns about function pointers that are the result of the ternary operator (?:)
  • Check ITR-uninit now works correctly for iterators that are initialized using operator=
  • Check RED-unused-param no longer warns for parameters that have the GNU attribute (unused)
  • Checks RED-cond-const-assign and EXP-cond-assign no longer consider “+=” and similar operators to be constant assignments
  • Non-system #include files are now included in the analysis of a file that includes them.

- Goanna Studio for Visual Studio

  • preprocessor macros within parentheses are expanded
  • macros in comments are not expanded

- Goanna Central

  • Cygwin support for windows, use –compiler-sort=cygwin to create a cygwin configuration
  • Other compiler sort added, use –compiler-sort=other for an empty configuration
  • Remove the dependencies on the hard to manage predefined_macro.txt files
  • Predefined macros are now stored in the Goanna resource files, which are generated during configuration

- Goanna Studio for Eclipse

  • Completely re-organised configuration
    • Per project configuration
    • File based (under a goanna directory in the project file system)
    • User editable, or use the Goanna Project Properties dialogs. (two way synchronisation)
  • Menu item “Run Goanna on Selected File(s)” now appears when right-clicking on folders, and will analyse all files found in the selected folder.
  • Cygwin toolchain support on windows

Goanna Studio and Goanna Central 2.4

In the latest round of development we aimed for the most rock solid and sane implementation of C/C++ static analysis that we could. We did this by working on three fronts: consistency, speed and features.

As usual, all current users get the upgrade for free. If you were a trial user in the past and need a trial extension visit: http://redlizards.com/trial-extension.

Whats new in 2.4?

General improvements:

  • We revamped the integer arithmetic interval analysis engine with a new algorithm which affords us increased speed and precision. So much speed that we have also added the ability to track pointer arithmetic as mentioned in the previous blog post.
  • Our interval analysis engine can handle new operations: Casts, Logical operators and shift operators.
  • We have enabled floating licenses in our Studio and Central products; please contact us if you are interested.

Goanna Central improvements:

  • The argument –parse-error=0 is now a default, this will cause Goanna to exit with an exit code of 0 when a parse error is encountered. We have also revamped the way parse errors are handled.
  • The interprocedural analysis in the Goanna Central command line is now even simpler to use and manage by conveniently specifying a user-defined folder with the argument –ipa=<project name>.  All new database files are then stored in ~/.goanna_project_store/<project name>.

Goanna Studio improvements:

  • In Eclipse there is now an option to not run the tool chain compiler during analysis.
  • In Eclipse there is a new menu item to jump to the currently selected projects Goanna Studio Properties page.

Renamed checks for consistency:

  • ATH-div-0-aft-assign -> ATH-div-0-assign
  • ATH-div-0-aft-cmp -> ATH-div-0-cmp-aft
  • ATH-div-0-bef-cmp -> ATH-div-0-cmp-bef
  • ATH-div-0-param-unchk -> ATH-div-0-unchk-param
  • PTR-param-unchk -> PTR-unchk-param
  • PTR-param-unchk-some -> PTR-unchk-param-some
  • RED-const-assign-cond -> RED-cond-const-assign
  • RED-const-expr-cond -> RED-cond-const-expr
  • SPC-ret-stack -> MEM-stack
  • builtin_ctor_dtor_leak -> COP-ctor-dtor-leak

New checks:

  • ARR-inv-index-ptr – A pointer is assigned to an array, static or dynamic, and it is accessed with an index that is out of the array’s bounds.
  • ARR-inv-index-ptr-pos – A pointer is assigned to an array, static or dynamic, and it is accessed with an index that may be out of the array’s bounds.
  • ATH-overflow-cast – An expression is cast to a different type, resulting in an overflow or underflow of its value.
  • ATH-shift-neg – The left-hand side of a right shift operation may be a negative value.
  • COP-dtor-throw – An exception is thrown, or may be thrown, in a class’ destructor.
  • CPU-delete-throw – An exception is thrown, or may be thrown, in an overloaded delete or delete[] operator.
  • FPT-arith-address – Performing pointer arithmetic on the address of a function.
  • FPT-literal – Dereferencing a function pointer that refers to a literal address.
  • FPT-misuse – A function pointer is used in an invalid context.
  • ITR-end-cmp-aft – An iterator is used, then compared with end().
  • ITR-invalidated – An iterator is assigned to point into a container, but subsequent modifications to that container have possibly invalidated the iterator. The iterator is then used or dereferenced, which may be undefined behavior.
  • ITR-mismatch-alg – A pair of iterators passed to an STL algorithm function point to different containers.
  • ITR-store – A container’s begin() or end() iterator is stored and subsequently used.
  • MEM-malloc-diff-type – A call to malloc tries to allocate memory based on a sizeof operator, but the target type of the call is of a different type.
  • MEM-stack-ref – A stack object is returned from a function as a reference.
  • PTR-arith-field – Direct access to a field of a struct using an offset from the address of the struct.
  • PTR-arith-var – Invalid pointer arithmetic with an automatic variable that is neither an array nor a pointer.
  • RED-cond-var-always – The value of the variable used as a condition will always evaluate to non-zero or true. This means the condition will always be met.
  • RED-cond-var-never – The value of the variable used as a condition will always evaluate to zero or false. This means the condition will never be met.

Goanna Studio for Eclipse and Goanna Central 2.3.1 – Linux 64-bit patch

Some of our users may have experienced a bug when using Goanna Studio for Eclipse or Goanna Central 2.3 on linux 64-bit systems. This error looked something like this:

internal error: assertion failed: conv_host_fp_to_double:
error on conversion of DBL_MAX: Numerical result out of range:
((double)1.7...e+308L) (float_pt.c, line 524)

This bug was caused by us upgrading our release build infrastructure that defined DBL_MAX to a value that contained a cast, and our parser was not able to deal with this. We are releasing a patch that will correct this issue. Please download version 2.3.1 if you are using goanna on a 64-bit linux system. Windows and 32-bit linux systems are not affected.

Goanna Studio and Goanna Central 2.3

For the past several months we have received a lot of great feedback and support requests which have helped us dramatically improve the stability of our products Goanna Studio and Central. And yesterday we pushed a new version to our website which bear the fruits of our labor. At the same time we are releasing a bunch of new features that we hope will help people understand what Goanna is doing and make people more effective at using Goanna.

As usual, all current users get the upgrade for free. If you were a trial user in the past and need a trial extension visit: http://redlizards.com/trial-extension

What is new in 2.3?

New Checks:

  • ATH-div-0-param-unchk: Dividing by a parameter value without first checking that it is not zero
  • MEM-stack-global-field: storing the address of a field of a local struct in a global variable
  • ITR-uninit: using (dereferencing or incrementing) an iterator that hasn’t been initialized
  • ITR-end-cmp-bef: using an iterator after it has been compared with end(). This can occur when using an iterator after it is used in a loop.

New Goanna Studio features:

  • access check descriptions and examples through the IDE
  • added ability to manage warnings by suppressing/unsuppressing
  • added ability to pass specific options to the Goanna command line that is used during analysis.

Goanna Studio for Eclipse:

  • New ability to right-click on a file/project and run the Goanna analysis over just that item
  • Project specific settings have been moved to the Properties of the project (out of the Goanna Preferences)
  • Right-click menu now includes a link to the Goanna summary page for projects
  • If you close the Goanna Warning view it can be reopened from the Goanna menu
  • Improved stability of the Goanna warnings pane (with regards to close/open of Eclipse)

Goanna Studio for Visual Studio:

  • redesign of menu structure, Goanna is now a top-level menu
  • auto updating: no more uninstall-install updating.
    Note: you cannot do this from version 2.2 -> 2.3. It will work from now on.

General:

  • Improved handling of floating point numbers in the data tracking analysis
  • Warning suppression captured in the database structure
  • Several bug fixes including handling of very large ASTs.

Goanna Studio and Goanna Central 2.2

I am thrilled to announce that Goanna Studio 2.2 is now the most stable version of Goanna Studio ever created. Goanna Studio 2.1 was an impressive piece of software, but thanks to all the great feedback we have received, we have nailed down all the outstanding issues and we are ready to let you reap the benefits of our efforts.

The major part of our efforts has been targeted at making the Goanna Studio experience much better. This means more features and more stability. On the Visual Studio front, Goanna Studio is now capable of fully handling the range of possible inputs that you can throw at it. That means full Unicode support in everything, from how we process filenames, to project names and folders names, even down to the default file name when you export your warnings to CSV format from the Goanna Summary web page. This means, Goanna works reliably now with, e.g., Korean, Chinese and Japanese Windows. In addition we have introduced a feature we are calling Macro Visualisation. It is a way of presenting to you exactly what your source code looks like underneath all that macro mess.

On the Eclipse side we have some amazing new features. Goanna Studio can now handle various custom CDT Tool Chain tools. It’s as simple as telling Goanna which Tool’s it should be emulating, and then the Goanna analysis in your build environment is one step closer to perfection. The long-standing annoyance that appears asking you to select a CDT Project when you first try to analyse something with Goanna is now fixed. In addition, Goanna Studio for Eclipse now works in Windows-based environments with the MinGW tool chain.

Another major feature we are pleased to announce is Flexelint and PC-Lint integration into the Goanna Studio for Eclipse package. This allows you to run your version of lint and Goanna analysis at the same time, and have all the warnings presented to you in one simple interface.

The fixed false positives and changes to checks include:

  • ATH-div-0 – now warns for any expression that evaluates to 0
  • ATH-div-0-* – these checks now only warn when there is a definite division by 0.
  • ATH-neg-* – these checks no longer give warnings about floating point operations.
  • ARR-inv-index – now warns about invalid global array accesses.
  • COP-assign-op-ret – no longer warns about operator==.
  • COP-copy-ctor – two false positives were fixed, no longer warns when the copy constructor is intentionally unimplemented, and now handles C++ structs properly.
  • COP-member-uninit – now gives one warning for each uninitialised class member and now will also analyse assignment operators.
  • LIB-* – we have added a number of new checks warning about the unsafe use of library functions
  • RED-no-effect now has a more precise warning message.
  • RED-unused-val – no longer warns when you return a value, and now warns more correctly and almost lines up with the MISRA coding standard rule: 0-1-6.
  • SPC-ret-stack – no longer gives a warning when you return static local variables.
  • MEM-double-free – gives less false positives due to assuming that delete operators can throw exceptions

Other minor improvements and bug fixes are:

  • better handling of gcc arguments (particularly include path specifications).
  • Goanna Central now smoothly handles multiple files on the same command line.

As usual, all current users get the upgrade for free. If you were a trial user in the past and need a trial extension visit: http://redlizards.com/trial-extension

Goanna Studio 2.1

Goanna Studio 2.0 has been a great hit, we have gotten a lot of positive feedback and we have also acted on very nearly all suggestions and bug reports to produce a shiny new Goanna Studio update 2.1. This means that a great many false positives have been eliminated, greater accuracy has been achieved and we have also fixed some bugs and there is even a performance improvement or two.

The fixed false positives and changes to checks include:

  • ATH-sizeof-by-sizeof – a false positive involving array sizeofs has been fixed.
  • FPT-cmp-null – a false negative when warning about using a function pointer directly in a condition, not using a condition operator.
  • RED-unused-var-all now considers sizeof(x) to be a use of x.
  • MEM-stack-global, MEM-stack-param and MEM-stack-param-ref now take into account re-assignments for globals or parameters
  • SPC-uninit-struct now considers a struct with a field which is an array to be accessible without warning.
  • ATH-cmp-unsigned-pos and ATH-cmp-unsigned-neg take into account comparisons with (unsigned)-1
  • MEM-free-some doesn’t warn when you check the result of a malloc and exit (or return) if it is invalid
  • RED-dead no longer warns about goto’s and breaks
  • PTR-null-const-pos doesn’t warn about string literals
  • EXP-null-stmt no longer warns about a non-empty else block or when there is an assignment or function call in the condition
  • SEM-nonconst-call has been renamed to SEM-const-call
  • SEM-global-write has been renamed to SEM-pure-global
  • SEM-impure-call has been renamed to SEM-pure-call
  • ATH-shift-bounds now warns when you shift by 64
  • COP-assign-op-ret now warns about assignment operators that do not return a non-const reference to this

The Goanna Studio for Visualstudio also has some improvements and bug fixes:

  • Support for additional options and CL environment variable support.
  • Fixed bug involving a non Win32 or Win64 build target and the verbose flag
  • Support for environmnent variables with non-alphabetic characters
  • ProjectDir and SolutionDir macros now expand with a trailing backslash

We’ve also worked hard to squeeze even more performance where we can, and overall the Goanna analysis engine now scales better with the number of checks that are being used, and Interprocedural analysis is now slightly faster.

Other minor improvements and bug fixes are:

  • Better handling of types that have been typedef’ed
  • Better handling of implicit this parameter in member functions
  • Better handling of simple conditions containing only variables
  • Better identification of compound declarations
  • Better handling of throw and catch statements
  • Support for -std=gnu++0x and -std=c++0x command line arguments
  • Improved support for c++0x standard (still incomplete)
  • With Goanna Central Linux – ability to use specific rc files
  • Goannacc now correctly identifies versions of GCC that are built for different targets

As usual, all current users get the upgrade for free. If you were a trial user in the past and need a trial extension visit: http://redlizards.com/trial-extension

When is a for loop like a do .. while loop?

At Red Lizard Software, we care about providing the most accurate static analysis for your cpu cycle. Therefore, we spend a lot of our time thinking about the nature of false positives (when Goanna gives a warning about completely reasonable code) and how to avoid them.

One class of false positives we have noticed recently happens when you want to warn about an action that must occur on all execution paths. These properties might be expressed as “you must initialise all variables on all paths before accessing their values” for some definitions of initialise and access. A problem with these kinds of requirements appears when the initialisation of a variable is performed within a looping construct, and then access after the loop. This loop is usually designed to execute at least once (thus initialising the variable at least once) and so the programmer knows that the access after the loop is perfectly valid. Goanna has historically not been very good at identifying this false positive and will often warn anyway because there is an execution path that might not initialise the variable, the path where the condition evaluates to false. This is probably a case where the programmer should have used a do .. while loop to convey the desired semantics of the loop, but given that do .. while loops are not as popular as for loops, Goanna needs to be able to deal with this scenario.

There are two steps to making Goanna more intelligent about loops. The first step is identifying when a for or while loop should be represented as a do .. while, and the second is presenting this information to Goannas internal analysis engine.

In order to determine that a loop will execute at least once, it may be simpler to ask the inverse question. When will a loop not execute at least once? A sub question of this is when will we not know if a loop can execute at least once? This is actually much easier to answer because it can be boiled down to a structural condition. If the condition of the loop contains global variable references or function calls, then it is almost impossible to determine if a loop will execute only once. So what is left? Loops that contain only literals and local variable references. Parameters are a trickier issue since each call to the function is potentially different. With additional interprocedural analysis it may be possible to determine the boundaries of function parameters accurately but at present these loops can be ignored as well. The only thing left to do is to determine the state of the variables used in the loop condition right before it is evaluated and then evaluate the condition.

The analysis engine of Goanna works upon what is known as a control flow graph. This graph is created by looking at the source tree and determining which operations happen in which order. So the best way to present this modification of a for loop is through modifications to the control flow graph. Specifically we would like to create a copy of the control flow graph of the loops condition and wire up the rest of the graph such that there is a direct path through this path to the body of the for loop. The graph must also go into this new condition instead of into the old condition in order for the modification to be complete.

After implementing this change we have noticed that there is a small drop in the number of certain types of false positives, specifically in the SPC-uninit-var-some, with no impact on the runtime performance of the Goanna analysis engine. We hope to roll this improvement into the next release of the Goanna static analysis product line.