Goanna 3.1 released
The Red Lizards team is delighted to announce the release of version 3.1 of the Goanna suite of static analysis tools across all product lines!
This release builds on the capabilities of Goanna 3.0, making huge leaps forward in terms of performance, support for C/C++ development environments, warning accuracy and detail, and Web Dashboard functionality. Goanna’s core C/C++ analysis engine is now more streamlined and accurate than ever before, providing more precise and targeted warnings for code requiring complex control and data flow analysis, and performing much better in many common situations such as very large functions and source files. In addition, for each warning there is now much more detailed trace information and a greatly improved Web interface including a full source code browser to help you pinpoint the root causes of bugs in your code.
- Configurable Parallel Analysis: Goanna 3.1 is now fully parallelized, and can take advantage of modern multi-core processors to deliver several times faster analysis than earlier versions.
- Enhanced Incremental Analysis: Goanna 3.1 includes a host of other performance optimizations, including greatly enhanced incremental analysis of a project with only small changes since the last run. Such analysis can be up to 20 times faster than re-analyzing the whole project from scratch.
- Build Specifications: Goanna Central 3.1 now lets you analyze whole projects from a build specification that captures all the analysis commands required to do a complete run of your project. Build specifications can be written and edited manually, or generated automatically with a suite of new and improved tools: goannamake for Makefile projects, goannascons for SCons projects, and goannaiarbuild for IAR Embedded Workbench projects.
- Auto-detection of C/C++ Dialect: Goanna 3.1 has vastly improved support for different C/C++ dialects including more C++11 constructs, GNU extensions, and Microsoft C++ language features than ever before. In most cases, you won’t even need to know precisely which dialect options to use, as the “goanna” suite of utilities can auto-detect all this information directly from your compiler.
- Interprocedural Traces: Traces for most warnings in Goanna 3.1 now show much more detailed information, including across interprocedural (function) boundaries where Goanna has detected that an issue spans multiple functions and files. On Goanna Studio traces are now displayed in a tree-like format for easier navigation.
- Dashboard Improvements: The Goanna Dashboard has a greatly enhanced Web interface, including a complete history of historical warnings and a full code browser with interactive trace highlighting and hyperlinks.
- New and Improved Platform Support: Goanna 3.1 adds support for more IDE’s and Toolchains including Atmel Studio 6.1, IAR 8051, IAR MSP430, ARM CC, Renesas H8, TI Code Composer 3.3, and the Wind River Diab Compiler.
Configurable Parallel Analysis
Goanna analysis is now fully parallelized. This means that Goanna 3.1 can now take advantage of unused cores on modern multi-core CPUs that would be otherwise idle. If your CPU has several cores available, then parallel analysis in Goanna 3.1 can yield 6 or more times speedup over earlier releases of Goanna.
By default, Goanna runs as many parallel processes as you have CPU cores. But you can change the number of parallel processes to any number you like by setting Parallel jobs to any number you like in Preferences for Goanna Studio, or by using the –jobs option in Goanna Central.
This gives you complete control over the amount of resources used by Goanna on your machine.
Enhanced Incremental Analysis
Goanna 3.1 not only parallelizes analysis into multiple threads, but has significantly optimized the performance of each individual thread. The optimizations are too numerous to go into detail here, but address common bottlenecks such as very large source files, functions with very many variables, and very complex expressions.
In particular, the common case where you have only made a few changes in your project since the last analysis run has been significantly optimized. Goanna 3.1 caches analysis information on all the source files in your project in the project database, and only re-analyzes the portions of your project that are actually affected by the change. This provides a performance boost for both individual developers and for build servers that are repeatedly analyzing the same project. Incremental analysis of a project can be up to 20 times faster than re-analyzing the whole project from scratch.
Enhanced incremental analysis is on by default and completely transparent, so you don’t need to do anything to enable it. Underneath, it works by hashing the content of your source files, and does not depend on file timestamps in any way. This means you should never need to force a “clean” analysis run as you sometimes do in build environments that only look at file timestamps.
If you want to experience how much performance improvement is due to Incremental Analysis in Goanna 3.1, try manually turning it off with the –force-analysis flag and experiencing the difference. You’ll never want to go back!
Goanna Central 3.1 now lets you analyze whole projects from a build specification that captures all the analysis commands required to do a complete run of your project.
Build specifications can be created in a variety of ways. They can be automatically generated for a project with the following new and improved tools:
- goannamake, which now also auto-detects the C/C++ compiler(s) used in your Makefiles, and transparently configures Goanna for them without any manual intervention.
- goannascons, which can now record build specifications for SCons-based projects.
- goannaiarbuild, a new utility for IAR Embedded Workbench projects.
- In addition, Goanna build specifications have a very simple, plain text format, so they are easy to edit manually or even write from scratch with an ordinary text editor.
Bringing it all together in Goanna 3.1 is a new, project-level utility called goanna. The goanna command is the one that now drives all the goannacc and goannac++ commands to analyze a whole project, and provides access to all the new features in this release. It provides all the configurable options and features including:
- Analyzing projects from a build specification,
- Configurable parallel analysis,
- Enhanced incremental analysis,
- Auto-detection of C/C++ dialect,
- Automatic publishing of results to the Goanna Dashboard and third-party integrations,
- Running goannacc and goannac++ with optimal analysis settings.
The goanna command is in fact common across all product lines, Goanna Central and Goanna Studio. So all the new features provided in Goanna Central are equally available in Goanna Studio, and you can expect goanna to provide even more powerful analysis capabilities in future releases.
Auto-detection of C/C++ Dialect
Goanna 3.1 has vastly improved support for different C/C++ dialects including more C++11 constructs, GNU extensions, and Microsoft C++ language features than ever before. The number of C/C++ toolchains supported has doubled, with the addition of ARM CC, more IAR targets, Renesas H8, TASKING C166, and Wind River DIAB among others. In addition the existing GNU and Microsoft dialect support has been improved to include the language features in the most up-to-date versions of these toolchains.
Goanna 3.1 also makes configuration and setup much easier than ever before with C/C++ dialect auto-detection. This means that whether you are using Goanna Studio or Goanna Central, and whether you have projects using multiple compilers, the same compiler with different target options, or even projects with any mixture of the above, Goanna 3.1 should auto-detect all your dialect and compiler-specific settings, and automatically configure itself for each target transparently.
This completely eliminates the manual “–configure” commands that were required in previous releases, and all the PARSE errors that resulted from incomplete information when configuring Goanna for each target compiler. The new auto-detection features in Goanna 3.1 greatly simplify the process of setting up Goanna in complex projects, for example, those that must cross compile for several target architectures.
In Goanna Central, the new C/C++ dialect auto-detection support is completely integrated with the “goanna” suite of utilities. So when using the tools to record build specifications and analyze whole projects, dialect auto-detection is enabled transparently. The new C/C++ dialect auto-detection support gives you complete control, so you can manually override the target dialect or tweak it in any way you like. It is also extensible, so that support for new dialects and compiler variants are now much easier to add in. Complete documentation on how Goanna 3.1’s dialect auto-detection works is provided in the User Guide. If you would like help in adding support for a new compiler variant, then please contact us.
Traces in Goanna 3.1 now contain interprocedural information when issues span function boundaries. This gives more detail when trying to track down exactly which execution path in your program causes the issue raised by Goanna and may help in isolating and fixing the issue.
The Trace dialog in Goanna Studio has been updated to show the new interprocedural information provided by Goanna. Traces are now displayed in a tree structure, with collapsible nodes on each function call boundary. This gives the ability to expand out each function and get the details of a trace, or collapse them all and just get an overview of how the trace works. As always, you can step through the trace side-by-side with your code, allowing you to replay the trace like a debugger.
The Goanna Dashboard also gives trace information through its new Source Code Browser. Just like in an IDE, you can step through trace of a warning via the Web interface, even when the trace spans many functions and source files.
In addition to interprocedural traces, we have gone to extensive effort to make the information presented in a trace clearer. When viewing a Goanna trace, each step will now give more details about how it was reached or what is happening at that line of code. This, in conjunction with interprocedural traces, allows you to gain much more insight into issues that Goanna finds.
The Goanna Dashboard has had numerous new features added, including:
Source Code Browser: The Goanna Dashboard now has a fully integrated source code browser across all product lines. The browser shows warning information side-by-side to your source code allowing you to visualize analysis results as easily as if they were in your IDEs. Trace information is also shown when available, so you can step through your code to see how the warning is found.
Historical Warning Data: The Goanna Dashboard now stores the full history of your warnings in past analysis runs, not just a summary. This allows you to track warnings through your development cycle to see when bugs appeared and disappeared.
Bug Statuses and Notes: It is now possible to classify all your warnings in Goanna Dashboard. You can classify your bugs as one of; Unclassified (default), Ignore, Analyse, Fix or Not A Problem. In addition to this, you can also write a note for each of your warnings. This status information will stay attached to your warning for all time.
New and Improved Platform Support
Goanna 3.1 now fully supports the Atmel Studio 6.1 IDE. All the new features of Goanna 3.1 including C/C++ dialect auto-detection are now available in all versions of Atmel Studio 6. Vendor toolchain plug-ins to Atmel Studio are also supported, provided the target dialect is on our supported dialects list.
In addition, IAR Embedded Workbench users can now integrate Goanna into the Workbench IDE much more easily using the new goannaiarbuild project-level utility. This command supports analysis of IAR Embedded Workbench project (.ewp) files natively, eliminating the need to create batch files or manual scripts. Goannaiarbuild also supports recording of build specifications and all the other features of the “goanna” suite of utilities, so you can also automate the analysis of IAR projects in a build server or command line environment. The IAR 8051 and MSP430 targets are now fully supported. Support for the other IAR targets is coming soon, so please contact us for more information.
Wind River Workbench support is greatly improved, including support for the DIAB toolchain and auto-detection of settings for projects using the Wind River Build System instead of the native Eclipse CDT builder.
In addition, toolchain support for the ARM CC, Renesas H8, TASKING C166, and TI Code Composer 3.3 has been added and/or improved. These C/C++ dialects should be auto-detected from Goanna Central and Goanna Studio Eclipse. If you would like to see support for a new platform not on this list, please contact us.
A whole host of other fixes and improvements have gone into Goanna 3.1. Here is a detailed summary:
- Check accuracy improvements: the following checks have had their accuracy improved:
- [MEM-lose-assign] now catches more memory leaks interprocedurally.
- [COP-dtor] no longer warns about certain kinds of class destructors leading to false warnings.
- [COP-dealloc-dtor] has an increased search depth. Freeing the member through other function calls within the same class can be detected.
- [ATH-shift-neg] no longer reporters false positives from complex macros.
- [RED-cond-const-expr] no longer reports false positives for dereference of a constant address declared as volatile.
- [PTR-array-indexing] no longer reporters false positive for class member dereferences in C++.
- [INT-octal] eliminates false positives on compiler synthesized enumeration constants.
- Resolved [PARSE] issues:
- Added support for __int128 type.
- Added support for c++11 final keyword.
- Added support for c++11 range based for loops.
- Added support for MSVC for each loops.
- Added support for GNU binary literals extension.
- Added support for the IAR extended C anonymous union.
- Fixed parsing of <type_traits> with use of __float128 in GNU C++ 4.6+
- Improved parsing of STL.
- Unusual values in the __aligned__() GCC extension are now supported.
- Unusual values in the #pragma pack directive are now supported.
- An illegal redeclaration of std::string::npos that’s accepted by the Microsoft compiler is now also accepted by Goanna.
- Dashboard improvements:
- Substantial improvement has been made to Goanna Dashboard server’s performance.
- Exporting now produces a CSV instead of a tab separated file.
- There is now the ability to log server activity to a file.
- Files analysed that contain no warnings now appear in the directory browser.
- UI has been made more compact.
- Bulk editing for changing bug status and deleting projects and snapshots is now available in the respective tables.
- Goanna Studio for Eclipse:
- A user friendly dialog is now available to send issue reports back to Red Lizards Software.
- Goanna Studio for Visual Studio:
- $(Configure) macro is now expanded correctly.
- Fixed parsing issues with size_t and va_list on x64 targets.
- Rare error dialogs that would occasionally pop up from Microsoft internal software components have been eliminated.
- Goanna Studio for Atmel Studio 6:
- More options are now auto-detected from a project’s toolchain settings.
- All Platforms:
- Template functions that have been automatically instantiated are now fully analyzed.
- A new diagnostic preference is available. This includes internal diagnostic information in issue reports to Red Lizards Software to debug difficult issues.
- Signals that occur due to external causes (such as an overly strict SELinux security policy) no longer cause goannacc to stop.
- Improved false positive elimination algorithm to improve accuracy in [COP-copy-ctor] and [COP-assign-op].
- Improved robustness of Goanna database transactions.
- Dashboard is now fully compatible with Internet Explorer 10.
- Goanna Dashboard’s database is now more resilient in the presence of system failures.
Update 3.1.1, 21 June 2013
This update fixes a few issues:
- Fixed a bug where timeouts could be extremely unresponsive, goanna will now timeout even under unusual circumstances.
- Fixed a bug in the [RED-local-hides-local] check where C++11 range based for loops appeared to conflict.
- goannamake now auto-detects all toolchain settings in recursive Make projects with different CC, CXX, etc. compiler variables. The old behavior (only auto-detect variables in the top-level Makefile, and assume all subdirectories inherit the same values) can be got with –no-recursive. (#837135)
- goannamake, goannascons, etc. now have a –record-incremental option which lets you append to an existing build spec instead of replacing it.
- goanna –recordwith no file argument now prints a usage statement, because it doesn’t make sense to record from DB to DB.
- goannamakeis no longer fooled by Makefiles that contain: CC = goannacc with extra space(s) after the = sign.
- goannamake auto-detects your C compiler correctly even if your Makefile contains definitions of the form: CC = @$(call ECHO,CC,$(MSG)); cc
- goannamake no longer takes a snapshot of the project if the only thing it did was to create or upgrade the project database, and nothing else. (#854939)
Update 3.1.2, 7 August 2013
This update fixes a few more issues, mostly related to parsing of C/C++ dialect features:
- Parse errors on various __builtin_HEXAGON_* functions have been fixed.
- Some parse errors on MIPS64 targets have been fixed.
- Parse errors due to incorrectly analysing C++ source files as C in certain situations have been fixed.
- Parse errors in the “armcc” dialect when cross compiling from 64-bit Linux have been fixed.
- Parse errors on various “armcc” intrinsic functions have been fixed.
- Parse errors in the “armcc” dialect on code that depends on the __GNUC__ and __GNUG__ predefined macros have been fixed.
- A parse error on reinterpret_cast<> in contexts where a constant expression is expected (which is valid C++03 but not C++11) is now fixed in Microsoft C/C++ (#5028, #863896).
- Parse errors when using command line -Define arguments that are surrounded by white space (e.g. “-D NAME =value”) have been fixed.
- Parse errors when using GNU Statement Expressions inside constructor initializers have been fixed.
- Parse errors in the “gnu” dialect on __atomic_… built-in functions have been fixed.
- Parse errors in Linux on Unicode UTF-8 and UTF-16 files in some situations have been fixed.
- Annoying warning messages if you analyze with `goannacc –with-cc=”env …”‘ have been suppressed.
- Parse errors in the “gnu” dialect when cross-compiling for targets with different integer and pointer widths in some situations have been fixed. (#5053)
- Parse errors in the “gnu” dialect due to failing to auto-detect old (2.x) versions of GCC have been fixed. (#1036055)
- Parse errors in the “qnx” dialect due to failing to auto-detect old versions of QCC have been fixed.
- Parse errors in the “microsoft” dialect due to mismatched “unsigned short” and “wchar_t” when “Treat wchar_t as built-in type” is set to “No” have been fixed. (#1037239)
- Parse errors due to a preprocessor symbol containing a # character have been fixed (#1068479).
- “goannamake”, “goannascons”, “goannaiarbuild”, and “goanna” now have a –no-goanna-path option, which causes all goannacc etc. subprocesses to be invoked with just the executable name, not the full path. This is useful to work around programs that can’t handle long path names, but does require you to place the Goanna bin directory in your PATH environment. (#4807, #1078769)
- Spurious warnings in the “iar-*” dialects when auto-detecting the IAR compiler path have been eliminated, and auto-detected include paths can now never pick up the wrong installation of IAR if there is more than one installed. (#1036709, #1067283)
- A “cosmic” dialect has been added for Cosmic Software’s C cross compiler tools. (#1153878)
- Problems when “goanna” invokes “goannacc” with a command line length close to the underlying OS limit (as low as 2047 characters) have been fixed.
- Problems when “goannacc” and “goannac++” are invoked in Windows with a command string containing leading or trailing space characters have been fixed. (#842175)
- An issue with # characters in build specifications in some situations has been fixed. (#1068479).
- A license issue in Windows when Central and Studio versions are installed on the same machine has been fixed.
- An issue loading pages in the Goanna Dashboard for projects containing no warnings in some situations has been fixed (#995789).
- An issue with some non-printable characters in Notes in the Goanna Dashboard has been fixed.
- The Note field in Source Code Browser can now be resized.
- In Eclipse, any manual modifications you make to the built in dialect files <workspace>.metadata.pluginsau.com.nicta.goannadialects*.goannadialect are no longer lost every time you restart Eclipse. Note that it is still possible to drop your own custom dialect files into <workspace>*.goannadialect and these will be available during auto-detection and from the drop-down Toolchain menu. (#1037239)
- goannacc now has separate –timeout and –timeout-per-phase options that allow you to set upper limits on analysis time for whole files and for individual phases of analysis within a file independently.
- An issue in the Linux install script when a suitable daemon directory (either /etc/init.d or /etc/rc.d) cannot be found is now fixed.
- A missing DLL issue in the license server has been fixed. (#1127381)
- All “goanna” commands now accept an –exclude=option which excludes source files and folders that match the specified pattern. Accepts * and ? as wildcards.
Update 3.1.3, 16 August 2013
This update fixes a few more issues and improves compatibility with a number of C/C++ development environments:
- An issue in Windows analyzing files across windows drive letter boundaries has been fixed. (#960715)
- Parse errors on Boost type_traits in the “microsoft” dialect have been fixed. (#1167720)
- Parse errors on some GNU __thread and __asm syntax have been fixed.
- Parse errors on “@ ADDRESS” syntax (as in the “iar”, “cosmic”, etc. dialects) in some situations have been fixed. (#1204304)
- A “c51” dialect has been added for use with the Keil Cx51 Optimizing C Compiler. (#1234653)
- A “metrowerks” dialect has been added for use with Freescale CodeWarrior (formerly Metrowerks) C/C++ development tools.
- The “ti” dialect is now compatible with TI Code Composer Studio build tools (at the command line level) as old as v2.2. The old “ti-3.3” dialect has now been superseded by “ti”.(#1200200)
- Some issues in dataflow analysis of typedef’d floating types and conversions between enums and signed integers have been fixed.
- An issue in Eclipse where linked source directories and files would be ignored has been fixed. (#960715)
- An issue in Eclipse where toolchain flags could get duplicated has been fixed. (#960715)
- An issue auto-detecting toolchain information from Wind River image projects has been fixed.
- A compatibility issue with “goannamake” and Microsoft NMake has been fixed. (#934298)
- An error “goreporter.exe does not exist, skipping snapshot generation” with –no-goanna-path has been fixed. (#1078769)
- A bug in recording build specifications containing command line arguments that end in “” has been fixed.
Update 3.1.4, 28 August 2013
- A parse error due to static assertions of sizeof(wchar_t)==2 in code intended to be compiled with g++ -fshort-wchar has been fixed.
- An issue auto-detecting include paths in the “cosmic” dialect has been fixed. (#1288642)
- The “goanna”, “goannamake”, etc. commands now honor the –quiet and –verbose options in more situations.
- The default Goanna database name (if no –db option is specified) now should always match the build specification filename.
- Added more options for using Sonar with Goanna: –sonar-db-user, –sonar-db-pass, –sonar-db-name allow users to use non-default Sonar database names, users and passwords, and –sonar-dir allows users to specify which directory of source files to send to Sonar for analysis and reporting. (#1274264)
- Extended the –exclude flag to pass exclusion patterns to Sonar so its analysis and reporting can also honor them. (#1274264)