Sprinting: Quality Assurance is Vital
One of the hallmark attributes of the Agile programming methodology is short development cycles with frequent, working deliveries. Quality Assurance is a special challenge in such a fast changing, short term production cycle; running code is mandatory. Semantic Designs' tools can greatly enhance the productivity of your automated testing. Each of our tools has a command line interface, as well as a GUI, to make integration with your test environment straightforward. Reports are available in XML format for custom processing and formatting.
Testing
Classic testing is the most obvious place to begin to address software quality assurance. One important metric is "How of the source code is actually exercised by my test system?" Unexercised code can mean that your test system is deficient, the source has "dead code" (code that cannot be reached by any path), or extra code is in the codebase that is not necessary (perhaps a feature was removed but not all of the associated code was deleted).
Test Coverage (Code Coverage)
One set of tests is based on user stories, a sort of blackbox testing. You run the tests and verify that the outputs match the inputs. Wouldn't it be nice to be able to look inside the blackbox code a bit and see just how much of the code was actually exercised by the test set? You can do this with SD's Test Coverage tool.
Our code coverage tool does not have a runtime environment. It provides branch coverage by instrumenting your source code. The tool reads your source code and adds a bit array, a line to set a bit in the array at every branch point, and code to dump the bit array at exit points. It is a very small addition to your source. You then compile and run the instrumented code in your test system. A display tool relates the bit array to your source code and graphically displays the coverage information. You can combine the data from several runs to see cumulative coverage.
Special Note to Embedded Systems Programmers: Because of our low impact approach to measuring code coverage, our tool works well in embedded system testing. It is easy to change the routine that writes the bit array on exit to open a channel and write the array back to a PC.
Performance Profiling
Functional correctness and completeness are vital. Most products also have performance goals. Even if the project does not have specific performance goals, efficiency is a measure of code correctness. It can also be quite beneficial to establish a performance baseline for an evolving project and then make sure that it doesn't unnecessarily slow down from one sprint to the next.
Similar to our code coverage tool, we have performance profiling tools. They come in two flavors: frequency count and timing. They work in a similar manner to our code coverage tool except that they collect execution counts or timings.
Memory Safety (for C)
If you are coding in C, we have a very useful tool for finding a wide variety of memory safety issues. Our tool performs dynamic code analysis and alerts you as soon as a problem is encountered, not hundreds or thousands of lines later when the problem finally manifests itself. Memory safety issues are generally very difficult to test even when looking for a specific problem. Our tool uncovers them at the earliest possible time as you execute your program.
Refactoring
A fundamental part of the Agile methodology is periodic refactoring: rewriting small sections of code to be functionally equivalent and better quality. There are three important tasks associated with refactoring: identifying what to refactor, rewriting the code, and testing the rewritten code. Testing the code (Is the rewritten code exercised by the test system? Has the performance degraded?) is covered in the section above, Testing.
One common way for programmers to make quick progress is to "clone" code: find something that does nearly what they want, copy it, paste it, and modify it to perform the desired task. Proper clone management is an important aspect to well written code.
Another good way to identify candidates for refactoring is by the use of metrics. Metrics provide an objective measure of the complexity of a piece of code. The complexity may be related to the piece of code attempting to perform too many functions or simply not well written.
Clone Detection
The identification of clones is not an easy task. They can be exact copies or, more likely, near copies that have three or four significant changes from the original. Or, a clone may be several hundred lines long. Our clone detection tool, CloneDR, can scan your entire codebase and find both exact and near copies of code fragments. It has parameters to allow you to control what is identified as a clone.
Dead Code Detection
Another candidate for refactoring is dead code - code that is not reachable by any valid execution path. Dead code can result from a previous refactoring that was not completed properly. In a large or complex system, it can be very difficult to conclusively determine dead code without the assistance of an automated analysis tool. As a result, it's easier to leave the code in place, contributing to code bloat, slower compilations, and additional unnecessary maintenance complexity.
Metrics
Overly complex code is another candidate for refactoring. An objective way to assess code complexity is through the use of metrics such as Halstead, Cyclomatic Complexity, or the Software Engineering Institute Maintainability Index. Our metrics tool can compute these measures across your codebase to readily allow you to assess your code and make appropriate refactoring prioritization decisions.
Coding Styles
Refactoring is frequently done during an Agile Sprint to improve the code. One form of code improvement is ensuring that all code follows the same style rules. An example would be that all integer variable names start with the same prefix. Another example is that all status values that are returned by a routine or method must be checked. Our Style Checker tools are able to automatically process an entire codebase and find style rule violations. In some cases the tool can automatically refactor the code.
We have a few Style Checking tools available "off-the-shelf". We find that most customers want their own custom style checker. We are quite willing to discuss your needs with you.
Smart Differencing Source Code
As a part of a refactoring effort you want to occasionally compare the base code with a revised copy. Most tools that display differences treat the code as just a large text file. Our Smart Differencing tool understands the programming language of your source code and only reports semantically meaningful differences.
For more information, please contact sales@semanticdesigns.com.
