CSC/ECE 517 Fall 2010/ch4 4h am

Static Analysis Tools for Ruby

Static analysis tools are used to create an interface between the program, analysis, and the user. These tools perform many different functions through simple commands. The results of the tools are shown in various ways; through graphs, underlines in code, pop-up text boxes, and many other possibilities. A downside to the use of static analysis tools is knowing whether or not the tool knows exactly what the coder intended. For example, a programmer may not care that the current piece of code is used in another section because it may be more important for it to be in two places, yet not important enough to become its own method.

Introduction

Static Analysis is a simplified analysis of a system’s change that does not take into account long-term system changes. Static Analysis for code is static analysis of the code as it is typed at that moment, which is without regard for how the code interacts with other parts of the code dynamically. This is similar to testing and it occurs sequentially with it as it is in the Waterfall Method (shown as Implementation). The process of coding, testing, and analyzing is depicted in the diagram below.

Static Analysis Diagram

Because Ruby is a dynamic language, static code analysis seems like an oxymoron; however, static analysis for Ruby takes into account best programming practices like duplicating code, too much code in one place, and many other checks. These checks are programmed into the tools and as such are flawed by not understanding when the author meant to have so many lines of code. In this regard, static analysis tools must be used with caution. There are times when it is purposeful to have these errors and it may become cumbersome to see errors with every run of the tools. Therefore, it is important to understand these limitations and how to tailor the tools to the code that is being written.

A Review of the Tools

Reek

"Reek v1.2.8 is a tool that examines Ruby classes, modules and methods and reports any Code Smells it finds." ^[1] This tool is useful to any programmer that needs to know code that the code is complicated to the point of near impossible fixing.

Code Smells

"Smells are indicators of where your code might be hard to read, maintain or evolve, rather than things that are specifically wrong. Naturally this means that Reek is looking towards your code’s future." ^[1] The following items describe the smells that Reek can find:

• Attribute - attr, attr_reader, attr_writer, and attr_accessor raise a warning
• Class Variable - these variables are used globally and can break many places where the variable is used
• Control Couple - the most common type of control couple is a conditional statement that determines the path of execution
• Data Clump - this occurs when a group of items appear in the same fashion in classes, parameter lists, or when instance variables contain similar substrings
• Duplication - this occurs when code fragments are similar or perform similar tasks
• Irresponsible Module - these are classes and methods that do not have comments preceding them that describe the purpose
• Large Class - this is a class with a large number of variables, methods, or lines of code
• Long Method - a large number of lines of code
• Long Parameter List - a method with more than two or three parameters
• Low Cohesion
  • Feature Envy - the use of other class variables or methods more times than its own class items
  • Utility Function - a function that is never called within the class it is defined in, but is called within other classes
• Nested Iterators - a block of code that includes another block of code
• Simulated Polymorphism - case statements with different types in each case, comparisons using if statements of the same variable with different types, etc...
• Uncommunicative Name - names of variables, methods, or classes that don't make sense for what it is being used

Ruby Version Compliance

Reek v1.2.8 requires Ruby versions - 1.9.1, 1.8.7, or 1.8.6

Installing Reek

From the command line or in eclipse, type

 gem install reek 

Running Reek

From the command line or in eclipse, type

 reek [options] [dir_or_source_file]* 

An example using this class' Project 1, which is a simple forum tool:
Reek is run on the entire controllers folder by the code: reek app/controllers/*.rb

 app/controllers/admin_controller.rb -- 4 warnings:
  AdminController#grading contains iterators nested 2 deep (NestedIterators)
  AdminController#index contains iterators nested 2 deep (NestedIterators)
  AdminController#upgrade calls @upgrade.first_name twice (Duplication)
  AdminController#upgrade calls flash twice (Duplication)
app/controllers/application_controller.rb -- 0 warnings
app/controllers/cheers_controller.rb -- 10 warnings:
  CheersController#cheer_now calls Post.find_by_id(["?", params[:post_id]]) twice (Duplication)
  CheersController#cheer_now calls Post.find_by_id(["?", params[:post_id]]).parent_id twice (Duplication)
  CheersController#cheer_now calls flash twice (Duplication)
  CheersController#cheer_now calls format.html twice (Duplication)
  CheersController#cheer_now calls format.xml twice (Duplication)
  CheersController#cheer_now calls params 5 times (Duplication)
  CheersController#cheer_now calls params[:post_id] 4 times (Duplication)
  CheersController#cheer_now calls redirect_to(:controller => "posts", :action => "show", :id => ("#{page_id}")) twice (Duplication)
  CheersController#cheer_now contains iterators nested 2 deep (NestedIterators)
  CheersController#cheer_now has approx 15 statements (LongMethod)
app/controllers/posts_controller.rb -- 37 warnings:
... and more totaling 72!

Note the output is purely textual. It contains information about nested iterators, code duplication, long methods, and many of the other problems within this project's code! There are ways of suppressing certain warnings to find more information use the following code: reek --help

---

Saikuro

Saikuro version 0.3 is a cyclomatic complexity analyzer,^[2] which essentially counts the number of independent paths through the code.^[3] The higher the number that is returned the more complex the code. This means complex code "is more prone to error, harder to understand, harder to test, and harder to modify."^[3] The output of the program gives you the complexity number for the file that is tested. It will also tell you the number of tokens per line.

How and what Saikuro counts to calculate the cyclomatic complexity

"Saikuro uses the Simplified Complexity Calculation, which is just adding up the number of branch points in a method. Each method starts with a complexity of 1, because there is at least one path through the code. Then each conditional or looping operator (if, unless, while, until, for, elsif, when) adds one point to the complexity. Each "when" in a case statement adds one point. Also each "rescue" statement adds one. Saikuro also regards blocks as an addition to a method's complexity because in many cases a block does add a path that may be traversed. For example, invoking the "each" method of an array with a block would only traverse the give block if the array is not empty. Thus if you want to find the basis set to get 100% coverage of your code then a block should add one point to the method's complexity. It is not yet for sure however to what level the accuracy is decreased through this measurement, as normal Ruby code uses blocks quite heavily and new paths are not necessarily introduced by every block."^[2]

Ruby Version Compliance

Saikuro v0.3 does not list the required version; however it has been tested in 1.8.7 and 1.9.2 and it does not work. The v0.3 was released on June 21, 2008, which corresponds to Ruby version 1.8.6 (according to http://rubyforge.org/frs/?group_id=426)

Installing Saikuro

From the command line, type

 gem install Saikuro 

Running Saikuro

From the command line, type

saikuro -c -p dir/fileName.rb

The results are saved in the current directory.

For more information on running Saikuro, on the command line type

saikuro -h

Again, the author's first class project was used to show what Saikuro looks like after running. Saikuro was run using the following command saikuro -c -t -i app -y 0 -w 11 -e 16 -o out/ The output of Saikuro is saved to the "out" folder in the project in .html format. The following table is the output of Saikuro on the users controller only. There were 32 other files analyzed, each with its own file.

Output of Saikuro

---

Roodi

Roodi, short for Ruby Object Oriented Design Inferometer, version 2.1.0 sifts through the passed code and performs various checks. The checks that are described below are verbatim from the Readme.txt file.^[4]

Checks

• AssignmentInConditionalCheck - Check for an assignment inside a conditional. It‘s probably a mistaken equality comparison.
• CaseMissingElseCheck - Check that case statements have an else statement so that all cases are covered.
• ClassLineCountCheck - Check that the number of lines in a class is below the threshold.
• ClassNameCheck - Check that class names match convention.
• CyclomaticComplexityBlockCheck - Check that the cyclomatic complexity of all blocks is below the threshold.
• CyclomaticComplexityMethodCheck - Check that the cyclomatic complexity of all methods is below the threshold.
• EmptyRescueBodyCheck - Check that there are no empty rescue blocks.
• ForLoopCheck - Check that for loops aren‘t used (Use Enumerable.each instead)
• MethodLineCountCheck - Check that the number of lines in a method is below the threshold.
• MethodNameCheck - Check that method names match convention.
• ModuleLineCountCheck - Check that the number of lines in a module is below the threshold.
• ModuleNameCheck - Check that module names match convention.
• ParameterNumberCheck - Check that the number of parameters on a method is below the threshold.

Ruby Version Compliance

Roodi v2.1.0 is compliant with Ruby v1.9 (Author's Blog)

Installing Roodi

From the command line

 gem install roodi 

Running Roodi

 roodi "rails_app/**/*.rb" 

---

Rufus

Rufus allows to check Ruby for unwanted/unsafe. The Rufus allows checking some Ruby code before loading it. Rufus is in fact a set of ruby gems derived from route. ^[5]

• rufus-decision - provides CSV decision table mechanism in Ruby (Rufus::Decision::Table)
• rufus-dollor - A one-method library for substituting ${stuff} in text strings.
• rufus-treechecker - For checking untrusted code before an eval. The treechecker uses ruby_parser to turn ruby code into s-expressions, the treechecker then checks this sexp tree and raises a Rufus::SecurityError if an excluded pattern is spotted. The excluded patterns are defined at the initialization of the TreeChecker instance by listing rules
• rufus-lru - LruHash class, a Hash with max size, controlled by a LRU mechanism
• rufus-lua - Lua embedded in Ruby, via Ruby FFI. (about Lua, Lua is a powerful, fast, lightweight, embeddable scripting language. Lua combines simple procedural syntax with powerful data description constructs based on associative arrays and extensible semantics. Lua is dynamically typed, runs by interpreting bytecode for a register-based virtual machine, and has automatic memory management with incremental garbage collection, making it ideal for configuration, scripting, and rapid prototyping. according to http://www.lua.org/about.html)
• rufus-mnemo - This gem provides methods for turning integer into easier to remember words and vice-versa.
• rufus-rtm - A Remember the Milk gem
• ruby-scheduler - rufus-scheduler is a Ruby gem for scheduling pieces of code or jobs. it understands running a job AT a certain time,

IN a certain time, EVERY x time or simply via a CORN statement

• rufus-sixjo - A 'Rack application' for RESTfully serving stuff.
• rufus-verbs - It is an extended HTTP client library (gem). It provides the four main HTTP "verbs" as Ruby methods : get, put, post and delete.It wraps a certain number of techniques that make it a decent tool for manipulating web resources.

Ruby Version Compliance

• rufus-decsion 1.3.2 is compliant with Ruby v1.8
• rufus-dollor 1.0.2 is compliant with Ruby v1.8
• rufus-treechecker 1.0.3 is compliant with Ruby v1.8
• rufus-lru 1.0.3 is compliant with Ruby v1.8
• rufus-lua 1.1.0 is compliant with Ruby v1.8.7
• rufus-mnemo 1.0.0 is compliant with Ruby v1.8
• rufus-rtm 0.1.3 is compliant with Ruby v1.8
• rufus-scheduler 2.0.6 is compliant with Ruby v1.8
• rufus-sixjo 0.1.5 is compatible with Rack 0.9.0
• rufus-verbs 0.10 is compatible with Ruby v1.8

Installing Rufus Gems

 gem install rufus-decision

 gem install rufus-dollor

 gem install -y rufus-treechecker 

 gem install rufus-lru 

 gem install rufus-lua

 gem install rufus-mnemo 

 gem install rufus-rtm 

 gem install rufus-scheduler --source http://gemcutter.org 

 gem install -y rufus-sixjo 

 gem install gem install rufus-verbs 

---

Metric_fu

Metric_fu version 1.5.1 combines many already created static code analysis tools in one package. "It uses Saikuro, Flog, Flay, Rcov, Reek, Roodi, Churn, RailsBestPractices, Subversion, Git, and Rails built-in stats task to create a series of reports."^[6]

A benefit of having a comprehensive tool like Metric_fu is being able to run many different tools from one command.

Ruby Version Compliance

Metric_fu v1.5.1 requires Ruby v1.9.1 and v1.8.7

Installing Metric_fu on Windows

To install metric_fu on windows, you have to install development kit (DevKit). DevKit is a toolkit that makes it easy to build and use native C/C++ extensions such as RDiscount and RedCloth for Ruby on windows. First download the latest DevKit from http://github.com/oneclick/rubyinstaller/wiki/Development-Kit and run the self-extractable file and then go to the extracted folder and run msys.bat file (this forks command prompt). To install the metric_fu gem run the following commands.

 gem sources -a http://gems.github.com
 gem install metric_fu 

Unfortunately the Rcov metrics won't be produced even with this fix.

Installing Metric_fu on Unix

 sudo gem install metric_fu 

Running Metric_fu

The gem installs a rake task that must be used to run it.

 rake metrics:all 

Note: if you get errors about needed gems, just install each gem that it tells you to. The author had to uninstall and reinstall several gems before metric_fu would complete a rake and produce the graphs.

Using the project 1 example, metric_fu was used to run Reek and show the output as a graph using the javascript Bluff project.

Graphical Result of Running Reek within Metric_fu

The other part of the output is a tabular result of the tool that was run, again from the example it was Reek.

Tabular Result of Running Reek within Metric_fu

It is not apparent at the output of running metric_fu where the files were put to find the graphs, but they are all located in the project's files under: project/tmp/metric_fu/output. You can then open the index.html file and use the provided links or you can open the individual .html or .png files that are created for each tool that you ran.

Conclusions

There are many tools that can perform static analysis of Ruby code. However, many of them are quite young in comparison to other languages. These tools each attack the problem of static analysis in different ways. Also, they each offer different ways and depths of completing the analysis. These tools cannot be compared directly to each other due to the differences of analysis methods. Nevertheless, Metric_fu is the most powerful tool available because it incorporates many analysis and metric tools in one package and it is highly configurable to the needs of the user. The advantages of using static code analysis include a repetitive, non-emotional check of the code, the ability to configure specific checks, and providing a quick output of the analysis. Of the disadvantages, the most prominent is the false positives that occur from needed detours from proper coding practices. Static analysis tools are great, but must be used with a certain degree of caution so that common sense can also be used during the programming process.

References

1. ^{a, b} Rutherford, K. (2010, April 26). Reek Wiki. Retrieved October 15, 2010, from Github: http://github.com/kevinrutherford/reek/wiki

2. ^{a, b} Blut, Z. (n.d.). Saikuro:A Cyclomatic Complexity Analyzer. Retrieved October 15, 2010, from Rubyforge: http://saikuro.rubyforge.org

3. ^{a, b} Watson, A. H., & McCabe, T. J. (1996). Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric. National Institute of Standards and Technology, Computer Systems Laboratory. Gaithersburg: National Institute of Standards and Technology.

4. Andrews, M. (n.d.). Roodi. Retrieved October 15, 2010, from Rubyforge: http://roodi.rubyforge.org

5. Mettraux, J. (n.d.). Rufus: a bunch of ruby gems. Retrieved October 18, 2010, from Rubyforge: http://rufus.rubyforge.org/

6. Skruggs, J. (n.d.). Metric_fu. Retrieved October 15, 2010, from Rubyforge: http://metric-fu.rubyforge.org/

Additional Resources

• Schuster, W. (2008, November 07). Static Analysis Tools Roundup: Roodi, Rufus, Reek, Flay. Retrieved October 15, 2010, from InfoQ: http://www.infoq.com/news/2008/11/static-analysis-tool-roundup
• Sidorov, D. (2009, June 29). Static Analysis for Ruby/Python. Retrieved October 13, 2010, from Klockwork: http://www.klocwork.com/blog/2009/06/static-analysis-for-rubypython/
• Group of Ruby code metrics tools, http://www.ruby-toolbox.com/categories/code_metrics.html
• Wiki document on static code analysis, http://en.wikipedia.org/wiki/Static_code_analysis

Other Static Code Analysis Tools

• Flay - analyzing ruby code for structural similarities - http://ruby.sadi.st/Flay.html
• Flog - showing the most complex code written - http://ruby.sadi.st/Flog.html
• Nitpick - a customizable, programmable, static checker for Ruby - http://github.com/kevinclark/nitpick/wiki
• Simian - a code duplication finder (not free) - http://www.redhillconsulting.com.au/products/simian/