Code to test ratio in bzr.

It's always interesting to see how the code changed as the revisions in repo increased. Actually, it's interesting enough to spend a while and generate some stats. And that's what I did :)

I wrote a simple ruby script that gathers stats from my app

  def file_stats(files)
    stats = {"lines" => 0, "comments" => 0, "loc" => 0, "classes" => 0, "methods" => 0}
    files.each do |filename|
      File.open(filename, "r") do |file|
        while line = file.gets
          stats["lines"] += 1
          next if line =~ /^\s*$/
            if line =~ /^\s*#/
              stats["comments"] += 1
            elsif line !~ /^\s*$/
              stats["loc"] += 1
              stats["classes"] += 1 if line =~ /class [A-Z]/
              stats["methods"] += 1 if line =~ /def [a-z]/
            end
        end
      end
    end
    stats
  end

Nothing special here. Further work was to write some script that browsed through code revisions and generated the stats for each rev. This was also pretty easy, even shell script would be sufficient, but I also ruby-scripted it:

if __FILE__ == $0
  revno = `bzr revno`.to_i
  1.upto(revno) do |i|
    `bzr revert -r revno:#{i}`
    stats = CodeStatistics.new(
      ["app/controllers", "spec/controllers"],
      ["app/models", "spec/models"],
      ["app/helpers", "spec/helpers"],
      ["lib", "spec/libs"],
      [nil, "spec/routes"],
      [nil, "spec/other"],
      [nil, "spec/views"])
    File.open("data.txt", "a") do |f|
      f.write("#{i}\t#{stats.code_loc}\t#{stats.test_loc}\n")
    end
  end
end

First it gets number of all revisions (237). Then iterates from 1 upto it and reverts the code to i-th revision. Simple. The output is appended to file in some easy-to-read-by-gnuplot format.

That part of work proved once again that bazaar version control is a great tool. It's very easy to use, no hard-to-remember switches, if you want to revert your code, just type bzr revert. And that's it, _all_ your code gets reverted. No 'recursive' switch required, no need to specify folder (like in svn) -- what would you expect to be done with bzr revert?! Revert first file, or some other repository?

Ok, back to generating stats. I was pretty much ready with everything I needed, opening gnuplot and writing

# set axis, ticks, png output etc.
plot 'data.txt' using 1:3 title 'Tests' with lines, 'data.txt' using 1:2 title 'Code' with lines

created this one:


As You can see the whole app is about 1.5k LOC, with 237 revisions, that gives us approx. 6.3 lines of code added per revision. Test to code ratio is currently slightly over 2.0.

Also interesting thing is the latest 20-30 revisions. I refactored a bit of code and removed unnecessary one reducing the line count adding few tests at the same time. This is motivating ;)



The test to code ratio throughout time shows that it's generally increasing, though there were some hops on the beginning. It's because there weren't lots of code there and the ratio could be changed a lot by adding few tests. It settled about 1.3 at rev 60-70 and increased since then to reach 2.0. You can see the refactoring here (slope is big at the end).

I'll try to come up with other things related to this topic as it's very interesting, though it would be even more if it was done on big project with more than one programmer, agile methodology (burndown chart for example).

Mutate, my code!

Today a bit about mutation tests. What are they?
Let's say you have a code and tests for it. They are passing, you are writing test _before_ implementation, everything is ok. Nothing to worry about? Ok, let's do simple logic ;)

If your tests cover whole code, then if you break the code, the tests should fail. Simple.

Here's what the mutation tests do: they take your precious code, modify it once every try and rerun tests for each mutant (modified code). If your tests fail, the mutated code is presented to you and you should now... write a testcase, because you don't have full coverage!

The mutations themselves are quite simple, see this example

def shiny(sun, moon)
  if sun.shines?
    moon.hide!
  else
    sun[:duration] -= 2;
    moon[:duration] += 3;
  end
end

Ok, don't lookt at the sense of code ;) We could mutate this code to look like this

def shiny(sun, moon)
  if sun.shines?
    moon.hide!
  else
    --- sun[:duration] -= 2;
    +++ sun[:wooot] -= 2;
    moon[:duration] += 3;
  end
end

As you can see, we've changed the symbol, method behaves different now and our tests fail. Or...? Quick! A testcase!

Other mutations could include

• changing comparison (for example from a == b to a != b)


• removing some lines


• swapping true with false


• ...and more

You can see that those are all simple operations, but don't expect that your tests will cope with them easily!

So, after the theoretical introduction, check out the tool, that helps you do the mutation tests in Ruby. It's called Heckle, and you can download and start using. There's nothing special about usage, it's worth saying, though, that Heckle is integrated with rSpec. Just

spec my_spec.rb --heckle MutateMeClass
spec my_spec.rb --heckle MyClass#mutate_my_method

and you ready to go. Heckle currently doesn't support class methods but it's still great fun! Good luck with adding new tests ;)

Sequel getting better

I've been trying to get sequel work with dataset methods from module. It's not such an obvious thing, as the dataset methods are defined

def dataset.some_method(args)
  # method here
end

which makes it impossible to do it the _classic_ ruby-module way. Jeremy Eveans (Sequel's creator) enlightened me ;) on the #sequel channel. The Plugin architecture provided by Sequel::Model::is method is the way to do it!

So basically it's

module Sequel::Plugins::Bunchy
  module DatasetMethods
    # define dataset methods here
  end

class ShinyModel < Sequel::Model
  is :bunchy
end

You define a module inside Sequel::Plugins scope and inside it define special modules which keep the methods.

• InstanceMethods


• ClassMethods


• DatasetMethods

Names are self-explaining. For some more info go http://sequel.rubyforge.org/classes/Sequel/Plugins.html

Also, after a short chat with Jeremy he told that Sequel is going to have built-in support for separate read and write connections! This means that SELECT will use different DB than INSERT/UPDATE/DELETE. Seems very handy in web apps to lower the DB load on each request. Keep up the good work!