Merb 1.0RC out!

So the release candidate is out!

sudo gem install merb

will do what you need to test and report any bugs. Meanwhile you can check wiki for known bugs. Good luck with testing :)

Version control at home

So here is the deal. You have your own project. You want it versioned. Yes, you *do*. What is your choice?
First thing that comes to your mind: SVN! Brilliant idea, if only you want less productivity.

• Start a repository.
  No, it's not THAT easy with SVN. You have to setup some repository server, then import files to it and checkout them in some directory. All these are mandatory, no escape.
  With bazaar all you do is type "bzr init" and "bzr add".


• Ignore files.
  This is actually the funniest thing. Ignoring files/directories in SVN is done through setting properties on them. Super-unintuitive, and very inconvinient. Other VCS-es like git use one, simple file which lists ignored files, directories (you can use simple regexes like "*.log").


• Branching.
  Now this is good. If you tame the complexity of merging and branching in SVN call me :) They complicated it so much not mentioning that the algorithm of merging is stupid (sees conflicts when it shouldn't). And by the way, if you branch your tree, the counter of versions goes up by 1. Even if your parent tree didn't change at al - this is cosmetics, but those matter!


• Alter files.
  You have to use some weird switches like recursive reverting. Why would you have to type it when this is the default behaviour you would expect?! This is all about usability. SVN has no idea about it. Weird switches, different revisions of working tree *without* "status" showing it (try to update some directory to different revision, you won't be able to commit this change!).


• Show log.
  No internet connection? Sorry, no log in SVN if you repository is out there, behind your ISP. SVN is not distributed, therefore everything is kept in repository, not your local tree.
  


• Commited something broken?
  Hmm, just commit a fix. With other VCSes you could just uncommit, but no, not in SVN.


• Copying directories.
  Don't bother moving folders from one repostory to other. Because svn keeps its config files in *each* folder you are versioning you will mix different configs and break repo. Also, when grepping through your code, you will get false results because the grep will search also in config files which include pieces of code.

If you are already working with SVN you can still move to Bazaar for example. Just use bzr-svn, it works fine, I'm using it with no problems with a repo with over 2400 versions.

If you know polish language you can read a small introduction to distributed verson control systems, it was prepared by Krzysztof Goj and me. Have fun :)

Developing alone

How developing alone is different from working with bunch of people?

I'm doing both now: my personal project and the project I do in the work (approx. 5 people involved). The personal project is a lot smaller than the work project, it uses a slightly different technology (though not _so_ different).

1. General knowledge of project.
   Working alone is a real benefit when it comes to knowing what is where. You develop it, you don't waste time on searching for some piece of functionality. It's a huge amount of time you're saving. This is connected to...


2. Reusing code.
   If you don't know of some code, you can't reuse it obviously :) This affects the quality and consistency of your application. Alone, you are almost 100% sure that you didn't create any useless code; in group, there is a good chance that you miss something, especially if your code isn't documented well.


3. Specifying requirements.
   I found out, that if I specify requirements for myself I tend to change them a bit. This is because I am a programmer and I have some general overview on implementation of something. This eases my work but can be tricky, because it can affect the user experience which is the most important thing.

Having someone who doesn't know programming specifying requirements is in this way better, you can be sure that he/she didn't change some functionality just because it is easier to code. On the other hand such people can be unfamiliar with user-friendliness topic for example, which affects the user-experience obviously. Then you have to talk over the specifications.

4. Issue tracking.
   This is very important when working in group. You have to be sure that you won't be doing something that someone is doing at the moment. This isn't an issue when you work alone. I found out that a simple TODO file is enough, I just take care of one task at the moment, prioritize the tasks simply by moving lines up/down. This is easy and effortless.


5. Quality of code.
   Alone, you are more error-prone. This is normal, and you don't have to be afraid of it :) When pair-programming you have two people looking at code, checking each other. You don't get this benefit when being alone. This is connected to code reviews/audits. They are also a way of keeping your code clean and of good quality, especially when it comes to some crucial parts of system -- this can be a way to decrease the impact of working in group on general knowledge of project and code reuse.

My general opinion is: don't expect to get 5-times-speedup on project with hiring 5-times more people -- it's just impossible, due to the communication issues. Instead, you should concentrate on improving the process to gain more.

Code to test ratio in bzr.

It's always interesting to see how the code changed as the revisions in repo increased. Actually, it's interesting enough to spend a while and generate some stats. And that's what I did :)

I wrote a simple ruby script that gathers stats from my app

  def file_stats(files)
    stats = {"lines" => 0, "comments" => 0, "loc" => 0, "classes" => 0, "methods" => 0}
    files.each do |filename|
      File.open(filename, "r") do |file|
        while line = file.gets
          stats["lines"] += 1
          next if line =~ /^\s*$/
            if line =~ /^\s*#/
              stats["comments"] += 1
            elsif line !~ /^\s*$/
              stats["loc"] += 1
              stats["classes"] += 1 if line =~ /class [A-Z]/
              stats["methods"] += 1 if line =~ /def [a-z]/
            end
        end
      end
    end
    stats
  end

Nothing special here. Further work was to write some script that browsed through code revisions and generated the stats for each rev. This was also pretty easy, even shell script would be sufficient, but I also ruby-scripted it:

if __FILE__ == $0
  revno = `bzr revno`.to_i
  1.upto(revno) do |i|
    `bzr revert -r revno:#{i}`
    stats = CodeStatistics.new(
      ["app/controllers", "spec/controllers"],
      ["app/models", "spec/models"],
      ["app/helpers", "spec/helpers"],
      ["lib", "spec/libs"],
      [nil, "spec/routes"],
      [nil, "spec/other"],
      [nil, "spec/views"])
    File.open("data.txt", "a") do |f|
      f.write("#{i}\t#{stats.code_loc}\t#{stats.test_loc}\n")
    end
  end
end

First it gets number of all revisions (237). Then iterates from 1 upto it and reverts the code to i-th revision. Simple. The output is appended to file in some easy-to-read-by-gnuplot format.

That part of work proved once again that bazaar version control is a great tool. It's very easy to use, no hard-to-remember switches, if you want to revert your code, just type bzr revert. And that's it, _all_ your code gets reverted. No 'recursive' switch required, no need to specify folder (like in svn) -- what would you expect to be done with bzr revert?! Revert first file, or some other repository?

Ok, back to generating stats. I was pretty much ready with everything I needed, opening gnuplot and writing

# set axis, ticks, png output etc.
plot 'data.txt' using 1:3 title 'Tests' with lines, 'data.txt' using 1:2 title 'Code' with lines

created this one:


As You can see the whole app is about 1.5k LOC, with 237 revisions, that gives us approx. 6.3 lines of code added per revision. Test to code ratio is currently slightly over 2.0.

Also interesting thing is the latest 20-30 revisions. I refactored a bit of code and removed unnecessary one reducing the line count adding few tests at the same time. This is motivating ;)



The test to code ratio throughout time shows that it's generally increasing, though there were some hops on the beginning. It's because there weren't lots of code there and the ratio could be changed a lot by adding few tests. It settled about 1.3 at rev 60-70 and increased since then to reach 2.0. You can see the refactoring here (slope is big at the end).

I'll try to come up with other things related to this topic as it's very interesting, though it would be even more if it was done on big project with more than one programmer, agile methodology (burndown chart for example).

Mutate, my code!

Today a bit about mutation tests. What are they?
Let's say you have a code and tests for it. They are passing, you are writing test _before_ implementation, everything is ok. Nothing to worry about? Ok, let's do simple logic ;)

If your tests cover whole code, then if you break the code, the tests should fail. Simple.

Here's what the mutation tests do: they take your precious code, modify it once every try and rerun tests for each mutant (modified code). If your tests fail, the mutated code is presented to you and you should now... write a testcase, because you don't have full coverage!

The mutations themselves are quite simple, see this example

def shiny(sun, moon)
  if sun.shines?
    moon.hide!
  else
    sun[:duration] -= 2;
    moon[:duration] += 3;
  end
end

Ok, don't lookt at the sense of code ;) We could mutate this code to look like this

def shiny(sun, moon)
  if sun.shines?
    moon.hide!
  else
    --- sun[:duration] -= 2;
    +++ sun[:wooot] -= 2;
    moon[:duration] += 3;
  end
end

As you can see, we've changed the symbol, method behaves different now and our tests fail. Or...? Quick! A testcase!

Other mutations could include

• changing comparison (for example from a == b to a != b)


• removing some lines


• swapping true with false


• ...and more

You can see that those are all simple operations, but don't expect that your tests will cope with them easily!

So, after the theoretical introduction, check out the tool, that helps you do the mutation tests in Ruby. It's called Heckle, and you can download and start using. There's nothing special about usage, it's worth saying, though, that Heckle is integrated with rSpec. Just

spec my_spec.rb --heckle MutateMeClass
spec my_spec.rb --heckle MyClass#mutate_my_method

and you ready to go. Heckle currently doesn't support class methods but it's still great fun! Good luck with adding new tests ;)

Sequel getting better

I've been trying to get sequel work with dataset methods from module. It's not such an obvious thing, as the dataset methods are defined

def dataset.some_method(args)
  # method here
end

which makes it impossible to do it the _classic_ ruby-module way. Jeremy Eveans (Sequel's creator) enlightened me ;) on the #sequel channel. The Plugin architecture provided by Sequel::Model::is method is the way to do it!

So basically it's

module Sequel::Plugins::Bunchy
  module DatasetMethods
    # define dataset methods here
  end

class ShinyModel < Sequel::Model
  is :bunchy
end

You define a module inside Sequel::Plugins scope and inside it define special modules which keep the methods.

• InstanceMethods


• ClassMethods


• DatasetMethods

Names are self-explaining. For some more info go http://sequel.rubyforge.org/classes/Sequel/Plugins.html

Also, after a short chat with Jeremy he told that Sequel is going to have built-in support for separate read and write connections! This means that SELECT will use different DB than INSERT/UPDATE/DELETE. Seems very handy in web apps to lower the DB load on each request. Keep up the good work!

The Beauty of Ruby

Well this is kind of weird:

swine:~ msq$ irb
>> $_ = "weird,stuff,comes,here"
=> "weird,stuff,comes,here"
>> nil.send(:split, ",")
=> ["weird", "stuff", "comes", "here"]

Looks very confusing, could be used as one of method to obfuscate code :)