ifacethoughts

Continuous Integration

Software projects vary widely in the scale – right from single-manned projects to the ones with hundreds of members working away at their keyboards across multiple shifts. The biggest thing that is affected by this is the management of source code. The unique thing about the source code is that it is just a single entity which can be built in the end, though it is worked upon by many. It has an attribute of being able to be split into multiple segments, but also requires that they should be integrated back to form that single entity.

Version Control System

A common problem is that sometimes we make wrong changes, and we want to roll them back. This problem is solved by using a version control system – something that can maintain versions of the source codes and allows to switch to any version. Today’s version control systems also include facilities like locking to avoid multiple people working on the same source code or like merging to merge different changes.

This seems to solve all the problems, and it does fix the visible problems. However, there is a more critical problem at hand in software project management. In traditional projects, software development is done in components, which are unit tested and integrated at the end to build the entire system. Unit testing covers only half the distance to the destination, the changes are right only when the entire system can be compiled, built and tested successfully. The entire system, not just individual components!

The version control systems do not ensure this, they only ensure physical integration of the source code. We need a stricter way of ensuring that the changes to the source code are right from the requirements perspective of the entire system. One solution is Continuous Integration.

Continuous Integration

Continuous Integration is a software practice where developers integrate their source code changes more frequently than the standard practice, at least once in a day. Note that it is about integrating the code, not just checking it in. Which means that it should be followed by the automated system build and automated self-testing which ensures that the code change is right.

OK, so we came across a bunch of automations here – the build and self-tests. The difference between compilation and building is that it is more than converting text source code to binary. Builds can include reorganizing, creating configuration files, or database resources. Martin Fowler suggests that builds should be inclusive, they should automate everything required to build the software.

I’ll elaborate my earlier rule of thumb: anyone should be able to bring in a virgin machine, check the sources out of the repository, issue a single command, and have a running system on their machine.

Build automation is typically done through scripts, and there are many tools out there that can help writing the scripts, like make, nmake, Ant.

These automated builds will notify errors because clash between developers. You can go one step further by including automated self-tests in the build so that the code is even tested as per the requirements. Of course, these tests are more inclusive than specific, and will not replace the various unit tests and system testing.

There are more recommended practices than these automations, which Martin lists in his paper. They include everything from maintaining a single source repository to making everything transparent to automated deployment.

Initial hurdles

I did have some hurdles when I started with Continuous Integration. First was that the list of recommended practices were intimidating. But, as Martin says, it is not important to start with everything. These are purely out of my experiences, something that probably others might not have faced.

Committing everyday

The biggest shift of paradigm for me was in committing the source code everyday. I was in the habit of having a logical point at which I would be satisfied with the source code and then change it. And this could happen anywhere between a day to a week. And it did work fine for some time. Until I realised that testing my code, by itself, without others’ changes, was not enough. I could move away from the logical closure thinking, so now I have broken down into more, which are manageable in a day.

You might wonder why is there a practice for checking in everyday. I had also questioned it when I started! I realised the benefit a little later, when I saw the change that was happening to the systems everyday, the change that was caused by integrating everyone’s code. And it continued to help me with my changes the next day. My cycles of redoing the same thing because something else had changed reduced.

Skills for automated builds and tests

Another problem I have seen with projects is that they do not have people who have enough information about the builds. It is not easy to automate everything and include it in builds. It might require shell scripting knowledge or knowledge of build systems like make or Ant. Similarly people who could write good automated tests were scarce. It is a skill to be able to write tests that could cover substantial amount of source code and expose the bugs.

Not enough priority

And of course, sometimes enough priority was not given to the source control or the build. There is always a system for structuring the source control tree, that can help you get what you want easily. This can in turn help you in building and testing the entire system quickly. But enough importance and priority is not given to these administrative tasks. However, these tasks require a one-time effort and give you benefits for the entire project life-cycle, probably beyond that too.

Epilogue

Continuous Integration is considered to be a key part of eXtreme Programming (XP), but I believe it is useful in all scenarios. Not only does it help in reducing effort of redoing a lot of work, it also helps in finding bugs much earlier in the cycle. There are a lot of tools available, most notably being CruiseControl. If you still have not given it a try, it is definitely worth it. You might end up making it part of your process.

Discussion [Participate or Link]

  1. Linus On SCM | iface thoughts said:

    [...] trying alternatives like Bazaar. However it has been difficult to migrate because I am exploring Continuous Integration to use with the version control system. Cruise Control, a continuous integration tool, integrates [...]

  2. Open Source Builds New Bugs | iface thoughts said:

    [...] code has been changed. I think this calls for automating the tests as well by using a process like Continuous Integration. Maybe tighter and more extensive tests should be used for release builds and milestone builds. [...]

Say your thought!

If you want to use HTML you can use these tags: <a>, <em>, <strong>, <abbr>, <code>, <blockquote>. Closing the tags will be appreciated as this site uses valid XHTML.

freshthoughts

contactme

Abhijit Nadgouda
iface Consulting
India
+91 9819820312
My bookmarks

badgesand...

This is the weblog of Abhijit Nadgouda where he writes down his thoughts on software development and related topics. You are invited to subscribe to the feed to stay updated or check out more subscription options. Or you can choose to browse by one of the topics.