The committer model solves a few problems of open source distributed development, but the primary one is that of trust: in a developers intent and in their competence. There are other issues that push different buttons depending on the project, but in the end, these are just indicators of intent or competence.

Personally, I think this is a bit of an extreme approach in the average commercial development project – certainly for teams under 10 developers I would call it excessive – as the same issues don’t apply in a commercial project.

Firstly, you must implicitly trust the intent of the developers which you are paying to write the software – and if you don’t, why are they working for you to begin with? Secondly, if you’ve hired them, then you already have made a decision based on their competence.

Fair enough, you don’t leave the keys to the castle with them on the first day, but not allowing them to check into a revision control system AT ALL is just asking for trouble in my opinion.

Still, the fact remains that to ensure quality, you can’t have code that was just checked in being rolled straight into a release and expect it to work. There needs to be some sort of gating to ensure that code has been relatively well exercised before it makes it into a release build. So how to compromise?

The way we approach this issue is to maintain a “trunk”, the latest code in development, and release branches, eg 1.0-release, 1.1-release, etc. We have a team of anywhere up to six developers working on various facets of the code at any time. The rules are simple:

1. don’t check into the trunk if your code won’t compile and pass our test suite.
2. if you have done significant work which has caused breakage, create a private branch until you can fulfill 1.
3. only one person is in charge of release “gating”, eg merging code from the trunk into a release branch.

This approach provides the ability for all developers to grab the bleeding edge code, whilst ensuring that code that gets merged into release branches is vetted accordingly. Fixes are applied within the trunk, and backported to release branches if necessary (eg for patch distribution)

In the case of simple changes, this probably takes all of 5 minutes to confirm that the fix does what it needs to do. Bigger changes usually require consultation or a walkthrough with the developer responsible.

In essence, this still implements the same safeguards as the committer model, however you also capture more change history as developers are able to commit small changes as they are made, reducing the risk of introducing widespread errors, or losing large amounts of work through equipment failure.

Personally, it’s rare for me to do a day’s work without committing it all – I’ve seen a hard drive die overnight enough times to be paranoid.

If JP Sartre was a coder, he would have actually said:

“hell is other people’s (undocumented) code.”

So for your amusement, here is the list of simple mistakes I’ve seen of late. Consider these some simple C++ programming sins you may use to torment those that will follow you for eternity.

Inappropriate use of delete when you really mean delete[].
Performing a scalar delete on a pointer to an array deletes the first element only and leaves the rest of the array hanging around in memory. What a nice mess that leaves behind.

Inconsistently implementing basic idioms
eg reference counting, across multiple classes. Decide on the “one true way” and then put it in a base class, template class, whatever. Just Don’t Repeat Yourself (DRY).

Using naked pointers.
If you must insist on using pointers, adopt one idiom and stick with it. Don’t leave dangling pointers, whenever you delete a pointer, set it to NULL and use this as a guard condition, so you don’t try to delete it twice. eg
if (pFoo) {
delete pFoo;
pFoo=NULL;
}
This is mostly useful in the case where you have pointers as member variables of a class, which may or may not have been allocated and/or cleaned up before the destructor is called.

Not commenting code.
At all. What more can I say really? Maybe it was obvious to you when you wrote it, but it beats the life out of me whether the code does what you meant it to do. I’m not a mind reader or a savant. You’re not a savant either, but chances are I think you’re an idiot now I’m knee deep in your code and it’s a mess that I can’t decode.

Smart programmers recognise their own fallibility
… as well as seeing that trait in others. We’re imperfect, we all forget things now and then, there’s no point in being macho about the way we program. Better to be defensive, when you revisit the code 6 months later, you’ll thank yourself.

Here are some simple remedies:

• Use STL containers wherever possible (and sensible), rather than dynamic arrays. The benefits outweigh the performance hit in most non-trivial cases. If you must use arrays, pay close attention to how you delete them, or you’re eventually gonna get bitten
• Pay attention to patterns – extract common code wherever you can to save repetition. Make this code testable, so when it’s right, it’s right through your entire project.
• Instead of naked pointers, consider smart pointers, such as STL’s auto_ptr. Boost also features some more flexible smart pointers, including shared_ptr. Sure auto_ptrs don’t work so good for arrays, and you can’t store them in STL containers, but if you find yourself in either of these situations, you should probably rethink what you’re trying to achieve anyhow. If you program inside of this constraint you’ll save yourself in the long term
• Document your code. It doesn’t have to be verbose, but leave a few clues to jog your own memory. At the very least, try to document expected functionality and keep it in synch with the code itself. If you say in the comments that a function does “A”, it better do “A” exactly as advertised.

Seriously, this is all stuff I learnt in first year software engineering? Why are so called professionals still not capable of doing this?

This is a vent/rant/gratitude post, based on what I’ve spent my last 2 weeks dealing with.

I’ll start with the gratitude, but it’s all downhill from there. I am immensely thankful for the developers responsible for the following tools which, IMO, no Linux developer should be without:

• Valgrind – helped me track down a bunch of obvious memory errors in the code
• dmalloc – helped me track down those which valgrind couldn’t
• gdb – the more I use gdb, the more effective a tool it becomes to me. Why aren’t you using it? At least learn to get a backtrace and inspect variables if you’re at all programming anything with GCC.
• distcc – chasing memory issues into base classes led to full recompilation more often than I would have liked. distcc let me put an extra 2 idle machines to work compiling the code.

What started as a rather curious crash in our dev branch a couple of weeks ago turned into a major audit of how we allocate, use and free memory in our codebase. When I did get to the bottom of the crash, it turned out not to be a memory issue, but a library dependency inherited from libstdc++.

So here’s a little nugget for you right off the bat:

If you use STL containers (or any code which uses STL’s default allocator) in a dynamic library which has been compiled with -MT, you must link any executable that dlopen’s said library with libpthread.

In hindsight this sounds fairly obvious, right? The problem seems to arise due to the fact that the gnu std lib stuff uses a multi-threaded allocator which is dependent on libpthread, yet the pthread_* functions are all weak references which will resolve to NULL if the actual functions can’t be found.

There are a few reasons why this issue was less than transparent.

• The error presented as a segfault in mt_allocator.h, rather than an undefined reference, which is usually an immediate sign that a library hasn’t been linked in.
• Our build system uses autoconf/automake, and we have it configured in a way which obscures the fact that we compile all dynamic libraries using -MT (although this is clearly obvious when you watch the compiler output)
• The test harness we experienced the issue in was compiled without -MT, although it shared identical AM_CPPFLAGS settings to the library in question.

In the end, I found myself here: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25409, and sure enough, adding -lpthread solved my problem. It sounds like they may have patched the mainline of libstdc++ to complain more about weakref’d symbols, which would be a good thing.

So my beef is not so much with libstdc++, but with how we set up the build system. However having said that, this code used to work fine so I suspect we’ve tripped over something that was recently changed either in our source, or in the library.

So that’s some gratitude and some venting. This is getting long… the rant will get its own post I think.