Warren Seen

A little something I cooked up, inspired by Rails’ ActiveSupport Numeric::Bytes and friends.
(more…)

For when it’s just as quick to roll your own …
(more…)

Interesting article about the use of the “Committer Model” in commercial software development, borrowed from the open source community.

The committer model solves a few problems of open source distributed development, but the primary one is that of trust: in a developers intent and in their competence. There are other issues that push different buttons depending on the project, but in the end, these are just indicators of intent or competence.

Personally, I think this is a bit of an extreme approach in the average commercial development project – certainly for teams under 10 developers I would call it excessive – as the same issues don’t apply in a commercial project.

Firstly, you must implicitly trust the intent of the developers which you are paying to write the software – and if you don’t, why are they working for you to begin with? Secondly, if you’ve hired them, then you already have made a decision based on their competence.

(more…)

My initial post was going to be a snark-filled venting of all that ails me with my “day job” at the moment. Specifically, the woes of maintenance programming when dealing with a codebase from 2000 and ported from Windows to Linux.

If JP Sartre was a coder, he would have actually said:

“hell is other people’s (undocumented) code.”

So for your amusement, here is the list of simple mistakes I’ve seen of late. Consider these some simple C++ programming sins you may use to torment those that will follow you for eternity. (more…)

I’ve been too busy lately with real work to keep up with the Ruby spider project, but I’m hoping to get back to it real soon.

This is a vent/rant/gratitude post, based on what I’ve spent my last 2 weeks dealing with.

I’ll start with the gratitude, but it’s all downhill from there. I am immensely thankful for the developers responsible for the following tools which, IMO, no Linux developer should be without:

• Valgrind – helped me track down a bunch of obvious memory errors in the code
• dmalloc – helped me track down those which valgrind couldn’t
• gdb – the more I use gdb, the more effective a tool it becomes to me. Why aren’t you using it? At least learn to get a backtrace and inspect variables if you’re at all programming anything with GCC.
• distcc – chasing memory issues into base classes led to full recompilation more often than I would have liked. distcc let me put an extra 2 idle machines to work compiling the code.

What started as a rather curious crash in our dev branch a couple of weeks ago turned into a major audit of how we allocate, use and free memory in our codebase. When I did get to the bottom of the crash, it turned out not to be a memory issue, but a library dependency inherited from libstdc++.

So here’s a little nugget for you right off the bat:

If you use STL containers (or any code which uses STL’s default allocator) in a dynamic library which has been compiled with -MT, you must link any executable that dlopen’s said library with libpthread.

In hindsight this sounds fairly obvious, right? (more…)

When I started this blog last month, I thought “Standard Deviation” was a snappy title. Of course, I also knew about standard deviation as a statistical tool, however I didn’t expect that this overlap would cause Google search to drive 50+ visitors a month here looking for implementations of the standard deviation formula.

So as a “public service”, here is some code to figure standard deviation in Ruby and Java.

(disclaimer: no warranties as to correctness, particularly to the nth decimal place, don’t use this to run your home made nuclear reactor or air traffic control system, blah blah, etc, etc )

(more…)

This is the second part of a series of posts covering the development of my web spider in Ruby. You can read about the initial idea here, and the architecture in Part 0: Concept.

You may also recognise some of the code in Scheduler#run from a short post I made to check that the syntax highlighting was working

First I want to recap the goal of the scheduler before getting into the code itself. Simply put, the scheduler exists to mangage the list of URIs (web pages, RSS feeds) that need to be spidered, and to manage the spiders themselves. In particular, we want to be able to limit the number of spiders working at any one time, out of politeness if nothing else.

I’m not going to make this a tutorial in Ruby syntax by explaining things line by line, if you haven’t used Ruby before and find something you don’t understand, the PragProg book, Programming in Ruby is the place to go look.

So let’s take a peek at some code!

(more…)

(I should probably mention that I have never written a spider or worked on a search engine before, so this is a learning process… I don’t pretend to be an expert on this – I picked this partly because it is far enough from my “day” job that I’m not going to inadvertently end up in a conflict of interest. The closest I’ve come in the past was working on a natural language interface to search engine queries, way back in 2001 while I was in my final year at UTas.)

So how did I start?

(more…)

The web spider writeup continues. I am no longer best friends with red wine after the weekend however.

Speaking of things that are red, I *love* the expressiveness of Ruby. The following is the run method of my Scheduler class, Ruby makes it a piece of cake to understand what is going on here, even without the full context, I think the following is fairly intuitive…

(more…)

No, it’s not that my pants will fall down if you do, but my Java code might get a bit less intelligible.

Ed Burnette blogs on a nasty little “feature” of Eclipse. First, a little background

Java, C, and C++ share a little syntactic quirk that has bitten all developers on the rear from time to time. Take a look at this code:

if (condition)
doSomething();
doSomethingElse();

The “doSomethingElse();” code will always be executed regardless of the value of the condition because only the “doSomething();” statement is considered part of the “if”.

This has bitten me on more than one occasion on source code that has been checked into our repository and edited by multiple people, causing the indentation to get screwed up (eg tabs and spaces). Especially when you’re scanning code you’re not familiar with, indentation is a key indication of code blocks and nesting.

(more…)