In the end, I opted not to host my own email, a strategic decision based on the time and effort required on my part to set up, configure and most importantly, secure and maintain a mail server. Instead, I spent fifteen minutes of my time setting up Google Apps. Being able to keep my personalised email address, with 6Gb of storage and full access via POP3 and IMAP (although it’s a tad quirky since GMail doesn’t support folders in the traditional sense) plus access via the GMail web UI on the go, it offers the best solution for my mail needs across multiple machines.

It’s been a busy time of year for me surprisingly, so much so that I’ve again picked up the GTD book and made another, more concerted effort to get my “things” done. It really feels like I don’t have enough hours in the day with all these different things on the go, and I’ve really noticed the relief that comes from being able to “braindump” everything that’s on my mind and start tackling it in a sensible way.

I’ve never been one for new year’s resolutions, but I definitely have started making plans to be more proactive this year, rather than just letting all this “stuff” control me. So here’s to new beginnings…

So as a “public service”, here is some code to figure standard deviation in Ruby and Java.

(disclaimer: no warranties as to correctness, particularly to the nth decimal place, don’t use this to run your home made nuclear reactor or air traffic control system, blah blah, etc, etc )

The algorithm I’ll be using is “borrowed” from wikipedia’s entry on Algorithms to calculate variance. Specifically, I’ll be using a variant of algorithm II, which is sourced from Knuth, except we’ll calculate the standard deviation for the population, rather than a sample.

As you should probably know, standard deviation is defined as the square root of the variance. If you didn’t know this, maybe you should go read about standard deviation first.

/**
 * @param population an array, the population
 * @return the variance
 */
public double variance(double[] population) {
	long n = 0;
	double mean = 0;
	double s = 0.0;

	for (double x : population) {
		n++;
		double delta = x - mean;
		mean += delta / n;
		s += delta * (x - mean);
	}
	// if you want to calculate std deviation
	// of a sample change this to (s/(n-1))
	return (s / n);
}

/**
 * @param population an array, the population
 * @return the standard deviation
 */
public double standard_deviation(double[] population) {
	return Math.sqrt(variance(population));
}

example usage:

double[] arr = { 1, 3, 24, 17, 12, 6, 14};
System.out.printf("%f", standard_deviation(arr));
  // prints 7.596992

  def variance(population)
    n = 0
    mean = 0.0
    s = 0.0
    population.each { |x|
      n = n + 1
      delta = x - mean
      mean = mean + (delta / n)
      s = s + delta * (x - mean)
    }
    # if you want to calculate std deviation
    # of a sample change this to "s / (n-1)"
    return s / n
  end

  # calculate the standard deviation of a population
  # accepts: an array, the population
  # returns: the standard deviation
  def standard_deviation(population)
    Math.sqrt(variance(population))
  end

example usage:

 puts standard_deviation([1, 3, 24, 17, 12, 6, 14])
 # prints 7.59699188589047

If you found this at all useful, (or have spotted a bug), please leave a comment to that effect…

Anyway, precis on BlogCode is fairly straight forward, drop the name and url (not the feed tho) of the blog you want to code into the UI, then score the blog on a range of sliding scales covering content, tone, etc.

Neat, but essentially useless in and of itself.

The trick however is that the sliding scale scores allow statistical analysis to determine the blogs that match according to the criteria rated. eg for this blog, a range of sites are returned with a 72-78% “match”.

The underlying idea is that over time, as more people code a blog, the ratings will become a more accurate and democratic opinion of the blog in question.

I noticed that they do collect the country that the blogger appears to be from, however it doesn’t seem that you can use this to narrow searches, etc.

The problem I can see with this system however is this: there are a LOT of categories to rank, and although it doesn’t take long with sliding scales to do this, the novelty of assessing a blog over 15+ criteria is going to get old pretty fast.

Additionally, there’s no real incentive to come back to the site again and again. This feature would be more useful as an element of existing blog search (eg Technorati) or built into an aggregator (eg Rojo) as a value add. Imagine you add a blog to your feeds, and get suggestions for similar blogs, with sample content so you can decide whether you want to read them too. Not to mention being able to sort blogs according to their BlogCode “match” level.
Alternatively, it could serve as the basis of a meme tracker in which you can influence the results you see based on blogs you’ve coded.

In short, an interesting statistical experiment, but the model (which has been carried over from storycode.com), doesn’t seem to be that tight a fit with the blogging state of the art. I mean, no RSS feeds, what were they thinking?

• Steve Rubel takes the classic high school debate approach of defining the word and building an argument from there. Conclusion – yes, it’s not authority, it’s popularity, people.
• Data mining agrees with Steve, “name things for what they are, not for what they are used for”. That is quite obviously right out of Usability 101.
• Jack Krupansky leaves an excellent comment on Scobelizer: “to Technorati, “authority” is simply popularity. That makes *no* sense.“

Consensus seems to be that tracking popularity but calling it “authority” muddies the waters… There is nothing wrong with the feature itself, it’s just the name that’s misleading.

Technorati has just added ‘authority’ filtering to their search (see Scoble, TechCrunch, Dave Sifry, et cetera).

My issue with this is simple – the number of inbound links does NOT necessarily qualify the source as being an “authority” on anything. This is nothing more than a popularity contest.

Ben Barren hits the nail on the head here.

So its very hard to determine relative popularity, on a regional basis

Before pointing to this New York Metro quote:

In the blogosphere, the biggest audiences—and the advertising revenue they bring—go to a small, elite few. Most bloggers toil in total obscurity.

Popularity, that’s all it is. And that’s sad, because some of the people who deserve to be held as authorities in their field are lost amongst the noise, while the “blogosphere” (gack, I HATE that “word”) becomes more and more like a conversation between a panel of “A-list” bloggers with everyone else on the sideline. Then you’ve flipped from being “citizen” media to just media.

Disparate thoughts to follow…

• Blog search the way Technorati does it is fundamentally broken. Ranking based purely on “link love” is primitive at best. Calling it “Authority” filtering is inaccurate. That’s a big call to make, but I’m going to be dumb and stand by that one…
• It’s broken because it causes new blogs to be put in a catch 22 – if you want to get noticed, you have to get links. If you want to get links you need to get noticed in the first place.
• Even if you can get some traffic driven to your blog from eg Technorati, how do you make it sticky? The majority of my page views reported by awstats are in the 30 second region. The next highest is the 30-60 second. It tails off quickly after that.
• Have you ever wondered about the great bloggers you haven’t read? How will you ever find them if no one links to them?
• Don’t you wish you could find blogs similar to the ones you already read? Or ones that provide a counter-point? ie “Show more like this…”
• I’m not talking about just keyword matching here, it’s a far more sophisticated problem than it appears. Memetrackers DON’T do this… bet you wish they could.
• What if your aggregator sorted items from your feeds by relevance? Think a mutant cross between reddit and amazon.com’s “for you” for your feeds. Key word filtered “smart” folders are dumb by comparison.
• What if technorati did the same? Why should I have to manually create my watch list, when technorati knows what I search for regularly? Hell, they know what I tag, that’s a BFC (big effing clue) right there…

time for the (long) tail to wag the dog.