Warren Seen

I mentioned Morfik a while back, the little Tassie company that was taking on Google with a number of patents.

Today they were featured on Qik, interviewed by Scoble himself. Check it out.

With a little hesitation, I cracked open the box, hoping that I’d gotten the resolution, colours and bleeds right. I had, they look excellent, if I do say so myself!

I’m the first to admit I’m no designer, and I’ll cop to the fact that the design was largely inspired by Dave Seah’s cards, whilst the loco image and ruby red was chosen to tie into the fact that I’m moving towards specialising in Ruby on Rails.

So, with new cards, I realised that the Wordpress template I’ve been using was pretty tired.

Time for some branding re-alignment!

A quick CSS change and a new header image later, here we are. I’ve kept the layout the same for the time being, I don’t want to sink a lot of time into a new Wordpress template as I’m thinking of packing this whole site up and moving to a Slicehost slice. If I do that, it will be goodbye Wordpress, hello to either Typo or Mephisto.

For the time being however, it’s steady as she goes!

Err, yeah. I remember, once upon a time, I used to have time to blog. That was before we built this:

Oh, and before he was born:

So despite the fact that I’m lying on the couch there, I’ve been far from idle.

My yard still looks like this:

so it’s not as though i don’t have enough to do! However, I *am* committing to finishing up, in the very near future, the Ruby Web Spider I began to write about some 5-6 months ago.

Consider this, a dusting off of the cobwebs kinda post…

This is the second part of a series of posts covering the development of my web spider in Ruby. You can read about the initial idea here, and the architecture in Part 0: Concept.

You may also recognise some of the code in Scheduler#run from a short post I made to check that the syntax highlighting was working

First I want to recap the goal of the scheduler before getting into the code itself. Simply put, the scheduler exists to mangage the list of URIs (web pages, RSS feeds) that need to be spidered, and to manage the spiders themselves. In particular, we want to be able to limit the number of spiders working at any one time, out of politeness if nothing else.

I’m not going to make this a tutorial in Ruby syntax by explaining things line by line, if you haven’t used Ruby before and find something you don’t understand, the PragProg book, Programming in Ruby is the place to go look.

So let’s take a peek at some code!

(more…)

(I should probably mention that I have never written a spider or worked on a search engine before, so this is a learning process… I don’t pretend to be an expert on this – I picked this partly because it is far enough from my “day” job that I’m not going to inadvertently end up in a conflict of interest. The closest I’ve come in the past was working on a natural language interface to search engine queries, way back in 2001 while I was in my final year at UTas.)

So how did I start?

(more…)

The web spider writeup continues. I am no longer best friends with red wine after the weekend however.

Speaking of things that are red, I *love* the expressiveness of Ruby. The following is the run method of my Scheduler class, Ruby makes it a piece of cake to understand what is going on here, even without the full context, I think the following is fairly intuitive…

(more…)

I mentioned before that I’ve been busy this last week, one of the things I’ve been working on in my own time is a web spider (written in Ruby) that can trawl both HTML pages and RSS feeds. I won’t say much about what I’m using it for, other than to say I’m testing some ideas out right now

Anyway, I’m almost at a point where I’m happy to share this code (probably under GPL) as it’s not exactly rocket science (and I’ve only invested a week or so of evenings into it), but it has a couple of neat tricks that made it a good exercise in Ruby. A short laundry list of features:

(more…)