Comments on: Ruby web spider Part 1: The scheduler http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/ freelance software developer Tue, 06 Jan 2009 23:57:54 +0000 http://wordpress.org/?v=2.5.1 By: Jim http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-87 Jim Sun, 14 May 2006 11:56:26 +0000 http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-87 Warren, I wanted to thank you for your piece on Ruby Spidering. I've been trying to find some sample code with little luck. I trying to learn Ruby and develop an application that scrapes horse racing pages and inserts the results into a MySql database. Your work have been very helpful. Thanks again. Can't wait for the next installmnet. Jim Warren,

I wanted to thank you for your piece on Ruby Spidering. I’ve been trying to find some sample code with little luck. I trying to learn Ruby and develop an application that scrapes horse racing pages and inserts the results into a MySql database.

Your work have been very helpful. Thanks again.

Can’t wait for the next installmnet.

Jim

]]> By: assaf http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-36 assaf Tue, 14 Mar 2006 00:04:51 +0000 http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-36 If the object is stateless, then does it really matter? You're not losing any state when it gets garbage collected, you're not building any state when you create it. The cost of creating a new object is lower than the cost of pooling it. It matters in EJB because a "stateless" session bean is void of application state, but is packed with container state. So it's not really stateless. What EJB does is ask you to help manage those "stateless" objects by writing code that pretends they are really stateless, but acting as if they're really stateful. How they managed to pull it off (and how I fell for it for so long) is still a mystery. If the object is stateless, then does it really matter? You’re not losing any state when it gets garbage collected, you’re not building any state when you create it. The cost of creating a new object is lower than the cost of pooling it.

It matters in EJB because a “stateless” session bean is void of application state, but is packed with container state. So it’s not really stateless. What EJB does is ask you to help manage those “stateless” objects by writing code that pretends they are really stateless, but acting as if they’re really stateful.

How they managed to pull it off (and how I fell for it for so long) is still a mystery.

]]> By: warren http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-35 warren Mon, 13 Mar 2006 13:08:27 +0000 http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-35 hi assaf, thanks for the advice. in particular, what you say about native threads makes perfect sense in hindsight. again, that's me still thinking in Java, ugh. It's true that object pooling is obsolete when you have relatively cheap object instantiation, but in this instance I was wondering whether repeatedly creating and deleting objects which are stateless is a waste of cycles that I can design out without any overhead... hi assaf, thanks for the advice. in particular, what you say about native threads makes perfect sense in hindsight. again, that’s me still thinking in Java, ugh.

It’s true that object pooling is obsolete when you have relatively cheap object instantiation, but in this instance I was wondering whether repeatedly creating and deleting objects which are stateless is a waste of cycles that I can design out without any overhead…

]]> By: assaf http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-34 assaf Sun, 12 Mar 2006 01:28:22 +0000 http://warrenseen.com/blog/2006/03/08/ruby-web-spider-part-1-the-scheduler/#comment-34 Ruby doesn't have native threads, so I don't see how thread pooling makes a difference. You'll spend more CPU handling the pool than on actual object creation. Java had us stop obsessing about memory allocation and deallocation. Then it reincarnated them as object pooling. Ruby goes a long way to let us stop obsession about object pooling. I hope they keep it that way. Similarly, don't worry too much about using blocks. Ruby doesn’t have native threads, so I don’t see how thread pooling makes a difference. You’ll spend more CPU handling the pool than on actual object creation.

Java had us stop obsessing about memory allocation and deallocation. Then it reincarnated them as object pooling. Ruby goes a long way to let us stop obsession about object pooling. I hope they keep it that way.

Similarly, don’t worry too much about using blocks.

]]>