Over the last few months traffic to Booko has been steadily growing – the kind of growth I like. Steady growth allows you to plan how to make things go faster without having to make crazy, on the fly tweaks to help things along (or stop them from falling over).  I’ve tried to design Booko to make it scale up easily – it’s basically split into two main components: the website is the front end, and the price grabber which is a separate bit of code, sits quietly in the background looking up prices. They talk to each other via the database – the website says “Hey Price Grabber, get me some prices for ISBN 123” (it does this by updating the database) , and the price grabber dutifully goes and grabs the prices and stores them in the database when it’s done, which the website then displays.

The idea was that I’d make the price grabber threaded, so that it could look up multiple books at once. Turns out, Ruby may not have been the best choice for a highly threaded, I/O based bit of code. Well, at least not Ruby 1.8. It’s interesting to see that JRuby may in fact be the best way to run this type of code.  Ruby 1.8’s threading model blocks on I/O – not great for my price grabber. The solution is pretty easy though – just run multiple price grabbers – which is what I’ve done. So, Booko can now have as many price grabbers as needed. They talk politely to each other via the database (so that, for example, two pricers don’t look up prices for the same book.)

This has been lightly tested but should be pretty stable – let me know if you have any problems getting prices.