All posts by Dan Milne

Alert for AbeBook users:


Epsilon Informs AbeBooks of E-mail Database Breach

We have been informed by Epsilon, a third-party vendor we use to send e-mails, that an unauthorized person outside their company accessed files that included e-mail addresses of some AbeBooks customers. Epsilon has advised us that the files that were accessed did not include any customer information other than email addresses.

As a reminder, AbeBooks will never ask customers for personal or account information in an e-mail. Please exercise caution if you get any emails that ask for personal information or direct you to a site where you are asked to provide personal information.


Australians cop it again

As if it’s not bad enough that whining Australian billionaires are complaining that international stores don’t have to pay GST, companies which can enforce regional pricing are going to town on Australian buyers.  Apple’s just released their new Mac App Store – the Mac equivalent of the iTunes store already available for iPhone / iPad / iPod devices.  Apple have regional pricing all worked out, so if you think we’ll catch a break with their latest online shopping experience, you obviously haven’t been paying attention.

Take one of my favourite Mac apps – Omnigraffle. Buying it direct from Omnigroup is $199 USD ~= $200.672 AUD

If you’re in North America and you use the Mac App Store US : $199 USD ~= $200.672 AUD

If you’re in the UK Mac App Store – $199 USD ~= $200.672 AUD

If you’re in the Finnish Mac App Store : €149.99 ~= $194.843 AUD

If you’re lucky enough to buy from the Australian Mac App Store, you’ll pay a whopping $249.00 AUD. That’s a 25% premium for living in the lucky country.   See JoshuaGan’s post for more info. (Update: This issue is mentioned in an Omnigroup blog post.)

It’s not just Apple though – check the Steam prices

Call of Duty:  Black Ops from the various international stores:

The UK Steam store : £39.99 GBP ~= $62.058 AUD

The US Steam store : $59.99 USD ~= $60.428 AUD

The Finnish Steam store:  €59.99 EUR ~= $78.447 AUD

And last, (and the opposite of least) the Australian Steam store : $89.99 USD ~= $90.647 AUD

The Steam prices are probably caused by Australian distributors taking some obscene cut. COD4: Black Ops is $89 AUD at both KMart and JB HiFi.

A new Booko feature

Booko now supports displaying “Works” – that is, a list of editions of the same book. A given work may have several editions – for example, a paperback, hardback or eBook version. Booko is starting to collect these various editions together as a “Work”.  We’ve had this feature for a while – but now Booko includes the minimum and maximum price for each edition of a work.  I think it should be a pretty useful feature.
Here’s the Top 10 works over the last few days. Check them out to see the new feature in action.
  1. The Girl with the Dragon Tattoo
  2. Tomorrow, When the War Began
  3. The Girl Who Played with Fire
  4. Eat, Pray, Love: One Woman’s Search for Everything Across Italy, India and Indonesia
  5. The Girl Who Kicked the Hornets’ Nest
  6. The Catcher in the Rye
  7. The Brain That Changes Itself
  8. Anna Karenina
  9. To Kill a Mockingbird
  10. Brave New World

Enjoy!

Calling leet logo designers

We’re looking for an awesome logo for Booko and figured we’d web2.0 / crowd-source / consult the cloud and use 99designs:

http://99designs.com/logo-design/contests/iconic-logo-book-price-comparison-site-50799

Are you a designer? Have at it!

On Daemons

As you might imagine (depending on just how nerdy and imaginative you are), Booko is a poster child for the concept of long running background tasks. Grabbing prices from 40 online stores isn’t a fast process and you certainly would not want your front end webservers making your users wait as long as the slowest of the 40 stores before responding to a user request.

Over the years, I’ve tried various approaches to running user level daemons. My first attempt was ok – I rolled my own and slowly improved it. It could handle HUP signals, write PID files, die gracefully and it knew if it hadn’t died properly and attempted to kill zombie versions of itself. It had stop / start / restart commands. But it wasn’t all sweetness and light.  What happens when it dies? This is probably the trickiest part of running daemons (Well, having to fork twice and make sure you have detached from the terminal is probably tricker, but still).

So, how do you make sure your daemon is running? Cron immediately springs to mind. So, part two of writing your own daemons is writing something to keep them going.  You may have found yourself in this position and felt a little tickle in the back of your mind when you setup a cron job to solve this problem. My cron job looked at the daemon’s log file’s modified time and if it was more than 5 minutes old, looked for the PID file and sent that process a KILL signal.

It’s an easy, stable solution to the problem at hand – albeit with a 5 minute lag to detect crashed daemons. It’s ok because I run multiple daemons which can take the load if one dies.  But, what happens if you only have a single daemon? Increase the frequency of checking?  Cron’s smallest resolution is 1 minute – that’s not really ok (depending on what your daemon does, it may be fine).  But now you have to make sure that your daemon’s writing to the log at least every minute.  Ugh.

This solution is starting to smell. So, what does everyone else do? Well, I checked out God – but it just doesn’t feel like an elegant solution to this problem. It may solve the problem nicely, but there must be a better way? Hard core nerds would probably move on to daemontools but it’s too much work for me.

That tickle you may have had in the back of your mind earlier was your subconscious telling you the problem is already solved and you already use it for your webserver, mail server, DNS server, ssh server and more. Your operating system can provide this exact service for you. Since I’m using Ubuntu that service is provided by Upstart.

Running your service with Upstart has two very nice consequences. Firstly – you can remove all the code used to manage daemonising. You can now write your code to hang around in the foreground. Leaving your code in the foreground while you’re in development mode is good anyway – you can watch it more closely. If you really want to daemonise in our dev environment, bang up a tiny ruby script with the Ruby Daemon gem which calls your actual script and manages PIDs, signals and a stop/start interface for you.

Setting up a service to run with Upstart requires just a config file – here’s one I prepared earlier:

description "Price Fetcher Upstart script"
author "Dan Milne"

start on startup
stop on shutdown

console output

respawn
instance $FID

script
env RAILS_ENV=production
export RAILS_ENV

exec sudo -u booko RAILS_ENV=production /opt/ruby-enterprise/bin/ruby /var/www/booko.com.au/booko/bin/fetcher.rb $FID
end script

That file gets named “fetcher.conf” and goes in the /etc/init/ directory. This has some nice features; the first of which is that once it’s started, it will keep running. If it dies, it’ll respawn (you can see the option right there in the script).  The fact that it died goes in /var/log/daemons – but what’s even awesomer, you can run multiple instances of the same script, by passing in FID=0 or FID=1 etc when you’re starting it. Finally, it gets the standard init features. You can start it with ‘service fetcher start FID=0’ for example.

The only missing feature that I can see, is that because I need to pass in FID=0 to the script, it doesn’t start at bootup. There appears to be no way of stating “Startup 2 of these at boot time”.

In summary, if you use your OS init services, you get to write simpler code, get respawning at an OS level and you get all the normal daemon control features.

Booko’s moved, features added.

I’ve been working on a beta version of Booko for, like, 7 months now.  I finally upgraded it while moving from Slicehost to Linode.  I’ve made a large number of changes to the way Booko performs long running tasks and how those tasks communicate. But I’ll leave the nerdy stuff for later.

The biggest change from a user point of view is some integration into Freebase.com. You’ll notice extra information appear in book listings now. For example, the Booko page for The Girl with the Dragon Tattoo now tells you:

  • that the book is part of the Millennium Trilogy
  • the other books in the series (The Girl who played with Fire & The Girl who kicked the Hornets’ Nest)
  • the other editions of The Girl with the Dragon Tattoo – hardcover and paperback.

The data at Freebase is a long way from complete, but it’s constantly growing. This should be a very useful feature. I’ll be doing more to integrate Freebase into the search results – for example, so a book shows up only once, listing the different editions in that single search result item.

List management also got an overhaul. That old list manager page was pretty bad. There’s still work to do, but it should be far more usable now. Log in and check it out!

There are still bugs to fix (soooo many missing images for cover art) and features to add (smarter list price calculation, used books).

Poor Australians

Another example of high prices in Australia. KitchenAid Mixer. With a RRP in the US of $349.99 USD or $382.67 AUD, Amazon sell it for $290 USD ~ $317.079. RRP in Aus is…. $729 AUD.

Amazon.com Vs Kitchenware Direct

I have nothing against Kitchenware Direct – I’ve bought stuff from them in the past and will do in the future. Just like high book prices, no doubt this has nothing to do with retailers, but probably with distributers. (Publishers in the case of books).