Wednesday, February 3, 2010

Finding Sales on Canon and Nikon Gear Just Got Easier

Canon version, Nikon version

I find that you can get some great prices on used camera gear on Amazon, but it's usually a single item or two priced well-below the rest that's only briefly available, so I thought I'd do something about it. I spent the past three weeks working on my cloud-based, Amazon price-checking engine. It scours Amazon hourly, checking for changes in new, used, and refurbished inventory and prices.

There's no limit to what it can track, but because I had to start somewhere, I just stood up its first two manifestations - Twitter feeds for each Canon and Nikon camera gear.

The bot currently tracks over 1,500 products hourly for each their list, new, used, refurbished, and collectible prices and availability. Adding new products to the database is still a manual step, but only requires a single click per product with a custom GreaseMonkey script. As I find the time, product imports will also be automated, and even personalized.

The two Twitter feeds are at:

Please follow them on Twitter, and tweet any suggestions or other feedback to either of the feeds.

The companion lens-buying guide for Canon is online here, with a Nikon one in the works:

Significant changes in price and inventory are sent through a list of notifiers - any one of those that decides to handle that update will queue up and send out its notifications. Currently, there are two sets of notifications - three (used, new, refurbished) for each Nikon and Canon, which are sent to their respective Twitter feeds with as much info as I can fit in to Twitter's 140 character limit, complete, of course, with a link to Amazon where you can grab that single used lens that's $50 cheaper than the rest.

Since this runs in the cloud, all processing is forked off into different tasks, and has a non-relational BigTable database, the only throttle I have to worry about is my contract with Amazon, which allows me to send out one batch of ten requests per second, or 36,000 products checked every hour. If Amazon wasn't throttling me, Google would allow me process those 36,000 in 3 minutes, and I'm confident the system could handle it.

Of course there are different strategies to handle many more products, but 36,000 should buy me a couple month's time to implement those work-arounds.

These Twitter feeds are only the beginning. I have a notebook full of ideas for directions to take this project. I've spent three weeks with my head in the cloud, and it's only getting more and more fun by the day.

More to come.


Anonymous said...

Hi. It's a nice idea. Is there any chance you could modify it for the UK Amazon store?

Blake said...

Yep, I'd love to. I've been so busy recently that I completely missed that Twitter changed how they're authenticating. The site hasn't been publishing to Twitter for a couple months now.

I'm hoping to bring this back online in the next month, and I'll look into country-specific ideas.


- Blake