Blag o' dkam

New post on Daemons over on the Booko blog.

Just posted on the Booko Blogo an article about implementing Users and Passwords.

Over on the Booko Blog.

When testing new stuff for Booko, I sometimes create a new slicehost and build myself a test box. In the past I’ve done this manually, which really isn’t very difficult, but it turns out that Slicehost have an API to allow you to automate this stuff.  With the API you can create, destroy, rebuild and reboot your VPS which is pretty cool.  It also lets you manipulate all your DNS settings. Check out the API for complete details. 

Here’s a script I put together to build a new dev host for me and set it up so it’s ready to use.  It performs the following steps:

1. Check the Domain name I’ve selected for the host is managed by Slicehost.
2. Create a new 256MB VPS with Ubuntu 8.04 installed.
3. Create the domain names for the host (including on the internal interface if required)
4. Wait for host to build and for Networking and SSH to startup.
5. Add my SSH key to the root user’s account for passwordless login.
6. Update and upgrade the host.
7. Install Puppet  

I plan on having a puppet server setup soon which will take over the rest of the setup, so I’ve installed that. I’ll probably use the Net::SSH stuff to get the host added to puppet (signing certificates and such.)

#!/usr/bin/env ruby

require 'rubygems'
require 'activeresource'
require 'net/ssh'
require 'net/scp'

API_KEY="your_key_goes_here"
SITE="https://#{API_KEY}@api.slicehost.com/"
DEFAULT_TTL = 300

host_name="hostname"
int_host_name="hostname-int"
domain_name="mydomain.com.au"

fqdn = host_name + "." + domain_name
origin = domain_name + "."

##
# Required definitions to access the Slicehost stuff
##
class Slice < ActiveResource::Base
  self.site = SITE
end 

# Address class is required for Slice class
class Address < String; end 

class Zone < ActiveResource::Base
  self.site = SITE
end 

class Record < ActiveResource::Base
  self.site = SITE
end 

##
# Create or update a DNS record
##
def create_host_record(zone_id, host_name, ip_address, ttl = DEFAULT_TTL)
	host_record = Record.find(:first, :params => { :name => host_name, :zone_id => zone_id  } )
	unless host_record.nil?
		host_record.data = ip_address
		host_record.record_type = "A"
	else
		host_record = Record.new(:ttl => ttl, :record_type => 'A', :zone_id => zone_id, :name => host_name, :data => ip_address)
	end
	host_record.save
end

puts "Getting Zone data for \"#{domain_name}\"."
dom = Zone.find(:first, :params => {  rigin => origin } )

raise "Domain \"#{domain_name}\" not found. Won't be able to create host record." if dom.nil?

puts "Domain exists. Creating slice."

slice = Slice.new(:image_id => 10, :flavor_id => 1, :name => host_name)
slice.save

puts "Slice created. Creating DNS records while it builds."

create_host_record(dom.id, host_name, slice.addresses[0])
create_host_record(dom.id, int_host_name, slice.addresses[1]) unless int_host_name.nil?

puts "DNS created. Waiting for host to build and become active."

while slice.progress != 100 && slice.status != "active"
	puts "Host is #{slice.progress}% complete - host status: #{slice.status}"
	sleep 10
	slice.reload
end

puts "Host built. Waiting for host to startup."

begin
	Net::SSH.start(slice.ip_address, 'root', {:auth_methods => ["password"], :password => slice.root_password}) do |ssh|
		puts "Connected to new host. Bootstrapping."

		puts "Creating .ssh directory and uploading public key."
		ssh.exec "/bin/mkdir -p /root/.ssh/"
		ssh.scp.upload!("/Users/dkam/.ssh/id_dsa_omena.pub", "/root/.ssh/authorized_keys")
		puts "Done!"

		puts "Updating apt and upgrading system."
		ssh.exec!("/usr/bin/aptitude update")
		ssh.exec!("/usr/bin/aptitude dist-upgrade -y")
		puts "Host has been upgraded and updated."

    		puts "Installing puppet."
		ssh.exec!("/usr/bin/aptitude install puppet -y")
		puts "Puppet Installed."
	end
rescue Errno::ENETUNREACH
	puts "Host network not up. Waiting 10 seconds, then retrying"
	sleep 10
	retry
rescue Errno::ECONNREFUSED
	puts "SSH not ready. Waiting 10 seconds, then retrying"
	sleep 10
	retry
end

puts "Done! Enjoy your new host #{fqdn}. You can now ssh root@#{slice.ip_address}"</pre>

Got myself an iPhone and so naturally it’s time to get the iPhone interface working. It’s a bit simpler – specifically the cart functionality has been disabled.

Additionally, I did a bit of refactoring which should speed things up.

asdf

I’ve taken Phil’s advice and removed the Javascript from the body’s onload event handler, replacing it with Prototype’s event handler.

    <script type="text/javascript">
    Event.observe(
             window,
             "load",
         <%=  remote_function :url => { :action => "get_prices", :isbn13 => @book.isbn13  },:method => :get %>
        );
    </script>

Guys at the Ruby meeting tonight mentioned another online bookstore — Boomerang Books. I’ve added them to the roster. Let me know if there’s any issues.

Fixed some Booko bugs/interface problems this weekend.

These ones were reported by Niall:

1. Search results report ISBN, but that value is listed as EAN on the next page. Fixed this to be consistent. I use ISBN to mean the 10 digit version, and EAN to mean the post January 1st 2007 13 digit version, although they are both technically ISBNs.
2. Searching for a valid ISBN which didn’t exist in Fishpond or The Nile doesn’t return any results. The spinning thing just kept spinning for ever. This bug was a bit surprising — thought I’d fixed this once before. Probably before a recent refactoring. Damn my lack of testing!

Made the message explaining when the price was last looked up more clear. Reported by Andy.

Improved a fragment caching problem reported by Joel. The problem was (as far as I can tell) that adding something to your cart, then moving to a different page before the cart is updated continued to show you the old version of the cart. This was because I was doing things in this order:

1. Expire Cache for the cart
2. Look up the book to add, then add the book to the cart and do a bunch of lookups and calculations,
3. Regenerate the cart html and send it.

#2 takes a while. If the user navigates away before #2 is done, then the cart html is regenerated without the book being in the cart. Subsequent adding / removing of books will refresh the html and your book will be there. I’ve tried to fix this by shuffling the order things are done to this:

1. Look up the book to add, then add the book to the cart and do a bunch of lookups and calculations,
2. Expire Cache for the cart
3. Regenerate the cart html and send it.

Hopefully this will reduce the problem, but it may not eliminate it.

This Simple XMPP client for Ruby looks like a great way to get some IM loving into an application. I’d love a monitoring system which used IM to alert me to failures. Since IM is not exactly a perfect medium, sending a copy of the message via Email for critical alerts is probably a good idea too.

Could be fun to ask your monitoring IM buddy “Status?” and get a status report. X Emails delivered, Y Emails rejected, Z books looked up on Booko.

Fun.

Well, apparently I’ve not been consistent enough — an important quality when one bags out others for being inconsistent. So — I’ve updated all references to “Bookie” to the now correct “Booko”.

While I was at it I updated the scraping code for Fishpond who have again updated their site. At least they’ve updated it for the better. Compare and contrast the old and new Hpricot XPath code for grabbing the book title and author.

The Old:

book.title     = (doc/"table/tr/td/div/h1").inner_html
book.author    = (doc/"table/tr/td/p[2]/a/font/u").inner_html

The New:

book.title     = (doc/"h1#product_title").first.inner_html
book.author    = (doc/"p#product_author/a").first.inner_html

Much nicer!