Tony's Place Random thoughts

My experience with Rackspace Cloud Sites

I've been using Rackspace's Cloud Servers for months now and I thought moving some of our standard PHP apps (like wordpress) to Cloud Sites would save me some time as a sysadmin/developer.  I also figured I would setup a database there and use it for my Rails application that runs on Cloud servers (instead of building my own or using Amazon's RDS).

I'm sorry to say that after a few days of working with Cloud Sites, I think managing the sites myself on Cloud Servers would be easier.   Fortunately, many of the problems I am having can be easily fixed.

My issues

1) No rsync or scp access.  Installing a wordpress site via FTP or Rackspaces file manager was just plain painful.  The file manager didn't allow me to move files and it always extracted zip files to the root directory.  Maybe it works correctly on IE?   I only run linux so I have no way of knowing.

2) No easy way to do backups.   Cloud Sites allow you to run cron jobs but the example backups build tarballs on the same host that the site is running on.  When I saw they had ruby as a cron shell option I assumed they had the CloudFiles gem installed,  but they didn't.  I want my backups off moved off the host.  I don't want to log into my account and download them manually.

3) No access to database binary logs.  I like to take a snapshot every 15 minutes or so in case I lose my database but this is not an option with the cloudsites database.  You could do a mysql dump from another host but you probably don't want to do this every 15 minutes.

4) For my Cloud Server rails app there was no LAN address to access my Cloud Site database so all my queries incurred bandwidth charges.

Feature Requests (easy to hard)

1) Give people a cron job (or simple backup tab like cloud servers has) that can be used to dump their database daily to Cloud Files.  Its a win for everyone.  The user gets automated backup and you get more Cloud Files revenue.   Here is a script I run from a Cloud Server to back up my Cloud Site database nightly (I can not run it as a Cloud Site job as the CloudFiles gem is not installed on the hosts..  as i mentioned above).

def run(command)
  result = system(command)
  raise("error, process exited with status #{$?.exitstatus}") unless result
end

cf = CloudFiles::Connection.new(@account_name, @cloud_key)
container = cf.container(@directory_name)
cmd = "mysqldump -C --opt -h #{@mysql_host} -u#{@mysql_user}  "
cmd += " -p'#{@mysql_password}'" unless @mysql_password.nil?
cmd += " #{@mysql_database} | gzip > #{@backup_directory}#{@db_file}"
run(cmd)
t = container.create_object(@db_file)
t.load_from_filename "#{@backup_directory}#{@db_file}"

2) Advertise the LAN address of the Database Hosts...  of course the address given now is probably a switch that hits several DB machines.

3) Fix the File Manager.   Maybe it's just me but on Firefox/Linux moving files doesn't work.  A crippled version of rsync may also be nice.  I suspect there are security issues here but shell access sure would be nice.

4) This is probably hard but it would be nice if I could get access to the MySQL binary logs.

Thanks for listening.  :-)

Why I chose The Rackspace Cloud over AWS

Last October at BarCamp DC 2 I ran a session called  “To Cloud or Not? AWS, EC2, S3 or build your own“.  Unfortunately the barcamp wiki died and my notes are gone but at the time it seemed that everyone loved Amazon's services.   I tried using EC2 in April and while the ablity to select from several pre-configured AMIs was nice, building your own AMI should have been easier.  I wanted to configure my machine and then push a button to have my image created.   With Amazon you needed to install tools and go through several steps to create an image.

Then I found Slicehost.  It was owned by Rackspace and had servers for as little as $20/month (for a 256MB instance).   A few weeks later I stumbled on Mosso, also owned by Rackspace and it had servers for about $11/month (plus bandwidth).   Since my applications were using very little bandwidth, I moved to Mosso which is now called The Rackspace Cloud.   With the Rackspace offerings you install your operating system image, configure it and then, from their control panel you can then back it up with 1 click.  You can also schedule backups.   This was so much easier than EC2.

Then there is the pricing.  Amazon's small instance is a big vitrual machine and at $0.10/hour it runs around $70/month (i think it was 0.12/hour when I 1st started using it).    This is probably a good price if you need that much horsepower.    What could you possibly run from a 256MB instance anyway?  Here's what I am running.

• A full rails app using Apache/Passenger and MySql (I had to remove several unused modules from apache config and my database is small at the moment)
• Apache PHP -- I don't have a database here but I suspect there is room

I suspect a 512MB instance a safe bet for most applications and I will lilely upgrade as my traffic and database size increases.   Depending on the situation, I may just spin up more instances of the same server as redunancy is a good thing.   Sure I could run everything on 1 AWS instance but if it dies I'm really SOL.

If you ever need a bigger slice you can upgrade in the control panel with 1 click.  All your configurations and IP address are kept the same. I usually make a backup (1 click) before doing this just in case something bad happens.

Rackspace is still making improvements to their APIs and Image Management so while they don't offer as many services as Amazon, they have offered all important features to make developer's lives easier, IMO.

For the record, I actually backup my Rackspace Databases to Amazon's S3, I feel better knowing my backups are in a different datacenter.  :-)

If you sign up (and found this post helpful) please use my referral code when creating your account.  It is REF-TONYCODE

Good Times at BarCampDC2

On Saturday, I spent the day at BarCampDC2.    Like last year there were plenty of great sessions.  I really wanted to discuss Amazon's EC2 as I think it is where most small companies should be moving their sites and I see huge business opportunities in this space.   As the session board was being built I kept looking for something on Amazon's EC2, S3 or AWS.  I had not used the services yet, but I was determined to discuss them as I know many of the companies present were using them.   In desperation, I grabbed a pen and a post it note and wrote "To Cloud or Not? AWS, EC2, S3 or build your own".   I owned the session and I hoped I would have a few experts in the room so I could act as a moderator instead of presenter.

The room was packed and I started out by telling everyone that I had no presentation or experience with these technologies and hoped we had some experts in the room.   As I expected there were plenty of experts in the room and we had a great discussion on what Amazon had to offer and other offerings that companies can leverage.

The room was filled with some of the greatest minds from the DC Tech scene.

More pics are here.

My notes from the session are here. Nikolas Coukouma helped clean them up and added some additional pointers.  

After my session I attended several hard core geek sessions, as usual there were many sessions I was unable to attend.   Maybe we can videotape the sessions next time?
I attended

• 11AM - Nitrogen Web Framework
• 1PM - Git
• 2PM - MySQL Optimization
• 3PM-5PM wandered through various sessions.
• 5PM - Beer!   Great presentation by Chris Williams.   I thought this was an early happy hour room but instead Chris schooled us on the history of beer.

Afterward we headed to McFadden's where I consumed several pints of  Guinness.   Fortunately I had taken the bus and metro from my house so getting home was a non-issue.

Thanks to Center for Digital Imaging Arts at Boston University for letting us have the conference there and thanks to all the folks that helped put this together.   Can we do this again in 6 months?  :-)

Building Bebo Apps. Part 2 (Auto-Updating the Profile Page)

Handles are the key to having applications update on the profile page without a user taking action. Some notes on how to do this on Facebook are here. Since Bebo has implemented most of the Facebook API this should be easy but I think some things (like Infinite session keys) are still missing. Hoping someone will leave me pointer proving me wrong on this. ;-)

By placing a handle on a user's page you can (in theory) push updates to a user's profile page without them interacting with the application. For example. If you had a "Washington Capitals Fans" application that pushed news to every user's profile that had the application, all these pages would have the same handle. Having a single handle means that a single update of the handle content would update all profiles using the application.

In other cases you might want a unique handle per profile. For example, if your application injected an RSS feed of the user's choice onto their profile.
Here is a simple example of setting a handle on a user's page. I use the user's UID as the handle in this case.

Set a handle on a user's profile

I then write an update.php file that injects the date on the user's profile

Update a handle on a user's profile

That's all fine and good but the update wasn't automatic. The user had to hit the update.php file. In a real application we would have a database of all the users and we could periodically update the handles. Assuming there is an infinite session concept in the Bebo API. Since I could not figure out how to implement an infinite session here is my hack.

I place a hidden image on the user's page that calls the update function.
Now when a user views their profile, the hidden image calls refresh.php and the handle content is updated. Unfortunately, the update happens after the page is rendered so you'll need to reload to see the change.

The code...

Set a handle on a user's profile and use hidden image for update

Unfortunately, the update happens after the page is rendered so you'll need to reload the page to see the change. It should also be noted that anytime someone views your page they will call the update.php. If they also have the application installed your handle will be updated.

Assuming infinite sessions keys are unavailable I would leverage app users to update their peers. In your database of users keep track of the last time they were updated. Anytime the update.php function is called do an update on all handles that haven't been updated in a specific period of time. As the user base grows this should allow for a more even distribution of updates. This might even be better than the cron jobs that infinite session key users use in their cron jobs.

Building Bebo Apps. Part 1

Last week I spent some time learning how to build Bebo applications. Their API is just like the Facebook API but it is not complete and the documentation has some holes in it. If you haven't built a Facebook application the entire process can seem awkward.

The Bebo Developer Page makes it sound like a simple 3 step process

• Get the library
• Install the app
• Start Creating.

If you don't know how to write web applications in PHP, Java or Ruby stop now. You need to know at least one of these systems to continue.

I downloaded the PHP library and put it on my host. Next step was to install the developer application. Why? Why do I need to install an app to build apps? Seems weird at 1st but basically this application is used to manage the keys, locations and settings for your apps.

Once you have the app launch it and click "Create new app" (way off on the right) . Give your application a name and a URL and a callback URL.

What is the callback URL? It is the URL that is hit when a user goes to the Application URL. http://apps.bebo.com/yourappname is just a proxy that adds headers and parameters to the request before it hits your application file. The bebo library uses this information.

For example. I created an app called sample with a URL of http://apps.bebo.com/sample. My callback URL is http://bebo.tonycode.com/apps/sample

Hitting http://apps.bebo.com/sample results in a call to http://bebo.tonycode.com/apps/sample where I have the following index.php

Display_All_My_Friends

This application simply prints out all your friends.

Add the application to your page by going to the developer application, viewing the app and clicking the profile page for the app. Now go to your profile page a voila.. the app is there but not your friends.

Huh? How do I get the content to show up on my profile page? If I don't show it on the profile page it seems pretty pointless. right?

To add content to the profile page you use profile_setFBML. Last I checked the return codes for this function are not defined. Seems to return a 1 when it works and an array when it fails. Nice. :-\

Here is the code updated to put friends on profile too.

Display_All_My_Friends_on_my_Profile

Weeks go by and you add new friends to your profile and you notice that the list of friends in your custom application aren't changing. profile_setFBML is only called when you visit the application page. Well that stinks you say. How can I make it update each time I visit the profile? I'll get to that in my next post.

Random Tips.

• Documentation is at http://developer.bebo.com/documentation.html
• The downloaded libraries have example code. Start there
• You may need to look at 2 different sections to piece together how a function works. For example user.getInfo has a fields parameter
• http://www.bebo.com/docs/api/UsersGetInfo.jsp
• Look at the SNQL users table to find what is available
• http://www.bebo.com/docs/snql/User.jsp