Rails and Amazon: Libraries and Plugins for EC2, S3, SimpleDB, SQS, FPS and DevPay Integration
Unless you've been hiding somewhere, you're probably aware that Amazon has built up a comprehensive cloud computing infrastructure featuring a raft of pay-as-you-go services. What you may not know is just how easy it is to integrate most of these services with Rails. Between Rails-specific plugins and Ruby projects, you can quickly take advantage of the services that Amazon offers. Here's a survey of the available integration pieces.
EC2 is Amazon's Elastic Compute Cloud - the overall image-based supplier of computing power. One way to manage these images is with the amazon-ec2 gem, which gives you full access to the EC2 API. This lets you control server resources on demand right from your application code. Alternatively, if you're looking for administrative control as part of a deployment process, take a look at capsize, a Capistrano extension for managing EC2 instances.
S3 is the Simple Storage Service - a web services interface that can be used to store and retrieve data to arbitrary buckets. There are a number of projects - such as Paperclip - that can optionally use S3 storage. For glue, many of them use the aws-s3 gem, which wraps S3 for any Ruby application. You might also find a use for s3storage, a dedicated plugin to put anything from Rails to S3. Another way to get at S3 data is with cliaws, a command-line client that speaks to both S3 and SQS.
SimpleDB is the core database functionality provided by Amazon, using a flat dataset (instead of relational) approach. amazon-sdb is a Ruby wrapper around SimpleDB. Because Amazon provides a RESTful API, the wrapper is fairly straightforward, although the project itself may have gone quiet before actually getting finished. aws-sdb is another attempt at a SDB wrapper that appears to be somewhat more up to date.
SQS, the Simple Queue Service, is the messaging infrastructure that can be used to communicate between EC2 and other web services. There's a gem, rufus-sqs, the lets you write to and read from SQS queues.
Flexible Payments Service (FPS) is Amazon's e-commerce infrastructure, letting you manage everything from micropayments to straight credit card and bank account billings. On the Rails side, amazon_fps_foo integrates FPS, though this is a new project that will need to develop a bit more to be really usable. An alternative is remit, which provides a general REST wrapper for FPS in Ruby code.
DevPay is Amazon's other billing project. DevPay uses Amazon Payments as the backend for an online billing system, letting users move from your site into the Amazon pipeline. If you want to use DevPay from Rails, check out the devpay plugin, which makes the whole process simple.
If you're considering payments, you should also know about Simple Pay, which is yet another way that Amazon helps you take in money. It's not part of the AWS system, coming from the Payments folks instead. Rails developers can get to Simple Pay via the simplepay plugin.
Finally, if you're seriously interested in using the Amazon Web Services infrastructure, check out the PoolParty project. It gives you a slick configuration file based way to build out your cloud and integrate with other services.
December 10th, 2008 at 7:20 am
What about RightAWS for EC2, S3, SQS, and SDB? It is a well-maintained alternative with some cool features like persistent connections and threading support.
http://rightaws.rubyforge.org/
December 10th, 2008 at 10:25 am
+1 RightAWS - I've found it to be the most well maintained for working with SDB. ActiveRecord2SDB is also worth checking out (http://activrecord2sdb.rubyforge.org) -it's built on top of RightAWS.
December 10th, 2008 at 11:22 am
Thanks, guys! I don't know how I missed including RightAWS, considering that I've used it in some of my own projects. And thanks for the pointer to ActiveRecord2SDB, Jonathan; I'd looked for that, but didn't find it. Looks a bit immature now but definitely promising.
December 10th, 2008 at 1:28 pm
Yeah I'd also recommend the RightAWS gem, it works really well for us and they're pretty good at updating it regularly
December 10th, 2008 at 3:27 pm
I've had problems with aws-s3 for lots of large files, started running out of memory. In the end I resorted to s3-bash, way faster and no wierd memory problems.
December 10th, 2008 at 4:19 pm
Is there a recommended machine image that is ready to go with rails and everything else that is normally needed to deploy a new rails app?
Thanks...!
December 10th, 2008 at 4:43 pm
Great stuff to add to my list. Thx...
Here is a list I put together the other day (it's not just Ruby. It includes other tools as well).
http://www.johnmwillis.com/other/tools-to-use-in-the-cloud/
John
johnmwillis.com
December 10th, 2008 at 7:07 pm
Another +1 for RightAWS. I've been using it with great success to integrate with EC2, S3, and SQS. It seems to be very stable and well maintained.
December 10th, 2008 at 7:09 pm
RightAWS is the way to go, for sure. It works very well.
Also not mentioned is the activemessaging gem, which can deal with ASQS, but I found it to be cumbersome to work with. RightAWS makes ASQS programming a breeze.
December 11th, 2008 at 1:12 am
Actually an small correction on Paperclip:
http://github.com/thoughtbot/paperclip/tree/master/lib/paperclip/storage.rb#L107-109
As you can clearly see there, it relies on RightAWS, not aws-s3.
And as was mention by Michael on first comment, is pretty good on what it does.
December 11th, 2008 at 8:48 am
If you want a free alternative to SimpleDB that looks and behaves identically (making it a useful test platform for Rails developers, for example), check out M/DB at http://www.mgateway.com/mdb.html
December 22nd, 2008 at 11:18 pm
Don't forget:
http://rubyforge.org/projects/amazon-fps-ruby
June 11th, 2009 at 8:29 am
Thank you so much for this post. I use the reviewazon plugin. Reviewazon is the easy and simple way to add amazon affiliate product to your website.