Sunday, November 16, 2008

Sinatra speaks a different language, Perl

So, if it has become obvious to some, I have a thing for web frameworks. I like to understand how and why they work, and for me that usually involves more than just reading the source code. My first attempt was different variations of Puddy -- Rails/Merb like framework in Perl. The second iteration has led me down the path of the Ruby Sinatra framework.

Sinatra is a minimalist framework for creating web applications. Its scope really only extends into realm of controllers. It does support views, but it is still far from the enormous support that Rails would provide in options. The controller aspect allows you to define the route in method call of the action -- often causing your entire application to be just one file. I have used it to experiment with building APIs for some of my projects.

In short, its a light, quick, and not at all a memory hog, so of course I wanted to dissect it. This lead me to just reading the source code, but I wanted to do more to understand how it worked. I should inform you that I do indeed know Ruby. I've used it on a daily basis for the past 2 years for my job(s) and have used well beyond just the Rails environment. Now, I have also been a big supporter of Perl, mainly because it was my first language beyond good old Q-BASIC.

With that in mind, I would like introduce a project that I have been working on Sinatra for Perl. Yes, it is a work in progress, but I have put a lot of effort into making sure that this code base was some what solid before I released it to the world. In the repository, a working example can be found and run on your local machine of the feature set.

I like the ease of of extending Sinatra in either Ruby and now Perl. I have added simple extensions to support page caching, running tasks from command line, and a simple background job server. Many of these are lacking optimizations, but I just wanted to show the ease of extending the framework.

The simplest example of a Sinatra app:

#!/usr/bin/env perl
require 'lib/';

use strict;

get('', {}, sub{
return 'Hello World';

get(':name', {}, sub{
my $r = shift;
return 'Hello, ' . $r->params->{name};
A simple DSL that defines what action to perform on a route. LOVE IT!

This has been an educational experience (albeit a nerdy one) and I hope to continue on this project. I think I will probably have to go through a name change in the near feature as not to confuse people on the Ruby version.

Monday, November 10, 2008

rails plugin to white label CDNs

This is a continuation of my previous post talking about using S3 as a CDN. This post covers some of the issues that were faced with making Rails work nice with a CDN.

Rails is made to do great things, but as many before have me said, handling concurrent connections is not one of them. Once a request comes into Rails, that process (Mongrel, FastCGI, etc.) is blocked till the request is done. Actions like sending emails, transferring files, large calculations need to be pushed away from the user request into another process. There are many solutions such as BackgrounDRb, Starling, etc, which allow you to load long running tasks from blocking Rails.

The task of handling files on a remote server is always a tricky one. Each CDN has there own interface on how to interact with the files -- delete, update, move, etc. This proved to be a problem when trying to test which one would work cleanly with our setup -- widgets stored in the database, which are updated immediately to all users on the Chumby network.

I took a top down approach to the problem. Designing how I wanted the widgets to move from our servers to a another server by building the API of methods. These methods were just a skeleton and did nothing. It just allowed to me to write the code I expected instead of working around another CDN or modules API. The methods could then be filled with the appropriate code to make it work with the CDN.

Originally, the CDN of choice was S3 -- not a true CDN, but suited our purposed of unloading our servers of loading dynamic content from our servers. The necessary API calls were filled to support the functionality of S3. Its then I realized that with the top down approach that the work I had could be easily modified to work with any CDN method. I've written extensions for rsync, ssh, and just S3, but it has supported our needs to be able to test multiple environments and services quickly.

This plugin data_fu is currently a work in progress, but I hope to modify for wide spread use and finalize it.

NOTE: At the time of writing this software that there are many solutions of CDNs that handle transparent proxy and caching of content. The reason these solutions were not used because the content of widgets need to be updated instantly. Since the solution was to use vanilla CDN mechanism so we could change CDN solutions if one went down and switch over was needed immediately.