Friday, December 14, 2007

deep eager loading with ActiveRecord

I was going through Rails today cleaning up some code and found and interesting thing with ActiveRecord. I have always known it was capable of eager loading of associations. For example, a blog post can have many comments.

Example:


@posts = Post.find(:all,:include=>[:comments])


Well a comment is associated is written by a user and say you want to eager load the user information too. Well it appears that ActiveRecord supports eager loading of associations with associations. Meaning, I can load the user information for each comment associated with a post.

Example:

@posts = Post.find(:all,:include=>[:comments=>[:user])


This is amazing. I had not known about this feature, but decided to give it a try. This saves in time with alot optimizations that I tried with eager loading and caching of associations' associations.

Update: It has been brought to my attention that this actually documented. This either proves I need to read the entire document or it needs to be better organized. :)

Sunday, December 9, 2007

bringing back reality into social networking

I found recently that social networking is no longer limited to the virtual world any more. In an amazing turn of events, people are meeting new people and making professional relationships by meeting face to face. This is all in respect to technology based fields. The days of Facebook, Myspace, and LinkedIn are numbered.

A (real life) social group I found called Jelly is nothing innovated, but something I think I have been missing in my work life. Its a simple concept. Biweekly someone opens up their place to strangers, offers free wifi, and works from home. Its great because people stop by to show what they are working on, from the trivial to the mediocre, but the most important thing is that you meet like minded people. How many people you know wants to leave their place open to complete strangers?

I have slowly learned since graduating college that the working world lacks the inspiration of personal creativity. I have heard the stories of Google that allow you to work on your own projects. I currently work at a start up that allows me work on my own schedule, so I can make time at those points in the day to work on my 'side projects'. Unlike my time in college though, I don't find the environment of people constantly coming up with new ideas and talk to people to throw ideas around. Am I missing something?

The idea behind Jelly is simple and I know its nothing new. It seems to be the simplest thing I've seen in real social networking. There is no one inviting heavy weight VCs, or the rockstar programmer, or the requirement of some sort of meal. One person opens up there place and invited (future) friends over, but with the intention of working on your own stuff. I'm going to have to give this a try. If anyone is interested in the San Diego area.

Friday, November 30, 2007

response to 'Hiring a *buzzword* Developer ... '

There are a lot of blog postings out there with the subject of to hire the perfect developer. The most recent article the sparked my interest was '11 Tips on Hiring a Rails Developer'. That describes a great or *shudder* rockstar Rails developer as a hard to come by. There are many articles like this in the Rails world. Describing how to get the Rails developer that goes to the RubyConf, goes to the local Ruby meetup, and dreams in Ruby.

In short, these articles often leave out the most vital argument of most programmers, engineers, scientists, etc. that have the willingness to learn and expand their expertise. At my current job, before I became the Rails developer/Software Engineer, my experience in Rails was next to nil. Mainly it was me tinkering around with the famous screencast showing how to make a blog. I was hired on as QA to test the currently developing website, but sometimes I would go in and tinker with the code, to make fixes that the other developer didn't have time for. Eventually I became the goto guy for the Rails development and I switched roles within the company.

My point is that there are people out there who are passionate about Python, but can pick up Ruby. There is a Java developer out there learning the potential of languages with Groovy. If you look to hard for the person whose main experience has been everything in one concentrated field, what happens when the next buzzworthy thing comes out? Will they be able to learn it, or will they stick to what they know because it can do everything they need it to do?

My belief if the true rockstar is able to adapt to anything.

*PS - In response to the article that sparked this annoyance, You can hire people with a degree too. I have a Masters in Computer Engineering with concentration on data mining.

Friday, October 26, 2007

capistrano deployment error 'Errno::EACCES: Permission denied - /.ssh'

If you are using a Mac with Tiger and are having an error with capistrano deployment such as:

[deploy:update_code] exception while rolling back: Capistrano::ConnectionError, connection failed for: localhost (Errno::EACCES: Permission denied - /.ssh)

Set your HOME environment variable. i.e. export HOME=/Users/monkeyboy

An hour later, I realized this. Hopefully someone else can find it here.

Monday, October 22, 2007

issues with Flash content within lightbox

The lightbox concept in webpages started around 2 years ago (as far as I can remember). It allows certain content to be focused in on a webpage, so that nothing else can be distraction. Its been used for photo galleries, ajax content, [new form] popup advertisement, etc. In my case, it needed to show Flash content for configurations of content providers (widgets).

Example screenshot (will be provided once I can get the image to upload)

The problem that was occurring was that the Redbox plugin (for Rails) could not provide the correct support for the Flash content. Redbox is still a nice plugin.

The following is a list of issues that were occuring, so that others may learn from my testing with Flash content in lightboxes:

  1. Redbox uses CSS transparency to creat the grayed out portion of the lightbox -- known as the background overlay. There were problems that occurred with it across the browsers with flash content. Mainly if the user tried to scroll the browser the Flash movie would disappear. The transparency problems were resolved by creating a alpha-PNG to be used for all browsers except IE6 that uses the CSS filter property.
  2. Some control was not provided by Redbox to handle the width and height of the lightbox. This was needed because the Redbox plugin sets the div that displays the lightbox to auto height and width, which is great for content such as images and text. Unfortunately, the width and height of the Flash movie do not get considered into this calculation (a DOM problem I assume) of the width and height of the div containing the lightbox. This would cause the lightbox display to be much smaller than it was supposed to be. The width and height need to be set manually.
  3. On the page where the lightbox was being displayed there are scroll bars that are part of a large select box. In some browsers, *cough* IE, like to overlay the scroll bars over all content, completely ignoring z-index. They need to be hidden while the lightbox is being displayed. It also needs to be taken into consideration that the scroll boxes within the lightbox are not removed.
  4. In Safari (and maybe some other browsers), content loaded from an Ajax request into lightbox needs to be removed from the lightbox even when the lightbox 'innerHTML' is set to empty string. This problem would occur if the same lightbox with Ajax request was performed again the content from the previous request would be there again. This is solved by removing the child HTML elements from the lightbox when the lightbox is no longer needed.
  5. Proper positioning of the lightbox to be centered with respect to the current browser windows scrolled at position, not with respect to just the top of the content. This one was pretty trivial to solve as it was a professional design choice for our content. I label it as an issue because it seems that lightbox should work with respect to the user's view and not force the user to scroll to where the lightbox appears in the browser window.
  6. In Mozilla browsers (FF2) on Mac, the Flash movie likes to disappear every once and awhile depending on the condition that there were two flash movies playing in the same window -- one in a lightbox and one in the background. The other condition is that the flash movie in the background had a flashvar 'wmode' set to 'opaque'. Set this 'wmode' to a different value and the problem no longer occurs.
I make no claims to knowing why some of these issues occurred either browser issues, users issues, etc. I just came up with the best solutions I could to solve the problem from not occurring again in the browsers.

With these issues, I took some time and built a Lightbox script that handles these conditions and supports Flash content plus any other content pretty well. I slapped it into a plugin a while back. I tried my best to make it backwards compatible with Redbox as much as possible, so we did not drastically need to change our code, so mostly its updated Javascript to handle the above conditions plus some extensions to the Helpers for width and height.

Note: I would just like to point out that this code is kind of old. Its still in use at my current employer's website. It works well and has proven to be quite stable. At the time of its creation there were no nice Rails plugins with lightbox support except Redbox, which did not work well with the Flash content.

Wednesday, October 10, 2007

timed out fragment caching in Rails

As you can probably tell I am a little into optimizations. In Rails, caching is one of those things that is made pretty easy, but can also be difficult when you want to do complex tasks with the caching -- such as expiring it. There are two ways with expire_fragment within the logic of your controllers or using an Observers. The Observers are useful, but require that much little effort to set things up, which personally I don't want to maintain. The expire fragment was my next target, but yet again a struggle because I had to write logic for it in certain places.

Hence the creation of my next caching mechanism. Rails supports many types of caching like file, memory, memcache, etc. The problem I have found is that memcache is the only that supports auto timeouts of the cache data. Tell me why I cannot have the same functionality built into Rails for the other caching mechanisms? I want to cache fragments in local memory of the server, which require quick access, and are small enough where they don't need to be shared like memcache.

For example I want to cache a fragment for 5 minutes on a page:


This is last 5 minutes of posts from our forums:
<%cache_timeout('forum_listing',5.minutes.from_now) do %>
<ul>
<% for post in @posts %>
<li><b><%=post.title%></b> - by <%=post.author.username%> at <%=post.created_at></p></li>
<%end%>
</ul>
<%end%>


This fragment will timeout after the page that contains it is loaded after the 5 minute mark and then will be reloaded. As you will noticed @posts are loaded, so is_cache_expired? method is provided in the controller and view to make sure if the fragment hasn't expired. This allows data to loaded only when needed.

The following is the code written for the timeout caching. Check back soon for plugin location.

module ActionController
module Cache
module TimedCache
#used to store the associated timeout time of cache key
@@cache_timeout_values = {}
#handles standard ERB fragments used in RHTML
def cache_timeout(name={}, expire = 10.minutes.from_now, &block)
unless perform_caching then block.call; return end
key = fragment_cache_key(name)
if is_cache_expired?(key,true)
expire_timeout_fragment(key)
@@cache_timeout_values[key] = expire
end
cache_erb_fragment(block,name)
end
#handles the expiration of timeout fragment
def expire_timeout_fragment(key)
@@cache_timeout_value[key] = nil
expire_fragment(key)
end
#checks to see if a cache has fully expired
def is_cache_expired?(name, is_key = false)
key = is_key ? fragment_cache_key(name) : name
return (!@@cache_timeout_values.has_key?(key)) || (@@cache_timeout_values[key] < Time.now)
end
end
end
end


module ActionView
module Helpers
module TimedCacheHelper
def is_cache_expired?(name = nil)
return false if name.nil?
key = fragment_cache_key(name)
return @controller.send('is_cache_expired?', key)
end
def cache_timeout(name,expire=10.minutes.from_now, &block)
@controller.cache_timeout(name,expire,&block)
end
end
end
end

#add to the respective controllers
ActionView::Base.send(:include, ActionView::Helpers::TimedCacheHelper)
ActionController::Base.send(:include, ActionController::Cache::TimedCache)


If anyone has any suggestions or comments please feel free to leave them in the comments.

Update: This is the link to the plugin.

Sunday, October 7, 2007

Puddy -- YAWFP

Yet another web framework for Perl, but I like to call it Puddy. A work in progress.

Short Description:

A project based on wanting a web framework built for speed and integration with standards. Currently modeling Rails in the area of application design, but improving areas with modules that are known to be faster and optimized (because I say so).

Model uses Rose::DB::Object for ORM support. It will handle to the association with the database that is well designed -- meaning the database handles the relationships not the ORM.

View uses a custom built XML::Builder and EPB that will handle templating that make incorporating Perl code into document easily. There is no language better than the langauge the application is built to use for the View. I am basing off the fact that out of my personal experience I have never met a 'designer' that just use a simple templating language.

Controller... well a work in progrees.

If anyone is interested in helping please message away at /[jtarchie][at][gmail][dot][com]/.

Thursday, October 4, 2007

XML fragment caching in Rails

Rails has been nice to use. The most powerful part of it is the templating engine for RHTML and RXML. A problem that exists is that they do not share some of the same functionality. Both can execute Ruby code, and output specific content, access helpers, etc. Yet, RXML lacks the ability for fragment caching.

The problem exists for fragment caching because it was especially designed for ERB fragments. ERB allows Ruby code to be use as a templating language -- the same way PHP does for HTML. The nice thing out ERB is that everything evaluated and rendered is plain text, but the caching relies on simple tricks with Ruby's eval to get the output of a block of ERB. The caching method cache (for views) even uses a function called cache_erb_fragment.

I use RXML alot, so the benefits of fragment caching should be available as they are for RHTML. Especially with the slow rendering time of the RXML documents (a future post). RXML relies on overloading of a method using method_missing to help create a DSL for an XML document, in short, Ruby code gets outputted to XML.

An Example RXML document:


directors = ['George Lukas','Steven Speilberg','Michael Bay']
xml.instruct!
xml.directors do
directors.each do |name|
xml.director :name=>name
end
end
With the output as:

<directors>
<director name="George Lukas"/>
<director name="Steven Speilberg"/>
<director name="Michael Bay"/>
</directors>
A very simple example, but lets pretend the each director is dynamically loaded and that there is more information like age, location, and the last film they made. Information that will not change much till a movie is made, they get older, or they move. Never know when a director's already render XML might be needed again.

It has actually proven to be quite a task to do XML caching. It appears to be a subject neglected in the Rails community -- any idea why? I have posted on forums here and here about the topic. I had some preliminary code with those posts too, but I was having problems.

I would like to give back my final results of XML caching. Something that I hope is useful to some.


The following is the code use for the xml caching:

module ActionView
module Helpers
module CacheHelper
def cache_xml(name ={}, &block)
@controller.cache_xml_fragment(block, name)
end
end
end
end

module ActionController
module Caching
module Fragments
def cache_xml_fragment(block, name = {}, options = nil)
unless perform_caching then block.call; return end
buffer = eval("xml.target!",block.binding)
if cache = read_fragment(name, options)
buffer.concat(cache)
else
pos = buffer.length
block.call
write_fragment(name,buffer[pos..-1], options)
end
end
end
end
end
It works just like normal caching in RHTML documents. This is an example that can be used with the directors XML provided above.

Example Usage:

directors = ['George Lukas','Steven Speilberg','Michael Bay']
xml.instruct!
xml.directors do
directors.each do |name|
cache_xml("director/#{name}") do
xml.director :name=>name
end
end
end
I hope to make this into a plugin very soon. Its easy to copy and paste for right now.

UPDATE: The plugin can be found here.

Saturday, September 29, 2007

Experience with the web (Part 1?)

I have been using the web since '94 in the 1900s. I wasn't apart of its original inception because I was too little and naive to know it even existed. Except for BBSs, those rocked, it still amazes me what an ANSI artist can do with TheDraw.

Currently, I am working for a company using Ruby on Rails as my primary development environment. Everything for the company is being written RoR. Its been interesting to use a framework to this extensive magnitude because you truly realize where its faults and advantages. With the changes, limitations, and additions I have worked into our implementation of RoR I thought that it would be nice to share in the case others might find it useful.

I claim to be no expert to programming. Anyone who make a claim to be an expert at anything better be ready to back it up. I will admit that I have experience though whether it be professional, amateur, or just plain tinkering. A lot of my interest in computers has always been the web. I like the power it has to be accessed by so many people from so many different places.

My first real website was way back in middle school. It was my play ground. Where everything from rants, drawings, hosting of others webpages, etc... were presented. It was my play ground for many year up till high school. Where I unsuccessfully tried starting my own cult. Unfortunately, a one person cult, well, is just lame.

From this website though, I picked up my first CGI scripts from Script Archive. This guy in my opinion really created the interest of CGI programs to provide web pages with interactivity and also was kind enough to lend his source code for everyone to use. This is what sparked my interest in Perl.

My most successful website to date incorporated alot what I had learned, but albeit still very crude. It was a fan site of South Park called killkenny. It had some of the copied content from many other fan sites, but also some original. I setup a weekly quiz, created with a friend of mine, about strange things we found in the episodes. This proved to be very popular because to date we had 100,000 unique hits / month. Keep in mind this is before Google Ads, so we made no money. It was for fun. Eventually it became, like other things, to difficult to maintain and was retired. From this I did the web design, content, hosting management, and interactivity B.A. (before ajax), during the year 1997 or 9th grade.

Well I am going to take a breather because this is much more than I intended to write. I might continue writing about this at a later time, but I would really like to write about issues I have faced with respects to web programming and show my solutions or questions.