Ruby 1.8 strftime works differently on Linux and Windows

Running Ruby 1.8 on Linux, cygwin, Mac, any POSIX environment:

now = Time.now
=> Mon Dec 27 18:28:28 -0500 2010
now.strftime("%l") # that's a lowercase "L"
=> " 6"

Under Windows using the exact same Ruby version and patch level:

now = Time.now
=> Mon Dec 27 18:28:28 -0500 2010
now.strftime("%l") # that's a lowercase "L"
=> ""

What? It turns out in Ruby 1.8 they cheated by simply calling the standard C strftime() function to implement Ruby’s strftime method. Because different compilers implement strftime() differently, compiling it under Windows results in a different set of supported % directives. Specifically, it looks like under Windows only the basic ANSI C version is implemented, but under POSIX compilers you get a mix of directives defined by ANSI and other standards like Gnu and Single Unix Specification. The “%l” directive was not even listed in the Ruby 1.8 docs — it worked by accident.

The good news is in Rails 1.9 they appear to have implemented their own strftime for consistency, instead of handing it off to the C compiler. If you need to fix this in the meantime, I provide the following monkey patch (which I simply place in my Rails config/initializers folder on any Windows developer machine):

# monkey patch Ruby strftime methods to implement
# %l directive that is missing in Windows Ruby
#
# as a monkey patch, just use this on your Windows
# development boxes to bridge the gap, it's not
# recommended for production
#
# notes:
# - you may want to add support for other missing directives
# - check your RUBY_PLATFORM and make sure it is in the regexp below
#
if RUBY_PLATFORM =~ /mingw32|mingw64|mswin32|mswin64/

  class Time
    alias_method :original_strftime, :strftime
    def strftime(fmt)
      hour12 = "%2d" % ((hour + 11) % 12 + 1)
      original_strftime(fmt.gsub(/%l/, hour12))
    end
  end
 
  class Date
    alias_method :original_strftime, :strftime
    def strftime(fmt)
      hour12 = "%2d" % ((hour + 11) % 12 + 1)
      original_strftime(fmt.gsub(/%l/, hour12))
    end
  end

end

Ruby SystemTimer gem for Windows

SystemTimer is a Gem that replaces timeout.rb and fixes some potentially serious flaws when accessing system resources (e.g., when communicating with remote APIs) in Linux/Posix environments. In a nutshell, timeout.rb uses Ruby pseudo-threads (“green threads”) that are not guaranteed to be given the same equal opportunities in the CPU as real system threads, resulting in synchronization problems. Visit http://ph7spot.com/musings/system-timer for all the details.

If you end up needing to use SystemTimer on your production environment and you develop on Windows, you have a problem. SystemTimer’s documentation says that it can be installed under Windows (providing the same interface for your code but not really changing anything — it just acts as a wrapper to timeout.rb). In practice, however, trying to “gem install SystemTimer” results in errors building the native extensions even if you have the RubyInstaller DevKit installed.

Solution? Simple! I don’t know who ghazel is but my hat’s off to them. They took the time to branch the SystemTimer project, patch it to install under Windows and simply provide the wrapper classes without trying to compile anything, and put it up on RubyGems.org.

Visit http://rubygems.org/gems/ghazel-SystemTimer or just “gem install ghazel-SystemTimer” and you should be good to go on your Windows development box in about two seconds.

The Health Care Reform Math Game

Normally I don’t do political stuff on my blog, but I shut down my rants & raves blog because I’m too busy coding and this is too entertaining to keep to myself.

Let’s play the Health Care Reform Math Game! To be more specific let’s horribly simplify the economics of health care reform in a way that would shock the pants off your average economist, just to see where it leads. Hey if conservatives can simplify it to one number (“health care reform will cost us a trillion dollars and bankrupt the country, oh noes!”) then I am allowed to grossly simplify it to a series of averages and swags, fair enough? Let’s play!

So big bad Obama spent 1 trillion on health care. Pretty bad huh? He is evil and should be in jail or sent back to Kenya where he’s really from, etc. Well, it is spread out over 10 years, so let’s call it 100 billion per year. Of course, we don’t intend to pay that all at once — it would probably become part of the deficit but just for fun let’s say we need to pay it off and not hand this monster of unholy evil to our children who will suffer the folly of our communist dictator.

There are about 150 million working Americans, so on average each of us would have to cover a terrifying 666 dollars per year to make this reform happen. Ohh noes! 666! That proves Obama is the devil! No, wait — that’s just an average. The rich would pay more and the poor would pay less — phew, may have dodged a bullet there. So this horrible abomination shoved down our throats that no one wanted nightmare basically amounts to $55/month for each working American to guarantee better access to health care and reduce health care costs by making sure everyone pays into the kitty instead of taking a free ride on Johnny Taxpayer’s nickel.

Wait a second… $55/month doesn’t sound very, well, socialist. Want to know what my health insurance costs me per month? $1,200 per month for my family. Should I be worried about $55 more to guarantee that if someone ended up with cancer the insurer wouldn’t dump us out on the street? Well, it’s the principal of the matter really — Obama and Pelosi are definitely criminal, evil, lying, socialists who should be in prison because any amount of money taken from the American people without their consent is wrong, even if it is to help the very people who whine about it like kicked dogs.

And hey if my math is wrong, show me your math instead of name-calling and whining! Game on!

(Seriously, feel free to correct me, but you can’t argue with the basic point that minimizing a complex economic issue into a single number and hyperbolizing that it will bankrupt the country is just plain wrong.)

Action and reaction: event-driven AJAX updates with jQuery

As I increasingly use AJAX to let users perform actions in a page without navigating away, I find myself constantly having to solve the problem of updating page content that depends on these actions. For example, a user logs in with a popup bubble, then parts of the page that depend on login state must be updated. This is a relatively trivial example and there are many approaches to handling this sort of update.

A straightforward method that is commonly used is to avoid AJAX in these cases–submit the login form and redirect back to refresh the entire page. While using AJAX sparingly is a worthwhile goal, avoiding it entirely doesn’t always result in the most engaging user experience. Another approach is to hard-code everything that needs to be updated after an action is complete:

// submit my login form with jQuery .ajax()
$.ajax({ url: login_url, data: login_data, complete: function() {
  // update page contents that depend on login state
  $('#mainmenu').load('mainmenu');
  $('#login_form').fadeOut('slow', function() {$(this).remove()});
  $('.welcome_text').show();
});

Fine for one or two actions, but this turns into a spaghetti mess pretty quickly in more complicated situations. A better solution is to have some way of registering update callbacks, e.g.:

$('#mainmenu').updateOnEvents('login', function() {
  $(this).load('mainmenu');
});

$('#login_form').updateOnEvents('login', function() {
  $(this).fadeOut('slow', function() {$(this).remove()});
});

$('.welcome_text').updateOnEvents('login', function(){
  $(this).show();
});

$.ajax({ url: login_url, data: login_data, complete: function() {
  // globally trigger the event 'login' somehow
});

So the question becomes, how do you globally trigger the event so any element at any level can register a handler? After considering a number of designs involving .live() or .delegate() (so the handler setups would affect both existing elements and new elements added later) I realized that good old .bind() would do the trick.

Solutions involving .live() and .delegate() are tricky because when you use something like $('.foo').live() or $(document).delegate('.foo'), you need to use $('.foo').trigger() for the event to be handled. Calling something higher-level like $(document).trigger() does not invoke the handlers. The trick is to roll your simple .live()-style function similar to the following:

$.fn.updateOnEvents = function(events, callback) {
  $(document).bind(
    events,
    { selector: $(this).selector, context: $(this).context },
    function(event, trigger_data) {
      $(event.data.selector, event.data.context).each(function(){
        if (typeof callback == 'function') {
          extra_data = callback(event, trigger_data);
        }
      });
    }
  );
};

// example usage
$(document).ready(function() {
  $('.welcome_text').updateOnEvents('login logout', function(event, data){
    $(this).text(data.message).show();
  });

  $(document).trigger('login', { message: "Welcome back!" });
  $(document).trigger('logout', { message: "See you next time." });
});

The trick is that when we call $('.foo').updateOnEvent(), the selector string ‘.foo’ is saved (and the context, if specified) and are passed to the handler function in event.data. The handler uses the selector and invokes the callback function for each matching element with .each(), so the value of $(this) in your callback is the element itself instead of $(document). Since the selector is evaluated at the time of the callback, any recently-added elements that match are included in the update.

The parameter trigger_data is whatever extra data you pass with the .trigger() call and your callback can take advantage of that data.

Commonly-used patterns can be expressed as a shortcut in the code. For example, in this variant if the callback parameter is a string instead of a function, the code treats the string as an AJAX URL and assumes you want to update the selected elements with .load():

$.fn.updateOnEvents = function(events, callback) {
  $(document).bind(
    events,
    { selector: $(this).selector, context: $(this).context },
    function(event, trigger_data) {
      if (typeof callback == 'function') {
        $(event.data.selector, event.data.context).each(function(){
          extra_data = callback(event, trigger_data);
        });
      } else if (typeof callback == 'string') {
        $(event.data.selector, event.data.context).load(callback, trigger_data);
        });
      }
    }
  );
};

// slightly absurd example of using AJAX to retrieve a welcome message
$(document).ready(function() {
  $('.welcome_text').updateOnEvents('login logout', '/ajax/welcome_msg');
  $(document).trigger('login', { type: "login" });
  $(document).trigger('logout', { type: "logout" });
});

In yet another variant where I always want to update the element with AJAX via .load(), instead of passing a callback function to perform the updates, I passed a callback function that returns the parameters for the .load() call — in this way, the .trigger() function doesn’t need to know anything about the parameters needed by the update callbacks:

$.fn.updateOnEvents = function(events, ajax_url, params_callback) {
  $(document).bind(
    events,
    { selector: $(this).selector, context: $(this).context, url: ajax_url },
    function(event) {
      var data = extra_data;
      $(event.data.selector, event.data.context).each(function(){
        // get AJAX request parameters from callback function
        if (typeof params_callback == 'function') {
          data = params_callback(event);
        }
        // convert params to string so .load() always uses GET method
        if (typeof data == 'object') {
          var new_params = "";
          for (var key in data) {
            if (new_params.length > 0) new_params += "&";
            new_params += key + "=" + data[key];
          }
          data = new_params;
        }
        // update the element with AJAX
        $(this).load(e.data.ajax_url, data);
      });
    }
  );
};

// another slightly ludicrous usage example
$(document).ready(function() {
  $('.welcome_text').updateOnEvents('login logout', '/ajax/welcome_msg', function(event){
    if (event.type == 'login')
      return { type: "login" };
    else
      return { type: "logout" };
  });
});

I have to profess that I am not a jQuery god (yet). I am certain that there are improvements that could be made to this function in terms of performance and simplicity, maybe a potential error or two, or a completely better way to approach the problem. I welcome your suggestions, comments, and criticism!

jQuery fadeIn / fadeOut problem in IE

Say you have a series of DIVs displayed side-by-side with the CSS style display:inline, and you want to use jQuery fadeIn and fadeOut to make the appear and disappear. This will not work under IE8, and I assume other versions of IE. To fix it, give your elements style="display:inline-block" instead.

Some people have reported that using absolute and relative positioning of elements causes jQuery animation problems in IE, but I have yet to see positioning cause any problems. I very commonly use a position:relative DIV, then position:absolute elements within the DIV (probably rely on it too much, never claimed to be a CSS designer!)

UPDATE:

I ran into a similar problem under IE8 where a DIV could be hidden and displayed with jQuery .hide() and .show(), and .fadeOut() and .slideUp() would work, but .fadeIn() and .slideDown() would not work at all. The HTML looked something like this:

<div id="page_help">
<div id="help_toggle">Click here to hide help</div>
<div id="help_text">This is some help text</div>
</div>

I had a CSS definition for the help_toggle class setting it to display:inline-block and the fadeIn/slideDown problems started appearing in IE8.  When I set it to display:block the problem went away. Bizarrely, if I inserted an HTML comment between the help_toggle and help_text DIVs, the problem also went away! E.g.:

<div id="page_help">
<div id="help_toggle">Click here to hide help</div>
<!-- bizarrely enough, this fixes the jquery fadein problem -->
<div id="help_text">This is some help text</div>
</div>

This is baffling. The only moral to the story I can figure out is that jQuery animations do not appreciate block and inline elements being siblings, and sometimes setting inline-block as the display style is not sufficient to get things working again.

Rails named_scope gotchas

There are two terrible gotchas in Rails named_scopes. These two have bitten many a named_scope newbie on the rear (including myself, except that my unit tests caught the problems before they went anywhere but my desktop, w00t for TDD!)

#1. Use length instead of size with the results returned by named_scopes.

Under certain conditions, particularly when using :limit in a named_scope, size and length will return different values. Example:

class Book < ActiveRecord::Base
  named_scope :top, lambda { |limit| {
    :limit => limit,
    :order => 'total_sales DESC'
    }}
end
# imagine we have 10 books with various values for total_sales...
>> Book.top(5).length
=> 5
>> Book.top(5).size
=> 10
# for some reason this only affects named_scopes:
>> Book.find(:all, :limit => 5).length
=> 5
>> Book.find(:all, :limit => 5).size
=> 5
# Go figure!

#2. named_scopes and AR calculations don’t mix

Named scopes are a joy to use, especially when you chain them. It’s completely addictive being able to use constructs like:

topten = Book.fiction.category('vampire').top(10)

(Hypothetical example, I do not read vampire novels…).

The arrays returned by named_scope are ActiveRecord arrays and support calculations such as .count and .maximum, so it’s tempting to want to try something like:

max_fiction_price = Book.top(10).maximum(:price)

Beware! You may not get the results you are looking for — in my experience, these will not work consistently. The problem is that ActiveRecord generates a query something like the following:

SELECT MAX(price) from books ORDER BY price DESC LIMIT 10;

As far as I can tell in researching it, this is not really valid SQL (at least not according to MySQL). LIMIT happens after everything else–similar to HAVING–and only filters the number of output rows that are displayed. Since the MAX() function tells SQL to aggregate the results into a single column, the ORDER and LIMIT clauses are meaningless.

Sometimes this type of statement works for me. I have tried it on SQLite3 and several different versions of MySQL for Linux and Windows. The database sometimes seems to understand that I want to order by price, take the top 10, then calculate the max price. It is entirely possible that in tests where the result appears to be correct, it is actually because the records were randomly ordered in a way that produced the correct result, but purely by chance.

Of course, Rails 3.0 is on the way and there may end up being changes in these behaviors since AR is being rebuilt from the ground up. Whatever the case, avoid mixing conditions and limits, and make sure your unit and functional tests are written to validate the number of results your named_scopes return.

My web site looks better in IE than Firefox

I noticed something unexpected today. A web site I’m working on looks better in IE 8 than it does in Firefox 3.6. It’s not because I did a bad job and the layout is significantly different on the two different browsers–they are almost identical down to the pixel. It’s the subtle things like how text, graphics, and default controls are rendered. Sadly, I can’t just direct you to the site to check it out for yourself because most of it is a private demo, but the following are some tiny screenshots taken out of context to illustrate my point.

Text Rendering

This obviously comes down to personal preference, but in general I think the anti-aliased fonts used by IE make for a more professional-looking site with less effort, using default fonts. Ironically I prefer un-smoothed fonts on my desktop, maybe because I’m an old-school DOS/Windows 3 guy. I have read that enabling font-smoothing in Windows will cause Firefox to use the anti-aliased fonts (along with everything else on your system) but regardless of how I change that system setting, the Firefox text does not seem to be smoother and the IE text is the same. I think IE must have its own font rendering engine aside from Windows ClearType. Example:

Firefox text

Internet Explorer text

Default Controls

When using default controls like buttons, the anti-aliased text sometimes gives the buttons a more polished look. Obviously, most web sites these day use custom control with their own graphics to achieve that too-kewl post-Web 2.0 look, but when you’re tacking together a quick site it’s nice to rely on good old default controls, at least to get started:

Firefox buttons

Internet Explorer buttons

Image Scaling Quality

The application I am working on allows the user to zoom images. Since the user is very likely to look at the 100% zoom view, we just send the full-size image and use Javascript to resize, letting the browser do the scaling work. While the images look exactly alike at full zoom, I have noticed that in some photos–mostly dark and grainy ones–IE’s graphics scaling is much smoother than Firefox. In the following example image, which is being scaled to 20% of original size by the browser, it almost looks like Firefox converted it to a dithered GIF:

Firefox scaled image

Internet Explorer scaled image

I’m not likely to change to IE because of these minor features, but it’s nice to see that there’s at least one or two good reasons that Microsoft’s beast is slower at rendering pages than the competition. In case you are curious how Google Chrome stacks up, in a nutshell its text and controls look more like Firefox, and its image scaling must use the same engine as IE because they appear identical. Speed-wise it’s no contest. Chrome loads and renders the fastest in my unscientific tests, but partly that’s because there’s so little going on–I don’t have AdBlock and my four other add-ons running in the background under Chrome. Subject for another post.

Is soft-delete fail?

It appears that the Rails community has hit the brakes hard on soft-delete (marking database records as ‘deleted’ instead of actually deleting them). Is the entire concept of soft-delete fail? Long before the Rails community embraced soft-delete, there were discussions of its various pros and cons.

I’m a big believer in the concept of never permanently deleting data. It’s a pretty easy bandwagon to jump on–why risk losing data when you can safely tuck it away somewhere or simply mark it deleted, then update all of your database queries to ignore the “deleted” rows? As a believer, I’ve worked quite a bit with soft deletion in both web and stand-alone apps, and it has always been an ongoing maintenance headache. Everyone in the organization who touches the data ends up needing to be aware of the soft-deletion scheme to avoid causing problems–programmers, DBAs, even support staff. Even if you move deleted records into a separate table, you are still likely to end up with a system where one forgetful programmer can cause headaches for everyone.

For a while it looked like Rails to the rescue, first with technoweenie’s (Rick Olson) acts_as_paranoid which started as an experiment back in 2005, and more recently semanticart’s (Jeff Chupp) is_paranoid, which cleverly exploited the new ActiveRecord default_scope feature to improve on the concept. Over the years, there has been some criticism of acts_as_paranoid’s problems and limitations but devotees have worked around them. More recently, some plugins and gems have popped up with alternative schemes such as moving deleted records to an archive table instead of marking them as deleted. In the last few months, however, Rails community support for soft deletion seems to be having a meltdown. Jeff of semanticart publicly stated that he is killing is_paranoid and rumored that technoweenie is no longer using acts_as_paranoid. A quick search of the net finds a number of very recent articles slamming soft-delete in general (for example, see Luke Francl’s list of softdelete Bookmarks) and there are an increasing number of Rails add-ons dedicated to the concept of moving deleted records to an archive table.

Ultimately, the argument for killing is_paranoid boiled down to how making soft-delete transparent added one too many layers of Rails magic, shielding the programmer from important data considerations that should have been kept explicit. Soft-delete doesn’t get off the hook that easily though–it still presented problems even when wrapped in Rails magic. For example, you can get into trouble if you use unique indexes in your database. Imagine a row is soft-deleted, then you try to create a new row with the same attributes that were supposed to be unique. Also, as Rails has rapidly evolved new and interesting methods of associating and finding data, the soft-delete add-ons required constant updating.

Personally, I’m starting to think the best answer is to remove all consideration of soft-delete / archive-delete from your apps and handle this externally. There are quite a few tools out there for backing up data in real-time as it is created, using SQL triggers so deleted rows are automatically journaled or moved into an archive, etc. There are also some pretty compelling articles asking an important meta-question, why are we deleting data at all?

For further reading with some interesting suggestions for alternatives, I recommend Luke Francl’s post on Rail Spikes.

If you use Rails, take a few minutes to learn Ruby

I’m sure I’m not the first person to code three or four complete Ruby on Rails projects before bothering to learn the finer points of Ruby, but after looking at some old code I’ve gotta say I wish I had spent a day just reading Pickaxe or Why’s (poignant) Guide before I jumped in with both feet.

There are dozens of examples of my former Ruby stupidity in virtually any module on my hard disk. They mostly fall into the category of “doing things the C++ way instead of learning the Ruby way”. For example:

  if !my_param.nil? && my_param.is_a?(Integer) && my_param >= 0 && my_param < 10
    # do stuff
  end

Tsk tsk. Terrible mess. Imagine an entire module full of that. The following is just so much better:

  if (0..9).include?(my_param)
    # do stuff
  end

This one method properly deals with nil and non-integer values. Nice!

Of course, I don’t blame C++ for my cluttered code — one could easily create their own syntactic sugar in C++ for common tasks like parameter-checking. The point is that Ruby has all these cute tricks already, and you should be making your life easier by using them!

If you happen by this post, I invite you to share your own favorite Ruby tricks.

Nginx gzip config and image problems in IE

I am running a web site under Nginx and noticed that when you right-click images on that site using IE, even if they are obviously JPEGs you can only save them as bitmaps (.BMP). If you get properties on the image the type comes up as unknown and the last modified date is missing. I tracked this down to my Nginx gzip configuration, specifically the option gzip_vary on;.

I won’t discuss what the gzip_vary option is supposed to do, partly because I don’t know and partly because I don’t care — it’s supposed to help tell browsers that gzip compression is available on the server or somesuch. After a little trial and error I realized that if I disabled this option, IE started treating my images normally. Apparently, if IE sees a HTTP response header that it doesn’t understand or is in the wrong place, it pitches a hissy fit and skips some of your headers. WTF.

I am using Nginx version 0.7.61. After reading the release notes of Nginx it appears this may have been a bug that is now fixed. One release mentioned that the gzip_vary header was being sent at inappropriate times. I can’t update that server right now because it’s on the critical path for a project at the moment and needs to be stable, but I will repost if updating Nginx solves the issue.