Loading Binary Data into Rails Fixtures

Posted by Peter Donald Wed, 05 Apr 2006 21:50:00 GMT

Loading image data into fixtures was a chore until recently as I had been using separate rake tasks to do the job. The following code demonstrates how easy it is to load binary data such as images into the fixtures via standard mechanisms. It loads some image data from within the $RAILS_ROOT/test/fixtures/ directory and puts it in database.

<%
def fixture_data(name)
  render_binary("#{RAILS_ROOT}/test/fixtures/#{name}")
end

def render_binary(filename)
  data = File.open(filename,'rb').read
  "!binary | #{[data].pack('m').gsub(/\n/,"\n    ")}\n"
end
%>
picture_data_1:
  id: 1
  picture_id: 1
  content_type: 'image/jpg'
  data: <%= fixture_data("picture_data_1.jpg") %>
picture_data_2:
  id: 2
  picture_id: 2
  content_type: 'image/gif'
  data: <%= fixture_data("picture_data_2.gif") %>

If you do not use two spaces as your indent then you will need to alter the line in render_binary(filename) that replaces newline so that every newline is replaced with two indents.

Easy peasy!

Update on 16th April 2006

It turns out that it was not as easy peasy under postgres as the driver did not know it had to escape the data as binary as fixtures don’t actually load the column type. The simplest hack around it is to add in the following bit of code somewhere that just patches the driver if a 0 is in the data. This may not always work but it works with my test data so that is good enough for me at the moment.

class ActiveRecord::ConnectionAdapters::PostgreSQLAdapter < ActiveRecord::ConnectionAdapters::AbstractAdapter
  def quote(value, column = nil)
    if (value.kind_of?(String) && column && column.type == :binary) || (value.kind_of?(String) && value.include?(0))
      "'#{escape_bytea(value)}'"
    else
      super
    end
  end
end

Posted in  | Tags , , ,  | 5 comments | 1 trackback

Rails Plugin to Help Debug Views

Posted by Peter Donald Mon, 20 Mar 2006 03:02:00 GMT

The debug_view_helper plugin was developed to make it easy to add debug information into your views. It was derived from techniques described in HowtoDebugViews and it makes it possible to expose the followng following debug data;

  • Request Parameters
  • Session Variables
  • Flash Variables
  • Assigned Template Variables

Typically you add code such as the following to the bottom of your layout that exposes the debug button in development mode.

<% if RAILS_ENV == 'development' %>
<center>
  <button onclick="show_debug_popup(); return false;">
    Show debug popup
  </button>
</center>
<%= debug_popup %>
<% end %>

You can grab the plugin from subversion at;

http://www.realityforge.org/svn/code/debug-view-helper/trunk/

Update: Added the ability to add inline debug information via the following. Suggestion by John Dell.

<% if RAILS_ENV == 'development' %>
<%= debug_inline %>
<% end %>

Posted in  | Tags ,  | 2 comments | no trackbacks

Rails Plugin to Validate (X)HTML and CSS

Posted by Peter Donald Wed, 15 Mar 2006 08:18:00 GMT

Here is an enahnced version of Scott Raymond’s assert_valid_markup plugin that I use in my projects. Below are the directions for use;

HowTo Validate (X)HTML

# Calling the assertion with no parameters validates 
# whatever is in @request.body, which is automatically 
# set by the existing get/post/etc helpers. For example:

class FooControllerTest < Test::Unit::TestCase
  def test_bar_markup
    get :bar
    assert_valid_markup
  end
end

# Add a string parameter to the assertion to validate 
# any random fragment. For example:

class FooControllerTest < Test::Unit::TestCase
  def test_bar_markup
    assert_valid_markup "<div>Hello, world.</div>"
  end
end

# For the ultimate in convenience, use the class-level
# method to validate a number of actions in one line. 

class FooControllerTest < Test::Unit::TestCase
  assert_valid_markup :bar, :baz, :qux
end

HowTo Validate CSS

# Pass a string parameter to the assertion to validate 
# a css fragment. For example:

class FooControllerTest < Test::Unit::TestCase
  def test_bar_css
    filename = "#{RAILS_ROOT}/public/stylesheets/bar.css"
    assert_valid_css(File.open(filename ,'rb').read)
  end
end

# For the ultimate in convenience, use the class-level
# method to validate a bunch of css files in one line.
# Assumes that the CSS files are relative to 
# $RAILS_ROOT/public/stylesheets/ and end with '.css'. 
# The following example validates 
#  $RAILS_ROOT/public/stylesheets/layout.css, 
#  $RAILS_ROOT/public/stylesheets/standard.css and 
#  $RAILS_ROOT/public/stylesheets/theme.css

class FooControllerTest < Test::Unit::TestCase
  assert_valid_css_files 'layout', 'standard', 'theme'
end

Most of the credit for this plugin goes to Scott for the initial idea! The modifications that I made include;

  • Validation of CSS files.
  • Caching of fragments occurs in $RAILS_ROOT/tmp according to the name of the test class + test method. This avoids filling up the system temp folder with expired cache files.
  • Ability to turn off validation by setting the “NONET” environment variable to “true”.

You can grab it from subversion at;

http://www.realityforge.org/svn/code/assert-valid-asset/trunk/

Update 12th of May, 2008

Steve Sloan has also added a Git repo for the code which you can grab from:

http://github.com/CodeMonkeySteve/assert-valid-asset/tree/master

Posted in  | Tags , , , ,  | 2 comments | no trackbacks

Upload a file via POST with Net::HTTP

Posted by Peter Donald Thu, 02 Mar 2006 02:57:00 GMT

To upload a file to a website I needed to supply the data with a content type of “multipart/form-data”. The Net:HTTP API does not offer any such functionality, it just accepts raw content data. So I needed to roll my own.

The “multipart/form-data” content type consists of a number of secions separated by --BOUNDARY\r\n and terminated by BOUNDARY--\r\n where BOUNDARY is a string that does not appear in the content of any of the data transmitted to the server.

Each section represents a form field and contains a number of headers, a \r\n, the content and finishes with a \r\n. Normal form fields look like

Content-Disposition: form-data; name="mykey"

mydata

while file fields must include a few more headers.

Content-Disposition: form-data; name="mykey"; filename="filename"
Content-Transfer-Encoding: binary
Content-Type: text/plain

DATADATADATADATADATADATADATA...

To construct the parameters in ruby I use the following code;

def text_to_multipart(key,value)
  return "Content-Disposition: form-data; name=\"#{CGI::escape(key)}\"\r\n" + 
         "\r\n" + 
         "#{value}\r\n"
end

def file_to_multipart(key,filename,mime_type,content)
  return "Content-Disposition: form-data; name=\"#{CGI::escape(key)}\"; filename=\"#{filename}\"\r\n" +
         "Content-Transfer-Encoding: binary\r\n" +
         "Content-Type: #{mime_type}\r\n" + 
         "\r\n" + 
         "#{content}\r\n"
end

To put it all together you need to join the parameters with boundary separators between each section. This can be done via

boundary = '349832898984244898448024464570528145'
query = 
  params.collect {|p| '--' + boundary + "\r\n" + p}.join('') + "--" + boundary + "--\r\n"

The last thing that needs to be done is to make sure that you set the HTTP Header Content-type to multipart/form-data; boundary=BOUNDARY.

A complete example that I extracted from code that uploads a css file to the w3c validator service is as follows.

params = [ 
  file_to_multipart('file','file.css','text/css',data),
  text_to_multipart('warning','1'),
  text_to_multipart('profile','css2'),
  text_to_multipart('usermedium','all') ]

boundary = '349832898984244898448024464570528145'
query = 
  params.collect {|p| '--' + boundary + "\r\n" + p}.join('') + "--" + boundary + "--\r\n"

response = http.start('jigsaw.w3.org').
  post2("/css-validator/validator",
        query,
        "Content-type" => "multipart/form-data; boundary=" + boundary)

It was a little bit painful to figure out “multipart/form-data” via Ethereal but relatively easy to implement. Hope this helps!

Update 3rd of October, 2006:

Slight correction supplied by Andrew Willis so that last boundary is '--' + boundary + '--' rather than just boundary + '--'.

Posted in  | Tags ,  | no comments | no trackbacks

Removing Stale Rails Sessions

Posted by Peter Donald Wed, 01 Mar 2006 00:53:00 GMT

By default rails does not clear out stale sessions from the session store. To implement this feature I added the following small snippet of code;

class SessionCleaner
  def self.remove_stale_sessions
    CGI::Session::ActiveRecordStore::Session.
      destroy_all( ['updated_on <?', 20.minutes.ago] ) 
  end
end

And then invoke the remove_stale_sessions method every 10 minutes via;

*/10 * * * * ruby /full/path/to/script/runner 
   -e production "SessionCleaner.remove_stale_sessions"

Posted in  | Tags ,  | no comments | no trackbacks

Older posts: 1 2