Vertical Tables in Rails

I just released a rails plugin that provides a mix-in so a class can get it’s attributes from an associated vertical table.

So if you have a table called ‘preferences’ or something that your other developers are constantly stuffing key-value pairs into, then go ahead and use this muh to stop writing a bunch of accessors. It wraps around an existing association to provide DRY access to that table, in a way you’ll never have to think about again.

Something cool about this – It basically de-schemafies one of (or all of) your models. You COULD, though I don’t know why you’d want to, do something obscene like this:

#  no attributes, just an ID and Type column
class DbObject < ActiveRecord::Base
  has_many :db_attributes, :autosave => true
  include VerticalTable::Attributes
 
  def self.has_attributes(*attrs)
    vertical_attributes_from(:db_attributes) do |v|
      attrs.each do |a|
        v.send(a, :key => a)
      end
    end
  end
end
 
# id, db_object_id, key, value
class DbAttribute < ActiveRecord::Base
  belongs_to :db_object
end
 
  Person = Class.new(DbObject) do
    has_attributes :fname, :lname, :phone
  end
 
  should "allow me to use a person as if it had real attrs" do
    p = Person.create(:fname => "Alex", 
      :lname => "Bartlow", :phone => '8675309')
    assert_equal "Alex", p.fname
  end

So there you go. Schemaless MySQL. The scary thing is, you might be able to cluster/partition this by the db_object_id and it might actually be performant.

Head asplode and all.

Add a comment

Rails3 XSS Protection

I’m fiddling about with a rails3 app after reading Yehuda’s 3.0 Announcement. One of the things that he talks about is Rails’ new paranoid XSS protection scheme.

I wasn’t able to find much about this, except This Slideshare which happens to be in German.

Here’s what I can grok:

  • Strings are now either safe or not.
  • Rails <%= foo %> escapes all non-safe strings
  • You can inquire about a string’s safety through the html_safe? method
  • You can mark a string as safe through the html_safe! method
  • You can intentionally display an unsafe, unescaped string in the view (if you’re damn sure) through the ‘raw’ function. (<%= raw foo %>)

Fragen?

Add a comment

The REST Virus

So I’ve been designing more and more REST based services, small and large, and I can’t get the following hypothesis out of my head:

Any business domain problem can be modeled by CRUD actions on domain objects, and a callback system for introducing domain logic around those actions.

No matter how complicated the problem I keep running into, I keep just breaking it down into domain objects and then injecting real code as callbacks around those objects. Since I also have timestamps on with all of those objects, I also have a built in logging system, so I can show when everything happened.

Can someone provide a use case where this idea breaks down? I find myself writing nothing but basically boilerplate rails code anymore.

Add a comment

Redis on Rails

Hello children! Today, we’re going to learn:

  • How to get an edge rails app up and running with a redis backend
  • How to create the familiar rails tutorial scaffold
  • How to extend the scaffold to make a simple calendar system

Before you follow along this code – it turns out that we don’t have a happy ending, and actually are going to suffer for the v0.0.2 dm-redis-adapter we’re using. So read and enjoy, but don’t expect to get it working on your machine just yet.

A Running App

The first thing we need to do is get a valid rails3 app up and running. Luckily, Yehuda got this done for us, and so we can find it here: http://weblog.rubyonrails.org/2010/1/1/getting-a-new-app-running-on-edge (Also found http://yehudakatz.com/2009/12/31/spinning-up-a-new-rails-app/)

After that, we’ll need to tell rails that we’re going to use our own database framework, so we’re going to make our Rails.root/app/config/boot.rb look like this:

# require 'rails/all'

# To pick the frameworks you want, remove 'require "rails/all"'
# and list the framework railties that you want:
#
 require "active_support/railtie"
 require "active_model/railtie"
# require "active_record/railtie"
 require "action_controller/railtie"
 require "action_view/railtie"
# require "action_mailer/railtie"
# require "active_resource/railtie"
 require "rails/test_unit/railtie"

I don’t plan on using active_resource for this app, so I’m going to leave that out. Then, delete your database.yml file (or keep it, to remember the bygone days.)

Also, if you comment out action_mailer be sure you go through your config/environments and delete out anywhere it says config.action_mailer, or you’ll crash.

After this is done, you should be able to run a basic script/server, and not have rails complain that you’ve done something wrong.

[alex-bartlows-macbook:: ~/Projects/rails3/redis_on_rails]
script/server
=> Booting WEBrick
=> Rails 3.0.pre application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2010-01-30 13:40:14] INFO  WEBrick 1.3.1
[2010-01-30 13:40:14] INFO  ruby 1.9.1 (2009-07-16) [i386-darwin9.8.0]
[2010-01-30 13:40:14] INFO  WEBrick::HTTPServer#start: pid=67476 port=3000

Woo!

Alright, now we’re getting somewhere. I’ll assume you’ve already installed Redis from http://code.google.com/p/redis/, and that you already have redis-server up and running happily in the background.

There are a couple options for your ORM. There’s an interesting project called ‘ohm’ (http://github.com/soveran/ohm) that I’ve forked (http://github.com/alexbartlow/ohm) to do some work in it’s associations, but it’s still in infancy, and it needs a bit more work until it’s fully production ready.

Datamapper, however, has a redis interface from whoahbot (http://github.com/whoahbot/dm-redis-adapter/), who is himself a luminous being, and looks like the ruby community’s _why replacement. Also, did you know there are no good GIS results for ‘Shekinah Glory?’

Anyway, so we’re going with datamapper. Let’s get it, and set up whoahbot’s adapter.

Add the following to your Gemfile:

gem "dm-core", "0.10.2"
gem "redis", "0.1.2"

And then run:

gem bundler

This is one of the most awesome things about Rails3. Native extension gems baked right into the app. rake:gems:freeze could be finnicky sometimes.

OK, now we need the adapter.

git submodule add git://github.com/whoahbot/dm-redis-adapter.git
vendor/plugins/dm-redis-adapter

I don’t know how you feel about git submodules, feel free to just copy the files. It might be a good call to keep these in your capistrano (http://capify.org) shared folder and just symlink to them. Anyway, you’re smart, and therefore capable of figuring this out yourself.

Finally, we’ll need to tell Rails to tell DataMapper that it should use Redis.

#config/initializers/datamapper.rb
require 'dm-core'
require 'dm_redis'
DataMapper.setup(:default, {:adapter => "redis"})

Setting up your model:

And so now we FINALLY can make a model.

touch app/models/person.rb
touch test/unit/person_test.rb

We’re using touch because the following doesn’t work:

[alex-bartlows-macbook:: ~/Projects/rails3/redis_on_rails]
script/generate model Person --orm=datamapper
       error  datamapper [not found]

But someday will!

In the true spirit of TDD, we will write some test cases before we write our application code.

require File.join(File.basename(__FILE__), '..', 'test_helper')

class PersonTest < ActiveModel::TestCase
  include ActiveModel::Lint::Tests
  setup do
    @model = Person.create(:name => "Alex Bartlow")
  end
end

This ensures that whatever model we create is compatible with the ActiveModel interface. Make sure redis is running, and run that muh!

Loaded suite /Users/bartlowa/Projects/rails3/redis_on_rails/test/unit/person_test
Started
FFFFFF
Finished in 0.101574 seconds.

Glorious failure! Lets make some of these pass.

#app/models/person.rb
require 'dm-validations'
class Person
  include DataMapper::Resource

  property :id, Serial
  property :name, Text

  def to_model
    self
  end
  def self.model_name
    self.to_s
  end
  validates_is_unique :name
end

I also needed to add the following to a rails initializer:

String.send(:alias_method, :human, :humanize)

String.class_eval do
  def partial_path
    self.downcase.pluralize << '/' << self.downcase.singular
  end
end

Though this should really be default on ActiveSupport’s monkeypatching of String.

Now we should be able to play with our Person in a script/console.

irb(main):005:0> Person.create(:name => "Whoahbot!")
=> #<Person @id=63 @name="Whoahbot!">
irb(main):006:0> Person.get(63)
=> nil
irb(main):007:0>

Crap! Looks like the dm adapter is borked. Or I’m just doing something wrong.

Anyone have any suggestions? Otherwise I’ll have to tweak whoahbot’s stuff and/or re-implement a dm adapter. Looks like it might not be that hard (http://www.theamazingrando.com/blog/?p=95)

Add a comment

Refactoring Inner Classes

If you’re like me, you don’t like massive class definitions in Rails.

Unfortunately, if you’re having to deal with a legacy schema, you have to deal with a bunch of little tables, so your top level models need to reference of bunch of tiny little classes. That’s fine an all, but you don’t want a bunch of top level classes polluting your namespace.

So, you turn to inner classes with set_table_name. That’s great, but then you get massive model definitions. What’s a boy to do?

# app/models/upper.rb
 
class Upper < ActiveRecord::Base
has_many :inners, :class_name => "Upper::Inner"
end
 
# app/models/upper/inner.rb
require 'upper'
Upper.send(:remove_const, :Inner) if defined?(Upper::Inner) 
Upper.const_set("Inner", Class.new(ActiveRecord::Base) do
  belongs_to :upper
end)

Voila! the class keyword in ruby just assigns an instance of Class to a constant. That’s it! Behold the elegance of the Meta Object Protocol!

Luckly, since Classes are also Modules, you can use Module#const_set to define a constant, and you can use Class#new to define a new class. So, you get the same effect as defining an inner class, in a separate file underneath your ‘upper’ folder in models.

The only tricky bit about this is that:

  1. You need to explicitly require the top level class. (hence require ‘upper’)
  2. You need to put in an explicit constant undefinition so you don’t get warnings about redefining constants

Enjoy your sexy refactored models!

Add a comment

ActiveRecord Single Table Inheritance using numeric types

The AR documentation talks about how you can change the column you use for single table inheritance, but not at all for how you can change the data-type contained within.

Looking into the Rails ActiveRecord::Base source, there’s a protected method called ‘compute_type’ that’s used to figure out into which type we’re supposed to cast the record:

  #activerecord/base.rb
          def compute_type(type_name)
          modularized_name = type_name_with_module(type_name)
          silence_warnings do
            begin
              class_eval(modularized_name, __FILE__, __LINE__)
            rescue NameError
              class_eval(type_name, __FILE__, __LINE__)
            end
          end
        end

Nothing special – basically just looks around the namespace for a class that matches it – so inner classes and what have you can be found here.

So that’s great, because it means if we’re using a numeric type_id field, to convince ActiveRecord to return the right kind of record, we just have to override that so that it transforms our ID into a type that makes sense.

# In the model for the STI parent - Lead in my case....
class << self
    def compute_type(type)
      if (type.to_i.to_s == type) && App.lead_type_hash[type.to_i]
        type = (App.lead_type_hash[type.to_i] + 'Lead') 
      end
      super(type.to_s)
    end
  end

So yeah, not much to it. My ‘App’ class holds a hash of ids to class names, and so all this does is transform the ID into a class name just before it hits the real compute type method. Overriding this on one class means that we can use this same technique in many different classes.

And we’re nearly done! If you do Parent.first now, you’ll see the record being ‘blessed’ into the right instance type. Now, you just need to tell ActiveRecord that it should consider the class’s name for STI purposes to be the integer, instead of the string name of the class.

# In the same model as before...
class << self
  def sti_name
    App.lead_type_hash.invert[self.name.sub('Lead', '')]
  end
end

Then, just declare the column you use for inheritance:

set_inheritance_column 'lead_type_id'

Done and done!

I’d have thought this would be a much more arduous process, but once delving into the AR code that handles the STI, it actually wasn’t that bad at all. I might have to put together a simple plugin (or a patch to rails itself) that allows this sort of behavior to be simply declared, but until then it’s < 20 LOC to get this working for one tree of your models.

So, enjoy your Single Table Inheritance on legacy schema!

Add a comment

A copy of XXXX has been removed from the module tree but is still active?

Being bit by this?

class ProblemModel < ActiveRecord::Base
  unloadable
end

Solved!

1 Comment

Ruby 1.9 Fibers + AJAX = HTTP Binding?

So, you’re probably aware that ruby 1.9 adds Fibers, a lightweight concurrency library.

There’s some cool stuff using this, and I’m sure that you could use that to implement the following, but it felt better rolling it from scratch.

So, the use case:

Assume that I am a user at a web frontend watching the status of my application. I don’t want to poll the app every N seconds to get the data from it, but I still want as-it-happens updates to the data on my screen. Say, every time a ‘Widget’ model is created I want to update a certain div with the new Widget.count.

To do that, we’d need to send an AJAX call to the server, which is going to purposefully hang the response and then wake it up again whenever our event occurs.

The following code should do just that:

require 'socket'
 
server = TCPServer.new('127.0.0.1', '8080')
#fork do
module FiberPool
  class << self
    def fire event
      $queued_fibers ||= {}
      $queued_fibers[event] ||= []
      while(!$queued_fibers[event].empty?) do
        x = $queued_fibers[event].shift
        puts "\tFiring #{x.inspect}"
        x.resume
      end
      $queued_fibers[event] = []
    end
  end
end
$events = []
$queued_fibers = {}
f = Thread.fork do
  loop do
    begin
      session = server.accept_nonblock
      $queued_fibers['respond'] ||= []
      $queued_fibers['respond'] << Fiber.new {
          puts "Firing a fiber: #{session.gets}"
          session.puts "Hello Fiber!"
          session.close
        }
    rescue Errno::EAGAIN, Errno::ECONNABORTED, Errno::EPROTO, Errno::EINTR
 
    ensure
      FiberPool.fire $events.shift unless $events.empty?
    end
  end
end
 
at_exit {Thread.kill(f)}

I’ve used some global variables here as a way of passing messages across threads, but in reality you’d use a Redis database or something instead.

So, every time we get a request on 8080, we hang the request into its own fiber. The server continually checks the $events variable (or just as easily, a redis event queue) for events. Then, as soon as we see the ‘respond’ event, we wake up all of those threads and fire them off.

For demonstration purposes, append the following code, giving the terminal control over the ‘event sending.’

begin
  loop do
    puts "Register an event:"
    event = gets.chomp
    puts "got event #{event}"
    if event == 'show'
      puts "Queued Fibers = #{$queued_fibers.inspect}"
      puts "Events        = #{$events.inspect}"
    else
      $events << event
    end
  end
rescue Interrupt
  puts "\nCaught interrupt, exiting"
end

Save all that to a file, run it, and point your web browser to localhost:8080. It should hang, until you go to your command line and type ‘respond’ as the event to raise, at which point the web browser should immediately see ‘Hello Fiber!’

Now, imagine that you’ve made the request through a new Ajax.updater and the response is a JSON map of DOMID => Value pairs computed by the server, and you see why this is exciting. You can set up your Widget.after_create to throw a new event to redis, at which point all of your clients will receive a response to this AJAX call, updating their page. You can set up the ‘onsuccess’ callback to make another request, and you’ve bound a dom object to a server-side map.

If you’re used to programming GUIs in the desktop, or IPhone, you’ll notice this is pretty much the way that those work. A change in the state of the bound object triggers an update in the view, and the view can send events to the controller to change the bound objet.

The gap between the real-time GUIs of application development and request-based transactions of web development just got a hell of a lot smaller.

References:
http://www.double.co.nz/pdf/continuations.pdf

2 Comments

Thoughtbot-Factory + Cucumber Tables = Crazy Delicious

Consider the following cucumber step:

Given /^the following "([^\"]*)" exist:$/ do |type, table|
  symbol = type.singularize.to_sym
  table.hashes.each do |hash|
    Factory(symbol, hash)
  end
  klass = symbol.to_s.capitalize
  if defined?(klass.classify)
    (class << klass.classify ; self ; end).instance_eval do
      define_method(:find_by_cucumber_handle) do |x|
        find(:first, :conditions => {table.headers[0].to_sym => x})
      end
    end
  end
end

Sure, no surprises. We use factorygirl to pump out a ton of models in the table. But what’s with all that metaprogramming crap in the middle? #find_by_cucumber_handle?

    # features/support/paths.rb
    # ...
    when /the "([^\"]*)" page for "([^\"]*)"/
      path = $1
      parameter = $2
      constant = path.capitalize.classify
      send (path.gsub(/ /, '_') + "_path", constant.find_by_cucumber_handle(parameter) )
    # ....

Which lets you do things like:

Background:
	Given the following "users" exist:
		|username|
		| alex   |

Scenario: User logs into site
	Given I am not logged in
	When I go to login
	And I fill in "login" with "alex"
	And I fill in "password" with "password"
	And I press "Login"
	Then I should be on the "user" page for "alex"
	And I should see "Logged in as alex"

Only for every model ever.

Tasty!

For extra credit, use the splat operator + regex-fu so you can do /user/products/commentaries/etc as the cucumber step:

Then I should be on the "user" "products" "commentaries" "etc" page
4 Comments

Even Easier XMLRPC Hosting

If you’ve thought about doing some SOA work and you know ruby, you probably realize that it’s blissfully easy to write a web service with the xmlrpc library:

1
2
3
4
5
6
7
8
9
10
11
12

require 'xmlrpc'
 
XML::Server.new(8080).tap{|s|
 
s.add_handler('alexbartlow.add') do |x, y|
x + y
end
 
s.add_handler('alexbartlow.sub') do |x,y|
x - y
end
}.serve

We didn’t even use variables! But this is a pretty un-maintainable approach, and there’s no way to generate Rdoc based on the service.

Enter APIServer!

require 'xmlrpc/server'
 
class APIServer
  attr :server
  class &lt;&lt; self ; attr :module end
  def initialize(port=8080)
    @server = XMLRPC::Server.new(port)
    @module.methods(false).each do |m| 
      @server.add_handler(m, &amp;@module.method(m).to_proc)
    end
  end
  def run!
    @server.serve
  end
  def self.boot!
    new.run!
  end
  def self.serves(aModule)
    @module = aModule
  end
end

This uses some encapsulation-breaking hackery to create a server based on a given module, like so:

module FooAPI
  def add(x,y)
    x + y
  end
  def sub(x,y)
    x - y
  end
end
 
class FooServer &lt; APIServer
  serves FooAPI
end
 
FooServer.boot!

So now you can generate RDoc based on the project, and even have a FooAPI::Base which includes all the other modules you might write, and this class should help you serve it all without coupling too heavily to the XMLRPC::Server.

Coming soon, removing the XMLRPC ::Server dependency altogether, to use SOAP or even a rails controller!

Add a comment