Saturday, November 3, 2007

bash aliases

Bash aliases are such lovely things, so I thought I would share some of my favorites and maybe even get some feedback.

Most of my aliases are there to make software development more enjoyable and less time consuming. As such, most of them deal with the projects I have in my project directories. Here are some of the more interesting ones:


for bigdir in $(ls ~/Developer/Projects/ | cut -d' ' -f7)
do
for dir in $(ls ~/Developer/Projects/$bigdir)
do
alias cd$dir="cd ~/Developer/Projects/$bigdir/$dir"
alias m$dir="cd$dir && mate ."
alias s$dir="cd$dir && ss"
alias g$dir="~/bin/svngrowl $dir"
alias o$dir="osascript -e 'tell application \"Terminal\"'
-e 'tell application \"System Events\"'
-e 'tell process \"Terminal\" to keystroke \"t\" using command down'
-e 'tell process \"Terminal\" to keystroke \"t\" using command down'
-e 'end tell'
-e 'do script \"s$dir\" in first tab of front window'
-e 'do script \"m$dir\" in second tab of front window'
-e 'do script \"g$dir\" in third tab of front window'
-e 'do script \"localhost\" in second tab of front window'
-e 'tell second tab of front window to set selected to true'
-e 'end tell'"
done
done

alias localhost="open http://localhost:3000"
alias addall="svn st | grep ? | cut -d' ' -f7 | xargs svn add"
alias removeall="svn st | grep ? | cut -d' ' -f7 | xargs rm -Rf"

for var in "server" "generate" "console" "destroy" "plugin"
do
alias s${var:0:1}="script/$var"
done


The first for loop is probably the most useful set of aliases that I have. It starts off by going through all the projects that live in my various project directories. Then, for each project it gives me a cdproject, mproject, sproject, gproject (very special), and oproject (less special, but still useful).

cdproject
  • cd's into the project directory

mproject
  • opens the project in TextMate

sproject
  • starts a server for the project by calling script/server (which is what the alias ss does in a rails project)

gproject
  • starts an svngrowl process for the project (which I'll explain later)

oproject
  • Uses the command line osascript, which invokes applescript, to manipulate Terminal and Firefox


First, an explanation of what oproject is doing. To start, for viewing purposes above I cut the oproject alias into more manageable pieces, but when used should be on one line (as would be the case if it were called from the command line).

This alias is kind of like the Ring of Power, in that it rules all the other aliases, and in the darkness (or the light) binds them... into one convenient call. It opens Terminal, and opens 2 new tabs for a total of 3 tabs (only for Leopard users though :-). It then opens a server in the first tab, opens the project in TextMate in the second tab, starts an svngrowl process in the third tab, goes back to the second tab and runs the alias localhost, and then gives focus back to the second tab (since it seemed to lose focus to the third tab, not sure why). This alias is a huge time saver when you think that every working day I spend about 10 to 15 seconds when I have to open everything up (which adds up to a lot when you have enough work to do!).

As for the gproject alias, this uses a cool little script in my ~/bin directory that starts a loop for the specified project, and every 3 minutes it runs svn up on the project and if there are any changes to the project, it sends out a pretty little growl notification with what was changed, who changed it, the commit message, and if any migrations were committed it changes the priority of the message to high (which for my personal growl settings turns the message text red).

Two scripts are involved in the growling, one bash script, and a ruby script (which is invoked by the bash script). First, the bash script (cut in half for easy viewing):


#!/bin/sh
source ~/.profile
alias | grep cd$@= | awk "{gsub(/cd$@='cd |'/, \"\"); print}" |
awk "{gsub(/~/, \"/Users/`whoami`\"); print}" | xargs ~/bin/svngrowl.rb

And now the ruby script:

#!/usr/bin/env ruby
begin
directory = ARGV[0]
project = directory.split("/").last
ignore = directory.gsub(project, "")

loop do
updates = `svn up #{directory}`.gsub(ignore, "").split("\n")
last_log = `svn log #{directory} --limit 1`

revision = updates.pop.scan(/\d+/)
committer = last_log.split("\n")[1].split(" | ")[1]
priority = updates.join("\n").include?("db/migrate") ? 1 : 0
message = last_log.split("\n")[2..-2].join("\n")

unless updates.empty?
`growlnotify -s -t "#{project} Update - #{committer}"
-m "#{message + "\n\n" + updates.join("\n")}" -p #{priority} --image ~/bin/mph.png`
end
sleep 180
end
rescue
`~/bin/svngrowl.rb #{directory}`
end


All the bash script does is finds the proper directory for the project in question and passes it to the ruby script. The ruby script in turn does an svn up on the directory, collects all the interesting data, and runs growlnotify (the command line growl utility) displaying all of the interesting data. There is also a rescue statement in there that just restarts the process if something goes wrong (an error is thrown if svn can't update properly, which can happen if the network drops out or if you do svn st at the same time that this process is doing svn up). I also used the --image options to pass in a beautiful picture of Mr. Potato Head to use as the icon shown on the notification.

Please feel free to use any of these to make your work/play more productive (except for the picture of Mr. Potato Head... that one is mine!)

Tuesday, October 2, 2007

sfight!

I played around yesterday when I got bored and wrote a little script which pits developers on a project against each other to see who has the most subversion commits. It doesn't really mean too much who wins since you can have plenty of little commits that fix things, but it is just fun to do to see who wins! This script assumes the developers on the project have separate subversion logins. Enjoy!


#!/usr/bin/env ruby
developers = ARGV.select {|arg| !arg.empty? && !arg.nil?}
raise "You need two developers to fight" if developers.size < 2

log = `svn log`
separator = log.scan(/^-.*?-$/).first
log_entries = log.split(separator)[1..-2]
names = log_entries.collect {|entry| entry.split(" | ")[1]}

numbers = developers.collect {|developer| names.select {|entry| entry == developer}.size}

results = []
developers.each_with_index {|developer, i| results << "#{numbers[i].to_s.ljust(3)}- #{developer}"}

puts results.sort {|x, y| y.scan(/\d+/).first.to_i <=> x.scan(/\d+/).first.to_i}


Use it like this (assuming you chmod'ed it to be executable, and put it into your ./script directory):

./script/sfight developer1 developer2 developer3


Tuesday, August 28, 2007

Rails and SSL (https://)

So I recently had to have some secure routes for a site I am currently working on. If you are using deprec and deploying to a slicehost, getting the SSL certificate stuff working is a snap. If you want your whole site to be over SSL, then all you do is put in a permanent redirect to socket 443 in your app_name.conf file. But, if you need to do something interesting like go back and forth between http and https, then you need to do a little digging.

Doing a google search doesn't bring up anything particularly useful in terms of going back and forth between http and https, but one thing did catch my eye. In your routes you can pass a protocol option to set which protocol you are going to use (something like http, file, svn, https, etc...). I also had to dig through some code to find out how to pass that option in, since it gets set two different ways depending on what kind of route you are using. If you do a map.resource(s) route, then you have to set the option as :requirements => {:protocol => ROUTES_PROTOCOL}, whereas if you just do a map.route_name type route, then you just pass in :protocol => ROUTES_PROTOCOL.

You should also be careful about when you are setting that protocol, as trying to access https://localhost:3000 just doesn't work (such as in development mode). I just set a conditional constant at the top of my routes file like so: ROUTES_PROTOCOL = (ENV["RAILS_ENV"] =~ /development/ ? "http" : "https").

Two more caveats:
  • First caveat is that if you want to go back and forth between http and https, you MUST explicitly set all of your routes to have http or https protocol. If you don't, then when rails generates your routes it will just use the protocol of the current page. This is annoying, but I can see why they would do this (if you are looking at something securely, then just look at everything securely).

  • Second caveat is that you must use the route_name_url throughout your application instead of route_name_path, as some are accustomed to. The _url named route gives you the full url (duh), which is what you want if you are on a page using http and want to go to https (whereas _path only gives you the uri, and will use the protocol and host of the current page). If you already use _url, then you are one step ahead in the game.


Here is a sample of some of my routes using the protocol option:

# Public routes using http protocol
map.with_options :protocol => "http" do |http|
http.login '/login', :controller => 'sessions', :action => 'new'
http.logout '/logout', :controller => 'sessions', :action => 'destroy'
http.resources :users, :sessions, :requirements => {:protocol => "http"}
end

# Public routes using https protocol
map.with_options :protocol => ROUTES_PROTOCOL do |https|
https.form_step '/form/*slug', :controller => 'forms', :action => 'show'
https.form_edit '/edit/*path', :controller => 'forms', :action => 'edit'
https.resources :forms, :requirements => {:protocol => ROUTES_PROTOCOL}
end

Wednesday, May 23, 2007

Different ways to program

I've been told that there are different ways to program. One way is procedural, where you work with something (let's say a shopping cart), and it does one thing, then it does another, then another, and so on until oblivion. Then there is object-oriented programming, where you have objects that you deal with (like in real life!) and not just random things that you happen to write code about. So your shopping cart has attributes, like it can hold items, you can remove items from it, and when it is time to checkout you remove all of the items from the cart and pay for the items. And just like at the grocery store, you leave the cart at the store (hopefully).

I guess the difference programming wise is that procedural programming you just write a whole slew of functions that can be applied anywhere and it is the programmers job to decide when and where he should use them, whereas in OOP you have classes for your objects, and each class has specific functions to deal mainly with instances (objects) of the class. I'm sure you can have the guise of OOP and still be doing procedural programming.

I only mention all this because I was looking through a website that the client I am working for said to look at and try to mimic what they were doing with some javascript showing and hiding. It all boiled down to that they were toggling the CSS display property on DOM elements between "block" and "none" (the show() and hide() functions in Prototype). Well, I finally found that out after looking on in amazement at a 1211 line function that seemed to show and hide everything on the entire website! Notice that I said function. They just used this massive catch all function to do all the showing and hiding, instead of trying to somehow abstract it into some showing/hiding class thingy (don't ask me how I would do it though, because I have no idea). Maybe I would have done it like that if I wouldn't have gone insane trying to remember exactly where I was in the 1200+ lines of code.

And as a side note, look at this code:



if (getFormObject("reslease","taxes")[0].checked) {
setVisDisp('taxesDiv',true);
if (getFormObject("reslease","taxesTenant")[0].checked) {
setVisDisp('taxesTenantDescDiv',true);
} else {
setVisDisp('taxesTenantDescDiv',false);
}

if (getFormObject("reslease","taxesGovt")[0].checked) {
setVisDisp('taxesGovtDescDiv',true);
} else {
setVisDisp('taxesGovtDescDiv',false);
}

} else {
setVisDisp('taxesDiv',false);
}


That was from line 1070 to 1086. After almost 1100 lines of madness, how do really know that you want the first element from the "reslease" form? And what is a "taxesTenantDescDiv"? My mentor has been drilling it into my head to make whatever your coding self-descriptive, so that when you go back you never have to wonder what the "taxesTenantDescDiv" is, or where it lives.

Wednesday, April 25, 2007

Regular Expression Finding in TextMate

Recently I had to convert a list of items in a word doc over to a yaml file in order so that I could load the data into our database. The word doc only contained the data necessary that a human could read and understand what was needed (meta data), but looked nothing like what the final yaml needed to look like. So I had two options, either sit there for 2 hours typing everything by hand over to yaml, or spend 20 minutes learning how to use the regular expression finding feature in TextMate. I went with the latter and am oh so glad I did. $1 is now my friend.


If you haven't tried it yet, play around with it ad see how awesome it can be!

Sunday, April 15, 2007

Selenium Remote Control Plugin!

I recently had a need to get the selenium testing working for another app I was working on and had to copy a bunch of files and whatnot, which I knew was bad. So I took about an hour the other morning and turned it into a plugin for all the world to enjoy.

Here's the repository. Enjoy!

Wednesday, March 14, 2007

ActiveRecord magic

Or so Josh Susser puts it. I think it is. It's dynamic finding. I had no idea this was possible until a few days ago when i wrote some moderately-ugly looking method like:



def find_or_create_order
unless @order = Order.find(params[:blah])
@order = Order.create!(params[:blah])
end
end

Then one of the geniuses (since I am working with multiple geniuses) I am working with right now came over and said "You can just use Order.find_or_create_by_blah(params[:blah])." I tried it out, and to my amazement (and I am easily amazed, and amused) it worked! I could try to explain it, but I think Josh Susser has a far greater understanding of how it works, so just go read his post if you're interested.


I did however do some playing, and found some really cool things you can do with these finders. Let's say you have an ActiveRecord model named Order. Let's also say that Order has 7 attributes on it, each labeled "1" through "7". You can write a massive line of code to find or create an Order as follows:



Order.find_or_create_by_1_and_2_and_3_and_4_and_5_and_6
_and_7("1", "2", "3", "4", "5", "6", "7")

The possibilities are endless!!!! Or so I imagine. One easy way to find out is to make an ActiveRecord model with 50 attributes and try to find_or_create one using the above method :-) I'm sure there are more sophisticated ways of using this, but just think of the shock value when you show this to a competent programmer. Their jaw will drop, and they'll say something like "You're smoking crack, man!"

Wednesday, February 14, 2007

Selenium and Ruby: They actually work together!

So we found out this week. Just wanted to do a brain-dump so I could remember how to configure the Selenium Remote Control to work with Ruby on Rails, and hopefully impart knowledge to those who wish to play with Selenium.

For a refresher, Selenium is a testing framework that allows you to test the clicking-around aspect of your web application. It allows you to record clicks and all sorts of actions that you may routinely perform to test whether you have broken something in your web application after changing some code. I recently started using this week after having some major changes in a application that I am working on with a friend.

A major portion of code was changed from being mostly server-side code (with a ton of SQL calls) to client side (using JavaScript). This threw a wrench into our testing of the application because we could no longer test the actions that one might perform on the screen because there were very few calls being made to the server through Rails (it was all being done with JavaScript in the browser). So to test this we decided to go with the most excellent program Selenium. The following is a somewhat detailed overview of how to get Selenium tests working with Ruby and Rails, and by that I mean being able to write your Selenium tests with Ruby code:

  • Get Selenium IDE (the Firefox extension).

  • Download the Selenium Remote Control and stick the unzipped folder into the lib directory of your application; you may wish to delete all the extraneous garbage, since all you will need is the ruby folder and the server folder. Also, in your lib directory make a file named selenium.rb, and the only thing needed in the file is require "selenium/ruby/selenium-ruby". You will see why later.

  • Start a test server (script/server -e test --port 3001) for selenium to work with so it doesn't try to use your development db. Choosing the port number is optional, but you will need to know which port this server is running on to use in your test file.

  • Start a java server (make sure you have JRE 1.5 or higher) by navigating to the selenium/server folder in your lib directory on a command line and type java -jar selenium-server.jar. This will start a Jetty server on port 4444.

  • Make a folder in your test directory named selenium. This is where you will put your selenium tests when you make them.

  • Save some recorded tests from the Selenium IDE in your test/selenium folder. You will need to perform the test in the IDE and then export the tests as "Ruby - Selenium RC". This will package the normal table tests into some handy Ruby code. You will need to rename the test class to be a proper name (something like MyNewSeleniumTest) and also whack the require 'test/unit' and replace it with require File.dirname(__FILE__) + '/../test_helper'.


Your selenium test should look something like the following, and should follow the normal Rails test-naming conventions "my_new_test.rb":


require File.dirname(__FILE__) + '/../test_helper'
require "selenium"

class MyNewSeleniumTest < Test::Unit::TestCase
def setup
@verification_errors = []
if $selenium
@selenium = $selenium
else
@selenium = Selenium::SeleneseInterpreter.new("localhost",
4444, "*firefox", "http://localhost:3001", 10000);
@selenium.start
end
@selenium.set_context("test_new", "info")
end

def teardown
@selenium.stop unless $selenium
assert_equal [], @verification_errors
end

def test_new
@selenium.open "/"
@selenium.type "b-query", "hey yo"
@selenium.click "b-searchbtn"
@selenium.wait_for_page_to_load "30000"
@selenium.click "link=Privacy Policy"
@selenium.wait_for_page_to_load "30000"
end
end


The line require "selenium" will call the selenium.rb file in your lib folder, which calls the selenium-ruby.rb file. This nice trick avoids having to load and use the selenium gem. You can, however, download the gem if you wish (gem install selenium) and kill the selenium.rb in your lib folder, and everything will still work properly.

Pay close attention to this line in the setup method: @selenium = Selenium::SeleneseInterpreter.new("localhost", 4444, "*firefox", "http://localhost:3001", 10000). The "http://localhost:3001" should be the port that your test server is running on. We banged our heads against a wall for quite a while until we figured that out. You can also choose Firefox or (::dread::) IE as the browser that Seleium will open. Mac users may want Safari, but we haven't checked to see about any surprises with Safari and Selenium.

That should be all you need (not that much, right?). Now you can run your tests from the command line like you would any other individual test (in Win-doze: ruby test\selenium\new_selenium_test). You can set up a rake task (to go into your lib/tasks folder) to run all of your tests for you pretty easily, like the following one we set up:


namespace :test do
Rake::TestTask.new(:selenium => "db:test:prepare") do |t|
t.libs << "test"
t.pattern = 'test/selenium/**/*_test.rb'
t.verbose = true
end
Rake::Task['test:selenium'].comment =
"Run the selenium tests in test/selenium"
end


Now you can simply call rake test:selenium from a command line (with your test server and Jetty server running) and your tests will pop up in a new browser window and spit out the results at the end. If you do run any of the tests individually without using the rake task you will be faced with the problem of your test db not clearing out (for some unknown reason Rails does not seem to want to treat the Selenium tests as normal tests, and thus does not prepare the test database before running each test). We struggled with this for quite a while and had to finally dig into the rake tasks to find out how to tell MYSQL what we wanted it to do. The result was a small smorgasbord of ActiveRecord::Base commands to make a schema of the current test db, clear the db completely, and then rebuild the db using the just made schema. The following code was put into a test helper (which was required by the test file by using the test_helper :selenium call) and called inside of the setup of the test:


def prepare_test_db
abcs = ActiveRecord::Base.configurations
ActiveRecord::Base.establish_connection(abcs[RAILS_ENV])
File.open("#{RAILS_ROOT}/db/#{RAILS_ENV}_structure.sql", "w+"
) { |f| f << ActiveRecord::Base.connection.structure_dump }
ActiveRecord::Base.connection.recreate_database(abcs["test"]["database"])
ActiveRecord::Base.establish_connection(:test)
ActiveRecord::Base.connection.execute('SET foreign_key_checks = 0')
IO.readlines("#{RAILS_ROOT}/db/#{RAILS_ENV}_structure.sql"
).join.split("\n\n").each do |table|
ActiveRecord::Base.connection.execute(table)
end
end


Now you're test db will be cleared out no matter how you invoke the test.

Now, go and use this information for the greater good. Share it and love it.