Tuesday, May 20, 2008

New blog

I'm ditching blogspot and using my own install of Radiant at iannopollo.com. Check it out

Sunday, February 17, 2008

Mail Fetcher - Fetch Email from Ruby and Rails

I recently had to write something to fetch emails from a GMail account and process the emails to check if the email was a bounce (bad email address). If so, the person in the system associated with the bounced email would be deleted from the database. I did some searching for such thing and found this:

How to receive emails with Rails

About half way down the page you see an example for polling emails from a POP3 server (edited for space):

require 'net/pop'
require File.dirname(__FILE__) + '/../config/environment'

logger = RAILS_DEFAULT_LOGGER

logger.info "Running Mail Importer..."
Net::POP3.start("mail.server.net", nil, "username", "password") do |pop|
if pop.mails.empty?
logger.info "NO MAIL"
else
pop.mails.each do |email|

begin
logger.info "receiving mail..."
Mailman.receive(TMail::Mail.parse(email.pop))
email.delete
rescue Exception => e
logger.error e.message
end

end
end
end
logger.info "Finished Mail Importer."

So try this with a GMail server and you get no connection. Why? Because Google uses SSL when logging in via POP3. So the I looked around and found someone using SSL to login via POP3. All they did was add this line right before trying to start the connection: Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_NONE)

Simple enough. I went ahead and tried that, and it blows up with the following error: undefined method `enable_ssl' for Net::POP3:Class. Now trying to find out why this didn't work was annoying, but I finally found out that in the current version of Ruby (1.8.6), Net::POP3 doesn't understand enable_ssl. The edge version, however, does. So, I got the net/pop.rb file and required it instead of the 1.8.6 version and voila! It works! So, instead of just posting this and letting everyone know that you have to do this to make SSL POP3 access work, I made a plugin:

Mail Fetcher

Now the whole world may enjoy SSL POP3 access to their hearts' content. And, it also works with IMAP. Checkout the README and mail_fetcher.rb to get all the details. Enjoy!

Monday, February 11, 2008

Talking to Gmail with Ruby and Rails

Recently I came across a dilemma in trying to send and receive emails from a Rails app: Either install a full-blown postfix server on a Linux box (by full-blown, I mean account handling, POP/IMAP interface, anti-spam, anti-virus, etc etc), or use Google to handle all the emailing. So, me not being a Linux/sys-admin junkie, I strongly opted for the Google route (and after some persuasion the customer, thankfully, agreed :-). So I went about to find the best practices for working with Google Apps (http://www.google.com/a which is where you sign up). I came across this very helpful post.

You just install the mentioned plugin, create the appropriate mailer.yml file and you're good to go. ActionMailer will login as the specified account and send emails through that account. Genius!

On a side note, I was able to get a Rails app to talk (no sending though, just receiving) to gmail by using the new net/pop library in the latest ruby and making the Mail Fetcher plugin.

But back to the main point, being able to send through the Google Apps account. The technical stuff seems really easy now. Just send your emails out like normal and have some way to check responses either manually or programmatically like I showed above. You're golden, or so it seems.

If you have an application like the one I'm working on, you have many emails being sent for single actions (one to the person who submitted a form, and then 10 - 100 emails being sent to others notifying them of the form submission as well). That seems fine up until you read through the Google Apps help center and find out that you can only send emails to 500 addresses each day per account (so either 1 email message with 500 recipients, or 500 emails each with one recipient).

If you have a commercial application, that is a serious limitation, and one that might stop you from using Google as your email hosting solution.

The important thing to note though is that the limitation is on one account per day. With Google Apps, you receive your Google Apps account which allows you to administer certain things like a calendar, file uploads (which at this point is pretty lame), chatting, and email. Google allows you to create users who can use these features in your domain. So you can create john@mydomain.com and mildred@mydomain.com, and they can have an email account, read your files, chat with each other, and check the mydomain.com calendar. The accounts that get limited are the accounts you create, not your Google App account. There is hope! Google allows you to create quite a few accounts for your domain (200 when you sign up, and you can always ask for more).

What that means is that you are not limited to 500 emails per day, but really 100,000 emails per day (200 * 500). That's a lot of traffic! The obvious way to use all of it is to create a bunch of accounts (don't forget to enable either POP3 or IMAP access!) and randomly send from them all. Just set up account names like "system1@example.com" and "system2@example.com" and randomly choose which one to send from each time you send out an email. It just works, and it distributes the load of sent emails pretty well across all email accounts. Implementation wise, I did something like this.

It ends up sending from email addresses "system100@example.com" through "system110@example.com". Also, in that example I put in the ActionMailer::Base.smpt_settings for clarity, but in reality I have them in the config/mailer.yml file used by the action_mailer_tls plugin.

Hopefully this will help someone who is on the fence about using Google Apps as a solution to email after finding out about the sending limit. Enjoy!

Saturday, November 3, 2007

bash aliases

Bash aliases are such lovely things, so I thought I would share some of my favorites and maybe even get some feedback.

Most of my aliases are there to make software development more enjoyable and less time consuming. As such, most of them deal with the projects I have in my project directories. Here are some of the more interesting ones:


for bigdir in $(ls ~/Developer/Projects/ | cut -d' ' -f7)
do
for dir in $(ls ~/Developer/Projects/$bigdir)
do
alias cd$dir="cd ~/Developer/Projects/$bigdir/$dir"
alias m$dir="cd$dir && mate ."
alias s$dir="cd$dir && ss"
alias g$dir="~/bin/svngrowl $dir"
alias o$dir="osascript -e 'tell application \"Terminal\"'
-e 'tell application \"System Events\"'
-e 'tell process \"Terminal\" to keystroke \"t\" using command down'
-e 'tell process \"Terminal\" to keystroke \"t\" using command down'
-e 'end tell'
-e 'do script \"s$dir\" in first tab of front window'
-e 'do script \"m$dir\" in second tab of front window'
-e 'do script \"g$dir\" in third tab of front window'
-e 'do script \"localhost\" in second tab of front window'
-e 'tell second tab of front window to set selected to true'
-e 'end tell'"
done
done

alias localhost="open http://localhost:3000"
alias addall="svn st | grep ? | cut -d' ' -f7 | xargs svn add"
alias removeall="svn st | grep ? | cut -d' ' -f7 | xargs rm -Rf"

for var in "server" "generate" "console" "destroy" "plugin"
do
alias s${var:0:1}="script/$var"
done


The first for loop is probably the most useful set of aliases that I have. It starts off by going through all the projects that live in my various project directories. Then, for each project it gives me a cdproject, mproject, sproject, gproject (very special), and oproject (less special, but still useful).

cdproject
  • cd's into the project directory

mproject
  • opens the project in TextMate

sproject
  • starts a server for the project by calling script/server (which is what the alias ss does in a rails project)

gproject
  • starts an svngrowl process for the project (which I'll explain later)

oproject
  • Uses the command line osascript, which invokes applescript, to manipulate Terminal and Firefox


First, an explanation of what oproject is doing. To start, for viewing purposes above I cut the oproject alias into more manageable pieces, but when used should be on one line (as would be the case if it were called from the command line).

This alias is kind of like the Ring of Power, in that it rules all the other aliases, and in the darkness (or the light) binds them... into one convenient call. It opens Terminal, and opens 2 new tabs for a total of 3 tabs (only for Leopard users though :-). It then opens a server in the first tab, opens the project in TextMate in the second tab, starts an svngrowl process in the third tab, goes back to the second tab and runs the alias localhost, and then gives focus back to the second tab (since it seemed to lose focus to the third tab, not sure why). This alias is a huge time saver when you think that every working day I spend about 10 to 15 seconds when I have to open everything up (which adds up to a lot when you have enough work to do!).

As for the gproject alias, this uses a cool little script in my ~/bin directory that starts a loop for the specified project, and every 3 minutes it runs svn up on the project and if there are any changes to the project, it sends out a pretty little growl notification with what was changed, who changed it, the commit message, and if any migrations were committed it changes the priority of the message to high (which for my personal growl settings turns the message text red).

Two scripts are involved in the growling, one bash script, and a ruby script (which is invoked by the bash script). First, the bash script (cut in half for easy viewing):


#!/bin/sh
source ~/.profile
alias | grep cd$@= | awk "{gsub(/cd$@='cd |'/, \"\"); print}" |
awk "{gsub(/~/, \"/Users/`whoami`\"); print}" | xargs ~/bin/svngrowl.rb

And now the ruby script:

#!/usr/bin/env ruby
begin
directory = ARGV[0]
project = directory.split("/").last
ignore = directory.gsub(project, "")

loop do
updates = `svn up #{directory}`.gsub(ignore, "").split("\n")
last_log = `svn log #{directory} --limit 1`

revision = updates.pop.scan(/\d+/)
committer = last_log.split("\n")[1].split(" | ")[1]
priority = updates.join("\n").include?("db/migrate") ? 1 : 0
message = last_log.split("\n")[2..-2].join("\n")

unless updates.empty?
`growlnotify -s -t "#{project} Update - #{committer}"
-m "#{message + "\n\n" + updates.join("\n")}" -p #{priority} --image ~/bin/mph.png`
end
sleep 180
end
rescue
`~/bin/svngrowl.rb #{directory}`
end


All the bash script does is finds the proper directory for the project in question and passes it to the ruby script. The ruby script in turn does an svn up on the directory, collects all the interesting data, and runs growlnotify (the command line growl utility) displaying all of the interesting data. There is also a rescue statement in there that just restarts the process if something goes wrong (an error is thrown if svn can't update properly, which can happen if the network drops out or if you do svn st at the same time that this process is doing svn up). I also used the --image options to pass in a beautiful picture of Mr. Potato Head to use as the icon shown on the notification.

Please feel free to use any of these to make your work/play more productive (except for the picture of Mr. Potato Head... that one is mine!)

Tuesday, October 2, 2007

sfight!

I played around yesterday when I got bored and wrote a little script which pits developers on a project against each other to see who has the most subversion commits. It doesn't really mean too much who wins since you can have plenty of little commits that fix things, but it is just fun to do to see who wins! This script assumes the developers on the project have separate subversion logins. Enjoy!


#!/usr/bin/env ruby
developers = ARGV.select {|arg| !arg.empty? && !arg.nil?}
raise "You need two developers to fight" if developers.size < 2

log = `svn log`
separator = log.scan(/^-.*?-$/).first
log_entries = log.split(separator)[1..-2]
names = log_entries.collect {|entry| entry.split(" | ")[1]}

numbers = developers.collect {|developer| names.select {|entry| entry == developer}.size}

results = []
developers.each_with_index {|developer, i| results << "#{numbers[i].to_s.ljust(3)}- #{developer}"}

puts results.sort {|x, y| y.scan(/\d+/).first.to_i <=> x.scan(/\d+/).first.to_i}


Use it like this (assuming you chmod'ed it to be executable, and put it into your ./script directory):

./script/sfight developer1 developer2 developer3