Sunday, February 17, 2008

Mail Fetcher - Fetch Email from Ruby and Rails

I recently had to write something to fetch emails from a GMail account and process the emails to check if the email was a bounce (bad email address). If so, the person in the system associated with the bounced email would be deleted from the database. I did some searching for such thing and found this:

How to receive emails with Rails

About half way down the page you see an example for polling emails from a POP3 server (edited for space):

require 'net/pop'
require File.dirname(__FILE__) + '/../config/environment'

logger = RAILS_DEFAULT_LOGGER

logger.info "Running Mail Importer..."
Net::POP3.start("mail.server.net", nil, "username", "password") do |pop|
if pop.mails.empty?
logger.info "NO MAIL"
else
pop.mails.each do |email|

begin
logger.info "receiving mail..."
Mailman.receive(TMail::Mail.parse(email.pop))
email.delete
rescue Exception => e
logger.error e.message
end

end
end
end
logger.info "Finished Mail Importer."

So try this with a GMail server and you get no connection. Why? Because Google uses SSL when logging in via POP3. So the I looked around and found someone using SSL to login via POP3. All they did was add this line right before trying to start the connection: Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_NONE)

Simple enough. I went ahead and tried that, and it blows up with the following error: undefined method `enable_ssl' for Net::POP3:Class. Now trying to find out why this didn't work was annoying, but I finally found out that in the current version of Ruby (1.8.6), Net::POP3 doesn't understand enable_ssl. The edge version, however, does. So, I got the net/pop.rb file and required it instead of the 1.8.6 version and voila! It works! So, instead of just posting this and letting everyone know that you have to do this to make SSL POP3 access work, I made a plugin:

Mail Fetcher

Now the whole world may enjoy SSL POP3 access to their hearts' content. And, it also works with IMAP. Checkout the README and mail_fetcher.rb to get all the details. Enjoy!

Monday, February 11, 2008

Talking to Gmail with Ruby and Rails

Recently I came across a dilemma in trying to send and receive emails from a Rails app: Either install a full-blown postfix server on a Linux box (by full-blown, I mean account handling, POP/IMAP interface, anti-spam, anti-virus, etc etc), or use Google to handle all the emailing. So, me not being a Linux/sys-admin junkie, I strongly opted for the Google route (and after some persuasion the customer, thankfully, agreed :-). So I went about to find the best practices for working with Google Apps (http://www.google.com/a which is where you sign up). I came across this very helpful post.

You just install the mentioned plugin, create the appropriate mailer.yml file and you're good to go. ActionMailer will login as the specified account and send emails through that account. Genius!

On a side note, I was able to get a Rails app to talk (no sending though, just receiving) to gmail by using the new net/pop library in the latest ruby and making the Mail Fetcher plugin.

But back to the main point, being able to send through the Google Apps account. The technical stuff seems really easy now. Just send your emails out like normal and have some way to check responses either manually or programmatically like I showed above. You're golden, or so it seems.

If you have an application like the one I'm working on, you have many emails being sent for single actions (one to the person who submitted a form, and then 10 - 100 emails being sent to others notifying them of the form submission as well). That seems fine up until you read through the Google Apps help center and find out that you can only send emails to 500 addresses each day per account (so either 1 email message with 500 recipients, or 500 emails each with one recipient).

If you have a commercial application, that is a serious limitation, and one that might stop you from using Google as your email hosting solution.

The important thing to note though is that the limitation is on one account per day. With Google Apps, you receive your Google Apps account which allows you to administer certain things like a calendar, file uploads (which at this point is pretty lame), chatting, and email. Google allows you to create users who can use these features in your domain. So you can create john@mydomain.com and mildred@mydomain.com, and they can have an email account, read your files, chat with each other, and check the mydomain.com calendar. The accounts that get limited are the accounts you create, not your Google App account. There is hope! Google allows you to create quite a few accounts for your domain (200 when you sign up, and you can always ask for more).

What that means is that you are not limited to 500 emails per day, but really 100,000 emails per day (200 * 500). That's a lot of traffic! The obvious way to use all of it is to create a bunch of accounts (don't forget to enable either POP3 or IMAP access!) and randomly send from them all. Just set up account names like "system1@example.com" and "system2@example.com" and randomly choose which one to send from each time you send out an email. It just works, and it distributes the load of sent emails pretty well across all email accounts. Implementation wise, I did something like this.

It ends up sending from email addresses "system100@example.com" through "system110@example.com". Also, in that example I put in the ActionMailer::Base.smpt_settings for clarity, but in reality I have them in the config/mailer.yml file used by the action_mailer_tls plugin.

Hopefully this will help someone who is on the fence about using Google Apps as a solution to email after finding out about the sending limit. Enjoy!