This mashup plugin allows your Rails website or application to have a proxy mailing list feature that includes email, SMS messages, and fax. A normal mailing list allows a website to send their messages to its list of visitors or users. A proxy mailing list allows a third-party user to send the messages to their own list of recipients, people who are previously unknown to the website, on behalf of the website.
Your boss calls you in to discuss a new feature for your company's website. There is a new marketing initiative for your company's products. You have been chosen to build in a new feature for the marketing folks on the 14th floor that allows them to send out regular email marketing messages to clients and potential clients. You dutifully write down the requirements for this mailing list feature:
Import contacts from a spreadsheet containing the name of the client or potential client into an internal database
Allow internal marketing users to define email campaign messages
Send email messages to the clients and potential clients at regular intervals
Doesn't look too hard! You walked away confidently.
The next day your boss calls you in again. Oh surprise—there is a new requirement. After some meetings, the marketing people decided that sending email messages is not enough, now they want to send SMS messages and send faxes as well. Because your clients are scattered around the world, the SMS messages and faxes also need to be sent around the world! Sighing expectedly, you write down the additional new requirements:
Allow internal marketing users to define SMS and fax messages
Send SMS messages to clients and potential clients
Send faxes to clients and potential clients
Messages need to be sent worldwide
You should have known better than to expect finalized requirements from the first meeting!
You were still figuring out how to send SMS messages the next day when your boss called for an emergency meeting with you. With a sinking feeling you walked into his room.
The marketing people originally wanted to consolidate the contacts data from your company's resellers around the world into an spreadsheet file, which is used to feed into the website. However because of data privacy (and you suspect, other commercial) issues, the resellers now refuse to give your company the contacts data directly. Instead, the marketing people have struck a deal to let the resellers send the marketing messages by themselves through your website. Worse, you are no longer allowed to store any contacts information in your database.
You're stuck now! You don't have the contacts data and now you need to let the resellers send the marketing messages? How can you do it?
You consolidated the following (hopefully final now) set of requirements:
Allow internal marketing users to define email, SMS, and fax messages in a template
Allow the external resellers to customize the pre-defined marketing messages according to their list of contacts
Send email, SMS, and fax messages to the clients and potential clients around the world on a regular basis
You cannot store any contacts information
Mashups to the rescue!
Let's see how we can use Rails and some mashup magic to build this new feature for your website.
Create a simple Rails web application that allows the marketing user to construct message templates that contain the messages to be sent via email, SMS text messaging, and fax. This template can be used later as the basis for sending various messages to the recipients of the marketing blitz.
We need to get the reseller's contact information without storing it in our database. To do this, we need to get the resellers to define or import their contacts into an online spreadsheet. After that, the resellers need to export the spreadsheet in a data format that is suitable for extraction and processing by our website. For this chapter, we will show how this can be done through Google Spreadsheet and EditGrid, two popular online spreadsheets. The reseller will upload his or her contacts from his or her spreadsheet into Google Spreadsheet or EditGrid, and then publish a link to this spreadsheet. Subsequently the reseller will select a message template and create a message-sending job by providing this link to his or her contacts.
The number of contacts provided by each reseller is normally high, so it is unrealistic to send the messages interactively (that is, to send the message on the click of a button in the user interface and wait for a response that indicates that the messages all sent). There are two alternatives for sending messages in a non-interactive way. One method is to create a threaded job that is separate from the main process and run it in the background when the reseller clicks on the send button. The other method is to create jobs that are stored in the database and retrieved separately by another process at regular intervals to be processed independently. I have chosen to use the second method of executing the send-message tasks in this chapter, as it is the simpler of the two.
Emails can be done easily through ActionMailer, with messages captured from the message template. However, SMS and fax messages are more complex. Let's go through some basic background knowledge on these types of messages before coming up with the strategies on sending them.
SMS or Short Message Service is a technology that enables the sending and receiving of messages between mobile phones. SMS was part of the GSM (Global System for Mobile Communications) standards at the beginning but was later ported to technologies like CDMA and TDMA. SMS is very popular and widely used as it is supported by all GSM mobile phones.
When an SMS message is sent from a mobile phone, it will reach an SMSC in the GSM network. The SMSC then forwards the SMS message towards the destination, passing through one or more network elements, including other SMSCs. If the recipient is unavailable (for example, when the mobile phone is switched off), the SMSC will store the SMS message and forward it when the recipient is available.
An SMSC normally belongs to a single network. For SMSes to reach mobile phones in different GSM networks, an SMS gateway is used to bridge between SMSCs in different networks.
SMS gateways are also used to act as a concentrator that can access multiple SMSCs. This allows applications that send SMSes to channel their messages through a single gateway to multiple SMSCs without the need to connect to each SMSC individually. This is the model many bulk SMS providers (including Clickatell, the provider we're using in this chapter) use.
One SMS message can contain at most 140 bytes of data, so one SMS message can contain up to:
Besides text, SMS messages can also carry binary data like ringtones, pictures, operator logos, wallpapers, animations, VCards, and WAP configurations to a mobile phone. However, such uses are vendor-specific and are not as widely supported as text-based SMS.
Probably the best way to send SMS messages in bulk is through a bulk SMS provider. A provider with wide global coverage will also allow us to send SMS messages around the world. As most bulk SMS providers also provide APIs for developers, this ties in nicely with our requirements to send SMSes to a large number of people globally. Clickatell, the SMS provider we're using in this chapter provides a bulk SMS gateway and numerous APIs to connect to it.
Fax or facsimile is a telecommunications technology used to transfer copies of documents over the telephone network. Fax is most commonly used to send documents between two fax machines connected to a telephone line. A fax machine is a three-in-one machine, with a scanner, a modem, and a printer rolled into one. The scanner extracts an image of the document in digital form, the modem sends it across to the other fax machine via the telephone network, while the printer reproduces it on the other end, and vice versa. However Internet-based faxing services and multi-function printers are fast replacing the traditional standalone fax machines.
Various Internet fax services now provide email and API services for users to send faxes directly from a file, removing the need for scanning and sending via a telephone network. The best way to programmatically send faxes would be through a fax service provider and for this chapter we will be using Interfax and sending faxes through an XML-RPC-based web service.
After reviewing the strategy and determining the best way to design for the requirements, we have established the following mashup APIs that we will use for this chapter:
Google Spreadsheet and/or EditGrid for the user to store and share the contacts information
Clickatell to send SMS messages
Interfax to send faxes
The APIs in this chapter have free developer trial accounts so you can experiment with them a bit. However, the APIs we are using in this chapter are not free for full commercial use and they have some restrictions on their usage.
We will not be using any specific Ruby libraries for this mashup as the APIs we use doesn't require them. Instead we will be using standard Ruby libraries that are available out of the box from any Ruby installation, in particular the Net::HTTP
module, which allows us to connect to XML-RPC web services and REST-based HTTP APIs. We will also use the built-in CSV module to simplify conversion of CSV formatted data into arrays.
Let's run through these online APIs as well as the Net::HTTP
module.
Google Spreadsheets (http://docs.google.com) is an online spreadsheet service offered by Google as part of its Google Docs and Spreadsheets offering. Google Docs and Spreadsheets is a Web-based word processor and spreadsheet application that allows users to create and edit documents and spreadsheets online while collaborating in real-time with other users. Google Docs and Spreadsheets combine the features of two services, Writely and Spreadsheets.
Users can create the spreadsheet online or upload spreadsheets of various formats including CSV, Excel, and OpenDocument (.ods for spreadsheets). There is a limit to each spreadsheet, which is 10,000 rows, 256 columns, or 100,000 cells, whichever limit is reached first. When importing spreadsheets, the import file cannot be larger than 1MB in size. Users are also able to export their documents in various output formats, which include CSV, Excel, OpenDocument, PDF, and so on. Google Spreadsheets is free, and any Google user can get it as part of the package.
Google Spreadsheets has its own API, which uses part of the Google Data APIs. This API requires user account authentication to access information on private spreadsheets. However for this chapter we will not be using access-controlled data in Google Spreadsheets and only the published spreadsheets in a comma delimited (CSV) format.
EditGrid is a free online spreadsheet service with paid subscription for commercial/enterprise users. EditGrid allows users to create an online spreadsheet or import spreadsheets in various formats including CSV, Excel, and OpenDocument. Users are also able to export their documents in various output formats, which include CSV, Excel, OpenDocument, PDF, and so on.
You will need to register for a free personal account, which requires only a login user name and a password: http://www.editgrid.com.
As with Google Spreadsheets, we will not use any of the private APIs but opt to access the publicly available published spreadsheet in a CSV format.
Clickatell (http://www.clickatell.com) is a bulk SMS provider that provides SMS messaging services and gateway for over 600 networks in almost 200 countries for outbound messages, and 100 countries for inbound (two-way) messaging. Clickatell allows developers to connect to its SMS gateway via various connectivity options including HTTP/S, SMTP (email to SMS), XML, SMPP (Short Message Peer to Peer protocol), FTP (file upload for SMS), and a COM object API. This provides applications with the ability to send SMSes globally, bypassing the need to hook up to local SMS providers individually.
Clickatell provides a trial account with 10 credits to allow developers to have a head start in trying out its services. To create a trial account, go to https://www.clickatell.com/central/user/client/step1.php.
Fill in the account creation form. You will be required to provide a mobile number. This is important as in Step 2 a confirmation code is sent to your mobile number, before a trial account is provided.
After entering your confirmation code from both your email and mobile phones, you will be presented with the homepage of your account.
Notice the balance you have at the top of the screen. This is number of credits you have been given for the trial account. If you run out you can click on a conveniently located button to buy more credits through various payment methods. Take note of the Client ID provided, you will need it when you log into the Clickatell management console. To allow for access to an HTTP API, click on the Manage My Products link.
Then from the drop-down list, select HTTP.
Select an appropriate name for this API. Optionally you can set the Dial Prefix if you are sending a lot of SMSes to a particular country. You can also set a Callback (to be explained in a later section) for Clickatell to call when reporting the status of the sent message. You can also specify that only a particular IP address, or set of IP addresses, can access this API. When you're done, click on the Submit button.
Take note of the API ID given here. We will need it when trying to access the HTTP API from our mashup.
Interfax is a fax service provider that has coverage in a large number of countries. Interfax provides a number of API interfaces to its fax server including web services, COM object API, and an email interface. Interfax also provides reports on faxes that are sent or received through its servers. With Interfax, an application is able to send text faxes, faxes from base-64 encoded binary documents like PDF documents, Excel spreadsheets, and Word documents as well as receive faxes as TIF formatted images.
Interfax provides a free trial account for developers but faxes sent from this trial account will only reach a single fixed fax number. To be able to send to any other fax numbers you need to subscribe to its commercial account. To sign up for its free trial account, go to http://www.interfax.net/Scripts/Reg_BP.asp and fill in the account creation form.
After creating the account, you will be given $10 for faxing pages and a business partner ID. The business partner ID is not needed for sending faxes.
Ruby comes with a standard library for managing HTTP connections. While it is not very sophisticated, it provides the essential and basic capabilities for an application to connect to web applications through HTTP.
A basic HTTP Get
command in Net::HTTP
goes like this:
require 'net/http' response = Net::HTTP.get_response(URI.parse ('http://www.packtpub.com')) puts response.body
This sends an HTTP Get
to Packt Publishing's website and returns what the web server delivers through that URL as a Net::HTTPResponse
object.
A basic HTTP Post
command in Net:HTTP goes like this:
require 'net/http' response = Net::HTTP.post_form(URI.parse ('http://search.yahoo.com/search'), {'p'=>'mashups'}) puts response.body
This code sends an HTTP Post to Yahoo's search engine with the parameter 'mashups' and it returns whatever search results are served from Yahoo, as a Net::HTTPResponse
object.
To react according to the returned Net::HTTPResponse
object, we need to inspect this object more closely. Net::HTTPResponse
is actually the parent object of a hierarchy of HTTP response statuses. In order to check the status of the sent command, we can run the response object through a case loop to check the actual subclass and respond accordingly.
require 'net/http' response = Net::HTTP.post_form(URI.parse ('http://search.yahoo.com/search'), {'p'=>'mashups'}) case response when Net::HTTPSuccess puts response.body else puts response.error! end
This code indicates that if the response object is of the class Net::HTTPSuccess
(HTTP code 2xx) it will print out the response body, otherwise it will just print out the error code.
We will be using Net::HTTP
extensively in this chapter. In later chapters we will introduce another built-in Ruby package that performs a similar function.
Although this is a mashup plugin, meaning it is normally added to an existing Rails application, we will be creating a new project to show how it can be used. This is the process flow of the mashup:
The marketing user will create a marketing message template with the email, SMS, and fax messages.
The reseller selects the message template to send and provides a link to the list of contacts to send the message to, then creates a message-sending job.
At regular intervals, the system will check for pending jobs, process them and send all messages to the respective contacts.
This is what we will be doing in the next few pages to implement this mashup:
Create a Rails project
Configure the database access and create the database
Create the standard scaffolding
Allow the marketing users to create the message templates
Allow the reseller to provide contacts data through a remote link
Create the rake script to send messages at regular intervals
This mashup's main processing is not in the web application itself. The Rails web application is used to get input from the various parties i.e. the message template from the marketing user and the contacts data from the external reseller. As explained earlier, the actual processing and sending of the messages is done outside of the web application in the rake script. The rake script is triggered periodically by a scheduler like cron
in Unix or at
in Windows.
As before, creating the Rails project is the easiest part.
$rails Chapter3
This will create a new blank Rails project.
The Rails web application is basic and the database needed to support it is simple as well. Change the necessary environment configuration file (development.rb
for a development environment) to configure access to the database. Then generate a migration file to create the database:
$./script/generate migration create_templates_and_jobs
This will create a file 001_create_templates_and_jobs.rb
in the RAILS_ROOT/db/migrate
folder. Ensure it has the following code:
class CreateTemplatesAndJobs < ActiveRecord::Migration def self.up create_table :message_templates do |t| t.column 'name', :string t.column 'sms_body', :text, :limit => 160 t.column 'email_body', :text t.column 'fax_body', :text end create_table :jobs do |t| t.column 'message_template_id', :integer t.column 'contacts_url', :string t.column 'status', :string, :default => 'pending' end end def self.down drop_table :message_templates drop_table :jobs end end
Now that we have the migration scripts, run migrate to create the tables:
$rake db:migrate
This should create the database tables needed. The data model in this mashup is quite simple. The MessageTemplate
is the model of the message templates created by the marketing people, while each Job
is created by the reseller.
Next, create the standard scaffolding for the tables we've just created:
$./script/generate scaffold MessageTemplate
and:
$./script/generate scaffold Job
This will create the standard controllers, views, and models for the two classes. Next, change MessageTemplate
and Job
(both located in RAILS_ROOT/app/models/
) to reflect their relationship:
class MessageTemplate < ActiveRecord::Base has_many :jobs end class Job < ActiveRecord::Base belongs_to :message_template end
The standard scaffolding should already allow the creation of the message templates though we might want to do it up a bit if we are giving it to our marketing users! For testing purposes, go to: http://localhost:3000/message_templates/list
and create some sample marketing messages for each type of message. Remember that SMS messages should be short and succinct and each message is up to a maximum of 160 characters (when using standard ASCII characters).
Note that in the example above, we have put in some Ruby-like variables in the messages such as display_name
. We'll see how this is used in the coming sections.
Next we need to let the reseller provide the contacts to our mashup. The general strategy is to allow the reseller to create message-sending 'jobs' that have the links to the contacts information. The reseller will select the message template to use and provide a link, so a simple job creation form can do this. The scaffold should have most of the code already in place, so just modify the _form.rhtml
partial (RAILS_ROOT/app/views/jobs/
) to link Job
to MessageTemplate
.
<%= error_messages_for 'job' %>
<!--[form:job]-->
<p><label for="job_message_template">Message template</label><br/>
<%= select('job', 'message_template_id', MessageTemplate.find(:all).collect {|t| [ t.name, t.id ] })%></p>
<p><label for="job_contacts_url">Contacts url</label><br/>
<%= text_field 'job', 'contacts_url', :size => 100 %></p>
<!--[eoform:job]-->
Open: http://localhost:3000/jobs/new
and put in the spreadsheet URL (see below).
The link we are providing here is for Google Spreadsheets but it should be similar for EditGrid. Note that the URL is a link to the CSV format of the spreadsheet and not to the main document.
Before going into how this link is generated, let's first see from the reseller's perspective how he or she will upload his or her list of contacts to the Google Spreadsheet. Log into Google and go to http://docs.google.com.
Click on the Upload link on the main page and follow the instructions on the page to upload a spreadsheet.
The spreadsheet that the reseller uploads must have the following columns in the following order:
Display name
Additional text (for the marketing messages)
Email address
Mobile number (for SMS messages)
Fax number
All the columns are optional but the display name should be there at the very least, otherwise the marketing message will have no addressee! If there is a value under the email column, an email will be sent, if there is a value under the mobile column, an SMS will be sent, and if there is a value under the fax column, a fax will be sent.
Now that the spreadsheet is loaded up in Google Spreadsheets, the reseller can proceed to generate the link that publishes the contacts in CSV format. To generate this format, go to the document page of the Google Spreadsheet. Then click on the publish tab at the top right of the screen on the document page.
An options pane will be opened to the right of the screen. Click on the Publish now button.
The document is now published. For additional publishing options, including the option for CSV format, click on the More publishing options link at the bottom of the options pane.
The reseller will then see a new pop-up window with several drop-down select fields. Choose CSV in the file format select field, and click on the Generate URL button. The reseller will be presented with a URL at the bottom of the window. This is the link we will ask the resellers to enter in order to retrieve the contacts and other information.
Uploading to and publishing from EditGrid is almost the same. After logging into EditGrid, the reseller will be presented with his or her workspace.
Click on the Upload button to upload a spreadsheet. The format of the spreadsheet should be the same as the one described above in the Google Spreadsheets section. The reseller needs to set the permission to public read-only to allow our mashup to read the contact information.
When the information has been uploaded into the spreadsheet, click on File in the menu bar and select the Permalinks menu item. Permalinks are permanent links on EditGrid. EditGrid provides permanent links support to allow you to access their spreadsheets through easy-to-understand URLs.
The reseller will see a list of formats in which data from this spreadsheet can be exported. The reseller needs to choose the CSV format link and use that as the link to provide as the input in the job creation screen. Make a record of the CSV link.
This is where the main action starts. As explained in Chapter 2, rake is a build program much like make, but one built with Ruby syntax. Rake is integrated and used extensively in Rails for various tasks including database migration (we used db:migrate
earlier on in this chapter). We will be using rake to run a processing script that will get the data from the remote site and send the messages.
Create a rake script named process_jobs.rake
in the RAILS_ROOT/lib/tasks
folder:
require 'net/http' require 'csv' require 'soap/wsdlDriver' namespace :chapter3 do # Clickatell credentials $clickatell_api_id = <your Clickatell API ID> $clickatell_login = <your Clickatell user name> $clickatell_password = <your Clickatell password> # Interfax credentials $interfax_login = <your Interfax user name> $interfax_password = <your Interfax password> $interfax_driver = SOAP::WSDLDriverFactory.new('http://ws.interfax.net/dfs.asmx?WSDL').create_rpc_driver desc "Activated regularly by AT or cronjob to process all jobs" task(:process_jobs => :environment) do $clickatell_session_id = get_clickatell_session begin pending_jobs = Job.find_all_by_status 'pending' if pending_jobs.size > 0 puts "#{pending_jobs.size} jobs pending processing ..." else puts "All jobs has been processed!" end pending_jobs.each { |job| # parse and get contacts data contacts_data = [] csv = parse_data(job.contacts_url) puts "Found #{csv.size} contacts for this job!" csv.each { |line| contact = {} contact[:display_name] = line[0] contact[:additional_message] = line[1] contact[:email] = line[2] contact[:mobile_no] = line[3] contact[:fax_no] = line[4] contacts_data << contact } contacts_data.each { |contact| # send email if !contact[:email].nil? t1 = Time.now sent = Mailer.deliver_mail(job.message_template, contact) t2 = Time.now if sent puts "Email sent to #{contact[:display_name]} in #{(t2 - t1)} seconds" end end # send sms if !contact[:mobile_no].nil? t1 = Time.now sent = send_sms(job.message_template, contact) t2 = Time.now if sent puts "SMS sent to #{contact[:display_name]} in #{(t2 - t1)} seconds" end end # send fax if !contact[:fax_no].nil? t1 = Time.now sent = send_fax(job.message_template, contact) t2 = Time.now if sent puts "Fax sent to #{contact[:display_name]} in #{(t2 - t1)} seconds" end end } job.status = 'processed' job.save! } rescue puts "Error during sending messages : #{$!}" end end # -- end of main task -- # send fax through Interfax def send_fax(template, contact) $interfax_driver.SendCharFax( :Username => $interfax_login, :Password => $interfax_password, :FaxNumber => contact[:fax_no], :Data => template.message_body(:fax, contact)) end # send SMS through Clickatell def send_sms(template, contact) begin res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/sendmsg'), {'session_id' => $clickatell_session_id, 'cliMsgId' => template.id, 'to'=>contact[:mobile_no], 'from' => 'Chapter 3', 'text' => template.message_body(:sms, contact), 'callback' => '3', 'deliv_ack' => '1', 'req_feat' => '8192' }) case res when Net::HTTPSuccess, Net::HTTPRedirection puts "Successfully sent message to #{contact[:display_name]}" return true else puts res.error! return false end rescue puts "## Cannot send sms to #{contact[:display_name]}! : #{$!}" end end # get the clickatell session needed to send SMS messages def get_clickatell_session res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/auth'), {'api_id' => $clickatell_api_id, 'user'=> $clickatell_login, 'password' => $clickatell_password}) case res when Net::HTTPSuccess, Net::HTTPRedirection return res.body.split(': ')[1] else puts res.error! end end # parse data from a CSV file published by Google Spreadsheet def parse_data(url) res = Net::HTTP.get_response(URI.parse(url)) case res when Net::HTTPSuccess, Net::HTTPRedirection csv = CSV.parse(res.body) header = csv.shift return csv else puts res.error! end end end
We will go through this script part by part. First, we will be using the Net::HTTP, CSV, and SOAP packages that are default in our Ruby installation so we will require them at the top of the file.
namespace :chapter3 do # Clickatell credentials $clickatell_api_id = <your Clickatell API ID> $clickatell_login = <your Clickatell user name> $clickatell_password = <your Clickatell password> # Interfax credentials $interfax_login = <your Interfax user name> $interfax_password = <your Interfax password> $interfax_driver = SOAP::WSDLDriverFactory.new('http://ws.interfax.net/dfs.asmx?WSDL').create_rpc_driver
The Web Services Description Language (WSDL) is an XML-based language used to describe web services. WSDL is often used together with SOAP to define web services as in the case of Clickatell and Interfax. A web service consuming application (such as our mashup) reads the WSDL to find out what is available from the web service and how to access it. From there we can generate the proxies that we use to access the web service as if it were a call to a local object.
Preset your global credentials for Clickatell and Interfax. We should have gotten these credentials when we registered for the developer accounts earlier on. The last line of this section creates a SOAP client to communicate with Interfax. The constructor for WSDLDriverFactory
takes in a WSDL file provided by Interfax and creates the necessary local proxy. We will be using this proxy to send our fax.
desc "Activated regularly by AT or cronjob to process all jobs" task(:process_jobs => :environment) do $clickatell_session_id = get_clickatell_session begin pending_jobs = Job.find_all_by_status 'pending' if pending_jobs.size > 0 puts "#{pending_jobs.size} jobs pending processing ..." else puts "All jobs has been processed!" end pending_jobs.each { |job| # parse and get contacts data contacts_data = [] csv = parse_data(job.contacts_url) puts "Found #{csv.size} contacts for this job!" csv.each { |line| contact = {} contact[:display_name] = line[0] contact[:additional_message] = line[1] contact[:email] = line[2] contact[:mobile_no] = line[3] contact[:fax_no] = line[4] contacts_data << contact } contacts_data.each { |contact| # send email if !contact[:email].nil? t1 = Time.now sent = Mailer.deliver_mail(job.message_template, contact) t2 = Time.now if sent puts "Email sent to #{contact[:display_name]} in #{(t2 - t1)} seconds" end end # send sms if !contact[:mobile_no].nil? t1 = Time.now sent = send_sms(job.message_template, contact) t2 = Time.now if sent puts "SMS sent to #{contact[:display_name]} in #{(t2 - t1)} seconds" end end # send fax if !contact[:fax_no].nil? t1 = Time.now sent = send_fax(job.message_template, contact) t2 = Time.now if sent puts "Fax sent to #{contact[:display_name]} in #{(t2 - t1)} seconds" end end } job.status = 'processed' job.save! } rescue puts "Error during sending messages : #{$!}" end end # -- end of main task --
We start off the processing run by getting a Clickatell session:
This session is needed by Clickatell to identify that we are an authorized user to send SMS messages. Next, we get all pending jobs in the system and for each pending job:
pending_jobs = Job.find_all_by_status 'pending'
we get the contacts URL and parse it:
csv = parse_data(job.contacts_url)
Remember that the contacts URL is actually a link to a CSV file published from an online spreadsheet. Parsing it returns an Array of Arrays that has the title row truncated. We will take this Array of Arrays and create an Array of Hashes to make the code simpler to read:
csv.each { |line| contact = {} contact[:display_name] = line[0] contact[:additional_message] = line[1] contact[:email] = line[2] contact[:mobile_no] = line[3] contact[:fax_no] = line[4] contacts_data << contact }
This is our list of contacts!
Now that we have the contacts data, we iterate through each one of them to send the messages. We run the contact through three if loops (one for each of the communications methods) to see if there is any contact information in that row. We will send the message if there is, that is, if there is an email given we will send the email message:
sent = Mailer.deliver_mail(job.message_template, contact)
If there is a mobile number given we will send the SMS message:
sent = send_sms(job.message_template, contact)
If there is a fax number given, we will send the fax:
sent = send_fax(job.message_template, contact)
This is the main loop for this script; let's see how we can extract the contacts information from the online spreadsheet and the send methods for each of the communications channels next.
We extract the data from the online spreadsheet (either Google Spreadsheets or EditGrid) through the link that is given by the reseller in the job. To do this we use Net::HTTP to send an HTTP Get
to the server, which should return a plaintext CSV string embedded within the response object.
# parse data from a CSV file published by Google Spreadsheets or EditGrid def parse_data(url) res = Net::HTTP.get_response(URI.parse(url)) case res when Net::HTTPSuccess, Net::HTTPRedirection csv = CSV.parse(res.body) header = csv.shift return csv else puts res.error! end end
Next, we use the CSV module (also built into Ruby) to parse the CSV string into an Array of Arrays. We also remove the header so that we get only data in the returned Array object.
Sending the fax through Interfax is relatively simple. Using the local proxy we have created from the WSDL provided by Interfax, we need only call any of the provided methods. In this example we will use SendCharFax
, which sends a text message to the recipient.
# send fax through Interfax def send_fax(template, contact) $interfax_driver.SendCharFax( :Username => $interfax_login, :Password => $interfax_password, :FaxNumber => contact[:fax_no], :Data => template.message_body(:fax, contact)) end
The username
and password
parameters are self-explanatory. The faxnumber
submitted, however, needs to be in the international notation. The format is: + <Country-Code><AreaCode><Local number>. For example: A number in New York, NY, USA will look like: +12123456789, where:
+ is a constant;
1 is the USA country code;
212 is New York area code;
3456789 is the local number.
The data
in this case is the text fax message stored in the message template.
The SendCharFax
web service is a basic one that only sends text messages. Normally this is not realistic, as this will only send an ugly string message to the recipient.
To improve on this, we can use either Sendfax
or SendfaxEx_2
to send files as faxes. We can send files with types including Word documents, Excel spreadsheets, Acrobat documents, or HTML-formatted text. To do this we can get the marketing user to upload a document to the database, which can then be sent to the contacts (we will skip this to keep this chapter simple). Check out the Interfax developer site at http://www.interfax.net/en/dev/webservice/reference.html to get the details of the various web services that are provided by Interfax.
Sending an SMS through Clickatell is only slightly more complicated. We will need to get a session key from Clickatell first by logging in and presenting our credentials, and then use this session key to send the SMS messages.
# get the clickatell session needed to send SMS messages def get_clickatell_session res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/auth'), {'api_id' => $clickatell_api_id, 'user'=> $clickatell_login, 'password' => $clickatell_password}) case res when Net::HTTPSuccess, Net::HTTPRedirection return res.body.split(': ')[1] else puts res.error! end end
Clickatell provides many different ways for developers to access its services, but for this chapter we will be using its HTTP APIs. To use the HTTP APIs we use the Net::HTTP package to send an HTTP Post command to http://api.clickatell.com/http/auth with the various credentials as part of a Post form in the request body to get a session key. Note that we need to provide an API ID to Clickatell—this is the number we get from Clickatell when we register as a developer for that set of API services (see section above). The session ID is retrieved then stored in a global variable for use when we send the messages.
# send SMS through Clickatell def send_sms(template, contact) begin res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/sendmsg'), {'session_id' => $clickatell_session_id, 'cliMsgId' => template.id, 'to'=> contact[:mobile_no], 'from' => 'Chapter 3', 'text' => template.message_body(:sms, contact), 'callback' => '3', 'deliv_ack' => '1', 'req_feat' => '8192' }) case res when Net::HTTPSuccess, Net::HTTPRedirection puts "Successfully sent message to #{contact[:display_name]}" return true else puts res.error! return false end rescue puts "## Cannot send sms to #{contact[:display_name]}! : #{$!}" end end
We also use Net::HTTP to send the SMS messages through the Clickatell SMS gateway by sending an HTTP Post request to http://api.clickatell.com/http/sendmsg with a set of parameters.
The Client Message ID or CliMsgId
is a parameter that's set by the external application (in our case it is our application), which Clickatell uses to group messages sent by different applications. Clickatell does not use it internally and we can use up to 32 alphanumeric characters though we cannot have any spaces in it.
The To
parameter is the mobile number we wish to send to. As with Interfax, we need to provide the full international number, together with country and area codes. However, unlike for Interfax we should not use '+' or leading 0's.
The From
field is an alphanumeric field. Normally when sent through a mobile phone, the From
parameter is populated with the mobile number of the sender. However Clickatell allows us to use any 11 character alphanumeric string or an international format mobile number that is between 1 and 16 characters.
The Text
parameter contains the text message. To send Unicode characters (for example, for a Chinese text message) we need to set the 'Unicode' parameter to '1'. For Unicode messages, the text message limit is 70 Unicode characters per message. SMS messages are limited to 160 characters for normal ASCII characters. If we wish to send messages longer than 160 characters, we will need to set the concat
parameter to '1', '2', '3' or N number of messages to send. Clickatell does not automatically do this for us, so we will need to check the length of the characters in the message and set this accordingly.
The callback, deliv_ack
, and req_feat
parameters are linked in usage. SMS messages are sent asynchronously if called by HTTP (since HTTP is a connectionless protocol). This means we have no idea if the SMS message really gets to the mobile user.
Clickatell uses the callback
parameter to inform our mashup of the status of the message. If callback is set to '0', no statuses are reported, if it is set to '1', only intermediate statuses are sent, if it is set to '2', only final statuses are sent and if it is set to '3', both intermediate and final statuses are sent.
Clickatell uses an HTTP Get
command to call a URL that we set in the user interface to inform us of the status of the sent message and will return the client message ID (CliMsgId
), To, From, api_id
, and timestamp
parameters for our application to identify the message sent. For the rake script earlier, we specified that we do not want to get the status of the message but under production conditions we should create a simple action for Clickatell to call.
The delivery acknowledgement parameter (deliv_ack
) is closely related to the callback parameter. If it is set to '0', Clickatell will not report the delivery status to the handset, but only to the SMS upstream gateway. If it is set to '1', Clickatell will report the delivery status to the final handset itself. However delivery acknowledgement is not a guaranteed service as not all upstream gateways report delivery acknowledgements to the handset.
The requested features parameter (req_feat
) is related to the delivery acknowledgement and callback features. By default Clickatell will send messages by best effort. However, if we set certain constraints in the requested features parameter, Clickatell will drop those messages that do not fit the requested features.
Hex value Decimal Feature Description
Hex Value |
Decimal |
Feature |
Description |
---|---|---|---|
0x0001 |
1 |
FEAT_TEXT |
Text—set by default. |
0x0002 |
2 |
FEAT_8BIT |
8-bit messaging—set by default. |
0x0004 |
4 |
FEAT_UDH |
UDH (Binary)—set by default. |
0x0008 |
8 |
FEAT_UCS2 |
UCS2 / Unicode—set by default. |
0x0010 |
16 |
FEAT_ALPHA |
Alpha source address (from parameter). |
0x0020 |
32 |
FEAT_NUMER |
Numeric source address (from parameter). |
0x0200 |
512 |
FEAT_FLASH |
Flash messaging. |
0x2000 |
8192 |
FEAT_DELIVACK |
Delivery acknowledgments. |
0x4000 |
16384 |
FEAT_CONCAT |
Concatenation—set by default. |
The requested features parameter is set by bitmask, that is we need to add the decimal values together to get the necessary final value to use. Some values are set by default as described above. As can be seen from the table, the value 8192 sets the delivery acknowledgements value, requesting Clickatell to only send messages to gateways that have delivery acknowledgements.
The values in the table are only a subset of the full set of features available from Clickatell. For more details on the settings and parameters, you should read the Clickatell HTTP APIs specification from http://www.clickatell.com/downloads/http/Clickatell_HTTP.pdf.
ActionMailer is the default framework in Rails used for sending and receiving emails. Before sending emails, we will need to configure ActionMailer first. Go to config
and open up the appropriate environment configuration file (in development mode we will normally be using development.rb
). Append the following to the end of the file and change the configuration appropriately:
config.action_mailer.delivery_method = :smtp config.action_mailer.server_settings = { :address => <your smtp server> , :port => <smtp server port>, :domain => <your server domain name> , :authentication => :login, :user_name => <your username> , :password => <your password> }
address
and port
are the address and port of the SMTP server that we're using to send the emails. Domain
is the domain the mailer uses to identify itself to the server specified in address
. You should normally use the top-level domain name of the machine that is sending the email. Use authentication
if your SMTP server requires authentication to log into the system. If you use authentication you should also enter your SMTP user name and password in user_name
and password
respectively.
Now that ActionMailer is configured, generate the Mailer
model through the generate script:
$./script/generate mailer Mailer
This creates a Mailer model in our app/model
folder. The Mailer class in mailer.rb
inherits from ActionMailer::Base
and is the class we will use to send emails out.
class Mailer < ActionMailer::Base def mail(template, contact) @recipients = contact[:email] @subject = 'Big Business Marketing Message' @from = <sender email> @body['text'] = template.message_body(:email, contact) end end
Each method in the mailer class sets up the environment for sending a particular email. In our case, we have only one email to send so we need only one method. The method sets up the environment by setting up instance variables containing data for the email's header and body. In our case, we set the recipient to the contact email and hard-code a marketing message as its subject. Next we set the sender email address and then the body of the email message from the message template. Body
is a hash used to pass values to the email template. In this case, the message template's email body is passed to the email template as a variable named @text
.
Let's create, then look at the email template named mail.text.html.rhtml
. (RAILS_ROOT/app/views/mailer/
). This file is in named in this format:
<method>.<content>.<type>.rhtml
The method is the name of the method in the Mailer
class, in this case, mail. The content and type are the MIME type of the text to be sent. In this case we're sending an HTML message so the content type is text/html
.
<div id='header'>Big Business Banner/div> <%= @text%> <div id='footer'>Copyright Big Business 2007</div>
Note that the email body from the message template has some HTML tags as well, and this goes well into this email template.
Finally, the messages in the message template can be customized according to the individual recipient. In this mashup we use the display name as well as additional text to be added for each recipient, if any. The text for the SMS message for example, goes like this:
Dear #{display_name}, New product just in! Check out our latest product offers at http://some.big_business.com! #{additional_message}
Notice that both display name and the additional text message customization look like the string replacement syntax in Ruby. This is because we evaluate this text as a Ruby string inside the Job class in RAIL_ROOT/app/model/MessageTemplate.rb
:
class MessageTemplate < ActiveRecord::Base
has_many :jobs
def message_body(type,contact)
display_name = contact[:display_name]
additional_message = contact[:additional_message]
case type
when :email : eval '"' + self.email_body + '"'
when :sms : eval '"' + self.sms_body + '"'
when :fax : eval '"' + self.fax_body + '"'
else 'Incorrect type of message requested'
end
end
end
By evaluating the text as a Ruby string, we replace the values specified in the message body with the required values from the contact information. Whenever we need the message text, we just call it like this:
res = Net::HTTP.post_form(URI.parse('http://api.clickatell.com/http/sendmsg'),
{'session_id' => $clickatell_session_id,
'cliMsgId' => template.id,
'to'=>contact[:mobile_no],
'from' => 'Chapter 3',
'text' => template.message_body(:sms, contact),
'callback' => '3',
'deliv_ack' => '1',
'req_feat' => '8192' })
This little trick gives us great flexibility when doing text replacement on the messages for individual recipients.
This wraps up the mashup. Run this script to send the messages once you have set up the templates and the jobs.
To recap the chapter, this is the sequence to follow in using the mashup:
Start up the server
Create one or more message templates
Create a job
Run the rake script to process the job
If you have set up things correctly you should receive the three marketing messages, through the email, SMS, and fax.
What we've learned in this chapter is to extract information from online spreadsheets through CSV and use that information to send messages through three different communications channels—email, SMS, and fax. We created a marketing message mashup using Google Spreadsheets and/or EditGrid, both online spreadsheets, Clickatell, a bulk SMS provider, as well as Interfax, an Internet fax provider.
3.149.243.32