Chapter 3. Proxy mailing list mashup plugin

What does it do?

This mashup plugin allows your Rails website or application to have a proxy mailing list feature that includes email, SMS messages, and fax. A normal mailing list allows a website to send their messages to its list of visitors or users. A proxy mailing list allows a third-party user to send the messages to their own list of recipients, people who are previously unknown to the website, on behalf of the website.

Building a proxy mailing list feature for your website

Your boss calls you in to discuss a new feature for your company's website. There is a new marketing initiative for your company's products. You have been chosen to build in a new feature for the marketing folks on the 14th floor that allows them to send out regular email marketing messages to clients and potential clients. You dutifully write down the requirements for this mailing list feature:

  • Import contacts from a spreadsheet containing the name of the client or potential client into an internal database

  • Allow internal marketing users to define email campaign messages

  • Send email messages to the clients and potential clients at regular intervals

Doesn't look too hard! You walked away confidently.

The next day your boss calls you in again. Oh surprise—there is a new requirement. After some meetings, the marketing people decided that sending email messages is not enough, now they want to send SMS messages and send faxes as well. Because your clients are scattered around the world, the SMS messages and faxes also need to be sent around the world! Sighing expectedly, you write down the additional new requirements:

  • Allow internal marketing users to define SMS and fax messages

  • Send SMS messages to clients and potential clients

  • Send faxes to clients and potential clients

  • Messages need to be sent worldwide

You should have known better than to expect finalized requirements from the first meeting!

You were still figuring out how to send SMS messages the next day when your boss called for an emergency meeting with you. With a sinking feeling you walked into his room.

The marketing people originally wanted to consolidate the contacts data from your company's resellers around the world into an spreadsheet file, which is used to feed into the website. However because of data privacy (and you suspect, other commercial) issues, the resellers now refuse to give your company the contacts data directly. Instead, the marketing people have struck a deal to let the resellers send the marketing messages by themselves through your website. Worse, you are no longer allowed to store any contacts information in your database.

You're stuck now! You don't have the contacts data and now you need to let the resellers send the marketing messages? How can you do it?

Requirements overview

You consolidated the following (hopefully final now) set of requirements:

  1. Allow internal marketing users to define email, SMS, and fax messages in a template

  2. Allow the external resellers to customize the pre-defined marketing messages according to their list of contacts

  3. Send email, SMS, and fax messages to the clients and potential clients around the world on a regular basis

  4. You cannot store any contacts information

Mashups to the rescue!

Design

Let's see how we can use Rails and some mashup magic to build this new feature for your website.

Define messages

Create a simple Rails web application that allows the marketing user to construct message templates that contain the messages to be sent via email, SMS text messaging, and fax. This template can be used later as the basis for sending various messages to the recipients of the marketing blitz.

Get contacts and customized message data

We need to get the reseller's contact information without storing it in our database. To do this, we need to get the resellers to define or import their contacts into an online spreadsheet. After that, the resellers need to export the spreadsheet in a data format that is suitable for extraction and processing by our website. For this chapter, we will show how this can be done through Google Spreadsheet and EditGrid, two popular online spreadsheets. The reseller will upload his or her contacts from his or her spreadsheet into Google Spreadsheet or EditGrid, and then publish a link to this spreadsheet. Subsequently the reseller will select a message template and create a message-sending job by providing this link to his or her contacts.

Send messages

The number of contacts provided by each reseller is normally high, so it is unrealistic to send the messages interactively (that is, to send the message on the click of a button in the user interface and wait for a response that indicates that the messages all sent). There are two alternatives for sending messages in a non-interactive way. One method is to create a threaded job that is separate from the main process and run it in the background when the reseller clicks on the send button. The other method is to create jobs that are stored in the database and retrieved separately by another process at regular intervals to be processed independently. I have chosen to use the second method of executing the send-message tasks in this chapter, as it is the simpler of the two.

Emails can be done easily through ActionMailer, with messages captured from the message template. However, SMS and fax messages are more complex. Let's go through some basic background knowledge on these types of messages before coming up with the strategies on sending them.

Sending SMS messages

SMS or Short Message Service is a technology that enables the sending and receiving of messages between mobile phones. SMS was part of the GSM (Global System for Mobile Communications) standards at the beginning but was later ported to technologies like CDMA and TDMA. SMS is very popular and widely used as it is supported by all GSM mobile phones.

When an SMS message is sent from a mobile phone, it will reach an SMSC in the GSM network. The SMSC then forwards the SMS message towards the destination, passing through one or more network elements, including other SMSCs. If the recipient is unavailable (for example, when the mobile phone is switched off), the SMSC will store the SMS message and forward it when the recipient is available.

Sending SMS messages

An SMSC normally belongs to a single network. For SMSes to reach mobile phones in different GSM networks, an SMS gateway is used to bridge between SMSCs in different networks.

Sending SMS messages

SMS gateways are also used to act as a concentrator that can access multiple SMSCs. This allows applications that send SMSes to channel their messages through a single gateway to multiple SMSCs without the need to connect to each SMSC individually. This is the model many bulk SMS providers (including Clickatell, the provider we're using in this chapter) use.

Sending SMS messages

One SMS message can contain at most 140 bytes of data, so one SMS message can contain up to:

  • 160 characters if the default GSM 7-bit character (ASCII) encoding is used

  • 140 characters if 8-bit character encoding is used

  • 70 characters if 16-bit Unicode UCS2 character encoding is used

Besides text, SMS messages can also carry binary data like ringtones, pictures, operator logos, wallpapers, animations, VCards, and WAP configurations to a mobile phone. However, such uses are vendor-specific and are not as widely supported as text-based SMS.

Probably the best way to send SMS messages in bulk is through a bulk SMS provider. A provider with wide global coverage will also allow us to send SMS messages around the world. As most bulk SMS providers also provide APIs for developers, this ties in nicely with our requirements to send SMSes to a large number of people globally. Clickatell, the SMS provider we're using in this chapter provides a bulk SMS gateway and numerous APIs to connect to it.

Sending fax messages

Fax or facsimile is a telecommunications technology used to transfer copies of documents over the telephone network. Fax is most commonly used to send documents between two fax machines connected to a telephone line. A fax machine is a three-in-one machine, with a scanner, a modem, and a printer rolled into one. The scanner extracts an image of the document in digital form, the modem sends it across to the other fax machine via the telephone network, while the printer reproduces it on the other end, and vice versa. However Internet-based faxing services and multi-function printers are fast replacing the traditional standalone fax machines.

Various Internet fax services now provide email and API services for users to send faxes directly from a file, removing the need for scanning and sending via a telephone network. The best way to programmatically send faxes would be through a fax service provider and for this chapter we will be using Interfax and sending faxes through an XML-RPC-based web service.

Mashup APIs on the menu

After reviewing the strategy and determining the best way to design for the requirements, we have established the following mashup APIs that we will use for this chapter:

  • Google Spreadsheet and/or EditGrid for the user to store and share the contacts information

  • Clickatell to send SMS messages

  • Interfax to send faxes

The APIs in this chapter have free developer trial accounts so you can experiment with them a bit. However, the APIs we are using in this chapter are not free for full commercial use and they have some restrictions on their usage.

We will not be using any specific Ruby libraries for this mashup as the APIs we use doesn't require them. Instead we will be using standard Ruby libraries that are available out of the box from any Ruby installation, in particular the Net::HTTP module, which allows us to connect to XML-RPC web services and REST-based HTTP APIs. We will also use the built-in CSV module to simplify conversion of CSV formatted data into arrays.

Let's run through these online APIs as well as the Net::HTTP module.

Google Spreadsheets

Google Spreadsheets (http://docs.google.com) is an online spreadsheet service offered by Google as part of its Google Docs and Spreadsheets offering. Google Docs and Spreadsheets is a Web-based word processor and spreadsheet application that allows users to create and edit documents and spreadsheets online while collaborating in real-time with other users. Google Docs and Spreadsheets combine the features of two services, Writely and Spreadsheets.

Users can create the spreadsheet online or upload spreadsheets of various formats including CSV, Excel, and OpenDocument (.ods for spreadsheets). There is a limit to each spreadsheet, which is 10,000 rows, 256 columns, or 100,000 cells, whichever limit is reached first. When importing spreadsheets, the import file cannot be larger than 1MB in size. Users are also able to export their documents in various output formats, which include CSV, Excel, OpenDocument, PDF, and so on. Google Spreadsheets is free, and any Google user can get it as part of the package.

Google Spreadsheets has its own API, which uses part of the Google Data APIs. This API requires user account authentication to access information on private spreadsheets. However for this chapter we will not be using access-controlled data in Google Spreadsheets and only the published spreadsheets in a comma delimited (CSV) format.

EditGrid

EditGrid is a free online spreadsheet service with paid subscription for commercial/enterprise users. EditGrid allows users to create an online spreadsheet or import spreadsheets in various formats including CSV, Excel, and OpenDocument. Users are also able to export their documents in various output formats, which include CSV, Excel, OpenDocument, PDF, and so on.

You will need to register for a free personal account, which requires only a login user name and a password: http://www.editgrid.com.

EditGridproxy mailing list, mashup pluginCSV format

As with Google Spreadsheets, we will not use any of the private APIs but opt to access the publicly available published spreadsheet in a CSV format.

Clickatell

Clickatell (http://www.clickatell.com) is a bulk SMS provider that provides SMS messaging services and gateway for over 600 networks in almost 200 countries for outbound messages, and 100 countries for inbound (two-way) messaging. Clickatell allows developers to connect to its SMS gateway via various connectivity options including HTTP/S, SMTP (email to SMS), XML, SMPP (Short Message Peer to Peer protocol), FTP (file upload for SMS), and a COM object API. This provides applications with the ability to send SMSes globally, bypassing the need to hook up to local SMS providers individually.

Clickatell provides a trial account with 10 credits to allow developers to have a head start in trying out its services. To create a trial account, go to https://www.clickatell.com/central/user/client/step1.php.

Clickatellproxy mailing list, mashup pluginEditGrid

Fill in the account creation form. You will be required to provide a mobile number. This is important as in Step 2 a confirmation code is sent to your mobile number, before a trial account is provided.

Clickatellproxy mailing list, mashup pluginEditGrid

After entering your confirmation code from both your email and mobile phones, you will be presented with the homepage of your account.

Clickatellproxy mailing list, mashup pluginEditGrid

Notice the balance you have at the top of the screen. This is number of credits you have been given for the trial account. If you run out you can click on a conveniently located button to buy more credits through various payment methods. Take note of the Client ID provided, you will need it when you log into the Clickatell management console. To allow for access to an HTTP API, click on the Manage My Products link.

Clickatellproxy mailing list, mashup pluginEditGrid

Then from the drop-down list, select HTTP.

Clickatellproxy mailing list, mashup pluginEditGrid

Select an appropriate name for this API. Optionally you can set the Dial Prefix if you are sending a lot of SMSes to a particular country. You can also set a Callback (to be explained in a later section) for Clickatell to call when reporting the status of the sent message. You can also specify that only a particular IP address, or set of IP addresses, can access this API. When you're done, click on the Submit button.

Clickatellproxy mailing list, mashup pluginEditGrid

Take note of the API ID given here. We will need it when trying to access the HTTP API from our mashup.

Interfax

Interfax is a fax service provider that has coverage in a large number of countries. Interfax provides a number of API interfaces to its fax server including web services, COM object API, and an email interface. Interfax also provides reports on faxes that are sent or received through its servers. With Interfax, an application is able to send text faxes, faxes from base-64 encoded binary documents like PDF documents, Excel spreadsheets, and Word documents as well as receive faxes as TIF formatted images.

Interfax provides a free trial account for developers but faxes sent from this trial account will only reach a single fixed fax number. To be able to send to any other fax numbers you need to subscribe to its commercial account. To sign up for its free trial account, go to http://www.interfax.net/Scripts/Reg_BP.asp and fill in the account creation form.

Interfax

After creating the account, you will be given $10 for faxing pages and a business partner ID. The business partner ID is not needed for sending faxes.

Net::HTTP

Ruby comes with a standard library for managing HTTP connections. While it is not very sophisticated, it provides the essential and basic capabilities for an application to connect to web applications through HTTP.

A basic HTTP Get command in Net::HTTP goes like this:

require 'net/http'
response = Net::HTTP.get_response(URI.parse ('http://www.packtpub.com'))
puts response.body

This sends an HTTP Get to Packt Publishing's website and returns what the web server delivers through that URL as a Net::HTTPResponse object.

A basic HTTP Post command in Net:HTTP goes like this:

require 'net/http'
response = Net::HTTP.post_form(URI.parse ('http://search.yahoo.com/search'),
{'p'=>'mashups'})
puts response.body

This code sends an HTTP Post to Yahoo's search engine with the parameter 'mashups' and it returns whatever search results are served from Yahoo, as a Net::HTTPResponse object.

To react according to the returned Net::HTTPResponse object, we need to inspect this object more closely. Net::HTTPResponse is actually the parent object of a hierarchy of HTTP response statuses. In order to check the status of the sent command, we can run the response object through a case loop to check the actual subclass and respond accordingly.

require 'net/http'
response = Net::HTTP.post_form(URI.parse ('http://search.yahoo.com/search'), {'p'=>'mashups'})
case response
when Net::HTTPSuccess
puts response.body
else
puts response.error!
end

This code indicates that if the response object is of the class Net::HTTPSuccess (HTTP code 2xx) it will print out the response body, otherwise it will just print out the error code.

We will be using Net::HTTP extensively in this chapter. In later chapters we will introduce another built-in Ruby package that performs a similar function.

What we will be doing

Although this is a mashup plugin, meaning it is normally added to an existing Rails application, we will be creating a new project to show how it can be used. This is the process flow of the mashup:

  • The marketing user will create a marketing message template with the email, SMS, and fax messages.

  • The reseller selects the message template to send and provides a link to the list of contacts to send the message to, then creates a message-sending job.

  • At regular intervals, the system will check for pending jobs, process them and send all messages to the respective contacts.

This is what we will be doing in the next few pages to implement this mashup:

  • Create a Rails project

  • Configure the database access and create the database

  • Create the standard scaffolding

  • Allow the marketing users to create the message templates

  • Allow the reseller to provide contacts data through a remote link

  • Create the rake script to send messages at regular intervals

This mashup's main processing is not in the web application itself. The Rails web application is used to get input from the various parties i.e. the message template from the marketing user and the contacts data from the external reseller. As explained earlier, the actual processing and sending of the messages is done outside of the web application in the rake script. The rake script is triggered periodically by a scheduler like cron in Unix or at in Windows.

Creating a new Rails project

As before, creating the Rails project is the easiest part.

$rails Chapter3

This will create a new blank Rails project.

Configuring the database access and creating the database

The Rails web application is basic and the database needed to support it is simple as well. Change the necessary environment configuration file (development.rb for a development environment) to configure access to the database. Then generate a migration file to create the database:

$./script/generate migration create_templates_and_jobs

This will create a file 001_create_templates_and_jobs.rb in the RAILS_ROOT/db/migrate folder. Ensure it has the following code:

class CreateTemplatesAndJobs < ActiveRecord::Migration
def self.up
create_table :message_templates do |t|
t.column 'name', :string
t.column 'sms_body', :text, :limit => 160
t.column 'email_body', :text
t.column 'fax_body', :text
end
create_table :jobs do |t|
t.column 'message_template_id', :integer
t.column 'contacts_url', :string
t.column 'status', :string, :default => 'pending'
end
end
def self.down
drop_table :message_templates
drop_table :jobs
end
end

Now that we have the migration scripts, run migrate to create the tables:

$rake db:migrate

This should create the database tables needed. The data model in this mashup is quite simple. The MessageTemplate is the model of the message templates created by the marketing people, while each Job is created by the reseller.

Creating standard scaffolding

Next, create the standard scaffolding for the tables we've just created:

$./script/generate scaffold MessageTemplate

and:

$./script/generate scaffold Job

This will create the standard controllers, views, and models for the two classes. Next, change MessageTemplate and Job (both located in RAILS_ROOT/app/models/) to reflect their relationship:

class MessageTemplate < ActiveRecord::Base
has_many :jobs
end
class Job < ActiveRecord::Base
belongs_to :message_template
end

Allowing the marketing people to create the message templates

The standard scaffolding should already allow the creation of the message templates though we might want to do it up a bit if we are giving it to our marketing users! For testing purposes, go to: http://localhost:3000/message_templates/list and create some sample marketing messages for each type of message. Remember that SMS messages should be short and succinct and each message is up to a maximum of 160 characters (when using standard ASCII characters).

Allowing the marketing people to create the message templates

Note that in the example above, we have put in some Ruby-like variables in the messages such as display_name. We'll see how this is used in the coming sections.

Allowing the reseller to provide contacts data through a remote link

Next we need to let the reseller provide the contacts to our mashup. The general strategy is to allow the reseller to create message-sending 'jobs' that have the links to the contacts information. The reseller will select the message template to use and provide a link, so a simple job creation form can do this. The scaffold should have most of the code already in place, so just modify the _form.rhtml partial (RAILS_ROOT/app/views/jobs/) to link Job to MessageTemplate.

<%= error_messages_for 'job' %>
<!--[form:job]-->
<p><label for="job_message_template">Message template</label><br/>
<%= select('job', 'message_template_id', MessageTemplate.find(:all).collect {|t| [ t.name, t.id ] })%></p>
<p><label for="job_contacts_url">Contacts url</label><br/>
<%= text_field 'job', 'contacts_url', :size => 100 %></p>
<!--[eoform:job]-->
Allowing the reseller to provide contacts data through a remote link

Open: http://localhost:3000/jobs/new and put in the spreadsheet URL (see below).

The link we are providing here is for Google Spreadsheets but it should be similar for EditGrid. Note that the URL is a link to the CSV format of the spreadsheet and not to the main document.

Uploading to and publishing from Google Spreadsheets

Before going into how this link is generated, let's first see from the reseller's perspective how he or she will upload his or her list of contacts to the Google Spreadsheet. Log into Google and go to http://docs.google.com.

Uploading to and publishing from Google Spreadsheets

Click on the Upload link on the main page and follow the instructions on the page to upload a spreadsheet.

Uploading to and publishing from Google Spreadsheets

The spreadsheet that the reseller uploads must have the following columns in the following order:

  • Display name

  • Additional text (for the marketing messages)

  • Email address

  • Mobile number (for SMS messages)

  • Fax number

All the columns are optional but the display name should be there at the very least, otherwise the marketing message will have no addressee! If there is a value under the email column, an email will be sent, if there is a value under the mobile column, an SMS will be sent, and if there is a value under the fax column, a fax will be sent.

Now that the spreadsheet is loaded up in Google Spreadsheets, the reseller can proceed to generate the link that publishes the contacts in CSV format. To generate this format, go to the document page of the Google Spreadsheet. Then click on the publish tab at the top right of the screen on the document page.

Uploading to and publishing from Google Spreadsheets

An options pane will be opened to the right of the screen. Click on the Publish now button.

Uploading to and publishing from Google Spreadsheets

The document is now published. For additional publishing options, including the option for CSV format, click on the More publishing options link at the bottom of the options pane.

Uploading to and publishing from Google Spreadsheets

The reseller will then see a new pop-up window with several drop-down select fields. Choose CSV in the file format select field, and click on the Generate URL button. The reseller will be presented with a URL at the bottom of the window. This is the link we will ask the resellers to enter in order to retrieve the contacts and other information.

Uploading to and publishing from Google Spreadsheets

Uploading to and publishing from EditGrid

Uploading to and publishing from EditGrid is almost the same. After logging into EditGrid, the reseller will be presented with his or her workspace.

Uploading to and publishing from EditGrid

Click on the Upload button to upload a spreadsheet. The format of the spreadsheet should be the same as the one described above in the Google Spreadsheets section. The reseller needs to set the permission to public read-only to allow our mashup to read the contact information.

Uploading to and publishing from EditGrid

When the information has been uploaded into the spreadsheet, click on File in the menu bar and select the Permalinks menu item. Permalinks are permanent links on EditGrid. EditGrid provides permanent links support to allow you to access their spreadsheets through easy-to-understand URLs.

Uploading to and publishing from EditGrid

The reseller will see a list of formats in which data from this spreadsheet can be exported. The reseller needs to choose the CSV format link and use that as the link to provide as the input in the job creation screen. Make a record of the CSV link.

Uploading to and publishing from EditGrid

Creating the rake script to send messages at regular intervals

This is where the main action starts. As explained in Chapter 2, rake is a build program much like make, but one built with Ruby syntax. Rake is integrated and used extensively in Rails for various tasks including database migration (we used db:migrate earlier on in this chapter). We will be using rake to run a processing script that will get the data from the remote site and send the messages.

Create a rake script named process_jobs.rake in the RAILS_ROOT/lib/tasks folder:

require 'net/http'
require 'csv'
require 'soap/wsdlDriver'
namespace :chapter3 do
# Clickatell credentials
$clickatell_api_id = <your Clickatell API ID>
$clickatell_login = <your Clickatell user name>
$clickatell_password = <your Clickatell password>
# Interfax credentials
$interfax_login = <your Interfax user name>
$interfax_password = <your Interfax password>
$interfax_driver = SOAP::WSDLDriverFactory.new('http://ws.interfax.net/dfs.asmx?WSDL').create_rpc_driver
desc "Activated regularly by AT or cronjob to process all jobs"
task(:process_jobs => :environment) do
$clickatell_session_id = get_clickatell_session
begin
pending_jobs = Job.find_all_by_status 'pending'
if pending_jobs.size > 0
puts "#{pending_jobs.size} jobs pending processing ..."
else
puts "All jobs has been processed!"
end
pending_jobs.each { |job|
# parse and get contacts data
contacts_data = []
csv = parse_data(job.contacts_url)
puts "Found #{csv.size} contacts for this job!"
csv.each { |line|
contact = {}
contact[:display_name] = line[0]
contact[:additional_message] = line[1]
contact[:email] = line[2]
contact[:mobile_no] = line[3]
contact[:fax_no] = line[4]
contacts_data << contact
}
contacts_data.each { |contact|
# send email
if !contact[:email].nil?
t1 = Time.now
sent = Mailer.deliver_mail(job.message_template, contact)
t2 = Time.now
if sent
puts "Email sent to #{contact[:display_name]} in #{(t2 - t1)} seconds"
end
end
# send sms
if !contact[:mobile_no].nil?
t1 = Time.now
sent = send_sms(job.message_template, contact)
t2 = Time.now
if sent
puts "SMS sent to #{contact[:display_name]} in #{(t2 - t1)} seconds"
end
end
# send fax
if !contact[:fax_no].nil?
t1 = Time.now
sent = send_fax(job.message_template, contact)
t2 = Time.now
if sent
puts "Fax sent to #{contact[:display_name]} in #{(t2 - t1)} seconds"
end
end
}
job.status = 'processed'
job.save!
}
rescue
puts "Error during sending messages : #{$!}"
end
end
# -- end of main task --
# send fax through Interfax
def send_fax(template, contact)
$interfax_driver.SendCharFax(
:Username => $interfax_login,
:Password => $interfax_password,
:FaxNumber => contact[:fax_no],
:Data => template.message_body(:fax, contact))
end
# send SMS through Clickatell
def send_sms(template, contact)
begin
res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/sendmsg'),
{'session_id' => $clickatell_session_id,
'cliMsgId' => template.id,
'to'=>contact[:mobile_no],
'from' => 'Chapter 3',
'text' => template.message_body(:sms, contact),
'callback' => '3',
'deliv_ack' => '1',
'req_feat' => '8192' })
case res
when Net::HTTPSuccess, Net::HTTPRedirection
puts "Successfully sent message to #{contact[:display_name]}"
return true
else
puts res.error!
return false
end
rescue
puts "## Cannot send sms to #{contact[:display_name]}! : #{$!}"
end
end
# get the clickatell session needed to send SMS messages
def get_clickatell_session
res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/auth'),
{'api_id' => $clickatell_api_id,
'user'=> $clickatell_login,
'password' => $clickatell_password})
case res
when Net::HTTPSuccess, Net::HTTPRedirection
return res.body.split(': ')[1]
else
puts res.error!
end
end
# parse data from a CSV file published by Google Spreadsheet
def parse_data(url)
res = Net::HTTP.get_response(URI.parse(url))
case res
when Net::HTTPSuccess, Net::HTTPRedirection
csv = CSV.parse(res.body)
header = csv.shift
return csv
else
puts res.error!
end
end
end

We will go through this script part by part. First, we will be using the Net::HTTP, CSV, and SOAP packages that are default in our Ruby installation so we will require them at the top of the file.

namespace :chapter3 do
# Clickatell credentials
$clickatell_api_id = <your Clickatell API ID>
$clickatell_login = <your Clickatell user name>
$clickatell_password = <your Clickatell password>
# Interfax credentials
$interfax_login = <your Interfax user name>
$interfax_password = <your Interfax password>
$interfax_driver = SOAP::WSDLDriverFactory.new('http://ws.interfax.net/dfs.asmx?WSDL').create_rpc_driver

The Web Services Description Language (WSDL) is an XML-based language used to describe web services. WSDL is often used together with SOAP to define web services as in the case of Clickatell and Interfax. A web service consuming application (such as our mashup) reads the WSDL to find out what is available from the web service and how to access it. From there we can generate the proxies that we use to access the web service as if it were a call to a local object.

Preset your global credentials for Clickatell and Interfax. We should have gotten these credentials when we registered for the developer accounts earlier on. The last line of this section creates a SOAP client to communicate with Interfax. The constructor for WSDLDriverFactory takes in a WSDL file provided by Interfax and creates the necessary local proxy. We will be using this proxy to send our fax.

desc "Activated regularly by AT or cronjob to process all jobs"
task(:process_jobs => :environment) do
$clickatell_session_id = get_clickatell_session
begin
pending_jobs = Job.find_all_by_status 'pending'
if pending_jobs.size > 0
puts "#{pending_jobs.size} jobs pending processing ..."
else
puts "All jobs has been processed!"
end
pending_jobs.each
{ |job|
# parse and get contacts data
contacts_data = []
csv = parse_data(job.contacts_url) puts "Found #{csv.size} contacts for this job!"
csv.each
{ |line|
contact = {}
contact[:display_name] = line[0]
contact[:additional_message] = line[1]
contact[:email] = line[2]
contact[:mobile_no] = line[3]
contact[:fax_no] = line[4]
contacts_data << contact
}
contacts_data.each
{ |contact|
# send email
if !contact[:email].nil?
t1 = Time.now
sent = Mailer.deliver_mail(job.message_template, contact)
t2 = Time.now
if sent
puts "Email sent to #{contact[:display_name]} in #{(t2 - t1)} seconds"
end
end
# send sms
if !contact[:mobile_no].nil?
t1 = Time.now
sent = send_sms(job.message_template, contact)
t2 = Time.now
if sent
puts "SMS sent to #{contact[:display_name]} in #{(t2 - t1)} seconds"
end
end
# send fax
if !contact[:fax_no].nil?
t1 = Time.now
sent = send_fax(job.message_template, contact)
t2 = Time.now
if sent
puts "Fax sent to #{contact[:display_name]} in #{(t2 - t1)} seconds"
end
end
}
job.status = 'processed'
job.save!
}
rescue
puts "Error during sending messages : #{$!}"
end
end
# -- end of main task --

We start off the processing run by getting a Clickatell session:

$clickatell_session_id = get_clickatell_session

This session is needed by Clickatell to identify that we are an authorized user to send SMS messages. Next, we get all pending jobs in the system and for each pending job:

pending_jobs = Job.find_all_by_status 'pending'

we get the contacts URL and parse it:

csv = parse_data(job.contacts_url)

Remember that the contacts URL is actually a link to a CSV file published from an online spreadsheet. Parsing it returns an Array of Arrays that has the title row truncated. We will take this Array of Arrays and create an Array of Hashes to make the code simpler to read:

csv.each { |line| contact = {}
contact[:display_name] = line[0]
contact[:additional_message] = line[1]
contact[:email] = line[2]
contact[:mobile_no] = line[3]
contact[:fax_no] = line[4]
contacts_data << contact
}

This is our list of contacts!

Now that we have the contacts data, we iterate through each one of them to send the messages. We run the contact through three if loops (one for each of the communications methods) to see if there is any contact information in that row. We will send the message if there is, that is, if there is an email given we will send the email message:

sent = Mailer.deliver_mail(job.message_template, contact)

If there is a mobile number given we will send the SMS message:

sent = send_sms(job.message_template, contact)

If there is a fax number given, we will send the fax:

sent = send_fax(job.message_template, contact)

This is the main loop for this script; let's see how we can extract the contacts information from the online spreadsheet and the send methods for each of the communications channels next.

Parsing data from the online spreadsheet

We extract the data from the online spreadsheet (either Google Spreadsheets or EditGrid) through the link that is given by the reseller in the job. To do this we use Net::HTTP to send an HTTP Get to the server, which should return a plaintext CSV string embedded within the response object.

# parse data from a CSV file published by Google Spreadsheets or EditGrid
def parse_data(url)
res = Net::HTTP.get_response(URI.parse(url))
case res
when Net::HTTPSuccess, Net::HTTPRedirection
csv = CSV.parse(res.body)
header = csv.shift
return csv
else
puts res.error!
end
end

Next, we use the CSV module (also built into Ruby) to parse the CSV string into an Array of Arrays. We also remove the header so that we get only data in the returned Array object.

Sending a fax with Interfax

Sending the fax through Interfax is relatively simple. Using the local proxy we have created from the WSDL provided by Interfax, we need only call any of the provided methods. In this example we will use SendCharFax, which sends a text message to the recipient.

# send fax through Interfax
def send_fax(template, contact)
$interfax_driver.SendCharFax(
:Username => $interfax_login,
:Password => $interfax_password,
:FaxNumber => contact[:fax_no],
:Data => template.message_body(:fax, contact))
end

The username and password parameters are self-explanatory. The faxnumber submitted, however, needs to be in the international notation. The format is: + <Country-Code><AreaCode><Local number>. For example: A number in New York, NY, USA will look like: +12123456789, where:

+ is a constant;

1 is the USA country code;

212 is New York area code;

3456789 is the local number.

The data in this case is the text fax message stored in the message template.

The SendCharFax web service is a basic one that only sends text messages. Normally this is not realistic, as this will only send an ugly string message to the recipient.

To improve on this, we can use either Sendfax or SendfaxEx_2 to send files as faxes. We can send files with types including Word documents, Excel spreadsheets, Acrobat documents, or HTML-formatted text. To do this we can get the marketing user to upload a document to the database, which can then be sent to the contacts (we will skip this to keep this chapter simple). Check out the Interfax developer site at http://www.interfax.net/en/dev/webservice/reference.html to get the details of the various web services that are provided by Interfax.

Sending an SMS through Clickatell

Sending an SMS through Clickatell is only slightly more complicated. We will need to get a session key from Clickatell first by logging in and presenting our credentials, and then use this session key to send the SMS messages.

# get the clickatell session needed to send SMS messages
def get_clickatell_session
res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/auth'),
{'api_id' => $clickatell_api_id,
'user'=> $clickatell_login,
'password' => $clickatell_password})
case res
when Net::HTTPSuccess, Net::HTTPRedirection
return res.body.split(': ')[1]
else
puts res.error!
end
end

Clickatell provides many different ways for developers to access its services, but for this chapter we will be using its HTTP APIs. To use the HTTP APIs we use the Net::HTTP package to send an HTTP Post command to http://api.clickatell.com/http/auth with the various credentials as part of a Post form in the request body to get a session key. Note that we need to provide an API ID to Clickatell&mdash;this is the number we get from Clickatell when we register as a developer for that set of API services (see section above). The session ID is retrieved then stored in a global variable for use when we send the messages.

# send SMS through Clickatell
def send_sms(template, contact)
begin
res = Net::HTTP.post_form(URI.parse( 'http://api.clickatell.com/http/sendmsg'),
{'session_id' => $clickatell_session_id,
'cliMsgId' => template.id,
'to'=> contact[:mobile_no],
'from' => 'Chapter 3',
'text' => template.message_body(:sms, contact),
'callback' => '3',
'deliv_ack' => '1',
'req_feat' => '8192' })
case res
when Net::HTTPSuccess, Net::HTTPRedirection
puts "Successfully sent message to #{contact[:display_name]}"
return true
else
puts res.error!
return false
end
rescue
puts "## Cannot send sms to #{contact[:display_name]}! : #{$!}"
end
end

We also use Net::HTTP to send the SMS messages through the Clickatell SMS gateway by sending an HTTP Post request to http://api.clickatell.com/http/sendmsg with a set of parameters.

The Client Message ID or CliMsgId is a parameter that's set by the external application (in our case it is our application), which Clickatell uses to group messages sent by different applications. Clickatell does not use it internally and we can use up to 32 alphanumeric characters though we cannot have any spaces in it.

The To parameter is the mobile number we wish to send to. As with Interfax, we need to provide the full international number, together with country and area codes. However, unlike for Interfax we should not use '+' or leading 0's.

The From field is an alphanumeric field. Normally when sent through a mobile phone, the From parameter is populated with the mobile number of the sender. However Clickatell allows us to use any 11 character alphanumeric string or an international format mobile number that is between 1 and 16 characters.

The Text parameter contains the text message. To send Unicode characters (for example, for a Chinese text message) we need to set the 'Unicode' parameter to '1'. For Unicode messages, the text message limit is 70 Unicode characters per message. SMS messages are limited to 160 characters for normal ASCII characters. If we wish to send messages longer than 160 characters, we will need to set the concat parameter to '1', '2', '3' or N number of messages to send. Clickatell does not automatically do this for us, so we will need to check the length of the characters in the message and set this accordingly.

The callback, deliv_ack, and req_feat parameters are linked in usage. SMS messages are sent asynchronously if called by HTTP (since HTTP is a connectionless protocol). This means we have no idea if the SMS message really gets to the mobile user.

Clickatell uses the callback parameter to inform our mashup of the status of the message. If callback is set to '0', no statuses are reported, if it is set to '1', only intermediate statuses are sent, if it is set to '2', only final statuses are sent and if it is set to '3', both intermediate and final statuses are sent.

Clickatell uses an HTTP Get command to call a URL that we set in the user interface to inform us of the status of the sent message and will return the client message ID (CliMsgId), To, From, api_id, and timestamp parameters for our application to identify the message sent. For the rake script earlier, we specified that we do not want to get the status of the message but under production conditions we should create a simple action for Clickatell to call.

The delivery acknowledgement parameter (deliv_ack) is closely related to the callback parameter. If it is set to '0', Clickatell will not report the delivery status to the handset, but only to the SMS upstream gateway. If it is set to '1', Clickatell will report the delivery status to the final handset itself. However delivery acknowledgement is not a guaranteed service as not all upstream gateways report delivery acknowledgements to the handset.

The requested features parameter (req_feat) is related to the delivery acknowledgement and callback features. By default Clickatell will send messages by best effort. However, if we set certain constraints in the requested features parameter, Clickatell will drop those messages that do not fit the requested features.

Hex value Decimal Feature Description

Hex Value

Decimal

Feature

Description

0x0001

1

FEAT_TEXT

Text&mdash;set by default.

0x0002

2

FEAT_8BIT

8-bit messaging&mdash;set by default.

0x0004

4

FEAT_UDH

UDH (Binary)&mdash;set by default.

0x0008

8

FEAT_UCS2

UCS2 / Unicode&mdash;set by default.

0x0010

16

FEAT_ALPHA

Alpha source address (from parameter).

0x0020

32

FEAT_NUMER

Numeric source address (from parameter).

0x0200

512

FEAT_FLASH

Flash messaging.

0x2000

8192

FEAT_DELIVACK

Delivery acknowledgments.

0x4000

16384

FEAT_CONCAT

Concatenation&mdash;set by default.

The requested features parameter is set by bitmask, that is we need to add the decimal values together to get the necessary final value to use. Some values are set by default as described above. As can be seen from the table, the value 8192 sets the delivery acknowledgements value, requesting Clickatell to only send messages to gateways that have delivery acknowledgements.

The values in the table are only a subset of the full set of features available from Clickatell. For more details on the settings and parameters, you should read the Clickatell HTTP APIs specification from http://www.clickatell.com/downloads/http/Clickatell_HTTP.pdf.

Sending an email through ActionMailer

ActionMailer is the default framework in Rails used for sending and receiving emails. Before sending emails, we will need to configure ActionMailer first. Go to config and open up the appropriate environment configuration file (in development mode we will normally be using development.rb). Append the following to the end of the file and change the configuration appropriately:

config.action_mailer.delivery_method = :smtp
config.action_mailer.server_settings =
{
:address => <your smtp server> ,
:port => <smtp server port>,
:domain => <your server domain name> ,
:authentication => :login,
:user_name => <your username> ,
:password => <your password>
}

address and port are the address and port of the SMTP server that we're using to send the emails. Domain is the domain the mailer uses to identify itself to the server specified in address. You should normally use the top-level domain name of the machine that is sending the email. Use authentication if your SMTP server requires authentication to log into the system. If you use authentication you should also enter your SMTP user name and password in user_name and password respectively.

Now that ActionMailer is configured, generate the Mailer model through the generate script:

$./script/generate mailer Mailer

This creates a Mailer model in our app/model folder. The Mailer class in mailer.rb inherits from ActionMailer::Base and is the class we will use to send emails out.

class Mailer < ActionMailer::Base
def mail(template, contact)
@recipients = contact[:email]
@subject = 'Big Business Marketing Message'
@from = <sender email>
@body['text'] = template.message_body(:email, contact)
end
end

Each method in the mailer class sets up the environment for sending a particular email. In our case, we have only one email to send so we need only one method. The method sets up the environment by setting up instance variables containing data for the email's header and body. In our case, we set the recipient to the contact email and hard-code a marketing message as its subject. Next we set the sender email address and then the body of the email message from the message template. Body is a hash used to pass values to the email template. In this case, the message template's email body is passed to the email template as a variable named @text.

Let's create, then look at the email template named mail.text.html.rhtml. (RAILS_ROOT/app/views/mailer/). This file is in named in this format:

<method>.<content>.<type>.rhtml

The method is the name of the method in the Mailer class, in this case, mail. The content and type are the MIME type of the text to be sent. In this case we're sending an HTML message so the content type is text/html.

<div id='header'>Big Business Banner/div>
<%= @text%>
<div id='footer'>Copyright Big Business 2007</div>

Note that the email body from the message template has some HTML tags as well, and this goes well into this email template.

Customizing text messages according to the individual recipient

Finally, the messages in the message template can be customized according to the individual recipient. In this mashup we use the display name as well as additional text to be added for each recipient, if any. The text for the SMS message for example, goes like this:

Dear #{display_name}, New product just in! Check out our latest product offers at http://some.big_business.com! #{additional_message}

Notice that both display name and the additional text message customization look like the string replacement syntax in Ruby. This is because we evaluate this text as a Ruby string inside the Job class in RAIL_ROOT/app/model/MessageTemplate.rb :

class MessageTemplate < ActiveRecord::Base
has_many :jobs
def message_body(type,contact)
display_name = contact[:display_name]
additional_message = contact[:additional_message]
case type
when :email : eval '"' + self.email_body + '"'
when :sms : eval '"' + self.sms_body + '"'
when :fax : eval '"' + self.fax_body + '"'
else 'Incorrect type of message requested'
end
end
end

By evaluating the text as a Ruby string, we replace the values specified in the message body with the required values from the contact information. Whenever we need the message text, we just call it like this:

res = Net::HTTP.post_form(URI.parse('http://api.clickatell.com/http/sendmsg'),
{'session_id' => $clickatell_session_id,
'cliMsgId' => template.id,
'to'=>contact[:mobile_no],
'from' => 'Chapter 3',
'text' => template.message_body(:sms, contact),
'callback' => '3',
'deliv_ack' => '1',
'req_feat' => '8192' })

This little trick gives us great flexibility when doing text replacement on the messages for individual recipients.

This wraps up the mashup. Run this script to send the messages once you have set up the templates and the jobs.

Using the mashup

To recap the chapter, this is the sequence to follow in using the mashup:

  1. Start up the server

  2. Create one or more message templates

  3. Create a job

  4. Run the rake script to process the job

If you have set up things correctly you should receive the three marketing messages, through the email, SMS, and fax.

Summary

What we've learned in this chapter is to extract information from online spreadsheets through CSV and use that information to send messages through three different communications channels&mdash;email, SMS, and fax. We created a marketing message mashup using Google Spreadsheets and/or EditGrid, both online spreadsheets, Clickatell, a bulk SMS provider, as well as Interfax, an Internet fax provider.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.149.243.32