The main strength of the Internet is its ability to communicate. This chapter looks at three ways in which CI makes communication easier.
First, we'll add to our testing toolkit by using CI's FTP class to access remote files directly.
Then, we'll use the email class to make our site automatically email us when certain conditions are met.
Lastly, we'll venture into Web 2.0 territory using XML-RPC to create a private 'web service' that allows our remote sites to take action and return information on a request form our testing site.
File Transfer Protocol (FTP) is a method of transferring files over the Internet. It's normally used to move files backwards and forwards to your website, using a special FTP program. It's something most of us only use occasionally, when we are putting up a new site.
You can, however, automate the whole process painlessly with CI. One use is to test the integrity of your remote site: are the files still there? As a website owner, you always face the possibility that someone will tamper with the files on your site. It may just be your ISP or your server admin, mistakenly deleting or over-writing something. (I had this happen to me once, when my ISP rebuilt their server and forgot to reload one of my application files. The file concerned wasn't used very often, but mattered a lot when it was. This led to an interesting error that took some time to track down!)
As one example of the power of the FTP class, let's build a regular test program, to check the files on a remote site. A few lines of code are all we need:
function getremotefiles($hostname, $username, $password)
{
$this->load->library('ftp'),
$config['hostname'] = $hostname;
$config['username'] = $username;
$config['password'] = $password;
$config['debug'] = TRUE;
$this->ftp->connect($config);
$filelist = $this->ftp->list_files('/my_directory/'),
$this->ftp->close();
return $list;
}
First, load the FTP library if you haven't already done so. Then, define the configuration parameters: hostname (e.g., www.mysite.com), username, and password for your FTP access.
Once connected, CI's FTP class gives you several options. In this case, we've used list_files() to return a list of files in the /my_directory/ folder. The function returns an array, and you can easily check this against an array of the files that you expect to find there. As before, we're trying to list all our tests in a database. So this time we need to list the FTP URL (or host name), the user name and password, and instead of a regex, the array of files to check against. To maintain the integrity of this array, if you store it inside your database, you will need to serialize it before you put it in, and un-serialize it when you take it out again.
Then it's easy to compare the $remotearray returned by the getremotefiles() function with the un-serialized $referencearray returned by your database:
function comparefiles($remotearray, $referencearray)
{
$report = "<br />On site, not in reference array: ";
$report .= print_r(array_diff($remotearray, $referencearray), TRUE);
$report .= "<br />In reference array, not on site: ";
$report .= print_r(array_diff($referencearray, $remotearray), TRUE);
return $report;
}
The PHP array_diff function compares the second array to the first, so it will list files present in the first, but not in the second. So, run the function twice, reversing the order of the array parameters: that way you get two lists, one of what isn't on your site (but should be) and one of what is on your site (but shouldn't be). The first should show any files that your ISP has accidentally deleted, the second any files that may have been added.
The CI FTP class also allows you to upload, move, rename, and delete files. Suppose that your test above reveals that one of the files in your reference array (let's call it myfile.php is missing from your site. You can use the FTP class to upload it:
$this->ftp->upload('c:/myfile.php', '/public_html/myfile.php'),
In this example, the local path is given first, and the path on the remote server second. Optionally, you can specify in a third parameter how the file should be uploaded (as ASCII or binary.) If you don't, CI makes its own decision based on the file extension which will usually be correct. If you are running PHP5, you can add a fourth parameter to set the file permissions, assuming you are uploading to a Linux server.
Be very careful about the delete option. As the user guide says, "It will recursively delete everything within the supplied path, including sub-folders and all files". Even writing this paragraph has made me nervous.
Using a combination of the FTP delete and upload functions, you could automatically update the files on your remote sites. List the files you need to update, and visit each site in turn, first deleting each old one, and then uploading each new one.
There is also an interesting 'mirror' function, which allows you to set up a complete duplicate of a website on another server.
If you are running PHP5, the FTP class also has a function that allows you to change file permissions.
As you can see, there's plenty of scope for expanding your application from testing your remote websites to actually maintaining or updating them. You could, for instance, write code to distribute updates automatically.
The Web 2.0 revolution is largely built on machine-to-machine interfaces, which allow mashups and APIs and all those good things.
This is the basis of 'web services'. You can offer an interface to your site that allows other people to use it to do something for them. To give a simple example, if you set up a 'web service' that converts temperatures in centigrade to Fahrenheit, the client sends in a request with one parameter (the temperature to be converted) and the server returns the converted value. So, anyone can add a temperature conversion function that appears to be on his or her own site, but is actually calling yours.
XML-RPC allows two machines to talk directly. The receiving site creates a simple API (application programming interface). Anyone who wants to talk to it needs to know that API what methods are available, what parameters they take, and what the syntax is for addressing them. Many major sites use this system: Google, for instance, allows you to make direct calls to its search engine or to Google Earth via a published API.
Setting up your own private API is relatively easy, thanks to CI. You need two websites to set this up and to test it, which makes it a little more complex than most things. One site (let's call it the 'receiving' site) is the one that offers the API, listens out for requests, and answers them. (In our example, this is one of the remote sites that we are trying to test and manage.) The other site makes the request using the API and gets the answer back. (In our example, this is the test site itself.)
In the XML-RPC protocol, the two sites talk by means of highly structured XML. (Hence the name XML-RPC it's short for XML Remote Procedure Call.) The client sends an XML packet to the 'receiving site' server, stating the function it wants to use and any arguments or parameters to be passed. The server decodes the XML and, if it fits the API, calls the function and returns a response, also structured as XML, which the client decodes and acts on.
Your API consists of the functions that the receiving site offers, and instructions for how to use them e.g., what parameters they take, what data type these should be, etc.
On the receiving site, we create an XML-RPC server, which makes the selected internal methods available to external sites. These 'internal methods' are actually just normal functions within one of your controllers: the server's role is to handle the interface between the external call and the internal function.
There are two sets of problems when you set up an XML-RPC process:
Getting the two sites to talk to each other
Making sure that the data is transmitted in a suitable format
Both rely heavily on multi-dimensional arrays, which machines can take in their stride, even if humans need to puzzle over them a bit. CI makes it a lot easier though it's still quite tricky to get right.
First, you have to set up a server on the remote site, and a client on the requesting site. This can be done with a few simple lines of code. Let's say we are doing the server in a controller called 'mycontroller' (on the receiving site) and the client in a controller called 'xmlrpc_client' (on the requesting site).
In each case, start off by initializing the CI classes within the constructor. There are two; for a client you only need to load the first, for a server you need to load them both:
$this->load->library('xmlrpc'), $this->load->library('xmlrpcs'),
Now, for the server. Close your constructor function, and within the 'mycontroller' index() function, define the functions you are offering up to the outside world. You do this by building a 'functions' sub-array (within the main CI $config array) which maps the names of the incoming requests to the actual functions you want to use:
$config['functions']['call'] = array('function' => 'mycontroller.myfunction'),
$config['functions']['call2'] = array('function' => 'mycontroller.myfunction2'),
In this case, there are two named function calls 'call' and 'call2'. This is what the request asks for. (It doesn't ask for the functions by name, but by the name of the call. Of course, you can use the same name if you wish.) For each call, you define a sub-sub-array giving the 'function' within the controller i.e. 'myfunction' and 'myfunction2' respectively.
You then finish off your server by initializing it and instantiating it:
$this->xmlrpcs->initialize($config); $this->xmlrpcs->serve();
and it is now ready to listen for requests.
Now you need to go to the other website the client and set up an XML-RPC client to make the requests. This should be a separate controller on your client site. It's quite short:
$server_url = 'http://www.mysite.com/index.php/mycontroller';
$this->load->library('xmlrpc'),
$this->xmlrpc->set_debug(TRUE);
$this->xmlrpc->server($server_url, 80);
$this->xmlrpc->method('call'),
You define the URL of the receiving site, specifying the controller that contains the XML-RPC server that you want. You load the XML-RPC class, define the server, and the method you want to use this is the name of the call you want to make, not of the actual function you want to use. If the function you are calling needs parameters, you pass them this way:
$request = array('optimisation','sites'),
As you see, we're passing two here.
Then, you check if a response has been received, and do something with it:
if ( ! $this->xmlrpc->send_request())
{
echo $this->xmlrpc->display_error();
}
else
{
print_r($this->xmlrpc->display_response());
}
The simplest option is to display it; but in a real application you're more likely to want the machine to analyze it, e.g., by using a regex, and then to act on the results. For instance, if the result contains an error message, you might want to record the error in your database, and take action to report it to the human user.
Let's use a real, if simplified, example. In this section, we will create an XML-RPC call/response that lets you remotely trigger a database optimization.
The client we wrote above, is asking for a method known as 'call' and supplying two parameters: 'optimisation' and 'sites'.
The server on the receiving site maps this request for 'call' onto a function called 'myfunction'.
Let's have a look at this function. It's basically an ordinary function within the controller. It attempts to optimize a MySQL database table, and returns 'success' or 'failure' depending on the result.
function myfunction($request)
{
$parameters = $request->output_parameters();
$function = $parameters['0'];
$table = $parameters['1'];
if ($this->db->query("OPTIMIZE TABLE $table"))
{
$content = 'Success';
}
else
{
$content = 'failure';
}
$response = array(
array(
'function' => array($function, 'string'),
'table' => array($table, 'string'),
'result' => array($content, 'string'),
XMLRPCexchanges, formatting),
'struct'),
return $this->xmlrpc->send_response($response);
}
Note the $request, set as the function parameter. This contains the $request array from the client remember, it had two values, 'optimisation' and 'sites'. CI has transformed the array into an object, $request. So you can't get the individual parameters by treating it as an array, instead you have to use the $request ->output_parameters() method of the $request object. This returns an array, which you interrogate in the normal way.
Using this, we have told the function on the receiving site which table we want to optimize, the 'sites' table. We've also told it what to call the function ('optimisation'). It adds a further parameter called 'result', gets the value, and returns all three to us.
The result it sends back to the client site looks something like this:
<?xml version="1.0" encoding="UTF-8"?> <methodResponse> <params> <param> <value> <struct> <member> <name>function</name> <value> <string>optimisation</string> </value> </member> <member> <name>table</name> <value> <string>sites</string> </value> </member> <member> <name>result</name> <value> <string>Success</string> </value> </member> </struct> </value> </param> </params> </methodResponse>
(Except it's not indented: I did that to make the structure clearer.)
As you can see, our simple three word response (optimisation, exercises, success) has been wrapped in verbose layers of tags, in a way sadly typical of XML, to tell a machine exactly what is going on. There are three<member></member> tag pairs. Each has a<name></name> pair ('function', 'table', 'result' respectively). And each of these has a<value></value> pair, which includes (as well as the data type) the actual information we want i.e. 'optimisation','sites','success'.
Never mind that I don't like it. Computers thrive on this sort of stuff: it is precise, unambiguous, and easy for a machine to read. This chapter is about computers talking to each other, not about user-friendly interfaces.
Now, your XML-RPC client function on your calling site can extract the values it wants and act on them. It's easy to do this with a regex, because each answer is clearly demarcated by XML mark-up brackets.
Note how CI spares you a lot of fiddling around with angle brackets you didn't need to write any of this stuff.
As soon as you start to test your client/sever combination, you will probably get this message:
The XML data received was either invalid or not in the correct form for XML-RPC. Turn on debugging to examine the XML data further.
Turn on debugging, by including the line:
$this->xmlrpc->set_debug(TRUE);
in your client. This allows you to see exactly what your client-receiving site combination is sending back to you. Be warned, this is where debugging gets quite frustrating.
There are several places where the exchange can go wrong:
The remote site is not responding properly. (You may have to temporarily set it to display errors in order to work out why it is not responding. This is annoying if it is an active site. The additional Catch 22 is that it will then display i.e. return as HTML error messages, which aren't part of the XML response your client expects, so you will get a second set of error messages, caused by the first set… ) Debugging this may involve quite a lot of FTP transfers back and forth, until you get it right.
You have got the URL wrong. (This needs to be CI's way of addressing the controller in which the XML_RPC server sits i.e.
http://www.mysite.com/ index.php/mycontroller. If you put all the server code in the controller constructor instead of in the index function, it will still work, but you need to address the function you want to call by name e.g.http://www.mysite.com/ index.php/mycontroller/myfunction).
The XML interchange may not be exactly right. The
set_debugfunction allows you to see what is being sent back, but you can spend quite a while staring at this trying to work out where it has gone wrong. (Believe me…)
However, once you get all this right, you've done something quite clever. You've built a function in a remote site, and called it remotely.
In other words, you've set up an application that can do maintenance or other operations on remote sites. If you have several remote sites to manage, you can easily replicate this across them, allowing you (for instance) to optimize all your database tables once a day by one action on just one site.
Security is an issue, of course. You would want to password-protect your function calls, so that the client had to send a password as a parameter before the receiving site responded. This can be done simply by sending the password as an additional parameter in the request, and having the called function check it before responding.
If you were exposing critical functions, you might want the whole thing to take place behind an SSL layer. Our example looks harmless you might not mind if a hacker repeatedly broke in to your site, but all he or she did was tidy up your database for you each time. On the other hand, it would be a good basis for a Denial of Service attack.
It has to be said that XML-RPC is frustrating and time-consuming to set up and debug, even with CI's very considerable help. You are writing and debugging two sites at once, and the XML format for transmitting data between them can only be called picky. It doesn't let you get away with even the smallest mistake.
Some would argue that XML-RPC is a superseded technology, with most new interfaces or APIs being written in more complex languages such as SOAP (which are even more time-consuming to set up) .
However, for our purposes, XML-RPC is ideal. It allows us to make our remote websites perform complex internal functions without bothering us with the details.
We've put together a lot of the building blocks for our web test site. We have a database of tests, and we've built functions to run different types of tests. We can access our site and check that we are seeing the right page; we can check that all the files are where we expect them to be on the remote server. We can automatically run functions on the site and get it to optimize itself. It's fairly simple to write code that uses these tools to run a suite of tests whenever we want, either when we log on or by some automatic reminder, such as setting a 'cron' job on a Linux server to start our program running at suitable intervals.
But it's not really enough to run tests and just store the results away in a database. If something is wrong, we need to know as soon as possible.
Here's where CI's email class comes in. It allows us to program our site to send us emails whenever certain conditions are reached. You might want to send an email for each failed test, or you might want to run a series of tests, collect the results, and then send just one email report.
To use the email class, first (as always) you have to load it.
$this->load->library('email'),
Then we have to set some configuration variables. This is where we can run into problems, because the class depends on the server that is hosting our code being able (and willing) to send email for us. Once again, we may have to check with the ISP. (It's also difficult to test this on a local site, because Xampplite, for instance, may not be able to offer you a mail server.)
However, once we've sorted out your ISP, we can easily configure the email class. There are a lot of options, all listed in the on-line user guide. The main ones are:
protocol: does your system use mail, sendmail or SMTP to send emails?
mailpath: where is your system's mail program stored?
You set them like this:
$config['protocol'] = 'sendmail'; $config['mailpath'] = '/usr/sbin/sendmail'; $this->email->initialize($config);
Other options, all of which have sensible defaults, include things like word-wrapping, character sets, whether you want to send text or HTML emails, and so on. Setting the options up and getting them working is the only (potentially) difficult part of using this class.
Once you've loaded the class and initialized it, using it is ridiculously intuitive.
$this->email->from('[email protected]'), $this->email->to('[email protected]'), $this->email->bcc('[email protected]'), $this->email->subject('Test message'), $this->email->message('Hello world'), $this->email->send();
will send me an email, copied to my client, reporting whatever message I want.
If you're sending more than one email, start each new one with:
just to make sure that you start with a clean slate each time.
You can also use the email class to send attachments. Remember that the attachment file must already be saved on the server that is sending the email, and you have to specify where, in terms of the server root file (giving the server address, not the web address).
Get its address and name like this:
$path = $this->config->item('server_root'),
$file = $path.'/my_subdirectory/myfile.htm';
then just add this line:
$this->email->attach($file);
before the $this->email->send();.
This simple CI function is so much easier than trying to write out the full PHP code to send attachments. It handles all the protocols involved, without you even having to be aware of them.
If you include the line:
$result = $this->email->print_debugger();
in your code, and print out the $result variable, you'll get a screenful of useful information, like this:
Your message has been successfully sent using the following protocol: mail User-Agent: Code Igniter Date: Wed, 18 Apr 2007 13:50:41 +0100 From: Return-Path: Bcc: [email protected] Reply-To: "[email protected]" X-Sender: [email protected] X-Mailer: Code Igniter X-Priority: 3 (Normal) Message-ID: <[email protected]> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="B_ATC_462614219d14d" This is a multi-part message in MIME format. Your email application may not support this format. --B_ATC_462614219d14d Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit test message hello world --B_ATC_462614219d14d Content-type: text/html; name="myfile.html" Content-Disposition: attachment; Content-Transfer-Encoding: base64
(etc. etc.)
If something went wrong, then the debug information will return any server error messages as well. For instance, if I set the delivery method to SMTP without setting the right host or permissions:
$config['protocol'] = 'smtp';
it can't send the message, and it tells me:
You did not specify a SMTP hostname Unable to send email using PHP SMTP. Your server might not be configured to send mail using this method.
Bear in mind, however, that 'sendmail' is potentially misleading here it returns a success message if it has passed the message on within the server, but this doesn't necessarily mean that the message has actually been sent. (So if you set the wrong 'mailpath' option, 'sendmail' may report that it has sent the email, when it actually hasn't.) CI relies on the messages it gets back from the mail sending application, so it can be fooled. As always with emails, they only way to be sure they have gone is to check that they've arrived but that's another story.
CI's email class includes several useful options, all explained in the online User Guide. For instance, you can set it to send text or HTML format mail if you chose HTML there's even a function to allow you to set a separate 'text' message for people who don't accept HTML email.
You can also set it to use different character sets, and to handle word-wrapping. You can set batch sizes, if you intend to send a lot of emails to a long mailing list, so that your server doesn't get overloaded. (Or your ISP doesn't panic and shut you down, thinking you are a spammer.)
We've now used CI to build some very sophisticated tools for our website, which give it some significant functionality.
Firstly, we used CI's FTP class to simplify and automate file transfer operations. Initially, we've just used this class to check that the files we expect to find on our site are actually there, and that noting unexpected has been added. This in itself is a valuable check, as many of the problems websites throw at you involve unexpected alterations of files, usually by site admins but sometimes by hackers. This function will check regularly. The CI FTP class also offers the possibility of remote maintenance and updating of sites.
Then we looked at developing our own private 'web services' using CI's XML-RPC classes. These allow us automatically to call functions on a remote site, pass in parameters if necessary, and have the results returned to us just as if we'd been logged on to the remote site instead of to our test site. We used this to have the remote site optimize a table in its database, and report back to us. Once again, we've gone beyond our original plan of simply monitoring the remote sites. Now we are able to instruct them to check or optimize themselves as well.
Lastly, we looked at the CI email class, which allows our testing site to generate emails. The CI code is extremely simple to use, and means that our site can notify us whenever it thinks there is a problem. CI makes it simple to build and send an email, and even to send attachments.