Chapter 13. Server interaction with files and HTTP

This chapter covers

• Running Dart scripts from the command line
• Interacting with the filesystem
• Serving content via HTTP

The Dart virtual machine is hosted in two different environments. In the previous part of the book, we looked at using Dart in the web browser, where the VM is embedded in the Dartium web browser and has access to the browser DOM via the dart:html library. In this chapter, we’ll start to explore the server-side Dart VM, which doesn’t have access to the browser DOM but instead has access to operating system I/O, such as files and network sockets via the dart:io library.

The Dart File Browser example project scenario for this chapter has a client-and-server solution that provides a browser-based text editor for editing Dart files, which we’ll tackle in three steps starting with the server side. The server-side Dart VM is hosted in a command-line executable, available for Windows, Mac, and Linux. We’ll begin by looking at how you can write a simple Dart script that interacts with the filesystem, passing command-line arguments to output a directory listing.

Next, we’ll look at serving HTTP to react to HTTP requests from a web browser. We’ll show you how to match URL request paths and serve static files from a server-side Dart application. This gives you the ability to develop a complete client-and-server solution in Dart.

Finally, we’ll combine the HTTP server and directory-listing script to build a client-and-server application to serve the directory listing to the browser as JSON and let users browse the filesystem and read .dart files via a client-side web application.

At the end of this chapter, you’ll have a client-and-server application that you can use to browse the local filesystem and load .dart files (or other text files) into the browser for viewing. Figure 13.1 shows the client-side part, which runs in the browser and communicates with a server-side Dart script.

Figure 13.1. The client side of the Dart File Browser app

But before you start getting fancy with the user interface, we need to look at the server part of the application and how you can interact with the computer’s operating system to read files and folders from it.

13.1. Running server-side Dart scripts

The Dart File Browser server-side application runs as a Dart script, either from the command line or from the Dart Editor, and can output text to the console using the print("") command. In this section, you’ll create a server-side script to output a file-and-directory listing for a given path provided as a parameter to the script. For example, the command

dart.exe DirectoryList.dart C:Dart

outputs the following:

<DIR>  c:dartcss
<DIR>  c:dartother_project
<DIR>  c:dartstatic

<FILE> c:dartDirectoryList.dart
<FILE> c:dartOptions.dart
... etc...

When you use the Dart Editor to create a new project, you get the option to create either a web application or a command-line application. Up to this point, you’ve been using the web-application option, which creates a .html file and a linked .dart file. But a command-line application doesn’t have an HTML component, so it only requires the .dart file.

A Dart script that runs on the command line is known as a server-side script. It’s the same as a Dart script on the client side in that it’s a .dart file containing a main() function, which is the function that executes first. As with client-side scripts, the server-side Dart script can also be a library, can import other libraries, and has an event loop, just like browser-based scripts, as shown in figure 13.2.

Figure 13.2. Dart scripts running from the command line have the same capabilities and structure as client-side scripts.

Figure 13.3 shows a simple server-side script running in the Dart Editor and from the command line. The Dart Editor uses the Dart VM executable you’ll find in the Dart SDK when you run a script, and the output is shown in the Dart Editor’s console. When you run a script from the command line, you see the output in the command-line output.

Figure 13.3. Running a simple server-side script

Tip

The Dart Editor, available free from dartlang.org, provides you with all the tools required to run and debug server-side scripts. This chapter assumes you’re using the Dart Editor, but you can equally run server-side scripts from the Dart executable file, available in the bin/ folder of the Dart SDK download.

The big difference between client-side scripts and server-side scripts is the hosting environment. Client-side scripts run in a web-browser-hosted VM and have access to the browser DOM. This allows them to import the dart:html library. Server-side scripts don’t exist in a web browser, and you’ll get an error if you try to use the dart:html library on the server side. But server-side scripts do get access to the dart:io library, which provides classes and functions for accessing the filesystem, serving HTTP and web sockets, and communicating across the network with network sockets. Table 13.1 summarizes the differences.

Table 13.1. Summary of the differences between the dart:html and dart:io libraries

One other difference between client-side and server-side scripts is that server-side scripts can access the command-line arguments passed into the script. In order to let your server-side Dart directory-listing application list a specific directory, let’s look at how to access those arguments.

13.1.1. Accessing command-line arguments

Dart provides the Options type to access command-line arguments. Creating an instance of Options with the new keyword gives you a fully populated object containing the path to the current Dart executable binary, the script file containing the main() function, and a list containing the arguments passed on the command line. Listing 13.1, which prints the following output to the console when run with the command, shows a simple Dart script to output the command-line arguments and information about the executing script:

>dart.exe TryOptions.dart hello world
C:dart-sdkindart.exe
C:dartTryOptions.dart
2
[hello, world]

Listing 13.1. Accessing command-line options

In the Dart Editor, you pass command-line arguments to the command-line script using the Run > Manage Launches dialog, shown in figure 13.4, which passes the two arguments hello and world into the script.

Figure 13.4. Passing arguments to the command line in the Dart Editor

When you run command-line scripts from the Dart Editor, the output from print("") commands appears in the Dart Editor’s console.

Now that you can access command-line arguments in your script, it’s time to use them in the Dart File Browser application, which will output a list of files and folders. To do this, you also need to start using some of the types provided in the server-side dart:io library.

13.1.2. Accessing files and folders with dart:io

The Dart File Browser server-side application will have two features available from the command line. If you pass in the command-line argument --list option, followed by a path, you’ll output to the console a list of files and folders in that path.

If, instead, you pass in the --out command followed by a filename, you’ll read that file and output it to the console. Listing 13.2 shows the starting-point code for the Dart File Browser application, which outputs the two command-line options if no arguments are passed into the application. You also import the dart:io library, which contains the File and Directory types you’ll be using as you add functionality to the listDir() and outputFile() functions in following listings.

Listing 13.2. Display the command-line options if none are provided

In DirectoryList.dart, you need to implement two functions: listDir(folderPath) outputs a list of child files and folders in the directory specified by folderPath, and outputFile(filePath) reads the file specified by the filePath parameter. In these functions, you’ll use the File and Directory types.

Listing Files and Folders with a DirectoryLister

The first use case is to provide arguments such as --list c:dart to your DirectoryList.dart script. The first argument determines that you’re in list mode, and the second argument provides the folderPath for your script to output the contents.

The Directory type provides a number of methods that are also common to the File type, such as exists(), create(), and delete(). The Directory type also provides a list() function that returns a DirectoryLister, which you’ll use to provide a file and folder listing. DirectoryLister provides asynchronous access to return a list of files and folders in a specific directory.

The filesystem I/O types provide both synchronous and asynchronous access to the filesystem, with a preference for async operations. Whereas the library provides both async and sync methods, the async option is the default, with the sync version of the method having a sync suffix, as in Directory.exists() and Directory.existsSync().

Async calls that expect to return a single value, such as exists(), return a Future whose value is populated in its then() callback function. Method calls that return multiple values, such as a list of files and folders in a directory, provide methods that are assigned a callback function that’s called as each entry in the list is returned. This lets you begin outputting data to the console while the filesystem is still in the process of returning data to your application.

The exists() function and the DirectoryLister type are shown in figure 13.5, which demonstrates how to open a folder and list the files and directories in it. This provides the implementation for the listDir(folderPath) function in the DirectoryList.dart script.

Figure 13.5. The listDir() function uses the async exists() function and a DirectoryLister to return the list of files and folders in a directory.

If you start the application by passing in the arguments --list c:dart, it will output a list of files and folders in the c:dart folder, as you were expecting earlier in the chapter:

<DIR>  c:dartcss
<DIR>  c:dartother_project
<DIR>  c:dartstatic
<FILE> c:dartDirectoryList.dart
<FILE> c:dartOptions.dart

Note

The example paths are shown on the Windows operating system, but they’re equally applicable for Mac or Linux systems. Try replacing c:dart with ~/dart to list the contents of a dart folder in your home directory.

Now that your application can return a list of files in a directory by using the Directory and DirectoryLister types, let’s see how to use the File type to read the contents of a file in an asynchronous manner.

Reading Files with the File Type

The second use case for the DirectoryLister script outputs the contents of the file when its name is passed as an argument along with the --out parameter, such as --out c:dartTryOptions.dart. This command should read the contents of the file and print it back out on the console, and for this you use the File type.

Like the Directory type, the File type also offers sync and async API methods, such as readAsString() and readAsStringSync(). The async version returns a Future<String> containing the file content and uses nonblocking I/O, whereas the sync version will block but returns a String value containing the file content. You can use the async readAsString() method, as shown in the following snippet:

This asynchronously reads c:dartTryOptions.dart. Once all the data has been read and returned from the filesystem, the Future value is populated, passing the complete content of the file into the Future’s then() callback function.

Behind the readAsString() method, File uses an InputStream type, which returns sequential data in a buffered, nonblocking fashion. The data returned by the InputStream’s read() method is a List<int>, but it represents an array of bytes, each byte representing a single byte in the TryOptions.dart file.

Note

Dart doesn’t have different numeric types, such as byte, word, and long. Instead, it has a parent type of num and two child types: int and double.

The String type provides a utility constructor called String.fromCharCodes() that allows you to create a new String object from a List<int>.

Listing 13.3 uses an InputStream to implement the outputFile() function. The InputStream’s onData() method is called repeatedly when data is available; you access that data by calling the InputStream’s read() function, which may return data or may return null if no more data is available, as shown in figure 13.6.

Figure 13.6. Reading data from a file’s InputStream

When no more data is available, the onClosed() method is called, allowing you to output the complete data to the console. You use a StringBuffer type to store the multiple strings created by each call to onData(). The contents of the StringBuffer’s internal list of strings is efficiently converted into a single string when you call the StringBuffer’s toString() function. The following listing uses a file’s InputStream in the outputFile() function and also contains the existing listDir() function you saw earlier in the chapter, for completeness.

Listing 13.3. Implementing outputFile() with an InputStream

Using an Outputstream to Write File Data

You can also use the OutputStream type to easily write data to a file. Data is retrieved from an instance of File and provides write(List<int> buffer) and writeString (String string) functions to provide nonblocking writes to the underlying filesystem. Once all the data has been written (using one or more calls to one of the write() methods), you use the close() function to indicate that all the data has been sent.

Although not required for your Dart File Browser application, the following snippet shows how you can create a new file and write data to it:

Although the underlying system behind the output stream takes the data as fast as it can, the data being written is buffered internally, if required. The write() functions return a bool value indicating whether this buffering is taking place.

The File and Directory types provide all the standard mechanisms for interacting with the filesystem in an efficient, nonblocking manner by using underlying input and output streams. We’ve covered a lot in this section, from reading arguments from the command line to accessing files and folders and looking at async methods for nonblocking I/O.

In the next section, you’ll start to send and receive HTTP data to allow communication with a web browser, which also uses input and output streams. We’ll also show you how to send data directly from one InputStream into another OutputStream, known as piping.

13.2. Serving browser HTTP requests

Although the Dart File Browser application is great as a command-line application and can output file and folder information to the command line, it would be even better as a web application, able to send file and folder information to multiple web browsers and other HTTP clients. In the next section, you’ll use Dart to build a RESTful API to send file and folder information to the web browser.

getFolderListAsJson("c:/dart")

Your Dart script is going to change, too, from a script that runs and exits to one that continues running until you forcibly kill it. This is because the HttpServer type that comes in the dart:io library reintroduces the event loop when it starts listening for HTTP requests.

You’ll make your new client-and-server Dart File Browser application from two Dart applications. First, you’ll create the server-side application, which is jointly responsible for serving the files that make up the client application and responding to requests from the client application. Second, you’ll create the client application that, once served, makes requests back to the server to load file and folder lists. The flow is shown in figure 13.7.

Figure 13.7. The flow of requests between the client and server parts of the application

This is where nonblocking I/O comes into its own. By using async APIs to ensure that the server part of the application doesn’t block, control returns to the event loop as fast as possible, allowing the server to handle other requests before the first has completed.

13.2.1. Using the Dart HttpServer

The dart:io library provides three key classes that you’ll use when serving HTTP requests: HttpServer, which provides handlers that are triggered when a request is received; HttpRequest, which represents the incoming data; and HttpResponse, which represents the outgoing data. Table 13.2 lists some of the common properties and methods for these types.

Table 13.2. Common methods and properties of the key classes that serve HTTP requests

The HttpServer class receives requests, tries to match a request handler by calling all the assigned matcher functions until one is found, and finally calls the default request handler if no others match. You can try this with a small amount of code, shown in listing 13.4, which looks for a GET: /echo/ request and returns the method and path properties of the request to the browser. If none are found, it returns the string “Hello World.” Figure 13.8 shows the browser response for two sample URLs.

Figure 13.8. The output you expect from your server

Listing 13.4 shows the complete code to return this data. Once you call HttpServer’s listen() function, the event loop gains control and calls back into the matcher and handler functions when a browser request occurs to the correct host:port combination. The examples that follow use port 8080, but you can replace it with another port that’s valid for your machine if 8080 is already in use.

Listing 13.4. Serving simple HttpRequests from Dart

Now that you’ve seen how to respond to browser HTTP requests, let’s look at how you use this technique to send static files to the browser.

13.2.2. Serving static files over HTTP

To form the basis of your Dart File Browser server-side application, you need to use HttpServer to send the files that make up the client-side application to the browser. These include the .html, .css, and .dart or .js JavaScript files containing your browser application’s application code. In the previous section, we covered how to read files from the filesystem with an input stream—the HttpResponse object requires an output stream to send data back to the browser. Fortunately, Dart provides a function called pipe() that allows you to send the data read from one InputStream directly into another OutputStream, closing the output stream when all the data is transferred. You’ll use this mechanism to serve files to the browser.

The server application includes the following files, which contain the client application in a client subfolder:

c:DirectoryServer
                  ServerApp.dart
                  client
                         index.html
                         ClientApp.css
                         ClientApp.dart
                         ClientApp.dart.js

To send the correct files requested from the browser, you need to use the HttpRequest.path property. In order to achieve this, you’ll create a matcher and a handler class that are responsible for handling your static file requests by checking to see if the requested file ends with one of the four file extensions you’re expecting: .html, .dart, .css, or .js. Listing 13.5 shows the StaticFileHandler class and its use in HttpServer. The matcher function returns true if the request is for one of your four file types. The handler gets the requested filename, appends it to the ./client subfolder, and pipes the file’s InputStream directly to the response’s OutputStream. Note that you don’t have a close command to close the output stream—the pipe() method does this for you.

Listing 13.5. Serving static files

This is a trivial example, and any real HttpServer should provide extra functionality such as error-handling and security. At the moment, if you request a file that doesn’t exist, the server will crash (you need to use file.exists()). And if you provide a request such as http://localhost:8080/../../someOtherFolder/someFile.txt, you can navigate to other files and folders on the same machine.

Now that the Dart File Browser server-side application can send the client-side application to the browser, you’ll build a RESTful API to send your folder and file list formatted as JSON.

13.3. Serving clients with a RESTful API

You now have enough information to combine the HTTP server code with the directory listing code from earlier in the chapter. You want to take a GET request for a specific folder on your local disk and send a list of files and folders in the requested folder. You’ll use DirectoryLister’s onDir() and onFile() functions to build two lists, and you’ll combine these using the dart:json library. The dart:json library is a good example of code that’s shared between the client and the server; it contains identical code that runs in both places.

You also want to be able to send the contents of a specific file to the browser; but instead of serving the file to the browser as a native file, this time you’ll wrap the file’s content as JSON data. This gives you two API calls that your application needs to support, shown in figure 13.9.

Figure 13.9. The two API calls to retrieve a folder listing and file content as JSON data

In order to achieve this, you’ll have three request handlers: one to serve the static client-side application files, one to serve the folder list, and one to serve the file content. You’ll use the StaticFileHandler class you’ve already seen, along with two new classes: FolderListHandler and FileContentHandler. Figure 13.10 shows the ServerApp outline code and how to use these classes.

Figure 13.10. The outline of ServerApp

The main() method using these classes is shown in listing 13.6. The listing also contains a utility function called addHeaders(). It adds two important headers to your application, allowing you to use the Dart Editor’s client-side debugger when you build the UI. Web browsers come with a security feature to prevent HttpRequests (AJAX requests) from accessing data on a different URL than the rendered HTML file. These two headers tell the browser that, in this instance, it’s valid to accept the script from a different URL. For more information, search the web for information about cross-origin resource sharing (CORS).

Listing 13.6. Updated main() function using all three handler classes

The main() function is fairly straightforward. Adding extra handlers would mean creating more handler classes. For the moment, all of these classes are in the same file, but in a large project you’d split the handlers among multiple libraries.

Next, you’ll send your file and folder list as JSON data with the FolderListHandler class.

13.3.1. Sending a directory list as JSON data

The FolderListHandler class provides two methods: matcher() and handler(). The matcher() method returns true if the request is a GET request, and it has a path that starts with a /folderList prefix. When matcher() returns true, the associated handler() function is called, and you can extract the request data from the request path. The request data follows the /folderList prefix on the path, so you’ll use the core library subString() function to remove the /folderList part from the path and extract the folder name to list.

Once you have the folder name, you’ll use the Directory and DirectoryLister classes that you used earlier in the chapter. As you receive file and folder names, you’ll add them to two separate lists; and in the DirectoryLister’s onDone() function, you’ll put these two lists into a map and convert the map to a JSON string. You’re only interested in showing .dart files in the file list, so you’ll ignore any others.

Finally, you’ll send that JSON string to the HttpResponse’s output stream and close the output stream. Let’s look at the code, shown in the following listing.

Listing 13.7. Implementing the FolderListHandler class

If you start running the server, you can browse to a URL such as http://localhost:8080/folderList/dart, and it will return all the files and folders formatted as JSON in the c:dart folder, as you can see in figure 13.11.

Figure 13.11. Returning JSON data from the server API

13.3.2. Sending the file content as JSON data

The file content is equally similar to the DirectoryLister command-line application that you built earlier in the chapter. You use the FileContentHandler class to match on a path prefix of /fileContent, extract the filename that follows, read the file’s content as text, and return the text wrapped as a JSON string.

A URL such as http://localhost:8080/fileContent/dart/Options.dart will return the contents of the Options.dart file wrapped as JSON data. The FileContentHandler class is shown next.

Listing 13.8. Sending data to the browser with FileContentHandler

You’ve finished the server-side part of the application. It can respond to requests for files and folders and request specific file content. Next, you’ll add a simple client-side user interface that will be served by, and interact with, your server.

13.3.3. Adding the client-side user interface

The client-side part of the Dart File Browser application loads into the browser and uses the HttpRequest that we covered in the previous section to make asynchronous requests to the server’s REST API for data. When the browser receives data from the server, the user interface is updated. The key actions are shown in figure 13.12.

Figure 13.12. The main actions of the client-side application request data from the server and render it in the user interface.

Because the browser is a single-page web application and doesn’t transition between different physical web pages, you need to manage the browser navigation history yourself by using the pushState() and popState() functions (covered in chapter 11). Listing 13.8 shows the complete client-side application’s Dart code: the navigate() function is responsible for adding browser navigation history with pushState(), and the popState() handler is declared in main() and is responsible for the browser’s back button click. This is the same thing you saw in the DartExpense example earlier in the book, but the code is now served from server-side Dart and talks to your server’s REST API by using GET requests. Like the outline of the server-side code shown previously, the ClientApp’s outline is shown in figure 13.13.

Figure 13.13. The outline of the ClientApp code

The other key functions in the client-side code (shown in listing 13.9) are loadFolderList() and loadFileContent(), which perform the GET HttpRequests to your server API to update the user interface when the data is loaded. The loadFolderList() function is called when you navigate to a different folder. The loadFileContent() function is called when you click a specific filename to load it.

Listing 13.9. The client-side part of the Dart File Browser application

For completeness, the associated client-side HTML is shown in listing 13.10. You can add your own CSS styles to the application to lay it out as you wish or grab the CSS file from the source code website that accompanies this book. The HTML provides boilerplate <div> elements to hold folderList, fileList, and fileContent and is served, along with the ClientApp.dart file, by the server-side part of the application.

Listing 13.10. Client HTML file hosting the client-side application

You now have a fully working, fully Dart client-and-server application. The server side serves static files and dynamic data read from the filesystem, and the client side requests data via a RESTful interface and displays it in the browser. Multiple browsers can request data from your server without being blocked by previous requests.

An exercise left for you is to add a Save button to the client-side application and transmit file content back to the server via an HttpRequest POST method. The server can then use the file’s OutputStream (covered earlier in the chapter) to write updated data back to a file. In this way, you’ll have transformed the Dart File Browser application into a file editor.

13.4. Summary

You’ve produced a complete Dart solution in this chapter, with Dart running on both the client and the server. This Dart server application uses the HttpServer type provided in the dart:io library and uses the File and Directory types to read from the filesystem using nonblocking, server-side async APIs.

We covered how you can access command-line arguments from server-side scripts and how to read and write file data using InputStream and OutputStream. We also reviewed some of the utility wrapper functions, such as readAsString(), which make working with streams easier.

You used HttpServer to send both static files and dynamic data into a client, and you learned how to pipe the InputStream being read from a file into an HttpResponse’s OutputStream to send the file data to a browser. You also used the dart:json library on both the client and the server.

In the next chapter, we’ll return to the DartExpense application you built in chapters 10 and 11, and you’ll hook it up with a server-side database. You’ll serve the DartExpense application from a server-side application, as you did for the one you’ve built in this chapter; but instead of interacting with the filesystem to return a directory listing, you’ll interact with a database. You’ll use HttpServer to send JSON data between the client-side application and a NoSQL database via HTTP.