Chapter 18
Network Programming
Computers connected to a network can communicate with each other only if they agree on the rules of communication, called protocols, that define how to request the data, if the data should be sent in pieces, how to acknowledge received data, if the connection between two computers should remain open, and so on. TCP/IP, UDP/IP, FTP, and HTTP are some examples of network protocols.
Local area network (LAN) refers to a computer network connecting devices in a small area — the same office or house, or a rack. Interconnected computers located farther apart or that belong to different companies are part of a wide area network (WAN). The Internet consists of millions of networks and individual devices. When connected networks belong to the same organization it is referred to as an intranet. For security reasons intranets are shielded from the rest of the world by special software called firewalls.
The World Wide Web (WWW) uses uniform resource locators (URLs) to identify online resources. For example, the following URL says that there is (or will be generated on the fly) a file called training.html located at the remote host known as mycompany.com, and that the program should use the HTTP protocol to request this file. It also states that this request has to be sent via port 80.
www.mycompany.com:80/training.html
The hostname must be unique and it is automatically converted to the IP address of the physical server by your Internet service provider (ISP), aka your hosting company. The IP address is a group of four numbers, for example 122.65.98.11. Most of the individuals connected to the Internet are getting so-called dynamic (not permanent) IP addresses assigned to their home computers, but for a small extra fee you can request a static IP address that can be assigned to any server located in your basement, office, or garage. In the enterprises, network computers usually get static (permanent) IP addresses. For individual use, it’s sufficient to have a dynamically assigned IP address as long as your ISP can find your server by resolving a domain name to the current IP address.
Finding a resource online is somewhat similar to finding a person by his or her address. The role of an IP address is similar to the role of a street number of a building, and a port plays the role of an apartment number in that building. Many people can live in the same building, just as many programs can run on the same server. A port is simply a unique number assigned to a server program running on the machine.
Multiple Java technologies exist for providing data exchange among computers in a network. Java provides classes for network programming in the package java.net. This lesson shows you how to read data from the Internet using the class URL as well as direct socket-to-socket programming. Lesson 19 is about sending e-mails with JavaMail API. Lessons 26 through 37 introduce you to other technologies that you can use over the network: Java Servlets, RMI, EJB, and JMS.
Reading Data from the Internet
You learned in Lesson 16 that to read local file streams, a program has to specify the file’s location, such as c:practice raining.html. The same holds true for reading remote files — the only difference is that you open the stream over the network. Java has a class, java.net.URL, that will help you to connect to a remote computer on the Internet and get access to a resource there, provided that it’s not protected from the public. First, create an instance of the URL of your resource:
try{
URL xyz = new URL("http://www.xyz.com:80/training.html");
...
}
catch(MalformedURLException e){
e.printStackTrace();
}
The MalformedURLException will be thrown if an invalid URL has been specified, for example if you typed htp instead of http or included extra spaces. If the MalformedURLException is thrown, it does not indicate that the remote machine has problems — just check the spelling of your URL.
Creation of the URL object does not establish a connection with the remote machine; you’ll still need to open a stream and read it. Perform the following steps to read a file from the Internet:
1. Create an instance of the class URL.
2. Create an instance of the URLConnection class and open a connection using the URL from Step 1.
3. Get a reference to an input stream of this object by calling the method URLConnection.getInputStream().
4. Read the data from the stream. Use a buffered reader to speed up the reading process.
While using streams over the networks you’ll have to handle possible I/O exceptions the way you did while reading the local files. The server you are trying to connect to has to be up-and-running and, if you’re using HTTP-based protocols, a special software — web server — has to be “listening to” the specified port on the server. By default, web servers are listening to all HTTP requests on port 80 and to secure HTTPS requests directed to port 443.
The program in Listing 18-1 reads the content of the existing or generated file index.html from google.com and prints this content on the system console. For you to test this program your computer has to be connected to the Internet. The program may not work properly if your connection goes through a firewall.
Listing 18-1: Reading the content of index.html from google.com
import java.net.*;
import java.io.*;
public class WebSiteReader {
public static void main(String args[]){
String nextLine;
URL url = null;
URLConnection urlConn = null;
InputStreamReader inStream = null;
BufferedReader buff = null;
try{
// index.html is a default URL's file name
url = new URL("http://www.google.com" );
urlConn = url.openConnection();
inStream = new InputStreamReader(urlConn.getInputStream(), "UTF8");
buff = new BufferedReader(inStream);
// Read and print the lines from index.html
while (true){
nextLine =buff.readLine();
if (nextLine !=null){
System.out.println(nextLine);
}else{
break;
}
}
} catch(MalformedURLException e){
System.out.println("Please check the URL:" + e.toString() );
} catch(IOException e1){
System.out.println("Can't read from the Internet: "+ e1.toString() );
} finally{
if (inStream != null){
try{
inStream.close();
buff.close();
} catch(IOException e1){
System.out.println("Can't close the streams: "+ e1.getMessage());
}
}
}
}
}
The code in Listing 18-1 creates an instance of the class URL, then gets a reference to an instance of URLConnection to open a connection with the stream, and, finally, opens InputStreamReader, which is chained with BufferedReader. Run this program and you’ll see the output shown in Listing 18-2.
Listing 18-2: The fragment of console output shown after google.com is read
<!doctype html><html><head><meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1"><title>Google</title><script>
window.google={kEI:"BHVpTNWrGoGclgeNrczDBA",kEXPI:"25976,26060",
kCSI:{e:"25976,26060",ei:"BHVpTNWrGoGclgeNrczDBA",expi:"25976,26060"},
ml:function(){},kHL:"en",time:function(){return(new Date).getTime()},log:function(b,d,c){var a=new Image,e=google,g=e.lc,f=e.li;
a.onerror=(a.onload=(a.onabort=function(){delete g[f]}));g[f]=a;c=c||"/gen_204?atyp=i&ct="+b+"&cad="+d+"&zx="+google.time();
a.src=c;e.li=f+1},lc:[],li:0,Toolbelt:{}};
window.google.sn="webhp";window.google.timers={load:{t:{start:(new Date)
.getTime()}}};try{}catch(u){}window.google.jsrt_kill=1;
...
</scri
pt>
The class WebSiteReader explicitly creates the object URLConnection. Strictly speaking, you could achieve the same result by using the class URL:
URL url = new URL("http://www.google.com");
InputStream in = url.getInputStream();
Buff= new BufferedReader(new InputStreamReader(in));
The reason you may consider using the URLConnection class is that it could give you some additional control over the I/O process. For example, by calling its method setDoOutput(true) you specify that this Java program is intended to write to the remote URL too. In the case of HTTP connections, this will also implicitly set the type of request to POST (see Lesson 27). The method useCaches() of URLConnection also allows you to specify whether the protocol can use a cached object or should always request a fresh copy. In general, if you are planning to write Java programs that will work using HTTP protocol, use the class HttpURLConnection, which supports HTTP-specific features, such as processing header fields, getting HTTP response codes, setting request properties, and so on.
Connecting through HTTP Proxy Servers
For security reasons, most enterprises use firewalls (see http://en.wikipedia.org/wiki/Firewall_%28computing%29) to block unauthorized access to their internal networks. As a result their employees can’t directly reach the outside Internet world (or even some internal servers), but go through HTTP proxy servers. Check the settings of your Internet browser to see if you are also sitting behind a firewall, and find out the hostname and port number of the proxy server if you are. Usually, web browsers store proxy parameters under the Advanced tabs of their Settings or Preferences menus.
If your browser has downloaded a page containing a Java applet, the latter knows the parameters of the proxy servers and can make requests to the remote servers through the firewall. But a regular Java application should specify the http.proxyHost and http.proxyPort parameters to “drill a hole” in the firewall. For example, if the name of your proxy server is proxy.mycompany.com and it runs on port 8080, the following two lines should be added to your Java application that needs to connect to the Internet:
System.setProperty("http.proxyHost","http://proxy.mycompany.com");
System.setProperty("http.proxyPort", 8080);
If you do not want to hardcode these values, pass them to your program from the command line:
java -Dhttp.proxyHost=http://proxy.mycompany.com
–Dhttp.proxyPort=8080 WebSiteReader
The other option for programmatically specifying proxy parameters is to do it via the class java.net.Proxy. The code for the same proxy server parameter would look like this (you can replace the name of the server with an IP address):
Proxy myProxy = new Proxy(Proxy.Type.HTTP,
new InetSocketAddress ("http://proxy.mycompany.com", 8080);
url = new URL("http://www.google.com/index.html" );
urlConn = url.openConnection(myProxy);
How to Download Files from the Internet
Combine the class URL with the reading files techniques and you should be able to download practically any unprotected file (such as images, music, and binary files) from the Internet. The trick is in opening the file stream properly. Listing 18-3 shows the code of a Java class, FileDownload, which takes the URL and the destination (local) file name as command-line arguments, connects to this resource, and downloads it into a local file.
Listing 18-3: Downloading an arbitrary file from the Internet
import java.io.FileOutputStream;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
class FileDownload{
public static void main(String args[]){
if (args.length!=2){
System.out.println(
"Proper Usage: java FileDownload URL OutputFileName");
System.exit(0);
}
InputStream in=null;
FileOutputStream fOut=null;
try{
URL remoteFile=new URL(args[0]);
URLConnection fileStream=remoteFile.openConnection();
// Open output and input streams
fOut=new FileOutputStream(args[1]);
in=fileStream.getInputStream();
// Save the file
int data;
while((data=in.read())!=-1){
fOut.write(data);
}
} catch (Exception e){
e.printStackTrace();
} finally{
System.out.println("The file " + args[0] +
" has been downloaded successfully as " + args[1]);
try{
in.close();
fOut.flush();
fOut.close();
} catch(Exception e){e.printStackTrace();}
}
}
}
Note how this FileDownload program starts by checking the number of provided command parameters: If the number is anything but two, the program prints an error message and quits. Here’s an example of how you can run this program to download a W-4 tax form from the Internal Revenue Service website:
java FileDownload http://www.irs.gov/pub/irs-pdf/fw4.pdf w4form.pdf
In this section you’ll learn how to write a program that can read stock market price quotes from the Internet. There are many Internet sites providing such quotes; the Internet portal Yahoo! is one of them.
Visit http://finance.yahoo.com, enter the symbol of any stock, MOT for example, and press the Get Quotes button. You’ll see the pricing information about Motorola, Inc. — a fragment of this web page is depicted on Figure 18-1.
But what’s more interesting for our goal is the URL that is displayed in the Web browser:
http://finance.yahoo.com/q?s=MOT
Right-click this web page and select View Source from the pop-up menu to see the HTML contents of this page; you’ll see lots of HTML tags, and the information about MOT will be buried somewhere deep inside. Modify the line in the class WebSiteReader from Listing 18-1 to have it print the content of this page on the system console:
url = new URL("http://finance.yahoo.com/q?s=MOT");
You can also store the whole page in a Java String variable instead of printing the lines. Just modify the while loop in Listing 18-1:
String theWholePage;
String txt;
while (txt =buff.readLine() != null ){
theWholePage=theWholePage + txt;
}
If you add some smart tokenizing (splitting into parts based on the specified tokens as in Listing 18-4) of theWholePage, to get rid of all HTML tags and everything but Last Trade info, you can create your own little Stock Quote program. While this approach is useful to sharpen your tokenizing skills, it may not be the best solution, especially if Yahoo! changes the wording it uses on this page. That’s why we’ll be using another URL that provides stock quotes in a cleaner comma-separated values (CSV) format. Here’s the URL that should be used for the symbol MOT:
http://quote.yahoo.com/d/quotes.csv?s=MOT&f=sl1d1t1c1ohgv&e=.csv
This URL produces a string that includes the stock symbol, last trade, date and time of the price quote, earning per share (EPS), opening price, day’s range, and volume. Compare the CSB line below with the data shown in Figure 18-1 — they are the same (the date is not shown on Figure 18-1).
"MOT",7.81,"8/16/2010","4:00PM",0.17,7.63,7.86,7.58,25934265
Now the task of tokenizing the entire web page comes down to parsing a short CSV line. The StockQuote class from Listing 18-4 does exactly this: It accepts the stock symbol from the command line, gets the data from Yahoo!, tokenizes the received CSV line, and prints the price quote on the console.
Listing 18-4: Printing stock quotes
import java.net.*;
import java.io.*;
import java.util.StringTokenizer;
public class StockQuote {
static void printStockQuote(String symbol){
String csvString;
URL url = null;
URLConnection urlConn = null;
InputStreamReader inStream = null;
BufferedReader buff = null;
try{
url = new URL("http://quote.yahoo.com/d/quotes.csv?s="
+ symbol + "&f=sl1d1t1c1ohgv&e=.csv" );
urlConn = url.openConnection();
inStream = new InputStreamReader(urlConn.getInputStream());
buff = new BufferedReader(inStream);
// get the quote as a csv string
csvString =buff.readLine();
// parse the csv string
StringTokenizer tokenizer = new StringTokenizer(csvString, ",");
String ticker = tokenizer.nextToken();
String price = tokenizer.nextToken();
String tradeDate = tokenizer.nextToken();
String tradeTime = tokenizer.nextToken();
System.out.println("Symbol: " + ticker +
" Price: " + price + " Date: " + tradeDate
+ " Time: " + tradeTime);
} catch(MalformedURLException e){
System.out.println("Please check the spelling of "
+ "the URL: " + e.toString() );
} catch(IOException e1){
System.out.println("Can't read from the Internet: " + e1.toString() );
}finally{
try{
inStream.close();
buff.close();
}catch(Exception e){
System.out.println("StockQuote: can't close streams" +
e.getMessage());
}
}
}
public static void main(String args[]){
if (args.length==0){
System.out.println("Sample Usage: java StockQuote IBM");
System.exit(0);
}
printStockQuote(args[0]);
}
}
If you’ve gone through all the previous lessons in this book, reading and understanding the code in Listing 18-4 should be a piece of cake for you. Test the StockQuote program by running it from Eclipse (enter MOT as an argument in the Run Configuration window) or run it from a command window as follows:
java StockQuote MOT
Java-based technologies offer many options for network communications, and one of the technologies to consider is sockets. In this section you learn how to use the Java classes Socket and ServerSocket from the package java.net. A socket is a connection end point in IP networking. TCP/IP protocol maintains a socket connection for the whole period of communication, while UDP is a connectionless protocol, which sends data in small chunks called datagrams.
The socket address is a pair: IP address and port. When a Java program creates an instance of the ServerSocket class, this instance becomes a server that just runs in memory and listens on the specified port for other program requests. The following lines create a server that is listening to port 3000:
ServerSocket serverSocket = new ServerSocket(3000); client = serverSocket.accept();
The client program should create a client socket — an instance of the class Socket — pointing at the computer/port on which the ServerSocket is running. The client program can connect to the server using hostnames or IP addresses too, for example:
clientSocket = new Socket("124.67.98,101", 3000);
clientSocket = new Socket("localhost", 3000);
clientSocket = new Socket("127.0.0.1", 3000);
While deciding which port number to use for the ServerSocket, avoid using port numbers below 1024 to avoid conflicts with other system programs. For example, port 80 is typically used by HTTP servers; port 443 is reserved for HTTPS; port 21 is typically used for FTP communications; port 389 is for LDAP servers, and so on. After creating a socket-based connection, both client and server should obtain references to its input/output streams and use them for data exchange.
Why Use Sockets?
Why even use manual socket programming if you can easily establish inter-computer communication with, say, HTTP-based protocol (it uses sockets internally), start one of many open-source or commercial web servers, and have clients connect to the server as shown in the preceding sample programs in this lesson? Because a socket connection has a lot less overhead than any standard protocol.
You can create your own very compact protocol that will allow you to send only the data you need, with no or minimal headers. Socket communication provides a duplex byte stream, whereon the data travels simultaneously in both directions, unlike protocols based on the request-response model. Think of financial trading systems: Speed is the king there, and the ability to send data up and down at the same time saves milliseconds, which makes a difference.
On the other hand, if you design your application to use sockets, the live connection will be maintained for each user connected to ServerSocket. If your program has to maintain several thousand concurrent connections it will require more powerful servers than programs using the request-response system, with which a connection is maintained only during the time of the client’s request.
The Stock Quote Server with Sockets
Let’s build a socket-based client/server application that emulates both a server providing fake price quotes for requested stocks and a client consuming this data.
The server will start and listen to requests on port 3000 (see Listing 18-5). The method accept() of the SocketServer class is the one that waits for the client’s requests. As soon as it starts you’ll see the message “Waiting for a quote request” on the system console, and nothing else will happen until the request comes in from the client. Creating a SocketServer instance binds it to the specified port, but if this port is already in use by another process you’ll get a BindException.
Listing 18-5: The server generating stock quotes
import java.io.*;
import java.net.*;
public class StockQuoteServer {
public static void main(java.lang.String[] args) {
ServerSocket serverSocket = null;
Socket client = null;
BufferedReader inbound = null;
OutputStream outbound = null;
try
{
// Create a server socket
serverSocket = new ServerSocket(3000);
System.out.println("Waiting for a quote request");
while (true)
{
// Wait for a request
client = serverSocket.accept();
// Get the streams
inbound=new BufferedReader(new
InputStreamReader(client.getInputStream()));
outbound =
client.getOutputStream();
String symbol = inbound.readLine();
//Generete a random price
String price= (new
Double(Math.random()*100)).toString();
outbound.write(("
The price of "+symbol+
" is " + price + "
").getBytes());
System.out.println("Request for " + symbol +
" has been processed. The price is " + price);
outbound.write("End
".getBytes());
}
}
catch (IOException ioe)
{
System.out.println("Error in Server: " + ioe);
} finally{
try{
inbound.close();
outbound.close();
}catch(Exception e){
System.out.println("StockQuoteServer: can't close streams"
+ e.getMessage());
}
}
}
}
}
When a client connects to the socket, the server gets references to its I/O streams and sends randomly generated quotes for the requested stock. In the real world this server would have to be connected to another program providing real-time market data, but for our purposes generating random numbers as “price quotes” will suffice. The client program shown in Listing 18-6 has to be started with a command-line parameter such as IBM or MSFT to produce a price quote.
Listing 18-6: The client sending requests for stock quotes
import java.io.*;
import java.net.*;
public class Client {
public static void main(java.lang.String[] args) {
if (args.length==0){
System.out.println("Usage: java Client Symbol");
System.exit(0);
}
OutputStream outbound = null;
BufferedReader inbound = null;
Socket clientSocket = null;
try {
// Open a client socket connection
clientSocket = new Socket("localhost", 3000);
System.out.println("Client: " + clientSocket);
// Get the streams
outbound = clientSocket.getOutputStream();
inbound=new BufferedReader(new
InputStreamReader(clientSocket.getInputStream()));
// Send stock symbol to the server
outbound.write((args[0]+"
").getBytes());
outbound.write("End
".getBytes());
String quote;
while (true){
quote = inbound.readLine();
if (quote.equals("End")){
break;
}
System.out.println("Got the quote for " + args[0]+":" + quote);
}
}catch (UnknownHostException uhe){
System.out.println("UnknownHostException: " + uhe);
} catch (IOException ioe){
System.err.println("IOException: " + ioe);
} finally{
// Close the streams
try{
outbound.close();
inbound.close();
clientSocket.close();
} catch(IOException e){
System.out.println("Can not close streams..." +
e.getMessage());
}
}
}
}
Have you noticed that StockQuoteServer appends the word End to indicate that the price quote has ended? The class Client also checks for this word. This is an example of a very simple custom-made networking protocol.
The goal of this exercise is to test the socket communication in action, even if you have only one computer.
Lesson Requirements
You should have Java installed.
You can download the code and resources for this Try It from the book’s web page at www.wrox.com. You can find them in the Lesson18 folder in the download.
Hints
In this exercise you’ll be using two separate command windows to run the client and the server. But Eclipse IDE also enables you to have more than one Console view. Find a little icon that looks like a monitor in the Console view toolbar and click the little triangle next to it to switch between toolbars.
Step-by-Step
1. Create a new Eclipse project called Lesson18. Copy into this project and compile the classes StockQuoteServer and Client from the DVD accompanying the book.
2. Even though you can run both programs from Eclipse, it’s easier to observe the entire process if you run them from separate command windows. Open two command windows and imagine that they belong to different computers.
3. In each command window, go to the bin directory of the Lesson 18. In one of them start the server and in the other start the client — for example
java StockQuoteServer java Client IBM
4. Observe that the server generates prices, and that both client and server print the same price. By starting client and server in different command windows you are starting two separate JVM’s, emulating network communication between computers.
5. Open a couple more command windows and start the client program in them, providing different stock symbols as arguments. Observe that the same server can handle multiple clients’ requests. If you have access to a real network in which each computer has Java run time installed, run the client and server programs on different computers — just replace the localhost in the class Client with the network name or IP address of the server’s computer.
Please select Lesson 18 on the DVD with the print book, or watch online at www.wrox.com/go/fainjava to view the video that accompanies this lesson.