One of the most frequent questions I hear from ASP developers is “How can I grab information from another web server?” Specifically, they’re interested in the HTML generated by a particular URL, be it a static web page or an ASP page.
There are many reasons a developer may be interested in using an ASP page to obtain a web page fresh from another server. One of the most popular reasons is to obtain some real-time data. For example, several of the portals maintain updated stock quotes and weather information. Many developers would like to be able to snatch this information and display it on their own web page.
Tip
Many sites that provide weather or stock information have the actual
temperature or quotes in a specific column and row in an HTML table.
An excellent article on 4GuysFromRolla.com,
“Grabbing Table Columns From Other Web Pages,” by Thomas
Winningham, presents an application to quickly and easily grab a
particular column and row from an HTML table residing on a different
web server. The article is available at: http://www.4guysfromrolla.com/webtech/031000-1.shtml.
To grab information from a web page, we need look no further than to Stephen Genusa’s excellent set of objects. The ASPHTTP object, available for $49.95 at http://www.serverobjects.com/products.htm#asphttp, allows developers to download HTML content from another web server. Furthermore, while using ASPHTTP, you can save the downloaded content to a file if you so choose!
When you download the ASPHTTP
DLL from ServerObjects.com, you will need to
move it to your Windows System directory and register it using
regsvr32. Once it is registered, you are ready
to start using the component in your ASP pages.
ASPHTTP contains a vast array of properties. A full listing of these properties can be seen at: http://www.serverobjects.com/comp/asphttp3.htm. Table 8.4 contains a listing of some of the more commonly used properties.
Table 8-4. The ASPHTTP Properties
|
Properties |
Description |
|---|---|
|
BinaryData |
Returns the data obtained by ASPHTTP in binary format. This is useful if you are using ASPHTTP to grab binary images, such as GIFs, from other web sites. |
|
Headers |
Returns the response HTTP headers sent by the web server contacted by ASPHTTP. |
|
Port |
Indicated what port to connect to on the remote web server (defaults to 80, the standard HTTP port). |
|
PostData |
Specifies the data to send through the HTTP POST. |
|
SaveFileTo |
Used to save the resulting HTML or binary data to a file. |
|
URL |
Specifies the URL to grab the information from (must start with
|
The most common ASPHTTP method is the GetURL method, which returns the response of the HTTP request to the URL specified by the URL property.
Since ASPHTTP is designed to request data through the HTTP protocol from other servers, let’s look at an example of doing just that. When using ASPHTTP, as with any of the other components discussed in this chapter, you must first create an instance of the object. To create an instance of ASPHTTP, simply use the following two lines of code:
DimobjASPHTTPInstanceSetobjASPHTTPInstance= Server.CreateObject("AspHTTP.Conn")
To retrieve the contents of an HTML page, all you need to do is set
the URL property accordingly and call the GetURL method. Example 8.3 contains ASP code that will grab the HTML
content from the LearnASP.com homepage and
display the HTML syntax.
Example 8-3. Grabbing the HTML of a Web Page on a Remote Web Server Is Easy with the ASPHTTP Component
<% @LANGUAGE="VBSCRIPT" %>
<% Option Explicit %>
<%
Dim objAspHTTP, strHTML
Set objAspHTTP = Server.CreateObject("AspHTTP.Conn")
'Grab the HTML from http://www.LearnASP.com
objAspHTTP.URL = "http://www.LearnAsp.com/default.asp"
strHTML = objAspHTTP.GetURL( )
%>
<HTML>
<BODY>
<H1>Here is the HTML to display LearnASP.com!</H1>
<XMP>
<%=strHTML%>
</XMP>
</BODY>
</HTML>
<%
Set objAspHTTP = Nothing 'Clean up!
%>Once the GetURL method is called, the
LearnASP.com web site is contacted through an
HTTP Get, and the response is stored in the variable
strHTML. Understand that
strHTML simply contains the HTML returned
by /default.asp on
LearnASP.com. When the contents of
strHTML are output, the HTML source is
shown due to the surrounding XMP tags. A
screenshot of the code in Example 8.3, when viewed
through a browser, can be seen in Figure 8.4.
ASPHTTP is not restricted to grabbing just textual HTML information from remote servers; binary data, such as GIFs, JPGs, and ZIPs, can also be obtained by ASPHTTP. Furthermore, with the SaveFileTo property, you can save these files to the web server.
Imagine that a News web site had a GIF that showed any major
accidents on the expressways in your location. This GIF was updated
as accidents occurred and were cleared, and was always stored in one
location, perhaps
http://www.TrafficNews.com/Chicago/map.gif. If
you wanted to provide a picture of this GIF on your site, you could
do so with a simple IMG tag, like so:
<IMG SRC="http://www.TrafficNews.com/Chicago/map.gif">
However, if your site attracts a lot of visitors during the day, the
webmasters at TrafficNews.com might not
appreciate your linking directly to their image and eating up their
bandwidth. To appease these webmasters, you might decide to copy this
GIF to your web server. However, if this traffic map is updated
frequently, the copy on your web server would become quickly
outdated.
The following situation would be ideal, and is quite possible using ASPHTTP:
A user visits a web page on your server that displays the accident map. Before displaying the map, the date and time the image was saved to the web server are compared to the current time. If the accident map on the local web server is less than one hour old, the user is shown that image.
If, however, the image is more than an hour old, the current accident map is downloaded from
TrafficNews.com, the user is shown the more recent map, and the downloaded accident map is saved to the local web server.
To accomplish this, we will use an ASP page,
/images/TrafficReport.asp, that outputs the
correct traffic report image (either the local copy or the one on the
TrafficNews.com web site) in binary GIF form.
Since /images/TrafficReport.asp will output the
GIF in binary format, you can display the GIF from any other ASP or
HTML page using the IMG HTML tag, like so:
<IMG SRC="/images/TrafficReport.asp">
In the file /images/TrafficReport.asp, we will
need to perform the following tasks:
Check to see if a version of the traffic report GIF exists on the local web server. If there is no local copy, proceed to Step 4.
Since the file exists, compare its date last modified property to the current time. If the file is more than an hour old, proceed to Step 4.
If we’ve reached this point, the traffic report GIF on the local web server is less than one hour old, so show this GIF and end.
If we’ve reached this point, either a local copy of the traffic report does not exist, or it is out of date. In either case, use ASPHTTP to grab the recent version from
TrafficNews.com, saving the GIF to the local web server.
The source code for /images/TrafficReport.asp
can be seen in Example 8.4. Note the comments in
Example 8.4 illustrating the start of the four steps
outlined.
Example 8-4. The Traffic Report GIF Is Only Downloaded from TrafficNews.com if the Local Version Is Outdated or Nonexistent
<% @LANGUAGE="VBSCRIPT" %>
<% Option Explicit %>
<%
'We are going to be outputting a GIF, so set the ContentType
Response.ContentType = "image/gif"
'strVirtualName is the virtual file name of the traffic report GIF
'on the local Web server
Const strVirtualFileName = "/TrafficReport.gif"
'strFileName stores the physical file name of the traffic report GIF
Dim strFileName
strFileName = Server.MapPath(strVirtualFileName)
'Create an instance of the FileSystemObject component
Dim objFSO
Set objFSO = Server.CreateObject("Scripting.FileSystemObject")
'******** STEP 1: Determine if the file exists ***********
If objFSO.FileExists(strFileName) then
'Since the file exists, we need to check when it was created
Dim objFile
Set objFile = objFSO.GetFile(strFileName)
'********* STEP 2: Compare the local copy's DateLastModified property
with the current date/time (Now( )) ******************
Dim dtDateModified
dtDateModified = objFile.DateLastModified
Set objFile = Nothing
If DateDiff("h", dtDateModified, Now( )) = 0 then
'****** STEP 3: The local copy is up to date - show it to the user! ******
Response.Redirect strVirtualFileName
Response.End
End If
End If
Set objFSO = Nothing
'********** STEP 4 **************
'If we get here, the file either doesn't exist on the web server, or it
'is outdated. Use ASPHTTP to grab the current version and save it to disk.
'*********************************
Dim objAspHTTP, strHTML
Set objAspHTTP = Server.CreateObject("AspHTTP.Conn")
'Grab the traffic report banner and save it to disk
objAspHTTP.URL = "http://www.TrafficNews.com/reports/TrafficReport.gif"
objAspHTTP.SaveFileTo = strFileName
objAspHTTP.GetURL 'Get the file
'Since we grabbed binary data from TrafficNews.com, we must output it using
'the BinaryWrite method of the Response object and the BinaryData property
'of the ASPHTTP component.
Response.BinaryWrite objAspHTTP.BinaryData
Set objAspHTTP = Nothing 'Clean up
%>