Now let's look at the curl
program as it is somewhat similar to wget
. One of the main differences between wget
and curl
is how they handle the output.
The wget
program by default displays some progress information on the screen and then downloads the index.html
file. In contrast, curl
normally displays the file itself on the screen.
Here is an example of curl
running on my system using my favorite website (screenshot shortened to save space):
Another way to get the output into a file is by using redirection like this:
You will notice that when redirected to a file the transfer progress is displayed on the screen. Also note that any error output goes into the file if redirected and not the screen.
Here is a very brief list of the options available in curl:
Curl options |
Explanation |
---|---|
|
Output filename |
|
Silent mode. Shows nothing, not even errors |
|
Show errors if in silent mode |
|
Verbose, useful for debugging |
There are many other options to curl
, as well as several pages of return codes. For more information consult the curl man
page.
And now here is a script showing how curl can be used to automatically get the current value of the Dow Jones Industrial Average:
#!/bin/sh # 6/6/2017 # Chapter 8 - Script 2 URL="https://www.google.com/finance?cid=983582" FN=outfile1.txt # output file TF=temp1.txt # temp file for grep loop=1 while [ $loop -eq 1 ] do rm $FN 2> /dev/null # remove old file curl -o $FN $URL # output to file rc=$? if [ $rc -ne 0 ] ; then echo "curl returned code: $rc" echo "outfile:" cat $FN exit 200 fi echo "" # carriage return date grep "ref_983582_l" $FN > $TF echo -n "DJIA: " cat $TF | cut -c 25-33 sleep 1h done exit 0
Here's what it looks like on my system. Normally you would probably leave the progress information out of the output by using the -s
option but I thought it looked cool and so left it in:
You can see that curl
and wget
work pretty much the same way. Remember when writing scripts such as these to keep in mind that the format of the page will almost certainly change from time to time so plan accordingly.
3.147.67.166