Webservers will not think anything is wrong if your webbots don't use cookies, since many people configure their browsers not to accept cookies for privacy reasons. However, if your webbot doesn't support cookies, you will not be able to access sites that demand their use. Moreover, if your webbot doesn't support cookies correctly, you will lose your webbot's stealthy properties. You also risk revealing sensitive information if your webbot returns cookies to servers that didn't write them.
Cookies operate transparently—as such, we may forget that they even exist. Yet the data passed in cookies is just as important as the data transferred in GET or POST methods. While PHP/CURL handles cookies for webbot developers, some instances still cause problems—most notably when cookies are supposed to expire or when multiple users (with separate cookies) need to use the same webbot.
One of the problems with the way PHP/CURL manages cookies is that as PHP/CURL writes them to the cookie file, they all become permanent, just like a cookie written to your hard drive by a browser. My experience indicates that all cookies accepted by PHP/CURL become permanent, regardless of the webserver's intention. This in itself is usually not a problem, unless your webbot accesses a website that manages authentication with temporary cookies. If you fail to purge your webbot's temporary cookies, and it accesses the same website for a year, that essentially tells the website's system administrator that you haven't closed your browser (let alone rebooted your computer!) for the same period of time. Since this is not a likely scenario, your account may receive unwanted attention or your webbot may eventually violate the website's authentication process. There is no configuration within PHP/CURL for managing cookie expiration, so you need to manually delete your cookies every so often in order to avoid these problems.
In some applications, your webbots may need to manage cookies for multiple users. For example, suppose you write one of the procurement bots or snipers mentioned in Chapter 19. You may want to integrate the webbot into a website where several people may log in and specify purchases. If these people each have private accounts at the e-commerce website that the webbot targets, each user's cookies will require separate management.
Webbots can manage multiple users' cookies by employing a separate cookie file for each user. LIB_http, however, does not support multiple cookie files, so you will have to write a scheme that assigns the appropriate cookie file to each user. Instead of declaring the name of the cookie file once, as is done in LIB_http, you will need to define the cookie file each time a PHP/CURL session is used. For simplicity, it makes sense to use the person's username in the cookie file, as shown in Listing 22-5.
# Open a PHP/CURL session $s = curl_init(); # Select the cookie file (based on username) $cookie_file = "c:ots".$username."cookies.txt"; curl_setopt($s, CURLOPT_COOKIEFILE, $cookie_file); // Read cookie file curl_setopt($s, CURLOPT_COOKIEJAR, $cookie_file); // Write cookie file # Configure the cURL command curl_setopt($s, CURLOPT_URL, $target); // Define target site curl_setopt($s, CURLOPT_RETURNTRANSFER, TRUE); // Return in string # Indicate that there is no local SSL certificate curl_setopt($s, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate curl_setopt($s, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirections curl_setopt($s, CURLOPT_MAXREDIRS, 4); // Limit redirections to four # Execute the cURL command (Send contents of target web page to string) $downloaded_page = curl_exec($s); # Close PHP/CURL session curl_close($s);
Listing 22-5: A PHP/CURL script, capable of managing cookies for multiple users