Fortunately, sourcing airfare data is somewhat easier that real estate data. There are a number of providers of this data, as well as paid and unpaid APIs. One challenging aspect of retrieving the data is that it requires a number of web requests. In the previous edition of this book, we outlined how to scrape data from Google's Flight Explorer page. This was the ideal presentation for seeing weeks of pricing data on one page for multiple cities. Unfortunately, that page has now been removed, and Google now provides a more typical search interface that requires the user to input the departure city, the destination city, start date, and end date. One fortunate feature that remains is the ability to input an entire region rather than a specific city. We'll make use of this in our scraping. An example of this can be seen in the following screenshot:

As you can see, we have input New York as our departure city and simply put Asia as our destination. This returns pricing for all the top cities in Asia (as well as the Middle East, for some reason). This is great news as we want to capture as many prices in one web request as possible.

While the interface still has some features that make it easier to scrape this data, we'll need to use some more advanced techniques than we've used in the past. We'll discuss that next.