9.3.2.1. Ridge regression

This method, which applies a penalty based on the size of the coefficients, is a technique that can be applied to the problem of multicollinearity. When fitting a linear regression model, the goal is to find the values of the parameters that minimize the sum of the squares. In Ridge regression, a penalty term proportional to the sum of the squares of the parameters is added to the residual sum of squares.

9.3.2.2. Lasso regression

Lasso regression (Least Absolute Shrinkage Selector Operator) is a related modeling technique wherein the penalty is proportional to the sum of the absolute values of the parameters.

The only difference from the Ridge regression is that the regularization term is an absolute value. The Lasso method overcomes the disadvantage of the Ridge regression by punishing not only high values of parameter β, but by setting them to zero if they are not relevant. Therefore, we can end up with fewer variables in the model, which is a huge advantage.

After discussing the basics of modelling, linear regression, logistic regression and the Ridge and Lasso extensions, we now turn to action.

9.4.1.1. Data preparation

The first thing to do in the data analysis process, as we saw in Chapter 6, is to prepare the material that we will use in order to produce good results. Data preparation therefore constitutes the first phase of our analysis. But before taking this step, let’s look at our database for a moment (Figure 9.3).

Now we will clean and remove some variables in this first phase of analysis. The goal is to keep those that are able to provide information to the modeling process.

As the database file contains 95 variables, we removed those that seem useless (like: id, name, host_id, country, etc.) and many more columns containing information about the host, the neighborhood, etc. Obviously, this type of variable will not be useful for this analysis.

All of the retained variables form the linear predictor function, whose parameter estimation consists of formulating a modelling approach using linear regression. From this, we conducted an initial review of 39 independent variables for which 59,881 observations would be used to model the expected value in euros for a night of accommodations in your apartment.

The nature of these variables is therefore both quantitative and qualitative. Quantitative variables contain, for example, the number of comments, certain characteristics such as: latitude/longitude, etc. Qualitative variables include the neighborhood, room_type, etc.

Now that we know our variables, we can proceed with data preparation.

Using the describe function in Python, we can identify the variables with missing values, which we described previously, as shown in Figure 9.4.

We note first that all non-numeric variables are eliminated. Due to the nature of the regression equation, the independent variables (x_i) must be continuous. Therefore, we must consider transforming categorical variables into continuous variables.

What about the other categorical variables (qualitative)?

You would like, no doubt, to find out which of the model’s variables are empty. For this, we will use the percent_empty function to allow you to see all the variables and to represent the results graphically (Figure 9.5).

From this figure, we can say that the square_feet variable does not contain any value (almost 100%).

Similarly, the license, security_deposit, and cleaning_fee variables contain missing values at a rate of more than 30%. Therefore, we will remove these variables. There are also other variables that contain missing values, namely: neighborhood, zipcode, and jurisdiction_names, but since the percentage is very small, we can incorporate them into the model.

For the remaining variables, we will eliminate the street variable because we already have information in zipcode, as well as the calendar_updated and calendar_last_scraped variables.

Thus, we can see that certain variables contain the variable “NA”, which we won’t use in the next part of the analysis because they have no value for modeling.

This phase allows us to obtain descriptive statistics for the dependent variable (price) in our model, such as: the mean, standard deviation (std), and the minimum and maximum values, as shown in Table 9.1.

These statistics, in addition to the graphics, of course, are very useful for understanding the data used. For example, we can say that the average price of apartments in Paris in all the observations is 110.78 euros per night.

Another important point to mention here is that we have a very high maximum value (25,000) compared to the mean. Our work is therefore not finished, and we have to look into this. Is this a false value? If this is the case, how should we handle this problem?

To answer these questions, we will use the sort_values function to pull data from the price variable in descending order.

The results in Figure 9.6 show that this is not an inconsistency, but only a high proportion of the apartment’s price.

After preparing the data, we will move onto the next stage!

9.4.1.2. Exploratory analysis

After determining the explanatory variables that will be used in the model, it is important to now consider whether each provides completely or partially overlapping information. This is defined in the regression process as “multicollinearity”, which exists when two or more independent variables are highly correlated.

To detect the existence of this type of regression problem, and before proceeding with modelling, the correlations must be analyzed. In other words, for you to better understand this data and to identify interesting trends by building our model, we will examine variables that may be correlated with each other.

It should be noted here that we will mainly use the pandas libraries to browse the exploratory data analysis workflow and to manipulate our data, along with seaborn and matplotlib to draw the necessary graphics.

For this, we used the corr function, and the results are presented in Figure 9.7.

Figure 9.7 outlines the Pearson correlation coefficient when examined by means of pairwise comparisons.

In this figure, we want to show you how to create a visual graph (data visualization), which will allow us to see the values more clearly. Two colors are used (red and blue) to help us better understand the correlations between the different variables.

The color red represents a negative correlation and the intensity of scaling is dependent on its exact value. The color blue, meanwhile, represents a positive correlation.

By analyzing the correlations, we found some moderately strong (positive or negative) relationships between the different features. The results of the correlation analysis, which varies from positive to negative correlation (see the measurement scale), reveals that there is a clear positive correlation, for example:

• – between availability_60 and availability_90 (0.94);
• – between host_total_listings_count and calculated_host_listings_count (0.88);
• – between accommodates and beds (0.86).

The other variables, however, are only moderately or weakly correlated with each other. So, we decided to delete some and keep others like availability_90 because this variable is weakly correlated with other variables.

Therefore, we’ll only keep the host_total_listings_count variable, etc.

We’ll also eliminate variables with NA results (empty lines or columns in Figure 9.7), which includes: requires_license, has_ availability, and is_business_travel_ready, which will not be useful for our model.

The selection of explanatory (independent) variables is now complete. We finally have a data framework with 20 variables and 58,740 observations (98% of all observations). And we have deleted a total of 75 variables.

Now we have to deal with the characteristics of our dependent variable, “apartment prices” (price). For this, we’ll import the shuffle function, which intermixes the data-set to achieve a series of good results, or model.

Let’s go!

First, it is important to examine the distribution of our dependent variable (price). Figure 9.8 provides us with information about its overall distribution.

We see that the prices in this data-set are highly asymmetrical (2.39). In Figure 9.8, we can also see that most of the data points are below 250 (A). We will use a subset of our database, where the price varies from 50 to 250, to remove the very high and low prices. We’ll also transform the target variable to reduce the asymmetry (B).

Now, let’s examine the effects of different variables that have an effect on prices.

In Figure 9.9, you will find box plots for some of the variables in relation to price. They show consistency among variables such as room_type, bed_type, and property_type, which could affect the price of the house or apartment (Figure 9.9).

This figure shows comparative box plots for these variables. We note that there is a price difference between the condition of the room or the apartment in general. Therefore, cleanliness, room type, and bed type are the most expensive options to select at Airbnb apartments in Paris.

Now, let’s look at other available features, such as bedrooms and bathroom. The results show that there is a positive relationship between physical characteristics and prices. This seems reasonable, because the best equipped apartments are undoubtedly the most expensive (Gibbs et al. 2018).

Figure 9.10 shows the effect of the number of bedrooms and bathrooms on the price per night for Airbnb apartments in Paris. These two features are very important for setting the price, as the correlation coefficient between these two variables confirms (0.62).

But are bedrooms and bathrooms the only features of an apartment? Of course not! We should also take into account another very important feature that can also influence the price. These are amenities, namely an apartment’s conveniences.

The database includes over 65 amenities, and Figure 9.11 shows the 20 most popular features. Heating, basic equipment, kitchen equipment, and television sets are among the most common amenities that Airbnb apartments provide for clients.

The apartment’s location is also an important factor that often has a significant impact on prices. For you to really see how important it is, we have shown price as a function of neighborhood in the following figure. Figure 9.12 shows the location of the apartment is important and that it strongly influences the price.

Before finishing this step and going to the next one, we selected numeric variables and explored them all together. Thus, we have transformed categorical variables into a numeric format. This process converts these variables into a form that can be read by Machine Learning algorithms to achieve a better outcome prediction.

Now that we have finished the exploratory data analysis phase, we can move to modeling.

9.4.1.3. Modeling

Exploratory analysis allowed us to conclude that the overall condition of the apartment affects the price. Other important features you should consider are cleanliness, type of room and bed, square footage, etc. Some additional information, such as access to shops and public transportation, is very useful for setting the apartment price.

In the end, we selected 20 independent variables (Table 9.2) that are the most promising on the basis of the exploratory analysis we performed.

In this step we wanted to manipulate certain characteristics of our dependent variable (price), since they are recovered in price format, and the data contains the thousands separator (“,”) and the “$” symbol. For this, we performed the following operation:

Then we moved on to the modelling step, which involves the evaluation of the model (among those tested) that gives the best results, making it possible to generalize the results to unused data.

Since the objective of this case study is to build a model to predict the prices of Airbnb apartments in Paris, the type of model we are using is a linear regression model. So, we chose to perform a linear regression to account for the various dependencies that may be present in the data.

To resolve the multicollinearity problem between some of the independent variables, we also used Lasso and Ridge regression.

Before turning to the application of the model, it is essential to divide (split) the data into testing sets (train), so as to create a set of unchangeable data for assessing the model’s performance.

Given that:

• – X_train: all predictors in the test data-set (train);
• – Y_train: the target variable in the test;
• – X_test: all predictors for the entire test;
• – Y_test: the target variable in the test set; in our case, it’s price.

In this context, we used sets of variables that were pre-processed and developed in the previous step to perform these operations, using different combinations of variables. Then we adjusted the parameters for each model that was developed; then we selected the model with the greatest available precision.

This optimization process is the final step before reporting results. It makes it possible to find a parameter or a set of robust and powerful parameters using an algorithm for a given problem.

For the Ridge and Lasso regression models, we tested different parameters α (0.001, 0.01, 0.1, 1, 10, 100). The hyperparameters for each model and their validation scores are described in Table 9.3.

We can see from the results in Table 9.3 that linear regression does not do a good job of predicting the expected price because our set of variables contains many correlated data points. It was over-adjusted for Airbnb apartments in Paris, which led to horrible RMSE scores.

Thus, we can infer that Ridge regression outperformed Lasso. Ridge regression seems to be best suited for our data, because there is not much difference between the scores of the (train) and (test) data-sets.

We can conclude that the prices of Airbnb apartments in Paris are heavily dependent on available equipment and location. Indeed, most of the variables that have led to price increases are related to equipment and location.

We can apply other algorithms, such as “random forest”, decision tree, etc. to see if these characteristics (amenities/location) are the main factors for predicting the prices of Airbnb apartments in Paris. But we’ll leave that to the next chapter, in which we’ll study these algorithms more thoroughly.