The Problem

“Have you seen the numbers sent by Nambi this morning?” asked Tamio.

Glanzo nodded and said, “I think they are pretty impressive given that we have had several storms around our farms in Brazil and South Africa this year.”

“You don’t understand my vision, do you?” asked Tamio, looking straight at his right-hand man. “My vision is to achieve a turnover of USD 500 million and reach USD 1 billion in six years. I told the board about this in the last AGM. You were there too,” said the boss.

Glanzo said, raising his shoulders, “We were all there, but I thought you were saying that to please the investors and the board.”

“No, that was a goal that I have been working towards for the past 15 years, ever since this company went public. This is not just a vision or a goal; this is a challenge that I have thrown up to you all,” said Tamio decidedly.

Glanzo retorted, “Yes, I understand the need to grow; otherwise, we will be eaten by the big fish in the industry. But you must understand the challenges that our operations are facing today, and unless we find real solutions for them, we will not grow at the rate that you want it to happen.”

Tamio now had a frown at his face as he sat down and rested his head on his huge, leather executive chair. He was listening intently as Glanzo spoke further. “The single thing that is preventing us from growing is our ability to predict the yield of crops in a given land.” Glanzo continued, “Our agriculture operations are a major problem for us in regions where we made a bad decision in buying infertile land that is low yielding. Low yielding land requires us to rectify it by applying different chemicals on hectares of land, which increases the cost of production and eats into our profitability. If we have to grow at the rate that you are spelling out, we need a way to determine which land is high yielding for a particular crop. If we are able to achieve this, we can avoid buying farms that are unproductive and low yielding, like the ones we have in Brazil where the soil is highly acidic and has to be treated with lime to make it more alkaline.”

Tamio raised his head slightly, signaling that he understood what Glanzo was talking about. He asked, “Do you know of a way to find out how to predict the yield for a particular type of land, Glanzo?”

Glanzo responded, “I have been looking at some of the research that has been happening in universities around the world; however, nothing concrete is available. But, in my opinion, data machine learning and AI show some promise to solve the problem. We can hire some machine learning engineers and data scientists who can help us create a model for ascertaining the yield of crop in a particular land location.”

“That sounds great. Why don’t you form a team to look into this problem and propose possible solutions?” asked Tamio, smiling at Glanzo.

Glanzo said, “Yes, that is what I intend to do. First, I’ll hire machine learning engineers and data scientist and then I’ll add people from our business operations to the team.”

“Send me an email for approval to go ahead,” said Tamio, picking up the eyeglasses from his desk as Glanzo rose to leave his cabin.

Machine Learning to the Rescue?

The machine learning team was formed in two months’ time with Hert Liu hired as the machine learning engineer for the pilot project. Along with three data scientists, he was made to take a robust tour of Aystsaga Agro’s agriculture operations in Brazil, South Africa, and India. The business operations team members were introduced to them once their induction program was over. With detailed briefings, Hert and his team had various new words typical to agriculture farming added to their vocabulary. They also understood the inside processes that went into producing crops. The detailed tours really helped the team in dig deeper into the company’s operations. However, what they lacked was the experiential knowledge, and that is the gap the business operations team members were going to fill in this pilot team.

Hert and his team met a couple of times after coming together. They met Glanzo several times too. He effectively communicated the company founder’s vision and the problem at hand, which was linked to its growth. Hert was an experienced machine learning engineer who worked in the insurance domain earlier. His only brush with agriculture was creating a model for predicting claims for agricultural farmers. So this was a very high learning curve for him, where he had to understand the intricacies of the commercial farming business and also the intricacies involved in managing it, such as crop failures due to insect infestations, changes in weather patterns, and the soil constitution and its effects on farm output. Hert and his team started gathering some parameters that they felt could help in predicting the farm output of a crop. They divided their analysis into weather, soil, and economic environments. They shared their understanding with Glanzo in bi-weekly meeting with him. Hert showed how weather was damaging crops in their Indian farms due to unpredictable rains and floods near the farm. Glanzo pointed out that the biggest problem they faced was determining the profile of high yielding farm land before purchase. “As per our founder’s vision, we are looking to buy a lot of farm land around the world such as in Australia, Thailand, and other regions; however, we can’t do this simply by blindly buying land and then finding out it yields less crop that the average farming operations. This means disaster to our ROI. You as a team should build a model that helps us in determining the profile of high yield cash crop farmland. To do so, you should look at the soil profile and what makes soil give high yield or low yield. Use whatever instruments you want; we can buy them. There is no shortage of funds but you must build a system that will benefit us the most in our bid to grow exponentially,” he said.

We are now going to build a solution for the problem of predicting the yield of a sugarcane cash crop based on the dataset from the file cashcrop_Yield_dataset.csv.

The Python Code

This is straightforward code. It first loads the dataset into a pandas dataframe and then checks data sanctity through df.size, df.dtypes, and other statements. The EDA is done with the df.mean(), df.median(), and df.mode() statements. The correlation is shown in Figure 9-1. In our case, we can have a look at the average mean values for soil pH and other soil nutrients in the output of Listing 9-2 and Figures 9-2 through 9-9. Outlier detection tells us that the yield data column has no outliers beyond the upper threshold limit; however, all the values are below the lower threshold limit. This can be seen visually by plotting the histogram using the command df.hist() in the code. After this, we can look at the correlation.

We can ignore and remove iron and copper because they do not show any significant correlation; in fact, it is negligible for them to be considered for our model building exercise.

        Crop  Center  FarmID  landsize_in_ha  Crop_in_tonnes  Yield_per_ha  

0  Sugarcane  Africa    1234            11.0          1235.3          112.3

   Soil_pH  Organic Carbon %  Nitrogen kg/ha  Phosphorus kg/ha  

0     7.81              0.88           355.0              36.0

   Potassium kg/ha  Iron mg/kg  Copper mg/kg  Sulphur kg/ha

0            485.0        9.73          4.73  33.0

6314

(451, 14)

Index([u'Crop', u'Center', u'FarmID', u'landsize_in_ha', u'Crop_in_tonnes', u'Yield_per_ha', u'Soil_pH', u'Organic Carbon %', u'Nitrogen kg/ha', u'Phosphorus kg/ha', u'Potassium kg/ha', u'Iron mg/kg', u'Copper mg/kg', u'Sulphur kg/ha'],

      dtype='object')

<class 'pandas.core.frame.DataFrame'>

RangeIndex: 451 entries, 0 to 450

Data columns (total 14 columns):

Crop                451 non-null object

Center              451 non-null object

FarmID              451 non-null int64

landsize_in_ha      451 non-null float64

Crop_in_tonnes      451 non-null float64

Yield_per_ha        451 non-null float64

Soil_pH             451 non-null float64

Organic Carbon %    451 non-null float64

Nitrogen kg/ha      451 non-null float64

Phosphorus kg/ha    451 non-null float64

Potassium kg/ha     451 non-null float64

Iron mg/kg          374 non-null float64

Copper mg/kg        337 non-null float64

Sulphur kg/ha       451 non-null float64

dtypes: float64(11), int64(1), object(2)

memory usage: 49.4+ KB

Training set has 360 samples.

Testing set has 91 samples.

('Coefficients: ', array([[-2.11198278,  0.02322665,  5.78698466,  3.69351487,  0.01497725,  0.17939622,  0.01021635,

         0.15396713]]))

mean squared error:  16.50

Variance score: 0.950297

Listing 9-2

Output of Code from Listing 9-1

In Figures 9-2 through 9-8, we can see that the bloxplot shows us a highly distributed value of Crop_in_tonnes. One way to avoid seeing this is to apply scaling to all of the numerical variables, like I did in the case study solutions for healthcare, retail, and finance in my book Machine Learning Applications Using Python. The histogram shows the distribution of variables, and we see that most of the soil nutrients are right-skewed except for copper and iron. We also note that the scatter plots for Yield_per_ha and Soil_pH are closely related through the Python code df.plot.hexbin(x='Yield_per_ha', y="Soil_pH", gridsize=20). After completing the usual data preparation steps by dividing it into a target variable named cropyield and a features variable with all the other features, we then split the dataset into training and testing datasets with 360 samples belonging to the training dataset and 91 samples belonging to the testing dataset. After this, we load the linear_model Python library to execute the linear regression algorithm on the training and testing datasets through the Python code regr.fit(X_train,y_train) , y_pred= regr.predict(X_test). Then we look at the accuracy score that we get with our testing dataset before starting to make a prediction through the code print("Variance score: %2f" % r2_score(y_test, y_pred)), which gives us 0.9502, or 95.02% accuracy. Since this is good, we can proceed with making a prediction. Please remember that this is fictitious data so we are able to achieve a good accuracy level. However, in the real world, you may need to run more regressors or fine-tune your data gathering efforts in order to achieve such an accuracy level.

Please remember that this program does not take into account other conditions such as weather and seed varieties, which also effect crop production and yield. Building such a dataset would definitely be a huge exercise beyond the scope of this book.