In the past, rankings were mostly reviewed as a reference by users. With this project, we found we are also in a position to assist our users in integrating our results with their decision-making tools, to help them utilize rankings better and also make their lives easier. For this, producing rules from rankings and also making scores behind rankings easily accessible became very important.
Because of the preceding reason, our deployment is still on to develop a set of rules and also to make all the scores available for decision makers, which include schools and some parents. Specifically, the main task of sending out a rule to alert users when some ranking changes dramatically, especially when a ranking drops down dramatically. Users of this project also have a need to obtain all the scores and rankings for their management performance.
Another purpose for this project is to produce good predictive models for the users to forecast possible changes of school rankings as per population changes using our developed regression models.
All the three needs for rankings, scores, and forecasting, mentioned in the preceding paragraph, are of value to various kinds of users who use various kinds of software systems for decision making. So, we need a bridge such as Predictive Model Markup Language (PMML), which is adopted as the standard by many systems. As discussed before, MLlib supports model export to PMML. Therefore, we export some developed models to PMML for this project.
In practice, the users for this project are more interested in rule-based decision making to use some of our insights and also in score-based decision making to evaluate their regional units' performance. Specifically, for this project, the client is interested in applying our results to (1) decide when an alert may be sent out if rankings have been changed or ranking changes will likely occur in the future, for which rules should be established, and to (2) develop scores and use them scores to measure performance as well as to plan for the future.
Besides this, clients are also interested in forecasting the attendance and other requests per ranking changes, for which R actually has a package called forecast that is ready to be used for this purpose:
forecast(fit) plot(forecast(fit))
To sum up, for our special tasks, we need to turn some of our results into some rules and also need to produce some performance scores for the client.
As discussed earlier, for R results, there are several tools to help extract rules from developed predictive models.
For the decision tree model developed to model whether or not a service request level exceeds a certain level, we should use the rpart.utils R package, which can extract rules and export them in various formats such as RODBC.
The rpart.rules.table(model1) * package returns an unpivoted table of variable values (factor levels) associated with each branch.
However, for this project, partially due to the data incompleteness issue, we will need to utilize some insights to derive rules directly. That is, we need to use the insights discussed in the last section. For example, we can do the following:
- If big mobility occurred and also parents' involvement dropped, our prediction shows rankings will go down dramatically and so an alert will be sent out
From an analytical perspective, we face the same issue here, to minimize false alerts, while ensuring adequate warning.
Therefore, by taking advantage of Spark's fast computing, we carefully produced rules, and for each rule, we supplied false positive ratios that helped the client utilize the rules.
With our regression modeling in place, we have two ways to forecast the ranking change at a specific time.
One is to use the estimated regression equations to do forecasting directly. Alternatively, we can use the following code:
forecast(fit, newdata=data.frame(City=30))
As long as we have obtained the scores, we can classify all the districts or schools into several categories and also illustrate them on a map to identify special zones for attention, such as the graphs as produced by R:
