RAPM: ISOLATING PLAYER IMPACT

As I discussed in my Wins Above Replacement (WAR) write-up, our WAR model evaluates players across six components. I use RAPM (regression) to isolate individual impact for four of the six WAR components:

The regression isolates a player's impact by accounting for various external factors that surround them. These factors differ depending on the component which I am evaluating.

1Even Strength Factors

For even strength offense and defense, I account for the following components:

2Power Play & Penalty Kill Factors

For power play offense and penalty kill defense, I account for the exact same components as even strength, except instead of a variable denoting whether a shift started as the result of an expired power play, I use a variable denoting whether that power play began on-the-fly as the result of an expired penalty where the previous game state was even strength.

Building the RAPM Dataframe

It's very easy for me to say "I account for all of these factors." It would also be easy for me to say "Once you account for camera angles, I'm better looking than Brad Pitt." Both of these would rightfully be met with a heavy degree of skepticism from those who hear them, which is why I've decided to be as transparent as possible about my process of isolating skater impact.

I begin the process by building a dataframe which contains one row for each shift and one column denoting the presence of every skater and contextual factor as a dummy variable with a value of one or zero. I define "shifts" as all instances of play where the skaters on the ice, goaltenders, score, and period do not change. When any one of these variables changes, a new shift begins.

Here's a partial view of what this dataframe for constructing even strength RAPM looks like:

This is a massive dataframe that holds roughly 600,000 rows and 1,800 variables depending on the season. In this example, not a single one of the visible dummy variables are present.

A Single Shift Example

To give a closer example, let's look at just one shift that occurred in the third period of the second game of the 2018–2019 season. Here's what you need to know about this shift:

If we built a dataframe for RAPM using only this shift, here is what it would look like:

As you can see, there are two rows for one shift. The top row is from the perspective of the home team and the bottom row is from the perspective of the away team. If we move across to the end of the dataframe, we can see the skaters marked with ones and zeroes for defense:

Why Regularization is Essential

The regression is run using every shift from the entire season. xGF/60 serves as the target variable and every other variable in the dataframe serves as a predictor variable. This regression, however, is not a typical linear regression; rather, it is a weighted ridge regression, where the length of each shift is used as a weight, and the "ridge" serves to shrink coefficients down to zero.

Why do we need this ridge, which shrinks coefficients to zero? Because if we do not have it, we will have wildly inaccurate values. Here is a snippet of what RAPM results look like for the 2018–2019 season if I were to run a standard weighted linear regression without the ridge:

L2 (Ridge) Regularization

With regularization, we implement a "penalty" to the dataset which minimizes the mean-squared error by shrinking coefficients towards zero. The method of regularization I've chosen is L2 (Tikhonov) Regularization, where every coefficient is shrunk towards zero, but never to exactly zero, since I want coefficient estimates for each player. L1 (Lasso) Regularization would not work here since it would drop some variables entirely.

The penalty that I use comes in the form of a lambda value, which I obtain through cross validation. This plot shows us where the optimal lambda value is:

Once cross validation is run and the optimal lambda value is obtained, we run the regression once again using this lambda value as a penalty for every skater in the data set. The results, run on the same exact data set, look far more intuitive:

Prior-Informed RAPM

However, I still felt that the outputs were generally a tad too low, and that there was too much variation in year-to-year player impact. I did some research on NBA RAPM and found that this was not an uncommon sentiment; many basketball analysts also felt that one year of RAPM was somewhat unreliable, which is why they generally incorporate prior knowledge into their regressions.

I actually made an account on an NBA analytics forum and spoke with Daniel Myers, the creator of the NBA's Box Plus-Minus, who helped guide me through the process of creating prior-informed RAPM.

How Prior-Informed RAPM Works

First, prior information for each shift is subtracted from the target variable. For example, if Connor McDavid is on offense and my prior knowledge tells us that his xGF/60 should be 0.5, I subtract 0.5 from the observed xGF/60 for that shift. Then I run the full regression — first obtaining lambda values through cross-validation and then running the actual regression. After this is complete, I add his prior value back to his observed value from the regression.

Calculating the Prior

I initially tested using a player's full estimated impact from the prior season as a prior and found that on a year-to-year basis, player impact estimates went from being too malleable to not being malleable enough. It was practically impossible for players who had a strong impact in year one to have a poor impact in year three or vice versa.

I tested some things out and ultimately found a compromise between incorporating no prior information and too much prior information by calculating the linear trend between vanilla RAPM in year one and year two, and then using that trend to calculate a "predicted RAPM" which was used as a player's proper prior.

The Daisy Chain Method

I calculated prior-informed RAPM using a method known as a "daisy chain" where I chained a player's impact in each season to their impact in the next season. For example, if a hypothetical forward had an xGF/60 of 0.5 in 2013–2014, I would use that to calculate their prior for 2014–2015. Then, I would use the prior-informed RAPM from 2014–2015 to calculate the prior for 2015–2016, and so on and so forth through 2019–2020. I began the process in 2007-2008.

Source Code & Transparency