Month: May 2021


  • The 10 Best NBA Impact Metrics

    The 10 Best NBA Impact Metrics

    It’s been a long, windy journey to get here, one that started with plans for a video. (That wouldn’t have been enough time to discuss all ten metrics.) Then I recorded a podcast that ended up being just under an hour and a half long. I’m hoping to fall somewhere in the middle here, to provide as much information possible in a digestible amount of time. Ladies and gentlemen, I present to you (finally…), the 10 best NBA impact metrics.

    Criteria

    Ranking impact metrics proved to be no easy task. To limit errors that would come from an arbitrary approach, I chose to run with a very strict criterion:

    • Descriptivity

    To qualify for the list, the impact metric had to (at the very least) be a measure of the past, or what has already happened. There will be some metrics in the rankings that enlist predictive techniques as well. But as long as past data is used to also measure the past, it checks this box. It’s also worth noting most metrics that aren’t strictly predictive don’t inject themselves with predictive power for traditionally “predictive” purposes. Model overfitting is a consistent problem among pure Plus/Minus metrics, meaning scores can severely change from team to team even if the quality of the player stays the same. Combatting these phenomena will actually create some clearer images of the past.

    • Type of Measurements

    Because almost every “good” modern metric is based in some way on Adjusted Plus/Minus (RAPM), which employs philosophically fair principles, I figured it would be “fairest” to evaluate metrics based on their adherence to the original “ideology” of RAPM: to measure a player’s impact on his team in a makeshift vacuum that places him alongside average teammates while facing similarly average opponents. Because this approach would, in theory, cancel out a lot of the noise that stems from extreme team circumstances to measure the player independent from his teammates, impact metrics are judged on how they align with these ideas. (Impact metrics are distinct measures of value to only one team, but some will be able to move the needle more in overcoming problems like these.)

    • No Sniff Tests

    A lot of NBA critics or fans who aren’t mathematically inclined will often skim leaderboards for a metric to see how it aligns with their personal lists or player rankings. Because this approach places too much stock in prior information, and a lot of the critics may not actually evaluate players well, the sniff test is not really going to help anyone judge a metric. For this list, all specific player placements are set aside to only view the metric from a lens that focuses on how they perform in the aforementioned criteria.

    • Availability

    This doesn’t concern how the metric is judged itself, but the last qualification for a metric to appear on this list is its current availability. A metric I reviewed for this list called “Individual Player Value” (IPV) may have held a spot on the list, but there were virtually no opportunities to view the metric’s results from recent seasons. Thus, all metrics on this list were available (not necessarily free to the public but in the public knowledge) through the beginning of the 2021 season. If it isn’t clear, I really wanted to include PIPM here.

    • Modern Era Metrics

    Not all metrics on this list can extend as far back in the past as others. Most will be available from the 2014 season (when the NBA first started recording widespread tracking data) onward, while some can be calculated as far back as turnovers can be estimated (basically as far back as the institution of the shot clock). Because this really takes a “modern era” approach to evaluating these metrics, only a metric’s performance and value in the 2014 season and beyond will be in consideration during these rankings. So, for example, PIPM’s shaky nature in season predating Plus/Minus data is out of the equation here.

    • Disclaimer

    People can use impact metrics improperly during a debate all the time, but the most specific case I want to show can be explained by the following example. Let’s say, hypothetically, LeBron James finished as +7 in LEBRON in 2021 and +8 in BPM. If someone instigates a conversation with the BPM score, the interlocutor may provide the +7 LEBRON as a “better” or “more meaningful” representation of James. This is not a good way to go about comparing scores in impact metrics. Different metrics sway toward various playstyles, team constructions, etc. Just because LEBRON is a “better” metric (this shouldn’t really be a spoiler),  it won’t measure every player better than, say, BPM.

    List Structure

    If only this were as simple as only needing one list… Because different metrics treat different sample sizes differently, and the time period during which a metric is taken affects its accuracy relative to other metrics, I’ll split this list into two. The first, which will include the condensed metric profiles, assesses the metrics’ performances across three (full) seasons or more. Three years is the general threshold for stability, a point at which scores aren’t significantly fluctuating. The second list will evaluate metrics in a sample taken within a single season. Since players will often be analyzed using single-season impact metrics, this distinction will hopefully separate some of the metrics’ strengths and weaknesses in various environments.


    10. Luck-adjusted RAPM

    Developer: Ryan Davis

    Based on the original works of Regularized Adjusted Plus/Minus (RAPM), Ryan Davis added a prior to his calculations as a “luck adjustment.” It’s not a traditional prior that would, for example, use counting or on-off statistics to bring in outside information we know to hold value. Rather, the adjustment normalizes a player’s three-point shooting to his career average based on the location of the shot. Free-throw shooting is also normalized to career average. I’m particularly low on the function of the prior because, to me, it would make more sense to adjust teammate and opponent performance instead (what is done in luck-adjusted on-off calculations).

    My largest concern is that long-term samples of LA-RAPM will struggle to capture improvements over time. And if someone were to map out a player’s career, it would probably be too smooth, and not in a good way. Because the career averages are used, it might be a bit better in evaluating career-long samples as a whole, but it’s not going to measure individual seasons or even long samples much better than RAPM with no prior. The ideology and the processes behind the metric are impressive, but their applications seem a bit wonky to me.

    9. NPI RAPM

    Developer: Jeremias Engelmann

    The predecessor to every single other metric on this list, non-prior-informed RAPM was Jerry Engelmann’s improvement on Daniel Rosenbaum’s Adjusted Plus/Minus (APM), an estimate of the correlation between a player’s presence and the shift in his team’s Net Rating. Although a promising metric, APM was never truly built to be used in practice because of its inherent noisiness and high-variance solutions to linear-system appeasements. Englemann employed ridge regression, an equal-treatment form of Tikhonov regularization in which a perturbation form of traditional OLS appeasement uses various degrees of lambda-values (nowadays found through cross-validation) that suppress the variance of APM coefficients and draw all scores toward average, or net-zero.

    A lot of great analytical minds will say long-term RAPM is the cream of the crop of impact metrics. However, as was with APM, it’s still unavoidably noisy in practice. And since players are treated entirely as dummy variables in the RAPM calculations, devoid of any causal variables, my confidence in the accuracy of the metric is lower than others. RAPM is built to provide strong correlations between the players and their teams, but due to a lack of any outside information creates a greater level of uncertainty regarding RAPM’s accuracy, I’m going to rank it near the back end of this list. However, I have it higher than Davis’s luck-adjusted version for the aforementioned reasons relating to career mapping.

    8. Basketball-Reference Box Plus/Minus

    Developer: Daniel Myers

    The signature metric of Basketball-Reference and probably the most popular Plus/Minus metric on the market, Daniel Myers’s BPM 2.0 is arguably the most impressive statistical model on this list. There are some philosophical qualms I have with the metric, which I’ll discuss later. BPM continues the signature Myers trademark of dividing the credit of the team’s success across the players on the team. However, this time, he updated the metric to include offensive role and position on his team to add context to the environment in which a player accrued his box score stats. This means assists are worth more for centers, blocks are worth more for point guards, etc.

    BPM incorporates a force-fit, meaning the weighted sum of BPM scores for a team’s players will equal the team’s seasonal Net Rating (adjusted for strength of schedule). However, a team’s NRtg/A uses a “trailing adjustment,” which adds a small boost to good teams and a downgrade for poor teams based on how teams often perform slightly better when they are trailing in the game. The aforementioned gripes are mainly based on how BPM treats offensive roles. The metric will sometimes treat increments in offensive load as actual improvements, something we know isn’t always true. There are also some questions I have on the fairness of measuring offensive roles relative to the other players on the team.

    7. Backpicks Box Plus/Minus

    Developer: Ben Taylor

    I’ve gone back-and-forth between the two major Box Plus/Minus models for some time now, but after learning of some new calculation details from the creator of the metric himself, I’m much more comfortable in leaning toward Ben Taylor’s model. He doesn’t reveal too much information (in fact, not very much of anything at all), even to the subscribers of his website, but he was willing to share a few extra details: BPM uses two Taylor-made (double entendre?) stats: Box Creation and Passer Rating, estimates of the number of shots a player creates for teammates every 100 possessions and a player’s passing ability on a scale of one to ten. This is a very significant upgrade over assists in the metric’s playmaking components and certainly moves the needle in overcoming team-specific phenomena that don’t represent players fairly.

    Backpicks BPM also trains its data on more suitable responses, something I didn’t mention in the Basketball-Reference profile. Myers’s model uses five-year RAPM runs, notably decreasing the metric’s ability to measure stronger players. Conversely, Taylor’s two-to-three-year runs include stronger levels of play in the training data, meaning All-NBA, MVP level, and beyond caliber players are better represented. Teammate data is also toyed with differently. Rather than measuring a player strictly within the confines of his roster, the Backpicks model makes a clear attempt to neutralize an environment. To put this into perspective, Myers’s model thought of Russell Westbrook as a +7.8 player in 2016 (with Durant) and a +11.1 player in 2017. Taylor’s model saw Westbrook as a +7 player in both 2016 and 2017.

    6. Player Impact Plus/Minus

    Developer: Jacob Goldstein

    As was with me when I was first learning about impact metrics, introductory stages to basketball data will often lead people to believe PIPM is arguably the best one-number metric in basketball. It combines the box score with on-off data and has very strong correlative powers relative to its training data. But when I started to look under the hood a bit more, it was clear there more issues than immediately met the eye. Compared to other box metrics, PIPM’s box component is comparatively weak. It uses offense to measure defense, and vice versa. It doesn’t account for any positional factors and the response is probably the most problematic of any metric on this list. I’ll discuss this more.

    Recently, I’ve been leaning away from on-off ratings. They’re inherently noisy, perhaps even more so than RAPM, easily influenced by lineup combinations and minutes staggerings, which can make an MVP level player look like a rotational piece, and vice versa. The luck adjustment does cancel out some noisiness, but I’m not sure it’s enough to overcome the overall deficiencies of on-off data. PIPM is also based on one fifteen-year sample of RAPM, meaning the high R^2 values are significantly inflated. Again, this means very good players won’t be well-represented by PIPM. This excerpt may have sounded more critical than anything. But the more I explore PIPM, the most I’m met with confounders that weaken my view of it. Perhaps the box-only metrics are slightly better, but I’ll give PIPM the benefit of the doubt in the long term.

    5. Augmented Plus/Minus

    Developer: Ben Taylor

    Augmented Plus/Minus (AuPM) similarly functions as a box score / on-off hybrid. It incorporates the Backpicks BPM directly into the equation, also roping in raw Plus/Minus data such as On-Court Plus/Minus and net on-off (On-Off Plus/Minus). It includes a teammate interaction term that measures the player’s Plus/Minus portfolio relative to other high-minute teammates, and the 2.0 version added blocks and defensive rebounds per 48 minutes. There’s no direct explanation as to why those two variables were included; perhaps it included the regression results a bit more, despite having likely introduced a new form of bias.

    Pertaining to the AuPM vs. PIPM debate, it should be abundantly clear that AuPM has the stronger component. And while PIPM bests its opponent in the on-off department, the inclusion of shorter RAPM stints in the regression for AuPM means more players will be measured more accurately. So, despite arguably weaker explanatory variables, the treatment of the variables leans heavily in favor of Augmented Plus/Minus.

    4. RAPTOR

    Developers: Jay Boice, Neil Paine, and Nate Silver

    The Robust Algorithm using Player Tracking and On-Off Ratings (RAPTOR) metric is the highest-ranked hybrid metric, meaning every metric higher uses RAPM calculations in its series of calculations, not just as a response for the regression. RAPTOR uses a complex series of box scores and tracking data paired with regressed on-off ratings that consider the performances of the teammates alongside the player and then the teammates of those teammates. The regression surmounts to approximate one six-year sample of NPI RAPM. My high thoughts of it may seem inconsistent with the contents of this list. However, one major theme has made itself clear throughout this research: tracking data is the future of impact metrics.

    Despite a “weaker” response variable, RAPTOR performs excellently alongside other major one-number metrics. During Taylor Snarr’s retrodiction testing of some of these metrics, which involved estimating a team’s schedule-adjusted point differential (SRS) with its players’ scores in the metrics from previous seasons (all rookies were assigned -2), RAPTOR was merely outperformed by two metrics, both prior-informed RAPM models. This is a good sign RAPTOR is assigning credit to the right players while also taking advantage of the most descriptive types of data in the modern NBA.

    3. Real Plus/Minus

    Developers: Jeremias Engelmann and Steve Ilardi

    Real Plus/Minus (RPM) is ESPN‘s signature one-number metric, touted for its combination of descriptive and predictive power. According to the co-creator, Steve Ilardi, RPM uses the standard series of RAPM calculations while adding a box prior. This likely means that, instead of regressing all the coefficients towards series as explained in the NPI RAPM segment, Engelmann and Illardi built a long-term RAPM approximation, which then acts as the datapoints the player scores are regressed toward. Value changes and visual instability aside, RPM is among the premier groups of metrics in their ability to divvy the right amounts of credit to players, having finished as the metric with the second-lowest SRS error rate in Snarr’s retrodiction testing.

    2. LEBRON

    Developers: Krishna Narsu and “Tim” (pseudonym?)

    BBall Index‘s shiny new product, as new as the latest NBA offseason, the Luck-adjusted Player Estimate using a Box prior Regularized On-Off (LEBRON) metric makes a tangible case as the best metric on this list. Similar to RPM, it combines the raw power of RAPM with the explanatory power of the box score. LEBRON’s box prior was based on the findings from PIPM, but upgrades the strength of the model through the incorporation of offensive roles, treating different stats differently based on how certain playstyles will or won’t accrue the stats. Three-point and free-throw shooting is also luck-adjusted in a similar fashion to PIPM’s on-off ratings to cancel out noise.

    Part of what makes LEBRON so valuable in a single-season context is its padding techniques, which involve altering a player’s box score profile based on his offensive role. For example, if Andre Drummond shot 45% from three during his first ten games, LEBRON’s box prior will take his offensive role and regress his efficiency downward based on how he is “expected” to stabilize. This makes LEBRON especially useful in evaluating players during shorter stints, and while these adjustments aren’t perfect, the metric’s treatment of box score stats probably uses the best methods of any metric on this list.

    1. Estimated Plus/Minus

    Developer: Taylor Snarr

    I don’t want to say Estimated Plus/Minus (EPM) runs away with the top spot on this list, because it doesn’t, but it’s clear to me that it’s the best widespread impact metric on the market today. Roughly as young as LEBRON, EPM is the product of data scientist, Taylor Snarr. As noted by RPM co-creator Ilardi, EPM is similar to RPM in that it uses RAPM calculations to regress toward the box score, but also includes tracking data. The tracking data, as has been shown with RAPTOR, makes all the difference here. During his retrodiction testings, Snarr constructed linear regression models to estimate the effect of lineup continuity on the metric’s performance.

    To the envy of every other metric, EPM’s reliance on lineup continuity was estimated to be roughly half of the runner-up metric, RPM. It may not sound like a crucial piece of information, but given EPM’s model strength and some of these types of metrics’ largest problems, EPM performs fantastically. It’s also worth mentioning EPM is predictive as well, having led the retrodiction testing in SRS error throughout the examined seasons. I allowed these details to simmer in my head for some time in case I was having some type of knee-jerk reaction to new information, but the points still stand tall and clear: EPM is currently unmatched.


    For the single-year list, I only made two significant changes. Because shooting is much noisier in smaller samples, such as a season or less, I’ll give the edge to Davis’s LA-RAPM over NPI RAPM. Additionally, PIPM’s response issues drop it down a peg for me in the one-year context. I did consider putting LEBRON over EPM due to its padding (EPM doesn’t employ any stabilization methods to my knowledge), but the tracking data leading to greater team independence is too large an advantage for me to overlook.

    Single-Season Rankings

    1. Estimated Plus/Minus
    2. LEBRON
    3. Real Plus/Minus
    4. RAPTOR
    5. Augmented Plus/Minus
    6. Backpicks BPM
    7. Basketball-Reference BPM
    8. Player Impact Plus/Minus
    9. Luck-adjusted RAPM
    10. NPI RAPM

    There was also some noise surrounding DARKO’s Daily Plus/Minus ranking on this list. I did evaluate the metric for this list despite its breaking of the criteria, simply to include it as it stacks up against the other metric in model strength. Based on the statistical model, I would rank it fifth on this list, bumping AuPM down a spot and slotting DPM right behind RAPTOR.

    To my surprise, some people saw DPM as the premier impact metric on the market today. Some questioning led back to DPM’s developer’s, Kostya Medvedovsky, game-level retrodiction testings during the tracking era, which saw DPM lead all metrics. However, DPM is specifically designed to act as a predictive metric, giving it an unjustified advantage in these types of settings. Based on how I “expect” it would perform in descriptive duties based on the construction of its model, I don’t really see an argument for it cracking the inner circle (the Final Four).


    Thanks for reading everyone! Leave your thoughts in the comments on these metrics. Which is your favorite? And which one do you believe is the best?


  • The Best Regular-Season Teams of 2021 Per Oliver’s Four Factors

    The Best Regular-Season Teams of 2021 Per Oliver’s Four Factors

    (? The Ringer)

    About a year ago, I trained Dean Oliver’s four factors (AKA the outcomes for a possession) on offensive and defensive ratings from 1974 to 2019 to estimate the per-100 performance of a team. The results were extremely promising (R^2 of 0.99 with no heteroscedasticity), so it’s fair to say most teams will be well-represented by this method. I’ve decided to return to this system at the closure of the 2021 regular season to look at how the model viewed teams this season!

    The Method

    As stated earlier, Oliver’s four factors are extremely predictive of how well a team performances in a given season. (This is expected because the outcome of a possession greatly influences how many points are scored and allowed.) The 2021 results are out-of-sample; the training data covered almost every season since the initial recording of the four-factor statistics, with a smaller sample validating the results. The model itself passed the standard criteria for use as an OLS regression model, with the final product being split into offensive and defensive components.

    The Results

    The 10 Best Teams:

    1. Utah Jazz (+7.5 Net)
    2. Philadelphia 76ers (+5.8 Net)
    3. Milwaukee Bucks (+5.5 Net)
    4. LA Clippers (+5.2 Net)
    5. Brooklyn Nets (+5.1 Net)
    6. Phoenix Suns (+5.0 Net)
    7. Denver Nuggets (+3.6 Net)
    8. Los Angeles Lakers (+2.9 Net)
    9. Dallas Mavericks (+2.4 Net)
    10. Atlanta Hawks (+2.3 Net)

    Unsurprisingly, the Utah Jazz remain the best team in this model, but with a significant error between their actual Net Rating and predicted Net Rating, straying 1.8 points away from the real mark of +9.3 Net. The Sixers sneak up from fifth to second here, with a +0.2 point overestimation ranking them as the Eastern Conference’s regular-season champion.

    The 10 Best Offenses:

    1. Brooklyn Nets (117.9 ORtg)
    2. Milwaukee Bucks (116.6 ORtg)
    3. Portland Trail Blazers (116.5 ORtg)
    4. Utah Jazz (116.2 ORtg)
    5. LA Clippers (116.1 ORtg)
    6. Phoenix Suns (116.1 ORtg)
    7. Denver Nuggets (115.7 ORtg)
    8. Dallas Mavericks (115.1 ORtg)
    9. Atlanta Hawks (114.8 ORtg)
    10. New Orleans Pelicans (113.6 ORtg)

    This mostly resembles the actual offensive rating leaderboard, with another Eastern Conference stand-out popping up to #2 and teams like the Pelicans narrowly stepping into the top-10 over the Philadelphia 76ers and their 113.1 predicted offensive rating.

    The 10 Best Defenses:

    1. Los Angeles Lakers (107.2 DRtg)
    2. Philadelphia 76ers (107.3 DRtg)
    3. New York Knicks (108.3 DRtg)
    4. Utah Jazz (108.7 DRtg)
    5. Golden State Warriors (109.9 DRtg)
    6. Miami Heat (110.2 DRtg)
    7. Memphis Grizzlies (110.4 DRtg)
    8. LA Clippers (110.9 DRtg)
    9. Milwaukee Bucks (111.1 DRtg)
    10. Phoenix Suns (111.1 DRtg)

    There are no real surprises in the defensive component, which posts the same ten teams in a slightly different order. As is with its offensive half, the model found floor efficiency and turnover rates were far more predictive of a team’s success than rebounding rate of FT/FGA, so the results may be a tad biased toward teams who contest well and induce a lot of turnovers.

    Discussion

    Because this used an ordinary least squares approach, each of the four factors was valued on how they correlated to teams’ actual Net Ratings. Using the correlation coefficients of each factor, we could estimate each statistic’s relative importance:

    to Offensive Rating

    • Efficiency (68%)
    • Turnovers (28%)
    • Rebounding (0%)
    • FT/FGA (1%)

    to Defensive Rating

    • Efficiency (74%)
    • Turnovers (19%)
    • Rebounding (0%)
    • FT/FGA (1%)

    Efficiency from the floor is the “most important” statistic, and this passes our intuitive checks, as field goals are the most frequent offensive action to end a possession in the sport. Turnovers happen about 10-15 times a game, so its presence is made clear despite a lesser occurrence. Rebounding is interesting. Because offensive rebounding is significantly less common than defensive rebounding, its importance won’t be really high. But this model doesn’t seem to think much of it at all. Because events like rebounding rates and free-throw percentages are more clustered and arguably have a lesser impact on the eventual outcome of a possession, they far less important relative to the other outcomes.

    I posted an interactive leaderboard for teams here, which covers each team and goes into more detail on error rates and which teams were over or under-represented.


  • 5 NBA Thoughts [#2]

    5 NBA Thoughts [#2]

    (? The Ringer)

    Inspired by the likes of Sports IllustratedSB Nation, and Thinking Basketball among countless others, I’m continuing here a “5 Thoughts” series. Watching and studying the NBA means there’s a constant jumble of basketball thoughts in my head. However, I rarely touch on them in detailed articles or videos, so that’s where this series comes in: a rough, sporadic overview of five thoughts on the NBA!

    1. Bradley Beal and the OPOY

    During a live chat discussion on Halftime Hoops, a surging topic was whether the league should implement an Offensive Player of the Year (OPOY) Award to counterbalance the defense-only counterpart. A contribution to the topic was that the OPOY Award would too closely resemble the scoring title leaderboard, stating players like Bradley Beal and Stephen Curry will inevitably win because of their scoring, so why bother creating a new award to recognize a pool we’re already aware of?

    The voter tendencies could support this claim; in other words, the patterns of the voters suggest scoring is the largest factor in offensive evaluation, so in a practical sense, the opinion makes sense. To me, the more pressing question here is what that says about voter criteria and how it influences the public eye in general. There’s always been a misstep in how offense is treated: teams win games by scoring more points, so the highest-volume scorers are often the best offensive players. The poor connection is here the exclusion of other skills; namely, shot creation, passing, screening, cutting, floor-spacing, and all the crucial qualities that go into building a high-level offensive team.

    Bradley Beal isn’t in the conversation as a top-5 offensive player in the league to me, and while that’s mainly in part to a lack of eliteness in multiple areas that would bolster his overall value on offense, his scoring doesn’t pop out as much to me as it does other. Efficiency is generally treated as a luxury piece, and that volume scorers with “good enough” efficiency (within a reasonable range of league-average) shouldn’t be penalized. However, an argument could be made to flip the script entirely: efficiency may be equally, if not more, important than volume.

    Beal is currently fourth in the league in scoring rate, clocking in at 30.3 points per 75 possessions. (That also brings up the issue with the scoring title being assigned to the points-per-game leader: it erroneously assigns credit to high-minute players on fast-paced teams.) Thus, there are too many hindrances in placing a heavy amount of stock in how many points a player scores each game. System design, quality of teammates, the plethora of other skills that amplify a team’s offense: all evidence points toward an OPOY Award being an entirely separate entity from the scoring title.

    2. Making teammates better is a myth

    Raise your hand If you’ve ever seen someone argue on behalf of one player over another because “he makes his teammates better.” (If you aren’t raising your hand right now, I’d be very surprised.) High-quality facilitators are often treated in this fashion, and the cause for belief isn’t way off track: When elite shot creators and passers are setting up teammates for open field goals that would’ve otherwise been contested, the teammate’s shooting percentages will increase. Thus, the illusion that a player can make his teammates better was inspired.

    The problem with this idea is that it treats a player’s “goodness” as dependent on who’s on the floor with him. If an elite spot-up shooter is paired with an all-time black hole like Stephen Curry, the openness frequency of his shots will likely increase. Conversely, if he were playing alongside someone antithetical to this style like Elfrid Payton, his shooting efficiency would probably go down. The big question here is whether the increase in scoring efficacy should be attributed to the shooter or the playmaker. To me, the answer is fairly simple given how complex most of these problems can be.

    The teammate’s shooting in a vacuum is unaffected by his teammates. Namely, regardless of lineup combinations and synergies, the true skill of his shooting remains constant. A three-point specialist doesn’t go from a 40% to a 45% shooter overnight because he’s going to start playing with better teammates. Therefore, a “better” interpretation of these shooting improvements is that the playmaker is bettering the team’s shot selection (per-possession efficiency), but he’s not bettering the teammate’s shooting skill. This case applies to every other skill that creates a similar illusion.

    3. Rim protection vs. perimeter defense

    After spending a good amount of time in the aforementioned Halftime live chat rooms, it’s become increasingly clear that a large chunk of NBA fanatics may solely focus on man defense when evaluating this side of the ball. This directly parallels the strong focus on isolation scoring when the general public evaluates offensive skill. Just as we discussed in the first thought, these types of approaches can be extremely limiting and leave out a lot of information and skills that go toward building strong team defenses. This creates an ongoing conflict between weighing the value of rim protection and general “perimeter” defense, so which one is better?

    A recurring theme that has followed with basketball analysis’s anti-education culture is the tendency to try and solve complex problems with the simplest solutions possible. (This led to the original reliance on the box-and-record MVP approach.) So when defensive skills outside of isolation defense and shot-blocking are brought up, they might be met with some reluctance. The opposing style will be used here: the most important piece of the premise here is that rim protection isn’t just limited to shot-blocking ability, as perimeter defense would be to “locking up” an opponent’s best perimeter player. The cause-and-effect relationships these skills have on the totality of defensive possessions can be much stronger than meets the eye.

    Strong rim protectors in the paint can drastically alter game-planning against the opposition, meaning an increased focus on a perimeter attack can deter a large number of field-goal attempts at the rim, thus taking away the sport’s (generally) most efficient shots. Similarly, perimeter defense doesn’t just refer to a player’s one-on-one defense. How do they guard the pick-and-roll? How do they maneuver around screens (and perhaps more importantly, ball screens)? Do they make strong rotations and anticipate ball movement well or is there some hesitation? (These are just a few examples. As stated earlier, the sport is much too complex to be solved through an ensemble of steals and blocks.)

    The simple answer is that it depends. A lot of people will say rim protection is generally more valuable, and in a broader sense, I would agree. The (generally) most efficient shots are taken in the paint, so to either contest, deflect, or deter these shots would certainly seem to have a greater positive effect on the team’s overall defensive performance. However, as the “truer” answer suggests, most of that value is dependent on the context of the game. During the Playoffs, we have seen a league-wide trend of non-versatile centers losing a certain degree of impact against certain postseason opponents. (I’ve discussed this a lot in recent weeks.) Versus great-shooting teams with a strong midrange arsenal, perimeter defenders will likely see an increase in value: defending some of the league’s best floor-raising offense (open midrange shots from elite midrange shooters) will put a cap on the opposition’s offensive ceiling.

    A total discussion wouldn’t even come close to ending here, but with this brief overview paired with other in-depth analysis, there are pretty clear indicators that rim protection will often be more valuable than perimeter defense. This explains why players like Rudy Gobert and Myles Turner will generally be viewed as stronger positive-impact defenders than ones like Ben Simmons and Marcus Smart. However, with the increasing rise in outside shooting efficacy and offensive development, defensive versatility ranging from multiple areas on the court may become more valuable than either of them.

    4. Building an RAPM prior

    Regularized Adjusted Plus/Minus is an all-in-one metric that estimates the effect of a player’s presence on his team’s point differential. I’ve written about the calculations behind RAPM before, but a topic I’ve never covered in-depth is how “priors” are constructed. For those unfamiliar with these processes, RAPM uses a ridge regression (a form of Tikhonov regularization) that reduces the variability among possible APM values, regressing scores toward average to minimize the higher variance in “regular” Adjusted Plus/Minus. A “prior” is used in place of an average (net-zero) to pair with pure APM calculations.

    Priors are often referred to as “Bayesian priors” because they align with the philosophy in Bayesian statistics that encourage pairing the approximations of unknown parameters with outside information known to hold value in the holistic solutions of these problems. The basketball equivalent to this would usually see APM regress toward the box score and past results, among others. Current examples include Real Plus/Minus, which regresses toward the box score. BBall-Index‘s LEBRON regresses toward PIPM’s box component. Estimated Plus/Minus regresses toward an RAPM estimator that uses the box score and tracking data (which the NBA started to track league-wide in the 2013-14 season).

    The composition of these priors has stood out to me, and during my research for an upcoming video podcast on ranking impact metrics, it become apparent the use of on-off ratings was often a significant hindrance to a metric’s quality. It’s long been clear the extreme variability of on-off data could limit a metric’s descriptive power in smaller samples because of its lack of adjusting for hot streaks, lineup combinations, and minute staggerings. These confounding variables can make an MVP-level player appear to be nothing more than a rotational piece, and vice versa. Because there’s simply too much on-off data doesn’t account for, even as the skill curves begin to smooth, it’s probably on a downward trajectory in its usage among modern impact metrics.

    Conversely, the more promising form of statistics seems to be tracking data. Although most of these stats weren’t tracked prior to the 2014 season, there’s a good amount of non-box data from resources like PBPStats, which could help in creating a prior-informed RAPM model that extends back to the beginning of the twenty-first century. These types of stats are estimated to have the least dependence on team circumstances among the “big” three that includes box scores and on-off ratings, which gives tracking data an inherent advantage. During his retrodiction testings of several high-quality NBA impact metrics, Taylor Snarr (creator of EPM) found evidence to suggest this pattern, and although this phenomenon hasn’t quite yet hit the mainstream, I expect the next generation of advanced stats is powered by tracking data.

    5. The statistical evaluation of defense

    Compared to the offensive side of the game, defense has always been a bit of a black box. It’s much harder to evaluate and even harder to quantify. (When the DPOY Award was instituted in the early 1980s, Magic Johnson would receive a good share of votes.) This is an even larger problem given the limitations of the current box score, which only extends to defensive rebounds, steals, blocks, and personal fouls. The majority of attentive NBA watchers knows the box score doesn’t track a lot of defensive information, so what statistical methods can be used to evaluate defense?

    One of the up-and-coming statistics in recent years has been opponent field-goal percentage; the proportion of defended field-goal attempts that were made. These marks are seen as especially useful by a community that mostly considers isolation defense; but as a measure of the quality of contests alone, these stats are often misused. Some will say a defender played exceptionally well against LeBron James because, in the two’s interactions, James made two of seven shots. (“Therefore, Andre Iguodala is the Finals MVP,” if you catch my drift.) Not only are these measures not a definitive representation of how well a defender contested a player’s shots (location and team positioning are big factors that affect these stats), but small samples are perennially mistreated.

    Anyone with some familiarity with statistical analysis knows the variability of a sampling distribution very often goes down as the sample size increases. (The common benchmark is n = 30 to satisfy a “Normal” condition for some types of stat tests.) Thus, when Kevin Durant shoots 21% on 12 attempts in a game versus Alex Caruso, it doesn’t mean Alex Caruso is therefore Kevin Durant’s kryptonite. The variability of opponent field-goal percentage, the lack of context surrounding the conditions of a field-goal attempt, and the infrequent instances in which they occur create a statistic that’s easy to abuse. If there should be one major takeaway from this post, it would be to KNOW AND UNDERSTAND YOUR DATA.

    Unfortunately, a lot of lower-variation and more descriptive defensive stats (defended FG% doesn’t confirm a cause-and-effect relationship) aren’t readily available to the public, but it’s worth noting some of the advancements in the subject. A more common statistic might be deflections, charges drawn, or total offensive fouls drawn (the latter two paired with steals created total forced turnovers). However, the most exemplary case of defensive measuring I’ve seen is through BBall-Index, whose data package comes with a multitude of high-quality defensive statistics, including box-out rates. rebounding “success rate,” loose ball recovery rate, rim deterrence, matchup data, among countless others. Seriously, if you’re looking to take your defensive statistical analysis up a notch, it’s the real deal.

    Impact metrics will always provide their two cents with their one-number estimates, and for the most part, they’ll be pretty good ballparking values. But intensive film studies (the building blocks for evaluating defense) may need to be paired with descriptive, intelligible defensive statistics to enhance the quality of analysis. This doesn’t make defense any easier to evaluate than offense, but relative to the alternative (steals + blocks), it’s a whole lot better.

    How important are skills like volume scoring and isolation defense? Are the right players credited with individual statistical increases? And how are the futures of RAPM and defensive measurements coming into play right? Leave your thoughts in the comments.


  • 5 NBA Thoughts [#1]

    5 NBA Thoughts [#1]

    (? The Ringer)

    Inspired by the likes of Sports IllustratedSB Nation, and Thinking Basketball among countless others, I’m introducing here a “5 Thoughts” series. Watching and studying the NBA means there’s a constant jumble of basketball thoughts in my head. However, I rarely touch on them in detailed articles or videos, so that’s where this series comes in: a rough, sporadic overview of five thoughts on the NBA!

    1. Curry’s offense vs. Jokic’s offense

    At this point, it’s safe to say the league’s two-best offensive players are Steph Curry and Nikola Jokic. But between the two, who’s better? The Playoffs will inevitably shed more light on this (while our knowledge of their postseason performances in the past is a positive indicator for Jokic), but with Jokic’s ascension, we may have enough to decipher a good amount of their skillsets.

    Curry functions in a high-movement offense that largely relies on finding him and open shot. That’s done primarily through Curry’s dynamic movement off the ball, darting through screens and coming up on pin downs, eventually culminating in an open jumper behind a ball screen. This style of offensive play may be considered a bit high-maintenance, but the potential it has to unclog other areas on the court (namely, the paint) is astronomical. Curry’s gravity was the main catalyst for a lot of Kevin Durant’s scoring bursts during the Playoffs, and that makes Curry one of the most adaptable offensive stars in the history of the game.

    Jokic is a passing savant with an off-ball repertoire of his own. His elbow game is perfectly designed for another style of high-activity offense. Rather than a long series of screening action, Denver uses an elaborate design of cutting to either unclog the paint or the corners, AKA prime real estate for Jokic’s passes and assists. The Nuggets’ offensive attack is so deadly in the half-court because of how distracting their cutters are. Defenses either collapse into the paint (leaving the main in the corner open) or maintain space (spacing out the paint). Pair that with Jokic’s midrange shooting, and you have the most electrifying team offenses in the game today.

    So which offensive game is better? Mentally projecting the Playoffs, I expect Jokic will be harder to gameplan against with his ability to pass out of traps inside the three-point line. (And those outlet passes are so phenomenal, half-court offense could be a contingency at times.) I don’t see Curry as poor in this regard, but his height definitely caps the ceiling in these types of possessions. I think they both fit alongside other star talents really, really well. I see Curry as the more “scalable” player considering Jokic’s defensive shortcomings and the increased difficulty in building a championship-level defense around a neutral-impact center.

    It’ll be extremely difficult to choose between Curry’s historical scoring and gravity and Jokic’s enigmatic passing and well-roundedness. I couldn’t sell myself on either one of them at this stage, but given out previous knowledge relating to their Playoffs adaptations, my have-to-make-a pick is Jokic.

    2. Is Harden being exposed in Brooklyn?

    I recently got around to eyeing Harden’s stint with the Brooklyn Nets, and one overarching thought put some concerns in my mind. During his day with the Rockets, he mastered spread pick-and-roll. He could create a ton of offense out of these spots with his stepback jumper, strength and finishing, and high-level passing and kick-outs, and this served as the primary source for his impact on offense. Now, if the last few seasons have taught us anything about Harden, a major sticking point has been that he provides nothing without the ball in his hands, and that may be a problem in Brooklyn.

    Because Harden absolved a decent amount of Durant’s and Kyrie Irving’s touches upon his arrival (refer to this graphic I made for a previous article on the Nets), his value in composite metrics and the box score seems fairly comparable to his level of play in Houston.

    A downward rate of change in their situational value for all three offensive stars is expected, but I wonder if there’s more cause for worry in the Playoffs. This is clearly a Harden-led offense, but his main offensive sets seem to have lost their big bite. Relative to his spread pick-and-roll possessions in Houston that I’ve watched this season, the Brooklyn counterparts seem like a notable downgrade. His handle has loosened and more point-of-attack defenders are poking the ball loose, fizzling out a lot of the action, and the aim and speed on his passes seem to have lost their touch. This may be a sampling issue, but the degree to which these plays declined seemed significant.

    As I let this thought simmer in my head for a while, I began to wonder if this was a game flow problem. With the Rockets, Harden could essentially call for these isolation possessions at will. But with the Nets, he has to concede a lot of those opportunities to Durant and Irving, which led me to believe the halt in constant ball-pounding left Harden in a bit of a funk. Does Harden absolutely need the ball in his hands throughout the game to exhibit his world-class value? Perhaps so, and perhaps not. Either way, this would indicate a really low ceiling on Harden’s fit alongside other perimeter isolationists.

    3. Are the Knicks legit?

    The New York Knicks have made one of the most sudden and unexpected team improvements in recent history. A season removed from a woeful -6.72 SRS and a seventh consecutive Playoff miss, they’ve made the astronomical leap to +1.90 through their 67th game of the season, very likely snapping their Playoff drought. (They did it for you, little Cody.)

    The 2021 season has been filled with unprecedented levels of confoundment (a trend that’s carried over from the 2020 bubble), and it’s made it a lot harder to separate lucky stints from tangible improvement. To me, that’s the biggest question with the Knicks. Are they a mere product of luck combined with a mild talent boost or one of the most improved teams in recent history?

    At this stage, it’s clear New York is thriving because of its team defense. At the time of this writing, their -1.5 relative Offensive Rating sits twentieth while their -3.7 relative Defensive Rating currently ranks as the fourth-best mark in the NBA. Because scoring and allowing points determine the winners of the games, the causal variables behind these ratings are crucial to simplifying the aforementioned “luck versus improvement” problem. The largest components of this will involve opponent free-throw shooting (as that’s something a team defense has no control over) and three-point shooting: the golden standard for noisy stats. To put this into perspective, take this quote from a Nylon Calculus study four years ago:

    “Let me begin this with some research background: open 3-point percentage is virtually all noise, and open three’s consist of a huge portion of the 3-pointers teams allow. There’s no pattern to individual 3-point percentage defense either — it’s noise too . Ken Pomeroy found that defenses had little control over opponent 3-pointer percentage in the NCAA as well.”

    From an intuitive perspective, this makes sense, especially on open three-point attempts. At that point, with no defender to dilute the quality of the shot, whether or not the shot is made is determined by the staggering variance in the bodily mechanics of the shot, which is influenced by factors almost entirely outside of the defense’s scope. Thus, these types of statistics will be used to ballpark the difference between shooting luck and systematic upgrades in the Knicks’ hardware.

    After looking at New York’s current statistical profile, I figured a lot of the data could be reasonably interpreted as noise. Last season, the Knicks shot 33.7% from three, ranking 27th in the whole league. That number has made the seemingly unparalleled jump to 39%, the fifth-best three-point percentage in the league. At the player level, there are a few data points that stood out in how drastic three-point percentage improvements were:

    • Julius Randle: 27.7% in 2020 | 41.7% in 2021
    • RJ Barrett: 32.0% in 2020 | 39.9% in 2021
    • Derrick Rose: 30.6% in 2020 | 38.3% in 2021
    • Kevin Knox: 32.7% in 2020 | 39.3% in 2021

    The Julius Randle uptick is the most prominent, not only because it was the largest change among moderate-volume shooters, but because he’s taken an extra 1.4 three-point attempts every 75 possessions compared to last season. Not only that, but BBall-Index‘s perimeter shooting data suggests Randle is taking pretty difficult shots (the 7th percentile in openness rating and the 2nd percentile in three-point shot quality). This is certainly good news for Randle and his MIP case, but it makes our questions about the Knicks even more difficult to answer.

    An even larger sticking point for me was how efficient their opponents were from three-point range. At the moment, this stands at 33.7%, good for the best opponent three-point percentage in the league. I like to use a ballparking tool in these types of scenarios by replacing the number of points the team limited in reality with a league-average result. So, for example, the Knicks’ opponents have shot 36.9 three-point attempts per 100 possessions at 33.9%, which means opponents generated 37.5 points every 100 possession from their perimeter shooting. A league-average percentage would have generated 40.6 points from these attempts in the allotted period.

    Namely, if the Knicks’ opponents were average three-point shooters, their new defensive rating would suggest they allow 111.7 points per 100, which would not be too far off from last season and would make the Knicks a net-negative team. If we use the same technique for their offense to estimate the effects of shooting luck (especially in this fan-free, short-schedule environment that’s seen a major spike in offensive efficacy), their new offensive rating would fall to 108.7. Using this stricter method, the Knicks would be a -3 team. I don’t think they’re quite that poor. There are positive signals for offensive shooting improvement and the defensive mind of Tom Thibodeau has really helped this team. But if I had to give my best guess, the Knicks are roughly an average team in 2021.

    4. The potential of Box Plus/Minus

    Box Plus/Minus (BPM) is perhaps the most popular plus-minus metric on the market aside from Plus/Minus itself. You don’t need to know anything about the metric other than its name to infer it’s calculated with the box score alone. Interestingly enough, a public that often doesn’t extend its thinking outside the box score will often criticize BPM for only using the box score. And that’s what I’ll discuss today, the “potential” of Box Plus/Minus.

    I’ve recently propagated the truest value from impact metrics comes from their ability to help us understand not necessarily who impacts the game most, but how much varying qualities of players can impact the game. BPM serves that purpose as well as any other one-number metric, but with an extra kick. Since BPM relies on counting stats alone, some of which quantify levels of skill, we can break down a player’s statistical profile to see which tendencies make up the bulk of his impact. More recent studies include Ben Taylor’s creations of ScoreVal and PlayVal, which estimate the plus-minus impact of scoring and playmaking as components of the Backpicks Box Plus/Minus model. Hybrid and APM metrics, which blend plus-minus data directly into the equation, can’t provide these types of analysis, giving BPM a grounded place among the major impact metrics.

    This is even more useful in career forecastings. While other impact metrics could use some type of role archetype extrapolated from its datasets or age to project careers, BPM allows for a much more granulated skills approach. For example, the Backpicks model uses shot creation and passing ability and teammate spacing and all other kinds of estimates based on the box score to help better the descriptive and predictive powers of the box-only metrics. So while some may see Box Plus/Minus as a dying art form, you could argue it’s actually ascending to its peak right now.

    5. A major analytics misconception

    For every analytical mind, there are five contrarians to oppose them. For the most part (emphasis on “the most part”), these oppositions don’t really know what they’re fighting against other than some vague advancement of NBA statistics. Due to this lopsided ignorance, there stem a lot of misconceptions as to how an analytical supporter interprets and use advanced stats.

    My experience conversing with the anti-analytics crowd has led me to believe they believe an analytical person will treat advanced stats as gospel, and that film study is some type of basketball sin that defies the analytics movement. The truth is really the exact opposite. Anyone who claims they can passively watch a game and then understand everything about it is overestimating themselves. It’s a simple function of cognition; we can neither track nor understand every single thing that’s happening on the court with our eyes alone. That’s where analytics comes in. (Because, honestly, if there weren’t a purpose for them, no one would have bothered to create advanced stats!)

    And that’s what I would like the major takeaway here to be. As someone who identifies as an analytics supporter, I can advocate on behalf of “us” and say the public eye is mostly wrong. Advanced stats aren’t treated as gospel. They’re estimates with varying error rates. Sometimes, advanced stats don’t capture players well at all! That’s why the fluidity of analytics and film is the actual driving force behind the analytics movement. As is with any other field, change causes pushback, and not everyone will be willing to evolve. But that’s the whole point of analytics, of advanced stats. They’re advancements.

    Who’s the better offensive player between Steph Curry and Nikola Jokic? Is Harden not serving the purpose in Brooklyn we’d all thought he would? Are the Knicks what their record says they are? Is Box Plus/Minus a dying breed? And how do analytical minds interpret advanced stats? Leave your thoughts in the comments.