I talked extensively with Mike and my dad about The Great Quinn’s performance in the Browns’ opener earlier today. Suffice it to say that I billed it as “even worse than I anticipated.” But knowing the fan base as we do, Mike and I also knew that the “Pro-Quinn” faction (an ironic name, since today Quinn looked decidedly amateur) would find multiple arguments for why his performance was not only OK but even encouraging: the Vikings are an elite defense, Quinn brought an end to the TD drought, it’s only his 4th pro start so you’ve gotta give him a few games, it’s Mangini’s fault for not giving him more reps with the first team throughout training camp…the list goes on.
Hilariously, we were both shocked to read an email from our dad that concluded Quinn deserved a ‘B’ - for some of the very reasons that we projected the Quinnbots would use.
The one that I want to focus on tonight, though, is Quinn’s passer rating.
This particular stat came up in the conversation because our dad insisted that he might change his grading of Quinn depending on his passer rating, which was “one of the few stats he understands.” We then quickly discovered that he had no real idea how passer rating was calculated or what about performance it signified, other than that the higher it was the better the QB theoretically played.
I don’t bring this up to rip on my pops (well, maybe a little). I bring it up because I think the vast majority of NFL fans has the same willingness to pretend the stat is useful, even though they have no real understanding of what it means.
To dispel the mystery, I dove into the research. (Thanks to The Quick And The Dead for a lot of the background info and formulas below.)
The passer rating system was devised by a guy named Don Smith, who was an exec at the Pro Football Hall of Fame. In 1971, then-Commissioner Pete Rozelle convinced Smith - a renowned statistician, at least in football circles - to come up with a new way to rate quarterback play. The NFL had previously tried out a variety of different methods, each with its own fundamental flaws. Rozelle believed Smith could do better.
In 1971, the NFL was only tracking limited stats for QBs: touchdown passes, interceptions, pass attempts, completions, and passing yardage. From these basics, they had also determined completion percentage and average yards per attempt.
Armed with this data, Smith devised a 4-step system based on what he believed to be the most important of those available stats: completion percentage, yards per attempt, TD percentage, and INT percentage.
The four steps to calculate passer rating are as follows:
Step 1: Start with completion percentage. Subtract 30 and divide the difference by 20.
Step 2: Yards per attempt. Subtract 3 and divide the difference by 4.
Step 3: Divide touchdown passes by pass attempts, then multiply by 20.
Step 4: Start with 2.375. Subtract from that the interception percentage (interceptions divided by pass attempts), then divide the difference by 4.
(**Note: Sum of each step cannot be greater than 2.375 or less than 0.**)
Add the sum of 1-4, multiply by 100 and divide by 6.
At first glance, the system may seem like it has a little merit. However, if you give it more than a cursory glance, the cracks in the foundation begin to show.
Perhaps the most egregious problem is that there is no statistical evidence that these 4 categories should have equal value in determining how well a QB has played. But if that’s the case, how did Smith come up with the math?
Like most statistical models, passer rating needed a baseline to measure performance against. It also needed an upper and lower limit. To determine these parameters, Smith looked at the numbers from the 1970 season and determined the average values in each of his four categories. He then went back into the record books to find the best performances in the history of the league for each of those individual stats.
Once he had that data he jerry-rigged the numerical values in the formula so that an “average performance” in any of the four categories would score the QB 1 point, and a “below average performance” would score 0 points. (Between each of these values, there’s a sliding scale, so decimals are in play.)
Smith then further manipulated the formula so that the cumulative ratings would reach pre-determined values. Specifically, he decided that a very good passer performance should hover around 100, whereas an average performance should be around 67.
However, Smith’s insistence on these values also meant the following: the maximum score for any one category was a bizarre 2.375; even though it was entirely plausible for a QB to perform well enough or poorly enough that an individual category could exceed 2.375 or 0, respectively, the numbers were to be rounded up or down to those pre-determined boundaries in either case; and the highest possible passer rating became an absurd 158.3.
Cosmetics aside, the simple fact of the system is this: today’s quarterbacks are consistently being rated based on the performance of a bunch of quarterbacks in 1970.
The problem here isn’t an age-ist bias. It’s that the conditions of the NFL have changed dramatically since 1970. Domed stadiums and turf have eliminated weather conditions and sped up the game, to the benefit of quarterbacks. Various rule changes - the “illegal chuck” rule, the “clearly in the grasp” rule - have made the NFL (in Mesa-favorite Michael Lombardi’s parlance) a “pass-first” league.
In 1970, the averages in Smith’s categories were: 50% completions, 7 yards per attempt, 5% TD, 5.5% INT.
The Quick & the Dead note that by 1979, the average completion percentage had risen to 54.1%. It has never dipped back below that figure since.
In fact, in 2008, the average completion rate was 61.2% - 11.2 percentage points higher than when the passer rating system was devised, or more than 1/5 better.
What does this mean? That quarterbacks throwing completions at a rate that is only average by current NFL standards will receive a significantly above-average mark in one category of the passer rating equation.
But arguably the biggest numerical distortion came from Bill Walsh: the invention of the West Coast offense, which of course focuses on quick passes for short yardage. If you run the numbers, you find that the passer rating system inherently favors this style of play. A series of 3 passes for 8 yards each is more valuable than 2 incompletions followed by 1 pass for 24 yards - despite that you’re looking at the same net gain in yardage.
The next issue is the entire concept of TD%. West Coast offenses should naturally rate lower in this category because they’re using a higher quantity of passes to methodically march down the field and into the endzone. But there’s not really a good argument to be made that a lower TD% is necessarily bad for a quarterback. Say, for example, that on every drive in a single game, your QB used short passes to drive the team from their own 5 to the opponent’s goal line. Then, on the final play, he used a quarterback sneak to get into the endzone. If he did this six times in a single game, he’d rack up 570 yards passing and score 36 points (42 if you assume the PAT is a given) but would theoretically be penalizd because he never threw for a TD. As a result, his TD% would be 0.
This is one of the other flaws of the passer rating system: it’s too true to its name. Though the term “quarterback rating” is often used to mean the same thing, that secondary term is deceiving. Passer rating is really only a measure of what the quarterback does through the air. Anything else that happens is irrelevant. A QB could take 10 sacks in a game for a total of 50 yards lost, or lose 3 fumbles - and neither would count against his performance.
In fact, if a QB feels the pass-rush closing in, he has a statistical incentive to take the sack rather than whip the ball into the sidelines for an incompletion. To draw a cross-sports analogy, this would be equivalent to a basketball player choosing to hold onto the ball for the closing seconds of a half rather than take a half-court Hail Mary, all for the sake of keeping his shooting percentage a little higher.
On the other hand, our hypothetical quarterback from the last paragraph could run for 6 TDs in a game, and it wouldn’t factor positively into his passer rating either. Considering the increased emphasis on running QBs over the course of the past five years or so, this seems like a glaring omission. Mike Vick rushed for over 1,000 yards in 2006 and created absolute havoc on the field in the process - but passer rating completely ignored that ingredient of his offense.
So when you put it all together, the underlying problem with passer rating is that it’s a relic stat - a model that may have fit the game in 1970 but which lost more and more of its value as the sport evolved. Nevertheless, modern sports pundits are still touting it as a relevant measure of performance.
To close the circle here, Quinn’s passer rating today was 74.1, or just slightly above average for a quarterback in 1970. However, the average passer rating for a starter in 2008 was 84.8. Quinn’s completion percentage was 60%, which is way above the 1970 average but slightly below the 2008 average. His yards / attempt were 5.9, which is significantly below average for 1970 AND in comparison to 2008 (6.8 yd / att average).
The rating number also doesn’t take into account that Quinn was sacked 5 times, coughed up a “fumble” while trying to throw on the run outside the pocket (in what was easily the most hilarious play by an NFL QB since Dan Orlovsky ran about 3 yards out of the back of his own end zone for a safety last season), or maxed out with a titanic 26-yard completion.
The point is: today Quinn played about as well as a 60 year old man.
I don’t want to nail the guy to the cross. My well-documented preference for the starting job is Derek Anderson, but most of all, I want to see the Browns win. If Quinn can get his act together and improve, then I’m all for it. But more of what we saw today is not the answer.
A final note: there’s a much larger discussion to be had at some point in the future about how QBs should really be rated. We’ll cover that in some form or another in the next week. Til then.
-T
(view comments)
