Deflategate and How Numbers Can Deceive

John David Mercer-USA TODAY Sports
John David Mercer-USA TODAY Sports

Super Bowl  XLIX is coming up -- and many people want to move beyond the New England Patriot Deflategate controversy.  However, before we get on with the game, I thought the following articles were particularly interesting.  

First, football analyst Warren Sharp put together a convincing article about the Patriots:  Sharp Football analysis and how the Patriots' prevention of fumbles is nearly impossible.   In particular, he writes:

One can CLEARLY SEE the Patriots, visually, are off the chart. There is no other team even close to being near to their rate of 187 offensive plays (passes+rushes+sacks) per fumble. The league average is 105 plays/fumble. Most teams are within 21 plays of that number.

I actually went back and researched 5 year periods for the entire NFL over the last 25 years. The Patriots ratio of 187 plays to 1 fumble is the BEST of ANY team in the NFL for ANY 5 year span of time over the last 25 years. Not was it just the best, it wasn’t close:

2010-2014 Patriots: 187 plays/fumble

2009-2013 Patriots: 156 plays/fumble
2006-2010 Colts: 156 plays/fumble
2005-2009 Colts: 153 plays/fumble
2007-2011 Patriots: 149 plays/fumble
2008-2012 Patriots: 148 plays/fumble
2010-2014 Texans: 140 plays/fumble
2004-2008 Colts: 139 plays/fumble
2006-2010 Jets: 135 plays/fumble
1999-2003 Chiefs: 134 plays/fumbl

This interesting analysis was followed by an article that discussed the iffy usage of some numbers and graphics to lean the story far to one side. 

Professor Matthew's rebuttal on how the Sharp article was misleading.  Matthews is a statistics professor at Loyola University Chicago.  Some good points include: 

... but statistics is hard, and this can lead to deception, either willful or otherwise. So let's walk through the curious inputs, outputs, interpretations, and statistical decisions that were made in the broadest of these posts...

The Patriots are indeed nearly off the chart, but that is partially because the author uses the smallest y-axis possible to demonstrate the largest effect that he could. It's generally preferred to use a y-axis that begins at 0, as any other scale is misleading and, in all likelihood, sensationalistic.

Given that the normality assumption is valid, we can calculate the Z-score for fumbles per play for the Patriots. To do this, we take the Patriots fumbles per play (0.00535) and subtract the mean of fumbles per play for each team (0.00983) and divide by the standard deviation of fumbles per play for each team (0.00165). This yields a Z-score of -2.71. Still assuming a normal distribution, a team would only be better than this (i.e. have a lower Z-score) about 0.336% of the time, or 1 in 297. That's pretty rare, but nowhere near 1 in 16233.77. So where did that 16000-ish number come from?
It's arrived at by using plays per fumble, rather than fumbles per play. If you use plays per fumble and calculate the same Z-score for the Patriots you get 3.84. We can then calculate that a team will do better than this (i.e. have a higher Z-score; higher is better for plays per fumble) about 1 in 16256. So we're guessing this is where that number comes from. 

In any case, I wanted to "bookmark" this set of arguments as a cautionary tale on numbers, statistics, and analytics.  Now, let's move on to the Super Bowl!

Popular articles

Super Bowl Square Pool Odds

Carlton Chin, CFA, is a fund manager and portfolio strategist who applies quant analytics to the financial markets and sports.  He has been quoted by the New York Times, Wall St. Journal and ESPN -- and has worked with sports organizations, including the Sacramento Kings for Draft 3.0.