Relationships between player actions and game outcomes in American football.
Abstract: Analysis of player actions in team sports can provide useful predictors of game outcome, but there have been few published analyses of games of American football. In this study we have analyzed publically available data from the American National Football League (NFL). Methods: The data consisted of over 100 variables from 1335 games in 2004-2008, Independent factors representing a specific aspect of football were derived by factor analysis and used in logistic regression models to determine their relationship to game outcome. Logistic models were also developed with the original variables. In a validation study, data from the 2008 NFL season were used to assess models created with the 2004--2007 data. Results: There were 14 factors, each with clear interpretations. Turnovers (large effect), rushing performance, passing and total offense/defense (moderate effects) were the best predictors of game outcome, while three factors related to ball control and possession change were trivial predictors. Logistic multiple linear regression models built from data of the 2004 through 2007 seasons predicted 92% of 2008 game outcomes correctly. Discussion: The identified factors have clear practical interpretations in football, and their relationships with game outcomes are not unexpected. In combination, the factors or original variables are very successful in predicting outcomes and could contribute to player personnel decisions, strategies for practice, and creating game plans. KEYWORDS: NFL, factor analysis, key performance indicators, logistic regression.
Article Type: Report
Subject: Football (Professional) (Analysis)
Football players (Behavior)
Group games (Analysis)
Authors: Cohea, Christopher
Payton, Mark E.
Pub Date: 01/01/2011
Publication: Name: Sportscience Publisher: Internet Society for Sport Science Audience: Academic Format: Magazine/Journal Subject: Health Copyright: COPYRIGHT 2011 Internet Society for Sport Science ISSN: 1174-9210
Issue: Date: Annual, 2011 Source Volume: 15
Product: Product Code: 7941020 Professional Football NAICS Code: 711211 Sports Teams and Clubs SIC Code: 7941 Sports clubs, managers, & promoters
Geographic: Geographic Scope: United States Geographic Code: 1USA United States
Accession Number: 297427060
Full Text: American football is a popular sport in the US and around the world. Rabid fans and professional pundits often debate what contributes most to the outcome of games in the National Football League (NFL), and opinions vary widely. In pregame shows it is not uncommon to have one expert speak of the virtues of a strong run defense, while another will marvel why a team with strong passing offense can't win a game. Evidence of the effect of player actions on game outcomes would obviously be of interest to all concerned with NFL: players, coaches, owners, and spectators.

There is limited published research on this topic in any of the football codes. James et al. (2001) developed performance profiles specific to positions in21 professional rugby union matches. In regards to American football, Stair et al. (2008) analyzed the effects of various factors pertaining to off-field conduct on the performance of NFL teams, with special attention to the number of arrests of team members (which was not statistically significant). Alamar and Weinstein-Gould (2008) investigated the contribution of individual NFL linemen to their teams' passing performance. White and Berry (2002) used logistic regression to rank NFL quarterbacks by finding a quantitative value for various plays, but not game outcomes, that occur in NFL games. However, a comprehensive analysis of the effects of common statistical measures in American football on the likelihood of winning has not been previously published. The aim of this study was therefore to perform such an analysis and to determine how the large number of inter--related statistical measures can be grouped into a smaller set of independent key performance indicators.

Methods

Over 100 variables were included from a sample of 1,335 NFL games spanning the 2004 through 2008 seasons. The data were collected from the website http://nfldata.com. Variables given in the original data set that do not represent in--game events (playing surface, temperature, weather conditions, predictive point spread, etc.) were removed, leaving 67 team--performance variables (Tables 1 and 2). Each game has data from two teams that mirror each other; therefore, data from only one team (selected at random) were utilized for each game to avoid duplication of data and problems with repeated measurement. For example, an occurrence of offensive rushing for one team is an occurrence of defensive rushing for the other. Terms such as defensive passing yards or defensive first downs refer to yards or first downs attained by the non-selected team when in possession of the ball. See the commentary by George Osorio for clarification of the rules of American football and an explanation of the terminology.

The principal-components version of factor analysis was employed to group the standardized game statistics into independent sets. The analysis was realized with Proc Factor in SAS using the defaults for identifying an appropriate number of factors and varimax rotation to combine the statistics into independent factors. The factors were named according to their perceived football-specific characteristics.

Univariate logistic regression using Proc Logistic in SAS was used to estimate the individual effect of each factor on the chances of winning. In these analyses, the values of the factor were scores from every game computed from the loadings and the values of the variables in the factor. As described elsewhere (Hopkins, 2010), the magnitude of the effect of the factor in question was estimated first as the ratio of odds of winning for games that differed by two standard deviations; that is, the difference between games with a typically low and high value of the factor. The odds ratio was then converted to a difference in the chances of winning centered on a 50% chance; for example, an odds ratio of 2.0 is equivalent to a difference of 18% in chances of winning (a 59% chance for a game on the high value of the factor vs 41% for a game on the low value). The difference in chances was then interpreted using the following scale: <10%, trivial; 10-30%, small; 3050%, moderate; 50-70%, large; 70-90%, very large; and >90%, extremely large (Hopkins et al., 2009).

The ability of all 14 factors to account for game outcomes was quantified by developing a logistic model with the 14 factors as main-effects predictors. The percent of game outcomes predicted correctly by this model was then calculated.

Univariate logistic regression using Proc Logistic in SAS was also used to estimate the individual effect of each game statistic on the chances of winning. For these analyses the one game resulting in a tie was removed from the dataset. A multiple logistic regression was also performed using all variables to ascertain the predictive power of all the original variables. Backward stepwise logistic regression was used to reduce the set of game variables to a more succinct set.

In a validation study, the multiple logistic regression analysis was repeated with data from the 2004--2007 seasons, and the resulting logistic model with all 14 factors was used to predict the outcomes of the 2008 games for comparison with the actual outcomes.

Because of the large number of effects estimated in this study, uncertainty in all estimates was calculated conservatively as 99% confidence limits. Outcomes were assessed using the paradigm of mechanistic magnitude-based inference. All outcomes were clear, owing to the large sample size, so the magnitudes of the observed effects were interpreted directly as population magnitudes without probabilistic qualifiers.

Results

The factor analysis yielded 14 factors; the loadings of the variables in the factors are given in Table 1. Fifty-nine of the 67 variables contributed to the factors, and none of the variables contributed to more than one factor.

The factors are numbered in descending order based on the amount of variation explained by the given factor. Offensive and defensive factors are those attained while the ball is in possession of the team or the opponent respectively.

The choice of 14 factors was based on the total number of eigenvalues >1. Other models with different numbers of factors were analyzed, but the factor loadings made factor interpretation difficult and the analyses are not shown here.

Table 2 shows the outcome of the logistic regression with the factors as predictors. Of the 14 factors, 11 have non-trivial effects. Turnovers is the most powerful predictor out of the 14 factors, and the rushing-related factors have slightly larger effects than the passing-related factors. When all 14 factors were included in a multiple logistic regression model, 91% of the 635 won games and 92% of the 699 lost games were predicted correctly.

Results of the logistic regressions using each of the original game statistics as a predictor of game outcome are shown in Table 3, with the predictors grouped according to the factors they contributed to. When all the game statistics were included as predictors in a logistic model derived for the 2005-2007 games, the resulting model applied to the 2008 games predicted 93% of the 122 won games and 92% of the 144 lost games correctly. A smaller set of 24 game statistics identified by backwards stepwise selection is shown in Table 4. This set predicted the same proportions of won and lost games as the full set.

Discussion

The 14 factors have relatively clear and concise interpretations, and they represent characteristics that in our experience are commonly used to describe a team's game performance. Logistic regression with the 14 factors showed that the factors related to ball control (offensive ball control, possession change, defensive ball control) had little relationship with game outcome (Table 2), but the other 11 factors all made substantial contributions. In particular, turnovers and rushing (offensive and defensive) were especially predictive of game outcomes, in addition to passing (offensive and defensive) and total offense/defense. The casual fan would probably agree that these five factors constitute the pillars of football success for a team. The remaining factors reflect the relatively lesser importance of sacks (defensive and offensive), penalties (good and bad), and 4th down performance (offensive and defensive).

When the results of the factor analysis are compared to those of the univariate logistic regressions performed on the original variables, it becomes evident that factor analysis can exclude variables that can be very predictive of the game result. For example, both offensive and defensive time of possession have large differences in chances of winning of approximately 75%. However, neither of these variables loaded significantly on a factor as a result of the factor analysis. This is most likely due to the level of correlation these variables have with other variables that did load on a factor. The predictive power of the factor analysis does exhibit its effectiveness of data reduction. The logistic regression using the factors was successful in predicting the results 91-92% of the time (only slightly less than what was exhibited using the original variables, 92-93%).

Owing to substantial correlations among the original variables, interpretation of individual coefficients from any of the original-variable logistic models was not attempted. However, the predictive power of these models is remarkable and could be useful, in the aggregate, in making predictions. Given that prediction was applied to game data mutually exclusive of the data the model was built from, more confidence is given to ability of this group of 60 variables to predict game outcomes. The results presented here suggest the in--game variables from an NFL game are powerful predictors of a game's outcome. Clearly, the models are not perfect representations of game outcomes due to unmeasurable variables and randomness. Future work will focus on building off these results to predict game outcomes before the game.

The stepwise logistic regression also yielded interesting results. Note that there were two variables that survived this method that did not load significantly on a factor in the factor analysis (Defensive Passing Attempts and Defensive Time of Possession). Not surprisingly, the predictive power of the stepwise method yielded identical results from the model utilizing all of the original variables. The results of the stepwise method also provided results that were somewhat similar to the factor analysis. All but 3 of the 14 factors were represented with one or more of its represented variables. The factors not represented in the stepwise method were the sack performance factors (both offensive and defensive) and offensive ball control.

In conclusion, we acknowledge that many important predictor variables are nearly impossible to collect and analyze. As examples, location (home or away), weather conditions, injuries, and time of year or playoff implications all impact outcomes of NFL games. In the analyses for this article we simply attempted to gain a better understanding of the subset of variables that were collected. It should also be noted that and impending outcome in a game will change players' behavior. For example, it is common for a football team, when leading late in the game, to run more rushing plays, which tend to expend more time and shorten the game. The effects of such changes in behavior need investigation.

References

Alamar BC, Weinstein-Gould J (2008). Isolating the effect of individual linemen on the passing game in the National Football League. Journal of Quantitative Analysis in Sports 4, bepress.com/jqas/vol4/iss2/10

Hopkins WG, Marshall SW, Batterham AM, Hanin J (2009). Progressive statistics for studies in sports medicine and exercise science. Medicine and Science in Sports and Exercise 41, 3-12

Hopkins WG (2010). Linear models and effect magnitudes for research, clinical and practical applications. Sportscience 14, 49-57

James N, Mellalieu S, Jones N (2001). The development of position--specific performance indicators in professional rugby union. Journal of Sports Sciences 23, 63-72

Stair A, Neral J, Mizak D, Day A (2008). The factors affecting team performance in the NFL: does off-field conduct matter? Economics Bulletin 26, 1-9

White C, Berry S (2002). Tiered polychotomous regression: ranking NFL quarterbacks. The American Statistician 56, 10-21

Published August 2011. [C] 2011

Christopher Cohea, Mark E. Payton

Department of Statistics, Oklahoma State University, Stillwater, OK 74078, USA. Email.

Reviewers: George Osorio, Pix & Flips Inc., Costa Mesa, CA 92627; Ken Quarrie, NZ Rugby Union, Wellington, NZ.
Table 1. The game statistics making up the 14 factors and the game
statistics that did not contribute to a factor. Numbers in parentheses
are the factor loadings.

Factor 1 - Defensive Passing & Total Defense

Defensive Passing Yards (0.92)       Defensive Yards / Play (0.71)

Defensive 1st Downs by Passing       Defensive Passer Rating (0.66)
(0.87)

Defensive Yards (0.81)               Defensive Passing TDs (0.63)

Defensive Passing Completions        Defensive Passing Completion %
(0.76)                               (0.63)

Defensive 1st Downs (0.75)           Defensive Red Zone Attempts (0.51)

Factor 2 - Offensive Passing & Total Offense

Offensive Passing Yards (0.91)       Offensive 1st Downs (0.74)


Offensive 1st Downs by Passing       Offensive Passer Rating (0.70)
(0.88)

Offensive Yards (0.83)               Offensive Passing TDs (0.69)

Offensive Yards / Play (0.76)        Offensive Passing Completion %
                                     (0.62)

Offensive Passing Completions
(0.75)

Factor 3 - Offensive Rushing

Offensive Rushing Yards (0.92)       Offensive Rushing Attempts (0.65)

Offensive 1st Downs by Rushing       Offensive Rushing TDs (0.61)
(0.84)

Offensive Rushing Yards / Attempt
(0.77)

Factor 4 - Defensive Rushing

Defensive Rushing Yards (0.90)       Defensive Rushing TDs (0.65)

Defensive 1st Downs by Rushing       Defensive Rushing Attempts (0.61)
(0.83)

Defensive Rushing Yards / Attempt
(0.78)

Factor 5 - Turnovers

Turnover Differential (0.91)         Takeaways (0.69)

Giveaways (-0.67)

Factor 6 - Offensive Ball Control

Offensive 3rd Down Attempts (0.80)   Offensive 3rd Down Conversions
                                     (0.70)
Offensive Plays (0.73)

Factor 7 - Defensive Ball Control

Defensive 3rd Down Attempts (0.83)   Defensive 3rd Down Conversions
                                     (0.65)

Defensive Plays (0.69)

Factor 8 - Defensive 4th Down Performance

Defensive 4th Down Conversions       Defensive 4th Down Attempts (0.79)
(0.94)

Defensive 4th Down Conversion %
(0.84)

Factor 9 - Offensive 4th Down Performance

Offensive 4th Down Conversions       Offensive 4th Down Attempts (0.80)
(0.94)

Offensive 4th Down Conversion %
(0.84)

Factor 10 - Good Penalties

Good Penalty Yards (0.91)            Offensive 1st Downs by Penalty
                                     (0.76)

Good Penalties (0.90)

Factor 11 - Defensive Sack Performance

Defensive Sacks (0.87)               Defensive Passing Yards /
                                     Completion (-0.52)

Defensive Sack Yards (0.87)

Factor 12 - Bad Penalties

Bad Penalty Yards (0.89)             Defensive 1st Downs by Penalty
                                     (0.76)
Bad Penalties (0.88)

Factor 13 - Possession Change

Defensive Punts (0.62)               Defensive 3rd Down Conversion %
                                     (-0.54)

Offensive Punts (0.62)

Factor 14 - Offensive Sack Performance

Offensive Sack Yards (0.89)          Offensive Passing Yards /
                                     Completion (-0.50)

Offensive Sacks (0.88)

Game Statistics not Loading on a Factor: Offensive and Defensive Time
of Possession, Offensive and Defensive Passing Attempts, Offensive
3rd Down Conversion %, Offensive Red Zone Attempts, Offensive and
Defensive Red Zone Conversions.

Table 2. Effect of factors derived from game statistics (Table 1) on
game outcome. Effects are expressed as odds ratios and the equivalent
difference in chances of winning for a team with a high value of the
factor in a game vs a team with a low value (a difference between the
teams of two standard deviations of the factor).

                                                        Difference in
                                           Odds ratio   chances (%)

Large Effect

Factor 5 - Turnovers                       19           63

Moderate Effects

Factor 4 - Defensive Rushing               0.17         -41

Factor 3 - Offensive Rushing               5.3          40

Factor 1 - Defensive Passing &             0.23         -36
Total Defense

Factor 2 - Offensive Passing &             4.3          35
Total Offense

Factor 11 - Defensive Sack Performance     4.0          33

Small Effects

Factor 14 - Offensive Sack Performance     0.35         -26

Factor 10 - Good Penalties                 2.2          20

Factor 9 - Offensive 4th Down Performance  0.51         -17

Factor 12 - Bad Penalties                  0.55         -15

Factor 8 - Defensive 4th Down Performance  1.6          12

Trivial Effects

Factor 6 - Offensive Ball Control          1.2          5

Factor 13 - Possession Change              1.2          5

Factor 7 - Defensive Ball Control          1.0          0

Magnitudes are based on the following scale for percent differences:
<10, trivial; 10-30, small; 30-50, moderate; 50-70, large; 70-90, very
large; >90, extremely large (Hopkins et al., 2009). Uncertainties (99%
limits) in the odds ratios are X/-M.5 (for the largest effect) to
X/-M.3 (for trivial effects). Uncertainties in the differences in
chances are  all approximately [+ or -]7%.

Table 3. Effect of each game statistic on percent
chances of winning derived by univariate logistic regression.
As in Table 2, effects are expressed as
difference in chances of winning for a team with a high
value of the statistic in a game vs a team with a low
value (a difference between the teams of two standard
deviations of the statistic).

Factor 1--Defensive Passing & Total Defense

Defensive Passer Rating (85)
Defensive Red Zone Attempts (74)
Defensive Yards / Play (64)
Defensive Passing Completion % (64)
Defensive Passing TDs (54)
Defensive Passing Completions (26)
Defensive Passing Yards (16)
Defensive 1st Downs by Passing (11)_
Table 3 continued.

Factor 2--Offensive Passing & Total Offense

Offensive Passer Rating (79)
Offensive Yards / Play (53)
Offensive Passing Completion % (49)
Offensive Passing TDs (48)
Offensive Passing Completions (27)
Offensive Passing Yards (10)
Offensive 1st Downs by Passing (7)

Factor 3--Offensive Rushing

Offensive Rushing Attempts (91)
Offensive Rushing Yards (76)
Offensive 1st Downs by Rushing (68)
Offensive Rushing TDs (67)
Offensive Rushing Yards / Attempt (16)

Factor 4--Defensive Rushing

Defensive Rushing Attempts (94)
Defensive Rushing Yards (79)
Defensive 1st Downs by Rushing (74)
Defensive Rushing TDs (71)
Defensive Rushing Yards / Attempt (18)

Factor 5--Turnovers

Takeaways (80)
Giveaways (74)

Factor 6--Offensive Ball Control

Offensive 3rd Down Conversions (40)
Offensive 3rd Down Attempts (8)

Factor 7--Defensive Ball Control

Defensive 3rd Down Conversions (50)
Defensive 3rd Down Attempts (1)

Factor 8--Defensive 4th Down Performance

Defensive 4th Down Attempts (61)
Defensive 4th Down Conversions (21)
Defensive 4th Down Conversion % (1)

Factor 9--Offensive 4th Down Performance

Offensive 4th Down Attempts (63)
Offensive 4th Down Conversions (25)
Offensive 4th Down Conversion % (4)

Factor 10--Good Penalties

Good Penalties (28)
Good Penalty Yards (20)
Offensive 1st Downs by Penalty (9)

Factor 11--Defensive Sack Performance

Defensive Sack Yards (70)
Defensive Sacks (67)
Defensive Passing Yards / Completion (59)

Factor 12--Bad Penalties

Bad Penalties (20)
Bad Penalty Yards (15)
Defensive 1st Downs by Penalty (12)

Factor 13--Possession Change

Defensive 3rd Down Conversion % (59)
Defensive Punts (32)
Offensive Punts (23)_

Factor 14--Offensive Sack Performance

Offensive Sack Yards (59)
Offensive Passing Yards / Completion (50)
Offensive Sacks (6)

Not Loading on a Factor

Offensive Time of Possession (75)
Defensive Red Zone Conversions (74)
Defensive Time of Possession (74)
Offensive Red Zone Attempts (73)
Defensive Passing Attempts (70)
Offensive Red Zone Conversions (68)
Offensive Passing Attempts (61)
Offensive 3rd Down Conversion % (47)

Table 4. A subset of game statistics identified by
backwards stepwise selection that correctly predicted
the same proportion of won games and lost games
as did the model developed with the full set of game
statistics.

Offensive 1st Downs by Passing
Offensive 1st Downs by Penalty
Offensive Rushing TDs
Offensive Yards/Play
Offensive Passing TDs
Offensive Passing Yards
Offensive 4th Down Conversion %
Offensive 4th Down Attempts
Offensive Punts
Bad Penalty Yards
Defensive Rushing TDs
Defensive Rushing Attempts
Defensive Passing Completions
Defensive Passing Yards
Defensive Passing TDs
Defensive Passer Rating
Defensive Time of Possession
Defensive Passing Attempts
Defensive 4th Down Attempts
Defensive 3rd Down Conversion %
Defensive 4th Down Conversions
Defensive Punts
Takeaways
Giveaways
Gale Copyright: Copyright 2011 Gale, Cengage Learning. All rights reserved.