The New York Times has a piece that calls into question the accuracy of past statements released by the NFL and USA Football in regard to the Heads Up Football program.
USA Football provided a $70,000 grant to Datalys to fund a study on the efficacy of the Heads Up Program. In February of 2015, Datalys provided preliminary results, and those were used to make claims that included saying that leagues with Heads Up Football had 76 percent fewer injuries, 34 percent fewer concussions in games and 29 percent fewer concussions in practices.
However, those weren’t the same results reached when Datalys actually published the study, and the results showed no significant concussion rate differences once the study accounted for whether the league was a Pop Warner league or not. (Pop Warner leagues have weight restrictions and several other safety measures that are not always included in independent leagues certified under Heads Up.)
From the New York Times:
As it turned out, only leagues that adhered to Pop Warner’s rules saw a meaningful drop in concussions. Leagues that used Heads Up Football alone actually saw slightly higher concussion rates, although that uptick was not statistically significant. The previously reported drops were clearly driven by a league’s affiliation with Pop Warner, not Heads Up Football.
Similarly, Heads Up Football leagues saw no change in injuries sustained during games unless they also used Pop Warner’s practice restrictions. The drop in practice injuries among Heads Up Football-only leagues was 63 percent, a very meaningful figure. But combined with the in-game injuries the total reduction became about 45 percent, far less than the 76 percent presented by U.S.A. Football and the N.F.L. for the past year and a half.
Both USA Football and the NFL stated they would update all claims and remove references to those incorrect numbers, when reached for comment.
These statements from Datalys, trying to justify what happened, are truly amazing.
Zachary Kerr, one of the researchers, said of passing on preliminary numbers that were then used by the NFL and USA Football, and never corrected: “The results were so compelling, we felt morally obligated to make the youth football community aware of the results.”
Kerr also said: “Datalys stands by our decision to release preliminary data in our Feb 2015 release because if we prevented even one youth football player from suffering an injury (sprain, fracture, strain, severe contusion, or concussion), then the release was a success.”
That’s not how research is done, and I’m not sure what they prevented. The release was inaccurate, and it was never corrected, and it will take a lot to undo it.