75 replies to this topic

[cage]

I'm surprised there isn't a thread on this yet, but one of Brandon's big announcements was the Bills will develop a sophisticated analytics operation to evaluate players....

NFL.com article

I'm interested in thoughts on several things.... does anyone know if any other NFL teams have dedicated any significant resources or had success with this approach?  Where and how would it be applied outside of salary negotiations?  I'm wondering what examples people can point to on the good/bad decisions that might be made with this approach?

If we were using sophisticated analytics I imagine that we would never have signed Fitz to a big contract, may have passed on Mario Williams in free agency and might have let Stevie walk?

[Al Cowlings]

Its a salary capped league, what are you talking about?  Everyone spends the same amount of $$.

[jr1]

Kyle Moore instead of Kelsey

[K-9]

I'm skeptical that baseball's "moneyball" metrics can work in the NFL. Baseball is the epitome of individual performance. Football is the exact opposite. Every play depends on the proper execution of several individuals on both sides of the ball. Failure of a play is often a failure of one or more people not executing their assignments properly. Within that context, I would think it's difficult to always ascertain individual performance measurements without deep analysis of each player's specific role during a play. While some measurements are easy, especially for skill players, quantifying others would depend on breaking down each play and assignment. Not impossible, but not as easy as simply inputting numbers.

GO BILLS!!!

[AlwaysBilleve]

Interesting post. I'm curious to what stats the management would key in on. In baseball it is pretty straight forward how they did it. On base percentage all the way. In football i don't think it is so cut and dry. What stat do you use? YAC, competition percentage, etc.? A cool idea but I think football is too complex of a sport that makes this system insignificant. I'd love to see if someone could figure out a system to weigh potential based on stats in football. Personally, I don't think it can be done. The intangibles that makes up this great sport is something that separates players at every position. How do you quantify this?

Al Cowlings, on 02 January 2013 - 01:41 PM, said:

Its a salary capped league, what are you talking about?  Everyone spends the same amount of $$.

I believe he is referring to the assessment of talent as opposed to the financial aspect.

[K-9]

Al Cowlings, on 02 January 2013 - 01:41 PM, said:

Its a salary capped league, what are you talking about?  Everyone spends the same amount of $$.

That's certainly true. But it's inarguable that some teams spend it smarter than others.

GO BILLS!!!

[playman]

yep. more than just  " eyeball "

[Best Player Available]

A couple of examples of what good teams use that for. Calculated odds of hitting a FG from the 34 yard line instead of punting. Going for 2 points on the first TD instead of kicking the PAT. Chip Kelly of the ducks is a master of those type of things and a reason the Ducks think they can still succeed without him. Since his analytics are a huge part of their program.

[Kelly the Dog]

One way is to discover how well an OL or a DL is performing. If you base a DL's performance on the number of sacks he gets or the number of tackles, it's a ridiculously inefficient way of looking at how well he plays. Obviously, they don't just do that, they study a lot of film on the guy and grade him. But breaking down a OL or DL's performance based on metrics, how many times he beats his man, gets a draw or gets beat, gives you a much better indication of his play. It can be broken down 20 different ways, and when combined with film study, seems to me to be a far, far superior way of evaluating him.

[dave mcbride]

San Francisco tried a moneyball approach around 2005 or so, and it failed for a number of years. The guy behind it is Paraag Marathe: http://sfist.com/200...yball_49ers.php . He's still one of the powers in the organization, though, so I guess one could argue that it paid off in the end. They were in the wilderness for a long time, though. Basically, it's really hard, not just because of the way you value players and assign credit on plays with 22 moving parts. Injuries are much more frequent in the NFL and careers shorter. Hence when you're thinking about signing someone long term the formula doesn't really work given the projection issues.

[hondo in seattle]

This is a good OP.

Here's the little I've gleaned from a variety of sources - hopefully credible.

*  Many, if not most, NFL teams use some sort of sabremetric-type analytics when evaluating player performance.  These might be used by scouts and/or by coaches, depending on the team and circumstance.

*  For example, I've read that some teams use their own in-house analytical metrics to measure QB performance (instead of using passer efficiency or QBR).

*  I've also read that no NFL franchise is as devoted to sabremetrics as some baseball clubs are.  The use in the NFL is far more limited.

*  Sabremetrics in football is difficult because, as mentioned, it's a team sport.  If the players around you are untalented or confused, your performance will suffer.  If you're coached poorly, your performance will suffer.

*  Sabremetrics is also logistically difficult.  Ideally, each play would be broken down 22 times - once for each player on the field.

*  Breaking down each player's performance on each play involves a lot of subjectivity.

*  NFL teams are secretive and it's hard to really know what they're actually doing behind closed doors.

Like the OP, I'd like to know more what  Russ intends.

[bigK14094]

K-9, on 02 January 2013 - 01:44 PM, said:

That's certainly true. But it's inarguable that some teams spend it smarter than others.

GO BILLS!!!

Yes, but remembering the Jason Peters case, the Bills were very smart.  Peters was locked up for three years.....but he decided that he wouldn't play for the paltry 3 mill/year he signed for, as he was an all pro.  So, he held out, and forced the trade to Philly (hows that working out btw)  Andhow, they had him, and he choose not to honor the contract.  Lots of cases like this in football.  I am thinking that moneyball won't work particularily well in the non contract compliance envieonment that is the current NFL.

[K-9]

bigK14094, on 02 January 2013 - 02:17 PM, said:

Yes, but remembering the Jason Peters case, the Bills were very smart.  Peters was locked up for three years.....but he decided that he wouldn't play for the paltry 3 mill/year he signed for, as he was an all pro.  So, he held out, and forced the trade to Philly (hows that working out btw)  Andhow, they had him, and he choose not to honor the contract.  Lots of cases like this in football.  I am thinking that moneyball won't work particularily well in the non contract compliance envieonment that is the current NFL.

Agreed. In my mind it's quite simple and it trumps any metrics used in quantifying a player's worth. If the player doesn't want to be here, I'm looking for a way to ship his ass out of town for as best a return as I can get, especially such an important cog on the OLine as the LT. As talented as he is, Peters' unprofessional behavior during that off season and subsequent sub-par performance while he got himself into game shape, was a slap in the face to his teammates and the fans alike.

GO BILLS!!!

[voodoo poonani]

The NE Patriots for one.  Here's an old article from MIT's magazine: link.

[OGTEleven]

I could see moneyball being applied somewhat readily to defensive players and not so much to offensive players.  Oddly enough, this is similar to baseball using it for offensive players over defensive players.  

A football defender and a batter in baseball are both reacting to something being thrown at them.  The defense (pitcher) initiates every play in baseball where the offense initiates it in football.  You can look at film and determine most of the time how the football defender matched up against the other team.  Did he draw a double team?  Penetrate? Cover his man?  Sometimes you won't be sure if he executed his assignment but most of the time you'll be able to tell by watching.  Even so, you can't have others do the watching for you by reading OBP stats like baseball.  That is a lot of people watching a lot of film and that makes it subject to varying interpretations.

I haven't read the book Moneyball (movie only) but I am a baseball fan.  I have not seen anything indicating that Beane used the metrics on pitching/defense but he didn't completely ignore them building that winning team.  They did have Hudson/Zito/Mulder after all.  Evaluating Scott Hatteberg as a value against Jeremy Giambi is one thing and I think it can be important, but you still need pitching.  That is where everything starts.  A quarterback is like a #1 starter that starts every game.  Take the Bills this year and plug in Manning for Fitzpatrick.  What would the record be?

I am not sure how to apply moneyball to WR (how do you know if a player operating from another team's playbook ran the right route?); RB (can apply it to rushes certainly but what about blocking/receiving?  And even rushing is dependent on the OL as has been pointed out).  I guess I can see applying it to the OL somewhat but again the assignments are not always clear.Moneyball worked (works) for the A's brilliantly with respect to hitters and production.  They have been able to develop pitching talent consistently too and make some timely trades to keep the farm stocked.  Some of these things could be done in football but some could not.

I am not against the moneyball concept but it would have to be a square one re-design for football.  I'm hopeful but skeptical.

Edited by OGTEleven, 02 January 2013 - 02:49 PM.

[Edwards' Arm]

cage, on 02 January 2013 - 01:30 PM, said:

I'm surprised there isn't a thread on this yet, but one of Brandon's big announcements was the Bills will develop a sophisticated analytics operation to evaluate players....

NFL.com article

I'm interested in thoughts on several things.... does anyone know if any other NFL teams have dedicated any significant resources or had success with this approach?  Where and how would it be applied outside of salary negotiations?  I'm wondering what examples people can point to on the good/bad decisions that might be made with this approach?

If we were using sophisticated analytics I imagine that we would never have signed Fitz to a big contract, may have passed on Mario Williams in free agency and might have let Stevie walk?

Developing a Moneyball tool for football would, as K9 has pointed out, be a difficult task. But not necessarily an impossible one.

If I was tasked with developing such a system, I'd do the following:

1. Define the variable I want to maximize
2. Develop metrics to measure players' contributions to those variables.
3. Measure each current and potential player on those metrics.
4. Obtain players who provide the maximum ratio of contribution to money spent.

For step 1, I think a fairly decent metric would be yards gained per play (offense) and yards yielded per play (defense). Those could be further broken down into yards/running play and yards/passing play. One could refine the above by also taking into account turnovers. Yards per passing play should be weighted three times as heavily as yards per running play or turnovers. (The number three comes from a regression analysis performed by The New York Times.)

For step 2, you define each player's task on a specific play. Then you rate how well he achieved his task. A quarterback's job is to throw the ball accurately, an offensive lineman's job is to win his battle against the guy across the line from him, etc. You could rate the accuracy of each pass on a scale from 1 - 5, the quality of a QB's decision on a scale from 1 - 3, etc. You could also rate the quality of a WR's catching on a scale from 1 - 5; where a 5 counts as catching a very poorly thrown ball, a 3 counts as catching a well-thrown ball, and a 2 counts as dropping a difficult-to-catch (but still catchable) pass.

Step 3 involves watching a ton of film. It will probably be necessary to watch the same game many times, just so that each player can be assigned a grade on the above-described scale. Ideally, you and your staff would watch at least 1 - 2 seasons' worth of games, with each player assigned a grade for each play. Once you had that data, it would be fairly straightforward to develop a statistical model which could answer questions like, "If an offensive lineman loses his battle against a defender, how many fewer yards is the running play likely to gain than would have been the case, had he won the battle?" Once you can answer questions like that, then you can determine your average expected yards per running play with Offensive Lineman A versus Offensive Lineman B.

For step 4, you look for guys who can provide the most yardage per play at the lowest possible cost. This does not necessarily mean weeding out star players and their high salaries. A guy like Aaron Rodgers or Peyton Manning almost certainly contributes enough to his team's yards per pass play to more than justify a princely salary.

[San Jose Bills Fan]

I can see where this discussion is headed.

You guys are gonna start communicating in algebraic equations, aren't you?

[BuffOrange]

Not sure what reason Brandon has to go public with this, but I'm glad to hear it nevertheless.
As a proud member of the "numbers guy" club I can't help but shake my head at a lot of the skepticism.

"Football isn't as easy quantifiable as baseball" - I don't think anyone on the planet including guys like Aaron Schatz or Bill Barnwell who write about it (football analytics) for a living would argue with this.  That doesn't mean it can't be a useful tool.

"Who has done this that's been really successful?"  If you lose $100 playing blackjack using basic strategy when you otherwise would've lost $500 playing on "gut feel" does that mean using basic strategy was a failure?  Obviously talent & QB are more important than everything else as it relates to football games.  That doesn't mean you shouldn't attempt to give yourself any little edge you can...especially when you're the Bills & don't have many edges.

Best Player Available, on 02 January 2013 - 01:46 PM, said:

A couple of examples of what good teams use that for. Calculated odds of hitting a FG from the 34 yard line instead of punting. Going for 2 points on the first TD instead of kicking the PAT. Chip Kelly of the ducks is a master of those type of things and a reason the Ducks think they can still succeed without him. Since his analytics are a huge part of their program.

This, exactly.  Which makes it a bit hard to believe that we're interviewing Doug Marrone, who punts from the opponent's 35 with the best of 'em.

[GOBILLS78]

Nerds!

(Chugs can of PBR, smashes it against forehead, coaches losing football team.)

[jjamie12]

Seriously?  No one here has heard of Football Outsiders?