I promise that I'm not really going anywhere.  I'll be around here and there doing Sabermetric stuff, sometimes in front of the camera and sometimes behind it.  Maybe I'll pull a Michael Jordan (I can't hit a curveball either) and re-appear here every now and again.  There are a few things up in the air right now, and I'm not sure where they will land.  And for right now, I kinda like things that way.

I owe a great deal of thanks to the other folks who have shared this space and collaborated with me behind the scenes: David Gassko, Sean Smith, Matt Souders, Michael Frain, Mike Fast, Eric Seidman, Brian Cartwright, Colin Wyers, Dan Novick, Jon Walsh, and Matt Swartz (not a bad list of alumni, with the exception of Seidman).  Plus there are all the folks who did roundtable last year before that had to be sacrificed to the time gods.  I also owe a big thank you to John Beamer, formerly of Chop-n-Change here on MVN, who recommended me for the gig here two years ago, and to MVN's then baseball director, now content director Cory Humes and president Evan Brunell for giving me the platform to begin with.  I even forgive Evan for being a Red Sox fan.

If you're reading this, thank you.  When I started doing my own Sabermetric work a few years ago, I didn't think anyone would ever read it.  The internet is a funny place like that.  What I thought would just be a little hobby has turned into a chance to interact with some really cool people who get just as geeked up about regressions and home runs as I do.  It means a lot to me personally that people actually liked my stuff.  So, from the absolute bottom of my heart, thank you.

;-)

Or maybe not.  Maybe on the side, you tinker a bit with Pedro's 1999 delivery and teach him to throw filthy side-armed sliders, under-hand rise-balls, and maybe help him out a little with that lagging fastball velocity.  You also teach Pujols how to hit the ball farther (he apparently needed help with that).  While you're in the neighborhood, you teach Frank Thomas, your team's designated pinch runner, a few tricks and he subsequently steals 100 bases and tries out for the US 4x100m Olympic relay team.  And makes it.  You even show Darrin Erstad (we need some HEART on this team) how to get a few more yards on his punts.  (I miss FJM.)  The problem is that no one really believes that you had anything to do with the World Series win.  After all, you just inhereited a bunch of superstars and went along for the ride.

The manager is always the first one to be blamed when the team is having a bad spell.  If you're angry about your team's performance and need to vent, "Fire the manager!" makes for a rather nice refrain, even if that manager is working miracles with what he has on hand... it's just he doesn't have much on hand.  On the flip side, a manager who skippered a World Series winning side can always say that he is a "proven winner" even if in reality he had nothing to do with it or the team succeeded in spite of him.  The problem is that in evaluating the manager's performance, there's a giant confound.  Managers can only use the players that are available to them.  You can't make a silk purse out of a sow's ear.  Just ask Manny Acta.

A manager, in my estimation, has three jobs.  He is responsible for game strategy decisions (should we hit and run here?), he is the head coach in charge of improving the players' performance, and he's the chief spokesman for the team both internally (keeping peace in the clubhouse) and externally (dealing with the media).  Most people looking to rate managers have looked at the question from the first perspective.  I've even seen a few attempts to develop rating systems.  The third really doesn't lend itself to measurement.  But what about the second job of a manager?  Major leaguers are gifted athletes, but there's always something that they can learn.  Otherwise, why have coaches?  True, the manager doesn't actually do all the coaching/teaching, but he is the man who is at the top and who hires the guys that do the teaching.  How can we tell if a manager is a good teacher/coach?

Why not look at managers the same way that we look at classroom teachers?  If you have kids, they probably took a state achievement test a few months ago.  (If you don't, you might remember taking the dreaded "Iowa Test" or the CATs at some point when you were in school.)  These tests are rather controversial, as is everything in education policy.  Most of the controversy has been around whether teachers are "teaching to the test" rather than encouraging creativity and innovation.  But, there's another hidden aspect.  The No Child Left Behind Act called for such annual evaluations as a way to test whether schools were "failing."  Then, there is the controversy of merit pay for teachers.  Teachers who have low-scoring kids must be poor teachers, right?  They should be paid less than those whose kids score better?

So, how to tell, in the parlance of former President G-Dubs, "Is our children learning?"  And while we're at it, which teachers and schools are doing good work.  Statistically speaking, that answer calls for a rather high-level statistical tool, hierarchical linear modeling (HLM).  (Note: technically, what I'm going to do is a related procedure, MLM or mixed linear modeling, but it's the same basic idea.)  Here's the intuitive introduction to HLM.  Imagine what factors influence a kid's score on a test like the CATs. 

• First off, the kid may be smart or... what's the polite term?... still developing proficiency (individual level variance).
• We can also track individual variance over time.  If a kid gets a 75 out of 100 one year, we don't expect a 30 the next year, assuming that the test has good reliability.  We can correct regression models for that.  It's called an AR(1) covariance matrix, which produces an intra-class correlation.  If you've read my stuff before, you've seen me use it extensively.  (And you're taking a shot in the StatSpeak drinking game.)
• But then there are some teachers who are really good and some that are... just plain awful.  (I suppose I could name a few names from my own educational past...)  If we see that all thirty kids in a class are all achieving at a high level and the kids in the other classes in the same school aren't, then the common factor there is the teacher (teacher level variance).
• In the same way that students are contained within classrooms, classrooms are contained within schools.  And schools within districts.  And districts... You can see why this technique is called "hierarchical."
• Now, the math is a little more complex, but it comes down to this: suppose that you have kid A is assigned to teacher B in school C.  What will his score likely be on "the big test" at the end of the year.  His result will be a combination of the effect of his own smarts + his teacher's ability + any school variables that factor in.

The parallels to baseball are pretty obvious.  The players are the students.  The manager is the teacher.  And every game, there is a test.  For a batter, the test is "please get a hit."  For a pitcher, "please get this guy out."  Now, are there specific managers who seem to have "students" who get better, even after we control for the fact that some of them come to the team really good and some really... developing?

I decided to look at pitchers.  I took twenty years worth of data (1989-2008) and isolated all pitchers who had faced 100 batters or more that season, and had played for one team (and one team only) who had one manager (and one manager only).  This is to prevent cross-contamination.  I also coded for his home stadium, age, and year-league context.  All of those will have effects and we need to include them in the model.  There were exactly 100 managers who had managed at least one full season with a team.

(Gory details: I created a mixed linear model with an AR(1) covariance matrix.  I use SPSS, but all you SAS junkies will recognize this as PROC MIXED.  I used a strictly fixed-effect model with manager, age, year-league, and stadium entered into the model.  I ran three different regressions, once for K rate, BB rate, and HR rate.  I saved the manager effect coefficients and normalized them so that they had a mean of zero.  This allows me to say things like ".75% above average.")

Now, as you might expect, the effects on these rates were fairly modest, usually on the order of a manager improving his players by something like 1-2% on walks.  A starter pitching a full season might face 800 or 900 batters, and he'd net 8 or 9 more walks (plus or minus the inevitable error of estimation) with these types of effects.  It's something that might not be visible to the naked eye.  But with a big enough data set, these things tend to shake themselves out.

I also didn't choose my stats randomly.  K, BB, and HR are the ingredients in putting together FIP.  Knowing that a manager generally improves his pitchers' rates on each by X%, and the knowledge that the average team faces 6250 batters per year over about 1445 innings.  At least that's what happened in 2008.  So, if I know that manager improves his pitchers (in general) by about 1% above average in K rate, then I know the team will likely have 62 more strikeouts, give or take, than if the average manager were in charge.  It's just a matter of plugging in some numbers and I can distill those manager effects into what they represent for a team's FIP.

Before I give you "the list", yes, I know that I should probably split this up by pitching coach instead of manager.  If I can find a good list of pitching coaches over the years, I will.

The five best managers (1989-2008)
(manager, delta FIP)

1. Buck Martinez, -0.90
2. Ned Yost, -0.79
3. Bobby Cox, -0.62
4. Larry Rothschild, -0.60
5. Davy Lopes, -0.55

*- these numbers should be read as "Buck Martinez, given the same staff as an 'average' manager, would have a predicted FIP 0.90 lower than the average manager.  It might be that if Buck were given a bunch of retreads and AAA pitchers, his team would have a 6.50 FIP, but the average manager would be predicted to have a 7.40 FIP."

The five worst managers
(manager, delta FIP)

1. John Russell, +0.73
2. Alan Trammell, +0.64
3. Dave Trembley, +0.61
4. John McNamara, +0.56
5. Cecil Cooper, +0.53

The smell test seems satisfied with the names on the list (Bobby Cox is a good pitching manager... makes sense).  I'm a little leary of the magnitude of those deltas though.  A change is FIP of 0.63 or thereabouts is worth 100 runs over the course of a season.  Did Alan Trammell cost his Tiger teams that much?  It seems a little extreme.  It might be that MLM is not as good at pulling apart the data as we had hoped.  Trammell managed three seasons in the majors, all with the Tigers (2003-2005) and he had some awful pitching on those teams.  Many of those pitchers never really got a chance to go to another team (and another manager) or hang around long enough for the Tigers to hire Jim Leyland.  Since the majority of the players managed by Trammell are guys who were a) awful and b) only ever managed by Trammell, the model may be over-blaming him.  The model has to put the blame somewhere.

But for some of the more established managers who have managed a bunch of pitchers who have been both in their care and with others, I trust those estimates of the manager effects.  The only real way to see what is the effect of a manager is to see what a pitcher does with another manager.  This just provides a systematic way to make those comparisons.

For the curious: the complete data file.

Strictly speaking (and statistically speaking), over a one-year period, there is very little in the way of a repeatable clutch skill.  No matter how you measure it, whether it's a leverage index based mathematical model of clutch or an intuitive "performance in 'close and late' situations" model, a player doesn't seem to perform better or worse in the clutch than he does overall.  At least over a one year period.  But then, perhaps we've over-stated our case.  It's one thing to say that a one year time period (or 700 PA or whatever) is not a sufficient sampling frame to attain proper reliability on a statistic.  It's another to say that the skill does not exist.  It might just be covered by a lot of noise.

One of the problems with measuring clutch is the way in which it's been operationalized.  The current best measure to be had is the method used at Fangraphs, which uses win probability added (WPA) and the Leverage Index (LI) to calculate a mathematically based clutch statistic.  The problem with WPA based versions of clutch is that WPA itself doesn't stablize over a short period of time, (and here, even a full year is a "short" period of time).  Many of the usual one-number performance indicators (the slash stats) are unreliable in small sample sizes as well.  Perhaps we need a better indicator of what's really going on in the clutch.

Let's go back to what the idea of clutch is.  People react to stress differently.  Some people seem to thrive under pressure.  Others seem to crumble.  Anyone who's ever had stage fright can attest to the latter.  The common thread is that performance suffers/improves under pressure.  But does behavior change under pressure?  There's a difference.  Behavior is what players decide to do, over which they have a great deal of control.  Performance is the result, and as we've seen time and time again, is often the result of luck mixed with behavior.

Let's take the decision on whether or not to swing at that pitch that's currently hurtling toward the plate, perhaps the one thing over which the batter has total control.  (Swing percentage also stablizes at ridiculously low levels of PA.)  Do players swing more (or less) with the game on the line?  Doubtless, some will not be affected by the pressure, some will start swinging at everything, and some will politely keep their bat on their shoulder more than they otherwise would.  This is assuming, of course, that players in general are affected by pressure.  But are players consistent in the way that they are affected by pressure?

I took all player-seasons from 2005-2008.  I split all plate appearances into non-clutch and clutch.  For my definition of "clutch", I went with a "close and late" definition (7th inning or later, score within 2 runs either way).  Why this and not a leverage-based method?  Because baseball players probably don't know that leverage exists.  It's a great mathematical tool, but it's probably not how players decide whether this is a big situation or not.  The reason behind the 7th inning cutoff is that even if one is leading off the seventh inning, there is the very real chance that this may be my last chance to do something.  If the team is retired in order over the next three innings, the guy leading off the 7th will be standing in the on-deck circle as the game comes to an end.  Plus, those "close and late" situations have pretty high leverage values to begin with.  I fully realize that it's not a perfect defintion of clutch, but I'm trying to model what goes on inside the head of a major league player.  I'm a psychologist after all.

I looked at each player's swing percentage when in the ho-hum basket and in the "clutch" basket.  I set a 50 PA minimum for the clutch basket.  I found the difference between his swing percentage for the clutch and non-clutch situations.  Indeed, some players swung more, some less.  Some stayed pretty much the same.  Since I had four years worth of data, I ran an intra-class correlation (StatSpeak fans, take a shot!), and came up with .24.  (short explanation: think of ICC as a year-to-year correlation... only better.  For more details, go here.)

That may not seem very big.  Papers have been written about less.  The difference may not be stable at a minimum of 50 PA of clutch at bats, but perhaps at a slightly bigger sampling, it may approach respectablility.  Given that swing percentage is a very stable statistic, and is something that is almost entirely in the control of the batter, it may even be possible to determine from minor league data what a player will look like on this measure before he gets to the majors.

The other thing to consider is that swing percentage matters.  Not directly, but in concert with other diagnostics, it makes a difference.  Low contact hitters (generally, power hitters) who swing more actually have lower rates of HR and higher rates of K's.  So, players who get nervous and start swinging more are much less likely to hit that homerun in the clutch.