Judging the Bias

Whether we know it or not, everyone plays favorites, even judges

Kelly Slater, throwing up a hail mary in an incredibly close final against Mick Fanning at Bells Beach. Photo: Joli

If you were born in America, odds are you grew up pledging allegiance to the flag, have at least a mild affinity for "Subterranean Homesick Blues," and feel an inexplicable draw toward In-N-Out (or Five Guys, for you East Coasters) whenever returning from abroad. I would assume that every nation has their versions of Bob Dylan and a Double-Double with cheese (in their minds, at least), and that shared experience gives nations a sense of unified identity. This is all well and good, but what happens when we drag this baggage into the world of professional surfing? More specifically, what happens when we drag it into the judge's booth?

Recently, University of Illinois PhD student Breno Sampaio finished his dissertation in the field of forensic economics, in which he discussed a bias that exists in the judging of professional surfing events. His research, which was conducted using a sample of wave scores from five events in 2010, showed a trend suggesting that ASP World Tour judges tend to give their fellow countrymen preferential treatment, but not in the way you might think.

Although the knee-jerk reaction is to rally as a community, light our torches, and sharpen our pitchforks for a good ol' fashioned ASP lynching, they may not quite have earned it in this case. According to the study, ASP judges aren't over scoring their compatriots but instead the study finds that they are underscoring the foreign opposition. The research found that on average, judges underscore the opponent by 0.133 for all waves, with the average rising to 0.3 on waves that will actually be counted toward their heat total. If their compatriot is winning, however, this negative effect disappears. In other words, judges aren't handing heat wins to their countrymen on a silver platter, they just want to keep them in the game if they fall behind.

For those of you who don't have a basic understanding of ASP judging, it goes something like this: Each heat has five judges to score waves, with a Head Judge supervising the panel. The panel is chosen from a pool of international judges who rotate throughout the event. During each heat, all five international judges contribute scores, but the highest and lowest are dropped, allowing the average of the middle three to make up each wave score--which the ASP contends is enough to eliminate any potential bias. Unfortunately, according to the study, a competitor still stands a better chance of winning with a compatriot in the booth even after dropping the outside scores.

When the ASP was pointed to the research and asked for comment, the feedback was surprising. "I don't rate any of it of any value," replied ASP Head Judge Richard Porta. "We have a guy that sits in the middle of America and plays with figures and formulas. Me, I have more important things to worry about than if a judge is 0.8 off the mark in a handful of heats out of the 510 mens heats that will happen this year. Plus the 91 women's events and the five Prime events I will be Head Judge at this year." Co-ASP Head Judge Pritamo Ahrendt was more pragmatic in his response to the numbers: "I believe that we have the best system in place to curb bias," he says. "We choose judges that are surfers and that love surfing as a sport and understand that being biased is the worst possible trait that any judge could possibly have. If we notice national pride or biased judging or just bad judging they will no longer work for ASP." It's clear that this is a topic they don't take lightly, but neither would admit that there is even a possibility of bias within the judging. The numbers from Sampaio's study, however, certainly beg to differ.

You may feel that these less-than-a-point discrepancies are trivial. But before you discount the research entirely, think about the final heat between Mick Fanning and Kelly Slater at the 2012 Rip Curl Pro at Bells Beach. That final was decided by less than a point, and it was a divisive win, with pros, fans, and pundits aligning themselves on either side of the debate. In a recent interview with STAB Magazine, Josh Kerr lent these words about the Bells final: "For Mick, I'm sure it helped that it was down at Bells and in Australia. I have a feeling if that final had been at Trestles, Kelly would've won."

Perhaps it's time that we find a neutral party--surfing's Switzerland, if you will. We train those mountainous folk to judge surfing from a live stream in a cozy little cabin in Zurich, and as long as they don't produce a world-class surfer themselves, our sport is safe. But even then we would not be out of the woods. Judges have to be lifelong surfers with a deep understanding of aquatic idiosyncrasies to be able to know for certain that a tail-high slob is far superior to a double-grab wheelie. Unfortunately, those individuals are and must be part of a developed surf scene--a scene that produces surfers they will more than likely be biased toward. In the end, no matter what we do in the pursuit of objectivity, surfing will never be the cut-and-dry sport of absolute winners and losers.

Watch Mick defeat Kelly by less than a point during the Bells final: