The H.U.M.A.N. poll revisited, part 2
Some additional stats and a salute to our champion ball-knowers
If you skipped our last post regarding the H.U.M.A.N. poll, no worries. It was a bit dense. The tl;dr was that the humans showed some limited skill in predicting errors in the preseason kenpom ratings. (If you’re have no idea what the H.U.M.A.N. poll is, you can read about it here.) Today, we look at some general summary statistics and announce our winner(s).
The H.U.M.A.N. poll turned into a celebrity-studded, red-carpet event. We got votes from a cross-section of kenpom subscribers, including numerous media members, assistant coaches, video coordinators, at least one head coach, and even a former #1 draft pick. If you voted, you can see your ballot here.
There were 719 participants in all, but 589 filled out an entire ballot. We’ll focus on them for the moment. The range of scores was from 18 to 41, with an average of 29.23. That means that the humans got 58.5% of their picks correct. The algorithm tried to serve up non-trivial matchups, so it’s encouraging to see the win rate north of 50% but not too high.1
For funsies, here’s what the worst ballot looked like (matchups are ordered by their difference in final rating):
This ballot does provide examples of how voters were, to some extent, at the cruel whims of the ballot-generating algorithm. Almost anyone that was served Auburn-Duke would have picked Duke, and if you got served Kansas-Wisconsin you would have thought it was an automatic win.
Even picking Hampton over High Point, though comical in hindsight, wasn’t that bad of a pick in October. My preseason ratings (though an outlier and completely unaware than Kimani Hamilton was a blossoming basketball legend) had Hampton about 20 spots higher. Clearly, this person was trying and just on the wrong end of some teams performing quite differently from what most people would have expected.
There was actually a worse ballot than this as someone went 15 of 49. In fact, 25 people only made a pick for 49 of the 50 matchups, surely due to some sort of display issue on mobile devices. Fortunately, none of them got within one win of finishing in the top ten which would have earned them some sort of reward. We’ll fix that for next season.
Here’s the distribution of scores in our group:
Let’s look at the teams that were the best for voters. These were the teams that were picked (or picked against) the most correctly when they showed up in a matchup:
UConn 259-73 .780
Duke 247-70 .779
Houston 273-79 .776
Purdue 261-76 .774
Gonzaga 236-77 .754
All five of these were squads that voters accurately thought would be very good. UConn’s record here matches the number of times they were picked and picked against in matchups. Because they finished the season at the top of the ratings, everyone who picked them was rewarded with a win and everyone that picked against them took a loss. There was one other team that had that distinction - McNeese State.
It wasn’t that easy for every team, though. Kansas was picked in 311 of the 341 matchups that were served up, but the humans selected correctly in just 49.6% of those matchups. But that wasn’t close to the worst percentage. Here are the teams that tripped up humans the most:
Tarleton State 52-91 .364
Central Connecticut 31-52 .373
LIU 21-35 .375
Oral Roberts 60-97 .382
UCLA 117-174 40.2
Not shown: Arkansas (12th) and Miami FL (13th)
This list is about the teams that humans mis-evaluated. For teams in the more obscure conferences, the humans’ opinions seemed to come down to random knowledge about them. Oral Roberts was in the tournament last year. They made it to the Sweet 16 in 2021. Match them up with another mysterious team, and the humans will pick Oral Roberts even though they went though a coaching change and had to replace Max Abmas and Connor Vanover. The humans ranked them 158th. The computer ranked them 194th. They finished 283rd, a resounding loss for all.
On the other end, there was Tarleton State, whom I feel most humans knew nothing about. It was the Texans’ fourth year playing a D-I schedule and next season will be the first time they are allowed to play in the NCAA tournament. The humans defaulted to picking against them regardless of opponent. Tarleton State was ranked 272nd by the humans, 220th by the computer and finished 123rd, 16-4 in the WAC. Billy Gillispie, in the right circumstances, can run a program. Tarleton State has now gone from 261 to 212 to 163 to 123 in his four years there.
Anyway, I’m happy that the computer generally served up competitive matchups. With every team having a success rate between 36 and 78%, no team was a gimme and no team was a guaranteed loss for those voting.
Now for our winners!
Our champion ball-knower of the year is Kevin Sweeney. As you surely know, Kevin is a writer for Sports Illustrated. Not so coincidentally, he is one of the few humans who does a preseason ranking of all of the teams in men’s college basketball. Kevin’s ballot is below:
Kevin declared this the “proudest honor of his career” which we only take to be slightly facetious since this kind of contest reflects skill much more than, say, filling out a bracket does. While other humans might do preseason polls, their target is whatever they want it to be. So when Jerry Carino ranks FAU #3 in the preseason, nobody can say he doesn’t know ball, even when the team ends up losing in the first round as a eight-seed.
Of course, there’s luck involved in this ballot, too. The UCLA-Arkansas matchup was a doozy considering both of those teams badly underperformed. Getting either of those teams separately had a high chance of a loss. But Kevin got them together and got the pick right. On the flip side, he was one of the few to pick Kentucky over FAU. That doesn’t sound like a risk in retrospect, but just five AP voters had Kentucky over FAU on their preseason ballot. (Although, I suspect if they went through an exercise like this, there would have been more.)
I’m fine with the hot-take industry and get that there’s a market for entertaining opinions even if there’s no actual accountability for being right more often. You have to attract eyeballs somehow. But I prefer the H.U.M.A.N. poll and how it better identifies skill in preseason prognostication.
Kevin wins a signed Chris Duhon jersey from the 2007 Chicago Bulls (estimated value: off the charts). I was not able to contact our second-place finisher, which should give you some reassurance about how little personal information we collect at kenpom.com. Just an e-mail address, which could be fake! Our third-place finisher was Matt Jones of Anaheim, California, who can often be found at UC Irvine games. Congrats to Matt who also wins some sort of artifact from my closet.
Overall, we had 14 people that finished in the top ten and earned a complimentary year of kenpom access.
Among those I could identify were a few borderline celebrities, including Jim Root of Three Man Weave; David Hess of TeamRankings.com; Mike Doyle, director of basketball operations at La Salle; Kory Keys, former grad assistant at Ole Miss; Eric Fawcett, video coordinator for the Edmonton Stingers; Isaac Schade of the Locked on College Basketball podcast; and Jacob Kohn, a data scientist at Sports Analytics Advantage, a company that more or less orchestrated Alabama’s Final Four run. Others include Mike Chmill of Harrison City, Pennsylvania; Travis Bowman, a Purdue grad; and Paolo Ciocco. Let it be known that these people are all expert ball-knowers.
That will conclude our discussion of the inaugural H.U.M.A.N. poll. This was one of the most fun basketball projects I’ve worked on and hopefully we’ll be back in a few months doing this again. It also forces me to recuse myself from commenting on anyone’s preseason ratings for the next few months to avoid biasing prospective voters. That is pretty liberating, actually. Hot-take world can churn out questionable content for the next few months without me.
The 130 incomplete ballots actually did slightly better than the complete ballots, hitting 59.4%. Although, this might not be surprising since they presumably avoided more difficult matchups.