News & Features
Championship Week
Revisiting the USTA Nationals Forecast
by Dallas Oliver, 15 August 2015
| Share: |  |
|  | |
|
|
Prior to players taking to the courts for the USTA National Championships. Tennis Recruiting put out a set of heat maps to forecast the various age groups. Those forecasts put our rankings on the line - estimating the number of upsets in each draw and showing the chances various players had to make runs through the events.
Yesterday we finished our roundup articles that highlighted the gold ball winners in our coverage of Championship Week, and now is the time to evaluate how we did in our predictions. For a detailed look inside the numbers, you can check out the 2015 USTA Nationals in our post-tournament analysis, which you can access by clicking here. But for those of us who don't want to think too hard, let's scattershoot over the numbers...
Expecting the Unexpected
Right at the top of our heat maps are predictions for the number of expected upsets. For example, there were 191 main-draw matches in Boys 18s, and our system predicted that in 38 of those matches the lower-ranked player would defeat the higher-ranked player. We also predicted 17 matches where seeds would be upset by unseeded or lower-seeded opponents.
These upset predictions are based on the match win probabilities shown in the heat maps. As we have discussed before, if Player A has an 80% chance to defeat Player B, we expect an upset 20% of the time.
Totaling up all upsets across all age divisions, we see that our predictions held up very well:
| Matches | Matches | Expected Upsets | Actual Upsets | Difference | | Main Draw | 1,272 | 243 (19.1%) | 243 (19.1%) | 0 (0.0%) |
| Involving Seeds | 713 | 139 (19.5%) | 130 (18.2%) | 9 (+1.3%) |
| All | 2,627 | 572 (21.8%) | 622 (23.7%) | -50 (-1.9%) |
Of the 1,272 main-draw matches, our system nailed the number of upsets with our estimate: we estimated 243 upsets, and there ended up being exactly 243. Getting it exactly right is actually a little bit spooky. Our estimates for the number of seeds losing to unseeded or lower-seeded players was also quite accurate - our prediction of 139 upsets (19.5%) was within 1.3% of the 130 upsets (18.2%) that actually occurred.
When considering all matches - both main draw and consolation matches - we estimated 572 of the 622 actual upsets, so our estimate of 21.8% was within 1.9% of the 23.7% that actually occurred.
Breaking It Down
The broad numbers look as good as one could hope for, but did we just get lucky? Let's take a closer look at different levels of competition.
We can break down the matches into five groups: 0-10% win probability, 10-20%, 20-30%, 30-40%, and 40-50%. If our forecast is accurate, we would expect underdogs in the 0-10% group to win about 5% of the time, underdogs in the 10-20% group to win about 15% of the time, and so on.
| Group (Win %) | Matches | Expected Upsets | Actual Upsets | Difference | | 0% to 10% | 772 | 35.6 (4.6%) | 46 (6.0%) | -10.4 (-1.4%) |
| 10% to 20% | 555 | 83.6 (15.1%) | 108 (19.5%) | -24.4 (-4.4%) |
| 20% to 30% | 454 | 112.6 (24.8%) | 133 (29.3%) | -20.4 (-4.5%) |
| 30% to 40% | 417 | 146.0 (35.0%) | 144 (34.5%) | +2.0 (+0.5%) |
| 40% to 50% | 429 | 193.7 (45.2%) | 191 (44.5%) | +2.7 (+0.7%) |
| All Matches | 2,627 | 572 (21.8%) | 622 (23.7%) | -50 (-1.9%) |
Again, the numbers work out as well as could be expected. Notice that the upset percentages fall right in line with where you would want them. All of our estimates were within 5% of the actual upsets. Our system once again does a good job knowing which match-ups are more evenly-matched and which ones have clear favorites.
Revisiting the Predictions
In addition to general statistics, we used our forecast to come up with four bold predictions for the Nationals. Let's take a retrospective look at our crystal ball...
i) Players with the best chance of advancing to the semifinals are Stefan Kozlov, Saud Alhogbani, Tornado Alicia Black and Alexa Noel.
Three out of four of these kids met our expectations. Stefan Kozlov, Saud Alhogbani, and Tornado Alicia Black all went on to reach the finals of their respective tournaments, and Alhogbani won a national title. Only Noel went out earlier than expected - losing a fourth round match to Blue Chip 8th-grader Naomi Cheong of Las Vegas who went on to reach the finals of the Girls 14s.
ii) The field in the Girls 16s looks to be wide open. Our forecast likes the chances for unseeded Abigail Desiatnikov to make a deep run, but it does not think any player has better than a 17% chance of winning the title.
We don't like to cherry pick data, but in this case we called the Girls 16s beforehand. As predicted, the tournament was a wild one with 21 upsets of seeded players - more than any other division - and 42 upsets overall. The Girls 16s was also the only age division to feature unseeded players in the semifinals - and there were actually two. Marlee Zein had an amazing run, recording five straight upsets to reach the semifinals.
The final in the Girls 16s featured 15th-seeded Whitney Osuigwe and unseeded Abigail Desiatnikov. While neither of these players was highly seeded, our system actually gave those two the highest likelihood of winning it all before the tournament began.
Desiatnikov, who turned fourteen earlier this year, came out on top. Our system considered her the favorite beforehand with a 17% chance to win it all. Of the 88 events that we have predicted, Desiatnikov is the only unseeded player our system has ever called out as the pre-tournament favorite. Although she came into the tournament unseeded - with 23 competitors in the draw holding higher UTR ratings - she knocked off five seeds en route to her second gold ball.
iii) Top seeds with the toughest roads to the semifinals are Daniel Sharygin, Nicole Mossmer, Phillip Deaton, and Jenna DeFalco. All four of these players had tough draws, and our predictions held true to form. Only DeFalco got as far as the quarterfinals, where she dropped a competitive 6-4, 6-4 decision to Sydni Ratliff.
iv) The draw with the most interesting first-round matchups is the Boys 16s. That draw features twelve matches between closely-ranked players where the probability is 60%-40% or tighter. We also project 17 upsets in the 64 first-round matches on Saturday.
The first round of the 16s saw plenty of unexpected results. Sixteen of the 64 first-round matches resulted in upsets. It is notable that in the first-round "coin-flip" matches that were tighter than 60%-40%, the higher-ranked player won eleven of twelve.
Updating the Numbers
We have added the USTA Nationals as the fourteenth tournament in our analysis of international, national, and sectional events. You can click here to see analysis for fourteen different tournaments. For each of these tournaments, we used player ratings immediately prior to the tournament start as the basis of our predictions, and the analysis shows how well the predictions held up.
These studies also show analysis for the USTA PPR rankings, Universal Tennis Ratings, and the International Tennis Federation where appropriate. As you can see, the Tennis Recruiting system compares favorably with other junior tennis ranking and rating systems.
That's a Wrap
This article concludes our coverage of Championship Week. As always, The Tennis Recruiting Network is proud to be able to bring you wrap-ups of all of these events, and we want to thank all of our great authors for their contributions. We hope you have enjoyed reading all of their quality content.
Take a few minutes to read over any of articles you might have missed - and celebrate the best of the best in American junior tennis!
Leave a Comment
More Ranking Articles
19-Sep-2017
Comparing Rating Algorithms
During the U.S. Open we posted information on social media about matchups involving American players - previewing matches by displaying rating and ranking differences and highlighting wins by underdogs. You might have noticed that the various rating systems often agreed on their favorites. But what about the close matches where they disagreed? Let's take a look ...
13-Jul-2016
Behind the Tennis Recruiting Rankings
This week marks the 569th consecutive week that Tennis Recruiting has
put out graduation-based rankings of American junior boys and girls.
Rankings are front and center on Tennis Recruiting, and people often ask
us how our rankings work. Today we describe our ranking system ...
19-May-2016
What Is An Upset?
Tennis Recruiting is a website that rates and ranks junior tennis
players. One of the questions we get most often from our users is,
"What exactly is an upset?" There are many possible
definitions of an upset - this article explores the question and
puts forward an answer.
The lifeblood of Tennis Recruiting is its rankings - and our team has been ranking tennis players for more than two decades. Learn about tennis rankings in general - as well as our best-of-breed ranking system.
