Thursday, August 19, 2010

The Fine Point In Wine Competitions

The yoga instructor this morning provided just the clarity I'd been looking for since returning last night from two days of judging wine in Lake County: "I can tell you where to go, but only you can experience the pose," she said. Substitute "wine" for "pose" and you have the essential goal of wine criticism, whether that judgment is handed down by a critic working in isolation or panelists who collectively come to a consensus at a wine competition. By whatever route, their opinions are cairns along a trail, providing direction to a peak experience that can leave the hopeful hiker enlightened, or maybe just thoroughly exhausted and exasperated. We've all been there. No method of evaluating wine and passing judgment is infallible, and no one, least of all the consumer, should look upon 95 points for this wine or a gold medal for that wine as the last word in quality and value. They are signs pointing to a wine that someone or some group, hopefully well informed and well intentioned, has found to be exceptionally expressive, and thus warrants discovery by other wine enthusiasts, who ultimately may or may not agree.

This year, several wine competitions, stung by criticism that their conclusions too often have been shaky, and mindful that entries generally have been down, have taken steps toward improving the consistency, reliability and relevance of their panels. In Dallas, judges tasted white wines after reds, contrary to the usual approach, and were given the chance to enroll in pre-dawn yoga sessions in hopes of keeping both body and mind limber and balanced. At Riverside and Long Beach, petite sirahs were grouped by place of origin in hopes of refining the standards by which wines generally are evaluated. And at Lake County, medals were eschewed altogether as the competition switched strictly to a point system. That is, judges were to assign each wine a value up to 100 points. Though the results are being reported in terms of points only, judges were mindful that 90 to 100 points was the equivalent of a classroom "A," or gold medal, while 85 to 89 points represented a "B" grade, or silver medal, and 80 to 84 points amounted to a "C," or bronze medal.

I'm not a fan of assigning points to wines, largely because the point approach suggests an objective precision to which the aesthetic appreciation of wine doesn't apply. I do appreciate that points constitute a shorthand means of conveying overall judgment, and I do recognize their popularity, so I looked forward to seeing how the approach would work in a group setting.

At most wine competitions, judges sit on panels that get to taste just a fraction of the wines entered. At Lake County, however, they did something else novel. Director Ray Johnson arranged the format so all judges got to taste all 146 wines, though not simultaneously. Two panels of five judges each were convened. Each panel came to a consensus on how many points should be awarded each wine, and their results subsequently were reconciled to come to one average score per wine. Under this format, the debates that ensued at the end of each flight seemed to take longer than they do when arguments focus on whether a wine deserves a bronze medal or a silver. At times, I found myself wondering whether mental fatigue from adding and subtracting all those numbers, then discussing whether a wine should get 83 points or 85, might be as distracting to concentrated wine evaluation as palate fatigue.

As I look over the results, I'm not sure whether wine consumers and Lake County vintners are better served by assigning points instead of medals. We ended up giving a lot of wines between 80 and 89 points, and consumers tend not to get excited by a wine if it scores anything less than 90 points. Furthermore, in at least two classes the overall results this year aren't as impressive as they were last year, when Lake County held its first modern commercial wine competition. Sauvignon blanc, for example, is a variety by which Lake County long has done exceptionally well. Last year, of the 21 sauvignon blancs that were judged, four got gold medals. This year, of the 20 sauvignon blancs judged, just one scored 90 or more points, the equivalent of a gold medal. Petite sirah is another variety for which Lake County historically has been recognized. Last year, five of the 13 petite sirahs in the competition got gold medals. This year, two of the 12 petite sirahs that were judged scored 90 or more points. Tougher judges this year? Weaker wines? Or did using points instead of medals somehow affect the perception or commitment of judges?

Unlike at most competitions, Lake County didn't have a taste-off among judges to determine the best wines in the judging. Instead, the sweepstakes winners were the wines that scored the most points in each of three categories - white, rose and red. For the record, the best white wine was the ripe, refreshing, apricot-scented and off-dry Shooting Star 2008 Lake County Devoto Vineyard Riesling (93 points, $12), a surprise, given that Lake County commonly is seen as too warm for this cool-climate variety. The rose sweeptakes went to the Gregory Graham 2009 Lake County Crimson Hill Vineyard Rose (88.3 points, $12), a forward and full-bodied interpretation of the genre, blended from grenache and syrah. The red sweepstakes went to the Six Sigma 2005 Lake County Diamond Mine Vineyards Tempranillo (93.2 points, $42), which while still firm with tannin remains a graceful and complex representative of the varietal, and easily could be misinterpreted as coming from Spain rather than California.

The competition formally is known as the People's Choice Wine Awards. The 10 judges who assembled at Langtry Estate and Vineyards in Middletown merely whittled the field of 146 entries to a more manageable 56 that consumers will be able to taste blind and vote on during a Sept. 26 tasting at Six Sigma Ranch & Winery in Lower Lake. Tickets for that event are $25 in advance, $35 at the door, and are available online here.


  1. Your problem is you've chosen effite Yoga rather than real labor. My suggestion is chop wood, dig ditches or (god forbid) pick grapes.
    You'll (I promise) enter into a whole new realm of tasting wine as well life.
    Plus, it may curl you hair.

  2. Hey Mike nice to see that the point system is a challenge on the mind as well as the soul. The thoughtful commentary says it all. So the highest score was a 93 for a tempranillo and a riesling...

  3. Right, Tim, the tempranillo got 93.2 points, the riesling 93, the rose 88.3. That preciseness reminds me that I could have been more precise in the original posting. The range of 90 to 95 points was the equivalent of a gold, the range of 96 to 100 points the equivalent of a double-gold. I don't know why no wine in the Lake County competition got a double-gold medal, but I have a hunch that when the ranking of 10 judges rather than the usual three to five is calculated there could be a flattening effect.

  4. Robert Hodgson (bob@fieldbrookwinery.comAugust 24, 2010 at 11:20 AM

    Based on some preliminary analyses, there seems to be considerable improvement in panel coherence when wine judges use a point-scale rather than a medal-driven one. Using a technique common in psychological and medical research, panel coherence may be expressed by an “intraclass correlation coefficient.” Using this method, judging panels at the Humboldt County Fair wine competition (albeit a very small competition and one in which you participated) achieved ICC’s of 0.6 to 0.7, considered “good” by technical standards. By contrast, the ICC’s at a large competition where judges used award-based criteria (No-Award to Gold) the resulting ICC’s were between 0.1 and 0.2, or “poor.”

    It would be nice to have access to the Lake County data to see if your impression of better panel coherence holds up under analytical scrutiny.