|State Fair's primitive scoring matrix in action|
This is how it goes at most wine competitions: Panels of three to five judges each taste through a flight of wines, make their notes, draw their conclusions, and gather to discuss and decide what kind of medal - if any - each wine warrants. It's worked that way for decades, with few complaints or issues.
Ignoring the maxim that "if it ain't broke, don't fix it," and hoping to impose more structure and standards on this practice, State Fair directors a few years ago came up with "the matrix." In short, it's a large chart that outlines every conceivable combination of medals that a panel of four judges could give a wine. Imagine: Three judges score a wine "gold minus," which basically means they like it a lot but have reservations about whether it really warrants a gold; the fourth judge thinks it worthy of only a bronze medal. According to "the matrix," the wine automatically would be awarded a silver medal. In times past, judges would hash out their differences and come to an agreement all on their own. Chances were pretty good that they also would concur that the wine deserves a silver medal. On the other hand, after debating and retasting the wine, they could agree that it really does warrant a gold medal after all. Or maybe most of them might realize that they were overly generous early on and now feel that the wine justifies just a bronze medal. That sort of second-guessing still is possible at the State Fair, but first "the matrix" has to be consulted, and by the time its determination is calculated judges are so distracted that they just want to get on to the next wine.
Conception of "the matrix," I suspect, was driven at least in part by the State Fair's desire to assign point scores to each wine. Wineries love to brag that this or that wine got 90 or more points, regardless of who gave the points. It's a marketing tool, going back to our school days, when 90 or more points was good enough for an "A" in a math test, the best a student could hope to achieve. By the same token, 90 or more points from the State Fair suggests that the wine got a gold medal, but that isn't necessarily the case; a wine awarded a silver medal also can get 90 or so points, according to "the matrix." Judges who face 80 or so wines during the day would rather spend their time and energy deliberating on the merits or shortcomings of each wine than parsing whether a given entry deserves 90 points or 91 points. "The matrix" takes that chore out of their hands by assigning points to each eventual outcome.
Can you tell that after I spent much of Wednesday at Cal Expo tasting and grading 84 wines I came away a bit miffed about "the matrix?" It's the same feeling I get when dealing with other government agencies - this whole process could be so much faster and simpler and just as reliable if only....
So, a modest proposal: Keep "the matrix," if you must. But bring it into the 21st century. Give each clerk who records each judge's score an electronic tablet programmed with "the matrix." He or she punches in each judge's score as it is given and instantly both clerk and panel will see the consensus medal and score. If they agree with the conclusion, move on to the next wine. If they don't, they can reconsider, debate and refine their judgment. Judges will be happier. State Fair officials should be as tickled; they would get results faster and no less reliably. Consumers? They may not recognize any change, but I've a hunch the results will be more consistent because judges will be devoting more time to pondering the wines and less to figuring their way through the labyrinth.