News about a tea competition results awhile back started me considering what a tea competition evaluation and scoring process might be like. As my mind works it would be interesting to compare some of the methodology and evaluation process to the very different but vaguely related process of judging teas against personal preference, when drinking them for enjoyment. I didn't expect a lot of telling overlap to emerge but did eventually get around to researching the subject.
Obvious enough, but evaluation for a tea competition would involve a few basic steps:
1. a consistent brewing methodology for evaluation, under carefully defined and controlled conditions
2. a set of review criteria, categories against which the tea would be evaluated (taste, color, body / mouth-feel, etc.)
3. a scoring system to enable judging which teas are evaluated as superior (although it seems conceivable that a tiered pass-through process could replace numerical scoring)
Really there would be a lot more to it, related to all sorts of controls of conditions, example integrity controls, methodology to ensure consistent judging related to throwing out some "outlier" results, and so on.
the real thing
I attempted to consult "tea experts" in one place such a request would make sense, in a tea professionals oriented LinkedIn group, but that didn't get far, so hopefully some gaps get flagged once this is published instead (the Cunningham's law idea).
The ISO standard
If there is such a thing this must wrap it all up, right? Not so much. ISO 3103:1980 does cover how to brew tea consistently for evaluation, though, described as such:
The method consists in extracting of soluble substances in dried tea leaf, containing in a porcelain or earthenware pot, by means of freshly boiling water, pouring of the liquor into a white porcelain or earthenware bowl, examination of the organoleptic properties of the infused leaf, and of the liquor with or without milk or both.
It is headed in the right direction, but the Wikipedia excerpt--a sample of partial detail summary, mind you--shows up why it just didn't get there (also noted by the Tea with Gary blog awhile back):
The pot should be white porcelain or glazed earthenware and have a partly serrated edge. It should have a lid that fits loosely inside the pot...
If a small pot is used, it should hold a maximum of 150 ml (±4 ml) and must weigh 118 g (±10 g).
2 grams of tea (measured to ±2% accuracy) per 100 ml boiling water is placed into the pot.
Freshly boiling water is poured into the pot to within 4–6 mm of the brim. Allow 20 seconds for water to cool.
The water should be similar to the drinking water where the tea will be consumed
Brewing time is six minutes....
If the test involves milk, then it is added before pouring the infused tea.
Milk added after the pouring of tea is best tasted when the liquid is between 65 - 80 °C.
It's not as bad as it looks. One possible way to go is brewing all the teas exactly in the same way and tasting around the broad difference in how personal preference brewing parameters would vary, and my understanding is that this is still essentially a common approach (maybe just not always using this set).
Tasting based on using milk might seem odd but I have it on good authority that some prominent tea purchasers do exactly that, judge teas in an altered final consumption form. A tea mentor (of sorts) once suggested that it's best to drink tea cooler to taste it, around 60° C, and I'm not certain about any optimum but of course he is right in principle (per my own experience, at least).
More developed standard references
The World of Tea site reference actually includes the ISO 3103 process as one alternative, and suggests using both that methodology and a second review based on more conventional brewing parameters, partially defined as such:
The article goes on to suggest evaluating the following criteria, along with reviewing leaf appearance:
So far so good; reasonable brewing and evaluation guidelines. I'd hoped for a bit more on scoring, some summary rating process, but regardless of what a reference set up for that it would be hard to really improve on a subjective over-all evaluation that compared teas directly. Direct evaluation of limited samples might just be limited when attempting to judge a large number of teas, or to combine multiple judgments from different judges.
This also seems like a good place for an aside about how limiting the number of factors evaluated would also limit the completeness. For example, that methodology suggested review against liquor color, liquor smell, taste, and number of steepings, four factors (and a good start). A more general Tea Vicious reference on how to evaluate teas suggests some others (only partially cited, with their section on methodology omitted entirely):
That would be way too much criteria for judging a lot of teas against each other but the general concepts of feel and finish (aftertaste) might be retained.
Maybe as well to cite a bit of two other references, and then wrap this up. One from Tearroir provided an example of the type of scoring system I'd expected:
The engineer side of me loves this approach. Based on how I evaluate teas myself, related only to preference, I'm not so convinced a lot wouldn't drop out in trying to weight groupings like this though, that it might fail to fully capture how a few strengths or weaknesses can really determine impression. I just tried a Hong Shui tea with an amazing range of great flavor characteristics, and a slight sourness issue that corresponded to how the feel was presented, so even the break-down might not be so simple.
Lastly, the North American Tea Championship, related to the World Tea News / Academy / Expo, provides another good example of evaluation methodology, which they summarize as such:
Each submission into the class is evaluated blind and through organoleptic analysis of the following characteristics: dry leaf, brewed color, brewed aroma, brewed flavor, brewed mouth-feel and brewed harmony. An overall numerical value on a 100-point scale is then calculated based on the ratings of each characteristic above.
Nice! They just don't mention how the actual scoring is calculated, but they do include a fascinating description of how they make judging more consistent by throwing out inconsistent / outlier scores (and you really should read that if you happen to love statistical sampling process). A sample of brewing parameters makes it clear they go with brewing methodology closer to ordinary brewing:
|Breakfast Blend||Origin Authentic||Boil||5||2.5g|
|Bai Hao/ Oriental Beauty||Breakfast Blend||Boil||2/4||2.5g||Steep open bowl; 2 infusions; 3 cupping bowls per tea|
|Chai||Masala Chai||Boil||5||3g||Add 20% warm milk/sugar mix|
That last reference didn't really seem intended to give the full details of the methodology and evaluation process but it's nice they passed on that much.
I had intended to circle this back more to a comparison against how people drink and evaluate teas for enjoyment but it seems as well not do much with that. It's just different.
It's interesting to consider how flaws in tea might factor in to these approaches, or how individual positive aspects might be rated versus a balance of aspects in a tea, but it also seems as well I that don't muddle the review here with too much of my own speculation. Besides, with more expert input or research about this much content I could always write up a "part two" post and ramble on in that one. Scoring systems never went far in these references; lots more room to add to that.