Tea evaluation and scoring

Originally published on the TChing site.

News about a tea competition results awhile back started me considering what a tea competition evaluation and scoring process might be like.  As my mind works it would be interesting to compare some of the methodology and evaluation process to the very different but vaguely related process of judging teas against personal preference, when drinking them for enjoyment.  I didn't expect a lot of telling overlap to emerge but did eventually get around to researching the subject.

Overview

Obvious enough, but evaluation for a tea competition would involve a few basic steps:

1.  a consistent brewing methodology for evaluation, under carefully defined and controlled conditions

2.  a set of review criteria, categories against which the tea would be evaluated (taste, color, body / mouth-feel, etc.)

3.  a scoring system to enable judging which teas are evaluated as superior (although it seems conceivable that a tiered pass-through process could replace numerical scoring)


Really there would be a lot more to it, related to all sorts of controls of conditions, example integrity controls, methodology to ensure consistent judging related to throwing out some "outlier" results, and so on.

the real thing

I suppose this is a good place to mention that I have absolutely no experience with such a process, which is why researching this subject appealed to me.  The prior categories and functions might benefit from restatement from someone with more experience and training, but that was one goal, to consult and summarize research sources.


I attempted to consult "tea experts" in one place such a request would make sense, in a tea professionals oriented LinkedIn group, but that didn't get far, so hopefully some gaps get flagged once this is published instead (the Cunningham's law idea).

The ISO standard

If there is such a thing this must wrap it all up, right?  Not so much.  ISO 3103:1980 does cover how to brew tea consistently for evaluation, though, described as such:

The method consists in extracting of soluble substances in dried tea leaf, containing in a porcelain or earthenware pot, by means of freshly boiling water, pouring of the liquor into a white porcelain or earthenware bowl, examination of the organoleptic properties of the infused leaf, and of the liquor with or without milk or both.


It is headed in the right direction, but the Wikipedia excerpt--a sample of partial detail summary, mind you--shows up why it just didn't get there (also noted by the Tea with Gary blog awhile back):


The pot should be white porcelain or glazed earthenware and have a partly serrated edge. It should have a lid that fits loosely inside the pot...
If a small pot is used, it should hold a maximum of 150 ml (±4 ml) and must weigh 118 g (±10 g).
2 grams of tea (measured to ±2% accuracy) per 100 ml boiling water is placed into the pot.
Freshly boiling water is poured into the pot to within 4–6 mm of the brim. Allow 20 seconds for water to cool.
The water should be similar to the drinking water where the tea will be consumed
Brewing time is six minutes....
If the test involves milk, then it is added before pouring the infused tea.
Milk added after the pouring of tea is best tasted when the liquid is between 65 - 80 °C.


It's not as bad as it looks.  One possible way to go is brewing all the teas exactly in the same way and tasting around the broad difference in how personal preference brewing parameters would vary, and my understanding is that this is still essentially a common approach (maybe just not always using this set).

Tasting based on using milk might seem odd but I have it on good authority that some prominent tea purchasers do exactly that, judge teas in an altered final consumption form.  A tea mentor (of sorts) once suggested that it's best to drink tea cooler to taste it, around 60° C, and I'm not certain about any optimum but of course he is right in principle (per my own experience, at least).

More developed standard references

There must be a lot more but Google did identify a few for a starting point.

The World of Tea site reference actually includes the ISO 3103 process as one alternative, and suggests using both that methodology and a second review based on more conventional brewing parameters, partially defined as such:

Tea Type	Temperature	Time for First Steeping
Green Tea	160F/ C	1min  (Yellow and White teas omitted)

Oolong Tea	190F/ 88C	1.5min
Black Tea	205F/ 96C	1min

The article goes on to suggest evaluating the following criteria, along with reviewing leaf appearance:

1.What color is the liquor?

2. What does the liquor smell like? (How does it change from steep to steep?)

3. What does the liquor taste like? (How does it change from steep to steep?)

4. How many steepings can the tea withstand and still produce acceptable flavor?

So far so good; reasonable brewing and evaluation guidelines.  I'd hoped for a bit more on scoring, some summary rating process, but regardless of what a reference set up for that it would be hard to really improve on a subjective over-all evaluation that compared teas directly.  Direct evaluation of limited samples might just be limited when attempting to judge a large number of teas, or to combine multiple judgments from different judges.

This also seems like a good place for an aside about how limiting the number of factors evaluated would also limit the completeness.  For example, that methodology suggested review against liquor color, liquor smell, taste, and number of steepings, four factors (and a good start).  A more general Tea Vicious reference on how to evaluate teas suggests some others (only partially cited, with their section on methodology omitted entirely):

• How does the inside of the mug lid smell?
• Does the aroma linger? Which part of it lingers longer? Which shorter?
• Does any part of the aroma remind you of another substance?
• Does any part of the taste come first and some other later?
• What do the different parts of the taste remind you of?
• Which part of your tongue gives you the sensation of the taste?
• How does the taste stay?
• Does the taste change during the process of tasting?
• What is the texture of the tea liquor?

That would be way too much criteria for judging a lot of teas against each other but the general concepts of feel and finish (aftertaste) might be retained.

Maybe as well to cite a bit of two other references, and then wrap this up.  One from Tearroir provided an example of the type of scoring system I'd expected:

Each tea receives a base of 60 points, and we award each tea up to 40 points based on the following categories:

Color and appearance of tea pre-brew: 5 points
Aroma: 5 points
Flavor and Mid Palate: 10 points
Mouthfeel and Finish: 10 Points
Overall quality level: 10 points

The engineer side of me loves this approach.  Based on how I evaluate teas myself, related only to preference, I'm not so convinced a lot wouldn't drop out in trying to weight groupings like this though, that it might fail to fully capture how a few strengths or weaknesses can really determine impression.  I just tried a Hong Shui tea with an amazing range of great flavor characteristics, and a slight sourness issue that corresponded to how the feel was presented, so even the break-down might not be so simple.

Lastly, the North American Tea Championship, related to the World Tea News / Academy / Expo, provides another good example of evaluation methodology, which they summarize as such:

Each submission into the class is evaluated blind and through organoleptic analysis of the following characteristics: dry leaf, brewed color, brewed aroma, brewed flavor, brewed mouth-feel and brewed harmony. An overall numerical value on a 100-point scale is then calculated based on the ratings of each characteristic above.


Nice!  They just don't mention how the actual scoring is calculated, but they do include a fascinating description of how they make judging more consistent by throwing out inconsistent / outlier scores (and you really should read that if you happen to love statistical sampling process).  A sample of brewing parameters makes it clear they go with brewing methodology closer to ordinary brewing:

Category	Type	Temp	Minutes	Weight	Special
Assam	Origin Authentic	Boil	5	2.5g
Breakfast Blend	Origin Authentic	Boil	5	2.5g
Bai Hao/ Oriental Beauty	Breakfast Blend	Boil	2/4	2.5g	Steep open bowl; 2 infusions; 3 cupping bowls per tea
Chai	Masala Chai	Boil	5	3g	Add 20% warm milk/sugar mix

That partial citation of "Signature Famous Teas" doesn't do parameter choices review justice but it's probably saying enough to add that green teas are cited as between 175 and 180, depending on type.  Interesting to also note World of Tea went down to 160 on their guidelines for green tea brewing; make of the difference what you will.

That last reference didn't really seem intended to give the full details of the methodology and evaluation process but it's nice they passed on that much.

Conclusion

I had intended to circle this back more to a comparison against how people drink and evaluate teas for enjoyment but it seems as well not do much with that.  It's just different.  

It's interesting to consider how flaws in tea might factor in to these approaches, or how individual positive aspects might be rated versus a balance of aspects in a tea, but it also seems as well I that don't muddle the review here with too much of my own speculation.  Besides, with more expert input or research about this much content I could always write up a "part two" post and ramble on in that one.  Scoring systems never went far in these references; lots more room to add to that.