Reviewers Roles and Responsibilities

Reviewers Roles and Responsibilities
Published December 7, 2017 8:19 am PT


The cigar industry avails itself of many product reviewers. Smoking a fine cigar is fun and everybody is entitled to their share of glory. We would like to deep-dive on some of the implications that different types of reviews have on consumers and manufacturers.

Solo reviewers roles and responsibilities

Solo reviews are subjective opinions of one person. An expert such as a wine sommelier is a solo reviewer when she communicates her opinion on a wine. She typically adds an array of flowery prose to the core description of the wine.  It depends greatly on her level of expertise and integrity.  Reading such a review, the consumer might feel more educated. She might feel the sommelier adds value to what she knows and this is a noble role for a solo reviewer.

Unfortunately not every solo reviewer has the objective to educate others, and each has different standards for both competence and humility. Based on our own individual standards, we may each regard some reviewers as more “noise” than signal.

Probably the extreme case in the soloist reviews production is when the author sells or has a benefit in promoting the “reviewed” cigars. In fact, rather than reviews, she writes sales pitches. Of course, the maxim caveat emptor applies.  In addition, such reviewer type often only reviews products she herself has a stake in.

Reviewers Roles and Responsibilities

Signals? To me they are “good, clean and fair” (to paraphrase Carlo Petrini, legendary founder of the Slow Food movement) messages that create a change in the consumer, which enable the consumer to know, trust and use her own power of discernment.

As it appears evident based on our data, The odds that any given cigar lover will like any particular good quality cigar are 60% or less Click To TweetThis is simply due to the intrinsic wide differences in everyone’s preferences.

You could say that a solo reviewer lowers those odds. A solo review is always a biased review, even if the reviewer tasted many samples of the same cigar before writing. Don’t get me wrong: everybody is biased. A professional taster has biases. The big difference between a professional trained taster and a non trained taster is that the trained taster has conscious competence and is aware of her own biases. This allows her to take corrective actions which, in the best case, consist in asking other tasters’ advice.

Group reviewers roles and responsibilities

Crowd-sourcing

If you are not familiar with the term, crowd-sourcing is the gathering of consumer opinions, and is often done for products purchased online. You might have seen a request for a review after having purchased cigars from some retailers.  The most critical aspect is the quality of the data that is gathered. I could have purchased a product to give as gift, and when the retailer asks me to review the product, I can choose not to do it, ask the person who actually used it to give me her feedback, or just enter anything. Or I could be asked to provide detailed input on a cigar and, if I am not skilled, I may in good faith provide inaccurate information.

The usability of crowd-sourced data depends greatly on how well it is consolidated and managed.  We can see numerous examples showing that if the crowd is large enough, it will collectively express every possible opinion, rendering the combined information only confusing and contradictory. An intelligent method is needed to parse such data and separate out a usable signal from the noise of the crowd.

Panel reviews

Panel reviews typically involve multiple tasters of different backgrounds and regions, tasting samples and providing structured feedback on their experience.  The data is then consolidated and managed to ensure that it provides useful insights into the differentiating characteristics of each product.  Further, any panel will benefit from the dilution of individual biases across samples tested by different tasters. Even in consumer panels, the drastic effect of one or few individuals not having liked the cigar at all will be smoothed out.

In addition to certain bias exclusion, one more reason panel reviews generally provide more reliable results is in the thoughtful design of their data collection and management process. In such a process, both tastings and publications have a higher cost, but they typically provide a more reliable guide to the consumer. The quality of structured input data enables, logically, good output data.

Blind tasting panel reviews

“The blindfolded Lady Justice represents blind incorruptibility and impartiality. The blindfold represents objectivity and the exclusion of fear or favor, of the influence of money, wealth, power, or identity. The balance scales represent the weighing of evidence.” I believe the quality of every cigar should be judged based on impartiality principles, by fairly weighing the evidence of multiple tests. This blind tasting process is key to ensuring that panelist opinions are not swayed by any individual prejudice or profit motive.

Biases need to be recognized and off-set. A good taster, reviewer or professional of any sort will constantly strive to know, challenge and balance her own biases.

Biases are not only in the knowledge and the use of the language, they are often in our cultural background, but can be simply genetic as well, presenting in various disconnects, including in our personal perception threshold. This means that some people will find a cigar (or, better, a similar sample of a cigar produced with high consistency standards) very spicy, some not at all spicy, some will not be able to accurately identify certain aromas, tastes, tactile perceptions, strength level, etc.

Some of these can be trained and improved with time, however everybody usually has a few perceptions they simply cannot perceive or improve with training – usually due to genetic variations between us.

The true expert learns what her personal shortcomings are, and understands them not as a weakness, but an important aspect of the human experience. An expert thus also knows that she has to balance her own, incomplete perception with the one of other experts in order to create a complete image.

You can be an expert, but you maybe focus on specific quality parameters, which is different from focusing on describing a cigar. For example, you could think that a good quality cigar always presents a good balance between certain qualitative taste and olfactory characteristics (which implies a yes or no answer or a short multiple choice). For me, in this case, what is important is that analysts are able to distinguish the individual taste and olfactory characteristics, describe them and accept that “balance” may be a personal, subjective thing.

Training and focus are the solutions.

Sight and touch are causes of initial biases when tasting a cigar. Technically speaking I should confine all analysts, each into an individual room, blindfold them (a numbered white band is not sufficient to eliminate all visual biases) and tie their hands behind their back. Then approach the cigar to their nose and to their mouth intermittently. This would allow controlling the tasting environment, the smoking pace, how they reset their palate and nose, how much time elapses from meals, making sure they don’t talk to each other, etc.

However, we are not machines and, as far as I know, there is no machine that reproduces the smoking retro-olfaction effects. Therefore we need to do what is humanly possible to do, in the best of our knowledge and honesty. It is up to each analyst to keep the smoking environment and parameters as constant as possible, which starts with a comprehensive knowledge of themselves.

How the data is consolidated is another aspect: it is preferable to be “blindfolded” in this process step too. This is possible thanks to software and does relieve the panel leader from biases that may arise from a good relationship with a manufacturer.

The good news is that there are many manufacturers seeking meaningful, comprehensive feedback beyond ratings, rather than dismissing such analyses.

Summing Up

We hope this article was of value.  We also hope that as reviewers give consideration to the impact they have on both consumers and manufacturers, it will encourage them to continue improving their methods, to the benefit of all.

 

References:
– Cigar Sensory Analysis Training, Cigar Sense, November 2017 (contributor: CH, anonymous analyst)
Wikipedia/Lady_Justice

Share this with your friends!