Today we begin a series of articles about personalized cigar recommendations and what we consider to be the key success factors for a personalized service. We also share with you some of the results of our latest customer satisfaction survey…
Science vs. Pragmatism
Some of you may already know that we operate as a lean-agile team, which means continuous improvement is a core value for us. In alignment with this value, we recently conducted a non-scientific survey among our members to understand where we’re doing well and where we can further improve our service. Based on a fascinating academic study we recently read, we selected the areas that are considered to be key success factors for any personalized service:
We thank our members for the terrific feedback we received on all these fronts. Today, we’ll focus a bit on the first one – Accuracy. For us, accuracy simply means that when we recommend a cigar, the consumer does in fact enjoy it.
Taking It Personally
One compelling competitive advantage that consumer businesses have today lies in providing customers a personalized experience, because it increases retention rate and repeat customers, which ultimately translates into higher revenue. Personalized recommendation engines, in fact, are often referred to as predictive offers or next best offers, and they’re everywhere you go online, from Netflix and YouTube to Amazon and Walmart. However, for a consumer, it’s difficult to differentiate what’s behind any recommendation. Let’s try to shed some light on the topic.
What drives “accuracy” in personalized cigar recommendations?
Typically in online businesses, there can be many complex and sophisticated recommendation algorithms. We believe that there are two basic questions that you can ask yourself when you are confronted with personalized cigar recommendations:
What motivations drive the selection of cigars recommended to me?
- The key motive for most e-commerce sites is revenue, hence the algorithm will be chosen to maximize the sales it generates from the recommendations
- The key metric for Cigar Sense is member satisfaction; the algorithm is chosen and optimized to help our members get spot-on recommendations that truly suit their preference. We don’t sell cigars (nor advertisements) and this makes a big difference in our focus.
What depth of knowledge of my actual preferences does the website use in formulating my recommendations?
On most sites, I could see a cigar recommended to me because, for example:
- based on very high-level product descriptions, it is similar to cigars I bought in the past,
- I have viewed that cigar more than once,
- other customers who like other cigars that I like, also like that cigar,
- that cigar is a “top seller”
- the recommender has a lot of that cigar in stock and wants to “move them”.
Recommendation engines using these rules are fraught with error, due to the gross inference that anything I buy is something I like, for example:
- I may have purchased cigars that I dislike, to give as a gift to someone else who likes them.
- My husband could have used my account to buy cigars that he likes, we have a very different taste!
- I may have liked one cigar with a Connecticut wrapper, but that does not imply I will like all the other Connecticut wrapped cigars in the store’s inventory.
The more a website knows about cigars, its customers and their interactions, the better the quality recommendations it can formulate. At Cigar Sense, we use:
- more than 150 attributes for each cigar, all coming from consolidated descriptive analyses of multiple samples blind-tested by a panel of international experts
- more than 65 attributes for each member, which are matched with the cigars’ attributes, and only those showing a high degree of fit are presented as personalized recommendations to our members
- direct feedback from our members on which characteristics they like or dislike in a cigar, which helps us improve the recommendations they receive.
But how well do we achieve the accuracy goal? We asked our members to rate the following statement:
Q: Cigar Sense recommends cigars that truly match my personal preferences
We have systematic ways for paid users to give us feedback on how much they like the cigars we recommend, and this is actually our most revered business metric. Whenever our members enter a Tasting Note, they indicate whether they liked the cigar and how much. We match that up with the Cigar Sense Fit % for that member and that cigar, and use this to track “true positives” meaning we recommended the cigar and the member liked it, as well as “true negatives”, meaning we do not recommend the cigar to the member, and the user does not like it. We also track each time we have a false positive or false negative, which fortunately is pretty rare. When it does happen, we analyze it carefully so we can learn from it, and find ways to further improve our methods.
In our recent survey, our members gave us an average score of 9 out of 10. This tracks very closely to our recommendation success metric described above, which routinely exceeds 90%. It confirms that our members agree that our service is a reliable predictor of whether they will like a certain cigar.
But that’s not all, folks…
While we’re happy with these results, we are not nearly satisfied yet. We continue to focus on ways to expand our database of fine cigars, to make the service better, and to bring it to more members. We recently rolled out a simplified 3-step on-boarding process, and made it free for everyone to get started with Cigar Sense. Btw, if you have friends and fellow cigar lovers who have not yet tried Cigar Sense, please give ‘em a nudge. It will cost them nothing to try it, and will help us make the service even better for everyone. We also have a friend referral program that can make you earn more money than you paid us!
We will also ship soon the beautiful ashtray to the randomly selected winner, among those members who responded to all questions of our survey.