Guide by: Jessyca Frederick
Inspired by frequent questions from friends and family about different wines and wine practices, I write Useful Wine Guides so that people I don’t know might benefit from my knowledge and desire to share information, too.
Thanks to The Wirecutter, the New York Times-owned reviews juggernaut, it has become a best practice in the review industry to share with readers how we evaluate products for review. Here’s what I do when I review a wine club.
When a wine club is interested in having their service reviewed at WineClubReviews.net, they typically reach out and ask if they can send a sample. Sometimes I reach out to wine clubs that I really want to work with to get those samples.
In some cases, like for personalized wine clubs like Firstleaf or Picked by Wine.com, I am asked to create an account and credits are added to the account so I can customize it as a customer would.
Wine samples are sent (and credits provided) at no cost to me. I do not subscribe anonymously to wine clubs and I don’t pay for my subscriptions. You can read more about why I do it this way here.
When I first receive a new shipment of wine, whether from a new wine club or an existing membership, I open the box of wine to check for damage. It happens rather infrequently, but it does happen as those FedEx and UPS delivery drivers aren’t always so careful with our packages.
If a package has damage, I alert the company. If they don’t immediately make it right, that’s a red flag, but honestly, that’s never happened. Sometimes they’ll ask for photographic proof — not because they don’t believe me (or the customer) but because they’re going to initiate some kind of claim process either with an insurer or the delivery company.
For safely-delivered packages, I carefully unbox and photograph them as I go. I rarely publish these photos because the packaging these days is almost all the same — molded fiberboard trays protect the wine inside standard-sized boxes. Sometimes wine comes in styrofoam to protect it from extreme heat or extreme cold. I live in the desert where extreme heat is the normal condition for six months out of the year, so I receive more styrofoam than the average customer would.
After the wine is unboxed it goes into our wine cellar (it holds around 400 bottles) for photography. These photographs appear on every review throughout the site.
After we evaluate what we’ve received, we determine a fair retail price for the wine and catalog it. I keep records of every wine I have received and consumed. The consumption records are available to the public here.
When wine critics and sommeliers taste wine, they often do so in warehouse or office settings. Where you are and what you’re eating (or have eaten) when tasting wines radically changes your opinion of them. With that in mind, we specifically review wines as our readers would – at home, with dinner and after, or sometimes at a party or a restaurant.
Each week my husband and I plan our meals for the week ahead of a weekly grocery shopping trip. At this time we slot wine into our tasting schedule, typically paired with our weekly meals. I cook three or four days a week and I carefully select meals and wines to go together because I like the experience of a well-paired wine and dinner. For nights where we have takeout that won’t go well with wine, we plan to drink the wine on its own after dinner (usually sparkling wines or sweeter wines, sometimes a beautiful red wine that deserves to be savored).
We bring wines to a monthly dinner party with friends who also weigh in on the wines. We bring wine to restaurants (something that is easy and common in California, but not everywhere). When my father-in-law is here in the winter, he gets to taste the wine, too. He’s a great litmus test for us because he pretty much only drinks wine under $15 a bottle.
Sometimes it takes us a week or two, or even three, to get through all the wines in a shipment.
We almost always drink wine at “the correct temperature.” For red wines, this is 55-65 degrees and for white wines this is 50-65 degrees (sometimes warmer depending on the varietal and producer). Refrigerator temp is too cold for red wines, it dampens the good flavors and hides the bad ones. For white wines, 10 minutes on the counter is good enough to get to drinking temp after storage in the fridge. Side note: don’t store wines with corks in the fridge for more than a month.
Before we drink a wine, we often decant it. We do this with the wines we’ve procured on our wine tasting trips and we do this with wine club wines. We generally decant young red wines 30-60 minutes, and sometimes older white wines (3-5 years old) get a quick decant (~15 minutes).
After decanting, it gets poured into a glass and photographed with the bottle. This is a newer practice I added last year, so many reviews on the site don’t yet have these photos. I also, even more recently, started photographing the back labels, too. This is an important part of marketing transparency, especially for wine where the front label isn’t even legally required, much less required to have useful information on it.
Then, we sit down to dinner with a glass of whatever we’re reviewing. We sniff, swirl, and sip before we taste any food, and then consider how the wine evolves through the meal. We discuss the wine as we eat, to see if either of us detects anything off, considering balance, acidity, fruit-forwardness, interest, or boringness.
Typically I head back to my desk for a while after dinner where I continue to sip on a wine — because that definitely makes it easier to work in the evening. Then it’s off to the couch to unwind for the day with the last glass of wine.
I sometimes write contemporaneous notes about these wines, but I sometimes don’t. It depends a lot on what kind of wine subscription I’m reviewing. For Wine Access, where the wines are very important and of very good quality, I will write notes while drinking the wine. For others, like Firstleaf, I go rate the wine on the site right away. For sites that don’t have ratings and ship wines under $20 a bottle, I don’t keep notes, just log my general impressions of whether the wine is a reasonable value or not (and whether it’s terrible or acceptable).
This may sound counterintuitive, but I historically have avoided reviewing the wine from wine clubs. Wine preference is subjective and highly personal. Just because I like a wine (or all of the wine from a club) doesn’t mean you will. Sometimes I like an oddball wine just because it’s different, but for someone who prefers familiar tastes, this would not be a good experience.
Most other wine club reviewers say “the wine is good” or “the wine is okay” or “the wine is great.” This is a disservice to you. My aforementioned father-in-law is very happy calling wines “good” when I call them “meh.” Neither of us is right and we’re both right.
Instead of focusing on how a particular wine tastes, which is the domain of people more trained than I am in picking out aromas and flavors, I prefer to discuss some of its more objective qualities. Typically I highlight:
Each wine I taste from a wine club contributes to my overall impression of quality and value, and this is how I build my scores for “wine quality,” “wine price,” and “wine curation.”
One of the criticisms of wine clubs in general is that you’re overpaying for wine somebody couldn’t sell at their winery. I’m not privy to the backroom dealings between wineries and wine clubs, so I can’t attest to how often this is actually true, but based on some of my own experiences, I don’t think it is often the case.
Most wine clubs act like buyers like any other retailer (restaurants and sommeliers included) and choose to carry wine they think is good and offers either good value for the money or some other important value, like prestige or uniqueness. In fact, because wine clubs buy such a limited set of wine and their business is based on loyalty, good ones are extra careful not to send out swill.
The other part of that complaint — that you’re overpaying — is the very reason a review site like mine adds value to your research process before joining a wine club. In some cases you absolutely are overpaying. In some cases you can get a seriously good deal. It depends a lot on the wine club and what their goals are.
Regardless, you may be curious about how I can tell if a wine club is charging too much for the wine they ship. To determine a fair retail price for the club wine we receive, we look at a variety of factors:
We also evaluate the total price you pay for both wine and shipping, for each bottle. We do this because some wine clubs offer “free shipping” (which is not actually free, it’s part of the price you’re paying per bottle) and some charge you extra for the shipping. For example, one wine club might charge $25 for a bottle with a suggested retail price of $20 while another charges a “discounted” $18 plus $7 to ship the same wine. Either way, you’re still paying $25 — and getting the same value for the same wine.
With all of the things you might be thinking about before joining a wine club, figuring out how the profit margins break down probably shouldn’t be one of them. You really just need to know if you’re overpaying or not, and whether you’re getting a deal or not.
Pro tip: the two best wine club values are: Wired For Wine’s 90+ Club (International selection) and the Case Club series from The California Wine Club (domestic, mostly California selection). Both ship wines for around $15 a bottle at steep discounts from retail.
When I first started this website, the web was still the wild west and very little was expected of reviewers, much less of wine club reviewers (there were maybe a half-dozen of us, mostly affiliate marketers who were more enamored with the big commissions than the wine itself).
I started this site to help promote a wine club owned by a friend of a friend. I love the Wayback Machine but it didn’t have any good screenshots of my site from those days. It was a one-page website with a grid of club comparisons. I can’t remember the grid elements, but in those days I worked with WSJwine, Laithwaites, Wines Direct (no longer in business), The California Wine Club, Gold Medal Wine Club, Monthly Clubs, and Oriel Wines (no longer in business).
I wouldn’t call the assessments from the early days rigorous, they were more gut checks. Also, back then, I hardly knew anything about wine.
In 2010 I decided to get serious and launched the first version of my multi-point review system. Over the years some criteria have been removed from the star rating (uniqueness) and some have been added (personalization rating for personalized wine clubs, a premium rating for premium clubs, etc.).
In 2019 I started writing reviews with much greater detail, specifically about membership benefits and more information about the wine itself (instead of just whether or not it provided value for the cost).
In April 2021 Google decided to start telling all reviewers what constitutes a good review and I have been transitioning my reviews to fit this model — I believe they have a good understanding of what should be in a review, though I’m not sure most of you want this much information.
All new and updated reviews have the following information:
A sample of how the star-ratings are calculated:
Wine Quality: Score is one of these: can add half-point where it's warranted
Wine Price: Score is one of these
Satisfaction Guarantee: Score is one of these
In order to get all of this information, I ask a lot of questions of the companies I’m reviewing. I find many wine club marketers leave a lot of information off their sites — they’re vague about which states they can ship to, they don’t talk about how they source their wines, they don’t disclose the specifics of their satisfaction guarantees, they don’t talk about their account management features, they hesitate to determine ideal customers or warn some potential customers off, etc.
This is the job of a reviewer and I try hard to be the best wine club reviewer out there. Even better than The Wirecutter. That said, not everyone agrees with my methodology, so here are some potential criticisms of how I do what I do, and my pre-emptive rebuttals.
I don’t pay for the shipments I receive
I’ve seen too many reviewers tout that they bought the wine from the wine club instead of receiving a sample shipment for review. This practice extends beyond just wine clubs, of course. The rationale behind this practice and claim is that you get a more authentic customer experience this way. I argue that’s not true.
For most personalized wine clubs (especially Firstleaf and Winc), I receive credits to my member account to pay for the wine I’ll review. I’m still getting the entire customer experience, but the wine club is paying for it.
With curated wine clubs, the concern is that I might be getting a cherry-picked selection of wines that make the wine club seem like a better value than it really is. Since most clubs take the time to create educational materials for the wines they ship, it’s fairly easy to verify if I’ve received the same wines as their paying customers.
Once in a while a selection seems too good to be true, which I can identify through experience, and I follow up with the company to see if I indeed received a cherry-picked selection. They typically acknowledge as much and I share those acknowledgements with my readers.
When I have a problem with a shipment, I don’t contact my marketing representative at the company, I contact customer service. This affords me the opportunity to both test the company’s customer service team.
I don’t have a formal wine education
I don’t believe this is a necessary credential to review wine clubs. I’m not a sommelier but I know a lot about wine. Lots of people know a lot about wine who aren’t sommeliers, wine critics, or winemakers. In fact, when it comes to wine subscriptions featuring wines under $20 a bottle, I’d argue it’s counterintuitive to get a sommelier’s opinion of the wine — that’s not the world they run in.
At the end of the day, only you can judge for yourself if you like a wine and if you think it’s worth it. For high end clubs I try to think like a sommelier; for less expensive clubs I try to think like a wine lover on a budget. As someone who has tried hundreds of wine clubs over the years, I can tell when the wine is worth your time and money and when it’s not. My goal is to help you figure that out for yourself before you buy — not to judge what you want to drink.
I get paid to write reviews
This is 100% true. Most of the time I get paid a commission when you sign up for a wine club after reading a review on my site. This is the revenue model called Affiliate Marketing.
Affiliate marketing is one of many revenue models which allow publishers to get paid for their work. Other common models are subscriptions (commonly used by print publications), display advertising (banner ads), paid placements or post sponsorships (used mostly by influencers on social media), and pay-per-click where a publisher gets paid a small fee when you click a link to an advertiser, whether you buy anything or not.
It used to be the case that mainstream publications like The New York Times or USA Today wouldn’t accept payment for product reviews (mostly books, beauty products, and wine). The concern was that the editorial position could be swayed by payola. Nevertheless, all of these publications now use affiliate marketing to boost their online revenue.
Here’s the truth about my reviews: I don’t accept payment to write a review, I don’t publish reviews for wine clubs I don’t recommend, I don’t accept payment for a more favorable review, and my opinions are 100% my own.
That said, my time is valuable and I don’t spend time reviewing wine clubs who don’t want to establish an affiliate program or offer to pay for the customers I’ll send them. This means that the reviews on my site are limited to the selection of wine clubs who have affiliate programs and that I’m not reviewing subscriptions from local wine stores and restaurants, wine clubs like Winestyr, Fat Cork, and SommSelect who don’t participate in affiliate marketing, and other small wine clubs for whom an affiliate program is too big of a financial imposition on their very narrow margins.
Guide by: Jessyca Frederick
Inspired by frequent questions from friends and family about different wines and wine practices, I write Useful Wine Guides so that people I don’t know might benefit from my knowledge and desire to share information, too.