Sentiment Tagging

The job

Your support team reads 200 messages a week. Some customers are delighted. Some are neutral. Some are frustrated. Some are one bad week away from leaving. The team feels which is which in real time. But “feels” doesn’t scale, doesn’t trend, and doesn’t route the frustrated customer to an account manager who might save them.

Sentiment tagging reads every incoming message and assigns a score. Happy (customer is pleased or solving their own problem smoothly). Neutral (transactional, no heat). Frustrated (customer is annoyed, feels stuck). At-risk (customer language signals churn threat, cancellation mention, contract review). The score lives on the ticket. The team can now filter. “Show me every at-risk message from the last 30 days.” A pattern emerges. Billing is tanking three customers a month. Product slow rollouts are triggering two more. These patterns never surfaced before because the signal was scattered.

Plated well: by month four, the account manager sees an at-risk tag and knows something specific to fix before the customer cancels. The team has acted on churn signals they didn’t know they had.

The recipe

All seven ingredients still apply. The leverage on this dish is Measurement (Ingredient #6). Sentiment tagging is useless without a scoreboard. You score every message. You measure whether the scores are accurate. You act on the patterns. Without measurement, tags pile up in a folder and nothing changes.

Training sets the definition of each sentiment tier. What language lands as frustrated versus at-risk. Examples show the station what edge cases look like (a sarcastic happy customer, a polite but clearly frustrated customer, someone escalating to lawyer language). Context matters because the same one-word reply ”?” means different things from a five-year customer versus someone on day three. Output Over Process means the destination is clear: every message gets one score so you can measure whether you’re missing churn signals. Guardrails are light here, but critical: the station should never tag a customer as at-risk based on speculation. If the language doesn’t clearly signal churn threat, mark as neutral or frustrated, not at-risk. Measurement is load-bearing because without it, the tags become noise.

How to build it

1. Define your four sentiment tiers. Write down what language lands as each one. Happy: “solved my problem,” “thanks so much,” “working great now.” Neutral: transactional, no emotional temperature. Frustrated: “still broken,” “been waiting,” “this is ridiculous.” At-risk: “considering other options,” “might cancel,” “contract renewal next month and we’re reviewing,” “switching to X,” actual cancellation language.

2. Create five example messages for each tier. Take real messages from your inbox. One clearly happy, one clearly neutral, one clearly frustrated, one clearly at-risk. Add a fifth edge case (sarcastic happy, politely desperate, quietly ominous). Show these to the station. These are the anchors.

3. Set the confidence threshold. The station should mark as at-risk only when the signal is explicit. If a customer says “we’re reviewing alternatives,” that’s at-risk. If a customer says “not sure this is the right tool,” that’s frustrated. Don’t let the station conflate the two.

4. Pick your measurement system. Zendesk tags, HubSpot pipeline, Intercom priority. The sentiment score lives where your team already works.

5. Run a validation sample. Have the station tag 50 recent messages. Compare against how your team would tag them. What’s the accuracy rate. Below 60%, sharpen the definitions. Below 75%, run the five edge cases again. If accuracy is above 80%, deploy.

6. Set the weekly review. Every Friday, run a report. How many at-risk messages landed this week. What’s the pattern. Is it the same issue or new threats. Is the team responding to at-risk flags or letting them sit.

What breaks it

• Sentiment scoring without action. The station tags fifty messages. Nobody looks at the at-risk pile. By month three, the at-risk tag means nothing because nothing changed after it was raised. Commit to acting on at-risk flags within 24 hours or don’t tag at all.

• Conflating frustrated with at-risk. Many customers are frustrated without being at churn risk. If you mark every frustrated customer as at-risk, the account manager ignores the score because it has no signal value. Keep the categories distinct. Frustrated is emotion. At-risk is behavior signal.

• No context window. The station reads only this message. It doesn’t see that this customer complained every month for six months, or that they just renewed, or that they’re on your lowest plan. Context matters for accuracy. Feed the station the account timeline, not just the message.

• Accuracy blind spot. The team assumes the tagging is right after week two and stops validating. By month three, the station has drifted because nobody measured whether it was actually catching what it’s supposed to catch. Run a validation sample every month.

When it’s working

By week two, every incoming message is scored. By week four, the at-risk queue exists and the account manager is reading it. By week six, patterns surface. Three customers at-risk are all on the same plan tier. Two more all complained about the same feature. The team fixes the pattern instead of firefighting the complaint. By month two, churn rate is measurable against the at-risk flags. If at-risk tagging correlates with actual churn 70% of the time, it’s working.

Measure it: tag 100 messages. Have your team tag the same 100. Compare. If accuracy is above 75%, deploy. If it drifts below 70%, retrain.

Monday Move

Tag yesterday’s inbox by hand yourself. Then have the station tag the same messages. Write down which ones it got wrong. Those misses are your sharpenings. Was it context it didn’t have. Was it a definition you need to clarify. Was it an edge case the examples didn’t cover. Fix one thing. Run it again on Monday.

Dish 4 of 10 on the Service Station. Build-note leverage: Measurement (Ingredient #6).