Classification

When to use this scenario

Sentiment analysis, topic tagging, intent routing, and toxicity detection all share the same profile: very short inputs, very short outputs, and very high volumes. At 5 million classifications per month, per-token cost dominates all other factors — quality differences between fast cheap models and frontier models on clear-cut cases are negligible.

DeepSeek V3.5 at $0.14/million input tokens costs $140/month for 5M × 200-token inputs. The same workload on GPT-5 ($1.25/million) is $1,250. On binary classification tasks (spam/not spam, positive/negative), the accuracy gap is rarely worth a 9× cost penalty.

Reserve more expensive models for edge cases and hard negatives: ambiguous sarcasm, domain-specific jargon, code-switching text. A practical architecture runs all items through the cheap model and routes low-confidence outputs (probability < 0.80) to a stronger model — a hybrid that typically covers 95% of volume with the cheap tier.

Common pitfalls

• Evaluating accuracy on a clean benchmark then deploying against real noisy user text — distribution shift is the number-one source of production surprises
• Using per-accuracy-point as the optimization metric without pricing the business cost of each error type (false positive vs false negative cost differently)
• Neglecting latency when classification is on the hot path — if a classifier gates real-time routing, 2s model latency is a problem regardless of cost
• Not rate-limiting per-provider: at 5M calls/month the inter-call spacing matters; some providers throttle sustained high-frequency traffic differently from burst