Anthropic’s “cheaper” AI models have a dirty little secret: they’re 20-30% more expensive than OpenAI’s—thanks to a tokenizer that turns efficiency into confetti.
The Great Tokenization Swindle 🎪
Anthropic’s Claude 3.5 Sonnet claims to undercut GPT-4o with 40% lower input token costs. But here’s the catch: it chews up 30% more tokens for the same damn text. That’s like bragging about a discount on gas while driving a Hummer.
- English articles? 16% more tokens.
- Math equations? 21% markup.
- Python code? A whopping 30% tax.
This isn’t competitive pricing—it’s a shell game.
Why Your “200K Context Window” Is a Lie 🕵️
Anthropic loves flaunting its 200K token context like it’s the Vegas Strip. But with their tokenizer’s bloat, GPT-4o’s 128K might actually fit more real content. It’s the AI equivalent of airlines shrinking legroom while advertising “more space.”
The Real Cost of Closed-Source Obfuscation 🔒
OpenAI’s tokenizer? Transparent (BPE,
tiktoken). Anthropic’s? A black box with 65K tokens vs. OpenAI’s 100K+—meaning it fractures words like a toddler with a dictionary. No wonder it inflates counts. Bottom line: If your AI budget matters, Claude’s “savings” are a mirage. GPT-4o wins on real-world cost, not marketing fluff. 🏆