Exact-match caching saves money when users repeat identical prompts. Real applications rarely do — but semantically similar prompts are everywhere.

Semantic caching hashes prompt meaning, not literal strings. When a new request is sufficiently similar to a cached response, you skip the API call entirely.

The tradeoff is staleness and false hits. Production systems need TTL policies, confidence thresholds, and invalidation hooks tied to your data refresh cycles.