Tag

Caching

10 posts tagged “Caching”.

When will my Valkey or Redis cache run out of memory? Sizing it with the Cache Size Simulator

used_memory tells you where you are, not where you're going. The Cache Size Simulator samples what your keys actually weigh, projects growth and TTLs forward, tells you when you hit maxmemory and what your eviction policy will do about it, and picks the smallest ElastiCache, MemoryDB, Memorystore or Azure node that still fits in six months. Works on Valkey, Redis OSS and other RESP-compatible databases.

Kristiyan Ivanov•October 28, 2026

How to Actually Cache an LLM App: Exact, Semantic, or Both

We ship two caching libraries and people keep asking which one to use. The answer is a decision tree, not a product name. Here is the playbook: what to cache exactly, what to cache semantically, how to tune the threshold, and the numbers behind every claim.

Kristiyan Ivanov•August 7, 2026

We benchmarked our semantic cache against Upstash. The thresholds did not transfer.

BetterDB vs @upstash/semantic-cache on four public datasets. Quality is at parity. The same embedding model produced different score distributions on each runtime, so the optimal threshold was different on each side, which is the whole argument for self-tuning. Latency is local Valkey vs a cloud REST API, so we report it as a deployment difference, not a library win. And BetterDB ships OpenTelemetry, Prometheus, and dollars-saved tracking in the MIT library, where Upstash ships none of it.

Kristiyan Ivanov•May 29, 2026

A week benchmarking LLM caches against RedisVL and Upstash. Here is what held up.

Over the past week we benchmarked our semantic cache against RedisVL and Upstash on four public datasets. Quality is at parity. Thresholds do not transfer between runtimes. Auto-tuning gains up to +2.8% F1 where the starting threshold was wrong. Latency is mostly about where you run the cache. And adversarial paraphrases beat every cosine cache.

Kristiyan Ivanov•May 29, 2026

We gave our semantic cache a self-tuning loop and benchmarked it on two academic datasets. Here is what happened.

BetterDB's Monitor-driven threshold autotuning on STSb and SICK: the cache loosened when too strict, tightened when too loose, and did nothing when it was already right. Up to +2.8% F1 on STSb and +2.1% F1 on SICK vs static thresholds, with zero degradation when the threshold was already optimal.

Kristiyan Ivanov•May 27, 2026

We benchmarked our semantic cache against Redis's. Here is what we found.

BetterDB vs RedisVL's SemanticCache on two public datasets, nine thresholds, and three configuration modes. Quality is at parity. First-query latency is at parity. On repeated queries BetterDB is 7x faster thanks to built-in embedding caching. Valkey and Redis 8 are indistinguishable. Full methodology and raw numbers included.

Kristiyan Ivanov•May 25, 2026

The cache that tunes itself: three autonomous optimization runs and what they found

A cache optimizer on a 6-hour Vercel Cron that issues proposals and either auto-approves them or queues them for human review in BetterDB Monitor: three runs against chat.betterdb.com, 15 tool calls down to 8, and what the repetition between runs 1 and 2 revealed about a problem no amount of TTL tuning can fix.

Kristiyan Ivanov•May 7, 2026

Two-tier caching for RAG: what we measured building chat.betterdb.com

I built a RAG chatbot over Valkey, Redis, and Dragonfly docs in a weekend, expecting to find that semantic caches silently corrupt answers when entity-swap queries collide. The data showed the opposite. Here is what we measured, what the architecture looks like, and where the real failure mode lives.

Kristiyan Ivanov•April 28, 2026

Your Agent Works Locally. Then You Deploy to AWS.

The quietest assumption in modern AI frameworks is that Redis means Redis 8 with modules. It doesn't, on any managed service you're likely to use. Here's why your agent cache breaks on deploy, and what to do about it.

Kristiyan Ivanov•April 20, 2026

The Glue Layer Most People Don't Design Intentionally: Caching, State, and Session Management in AI Apps

Your AI app works great in dev. Then LLM costs spike, latency doubles, and your agent forgets mid-task. The problem is the layer between your app and your model.

Kristiyan Ivanov•March 21, 2026