Two ways to move a live Valkey cache: proxy cutover vs operator cutover
TL;DR: both tools replicate with PSYNC. The difference is who owns the cutover. AWS put a proxy in front of your app so the flip is invisible, and paid for it with serialised pipelines, no reader endpoint and no rollback. Without a proxy you own a write pause of a few seconds, keep pipelining, and roll back by pointing the app at the old host. What we lacked was a lag number and a cutover button. Both are coming.
On October 21 AWS announced online migration into ElastiCache Serverless for Valkey. Zero downtime, no third-party tooling, three steps. We ship a live migration tool for Valkey and Redis, so I read the whole documentation set the same evening to find out what they did that we don't.
The replication part is the same. The cutover part isn't, and the reason is a design choice, not a feature we forgot. By the end of the docs I also had a list of two things ours does badly, which is the real reason this post exists.
1 - What AWS shipped
Four steps. Three are API calls, one is yours.
- Prepare. ElastiCache checks your source, provisions migration infrastructure inside your VPC, and gives you a migration endpoint. That endpoint is the serverless cache's normal endpoint, so after the cutover nothing changes for your app.
- Redirect. You point your app at the endpoint. A proxy forwards every command to your existing source. The docs say outright that AWS cannot tell whether you did this.
- Sync. ElastiCache connects to each source primary as a replica, takes a full RDB snapshot, then streams the replication backlog for as long as the migration runs.
- Cut over. Manually or automatically, at the next "fully synced window", the proxy flips each shard from source to target.
Sources: self-managed, cluster mode, Valkey 7.2 to 8.1 or Redis OSS 4.0 to 7.2, on EC2. Targets: an empty Serverless cache on the same or a higher version. "Zero downtime" is defined the way the ElastiCache SLA defines it, zero errors returned to the client.
2 - Same primitive on both sides
Take away the console and both tools are a process pretending to be a replica.
It sends REPLCONF and PSYNC to each source primary. The primary forks, writes an RDB, streams it, then keeps streaming every write from its backlog. The migrating process parses the RDB, replays it onto the target as normal commands, then replays the backlog as it arrives. That is how RedisShake's sync reader works, how our sync mode works (we run RedisShake), and, going by the prerequisites page, how the AWS migration nodes work. The page even lists the two commands you are not allowed to have renamed.
Because it is a replica connection, the source pays the same bill in both cases:
- The fork needs memory headroom on the primary, and more of it the higher your write rate is during the snapshot.
- The primary keeps serving your whole workload for the whole migration, so it needs CPU headroom too. AWS says it plainly, an overloaded source primary "can compromise the health of the migration".
- Throughput scales with the number of source shards, one replication stream per shard. A hot source with few shards is the worst case for both.
Everything below is about the one place they split.
3 - The flip, with and without a proxy
At some point the target has caught up and the app has to start talking to it. There are two ways to do that, and only two.
With a proxy. The app has been talking to the proxy the whole time. The proxy watches replication offsets, waits for a moment when every shard's target matches its source, and starts forwarding to the target instead. Your connection string never changes. That is what AWS built, and it is why there is nothing to install on your side. The proxy was already there, ElastiCache Serverless has always fronted every connection with one.
Without a proxy. The app talks to the source directly. When you decide to cut over, you stop or pause writes on the source, wait for the replication diff to hit zero, stop the replicator, and repoint the app through whatever you already own, a config push, a DNS flip, service discovery. That is what RedisShake-based tools do, ours included.
So the proxy moves the write pause from you to the machine. It doesn't remove it. AWS documents what your app sees while the proxy flips, and I'd rather quote their table than paraphrase it:
| What the application sees | Why |
|---|---|
| Reads and writes keep flowing, tail latency rises for roughly 2 to 10 seconds | Each shard's flip holds commands briefly |
| A connection with a write in flight is closed | ElastiCache closes it rather than replay a write it cannot confirm |
| Blocking commands and pub/sub subscribers lose their connection | They have to reconnect and re-subscribe |
| Clients attached directly to the source are disconnected | ElastiCache ends the source's client connections as part of the cutover |
Read that list again as an operator doing a manual cutover. It is your checklist. A closed connection with an in-flight write is a closed connection whether a proxy or a config push caused it. The proxy makes the window short and predictable. It does not make it zero.
4 - What the proxy costs you
A proxy in the path for the length of a migration has consequences. To AWS's credit, they list them.
- Pipelining is serialised. A pipeline of 100 commands becomes 100 round trips while the migration runs. If your client relies on pipelining for throughput, you need more connections to compensate, and the docs tell you to raise your pool size.
- No reader endpoint. Everything goes through the migration endpoint.
- Command support is the target's, not the source's. Commands Serverless does not support are rejected at the proxy from the moment you redirect, before a single key has moved.
FLUSH,CLUSTERand pub/sub behave differently too. - A write that bypasses the proxy can poison the migration. If some client sends a command straight to the source that Serverless does not replicate, the target diverges. ElastiCache reports an error, and after that the migration cannot be restarted. Your only exit is to complete it with force and accept the inconsistency.
- Errors are permanent. A dropped network path or a rotated credential, even one you fixed five minutes later, leaves the migration in force-complete-only state.
- No rollback after cutover. Once the proxy flips, the target is authoritative. Going back means repointing the app yourself, and the source has had no writes since the flip.
- One direction only. Serverless is never a source. Node-based ElastiCache and MemoryDB are never sources either, so a node-based ElastiCache cache can't use this to move to Serverless.
That is the price of making the app not notice. Pipelining in particular is hard to keep when a proxy has to know, per command, which backend answered during a per-shard flip.
5 - What no proxy costs you
The operator-owned cutover has its own bill.
- Somebody owns the write pause. It is seconds, but it is yours to schedule and run. Repoint while writes are still landing on the source and those writes are gone.
- Lag has to be visible. You can't pause writes and then guess when the diff reached zero. The replicator has to tell you, per shard, and the tool has to put it on the screen instead of in a log line. This is the part most RedisShake-based tools do badly. Ours included, until now. The number is in the periodic stats line and nowhere else.
- The repoint is yours. Config, DNS, service discovery. If your clients cache node IPs, AWS's warning about that applies to you too.
What you keep:
- Pipelining works the whole time. Nothing sits between client and server.
- The source is untouched. Rollback is pointing the app back, and the source has every write up to the pause.
- Any topology on either side. Standalone to standalone, standalone to cluster, cluster to cluster, Redis to Valkey and back, across clouds, out of a managed service as well as into one.
- Nothing extra in the data path to pay for, and no permanent-error state. A run fails, you fix the cause, you run it again.
Which bill is cheaper depends entirely on where you are. Already on AWS, moving into Serverless, cluster mode, client doesn't pipeline much? The proxy is easier, take it. Any of those false? The operator approach is the only one on the table, and it is fine as long as you can see the lag.
6 - What we are fixing in ours
Reading the AWS cutover page next to our own sync mode made two gaps obvious, both on the operator side of the ledger.
First, lag as a real field. RedisShake prints diff=[N] per source shard in its stats line. We were parsing that line for the write count and throwing the diff away. It becomes a per-shard lag value on the execution status, so the UI can say "fully synced" the way the AWS proxy decides it, instead of telling you to read the log (issue).
Second, a cutover button. Today "Stop migration" is both cancel and cut over, and the actual procedure lives in the docs as prose: pause writes, wait for zero, stop, repoint. It becomes "Complete cutover", which waits for every shard's diff to hit zero and then stops the replicator, with the same force override AWS has for when you've decided the data is good enough (issue).
They don't make ours a proxy cutover. They put the one question you have to answer, "is it safe to flip now", on the screen. That is the part of the AWS launch worth copying, and it doesn't need a proxy.
So, which one?
If you're moving a self-managed cluster on EC2 into Serverless and your client isn't pipelining heavily, use AWS's tool. Before you prepare anything, read their prerequisites page end to end. Check your key names against the 4 KiB limit, reload your Lua scripts through the migration endpoint, and make sure nothing writes to the source directly. Those three are what turn a healthy migration into a force complete.
For everything else, standalone sources, managed sources, other targets, the other direction, cross-engine, that is what our migration is for. Analyze first, copy, then verify the target against the source, so "done" means "correct".