I've watched a lot of good changes not happen because nobody could promise they'd be reversible.
An over-permissioned role sits open for months. A stale firewall rule nobody will touch. A credential everyone knows should be rotated. The reason is almost never that the change is hard to make. It's that nobody can say, with confidence, what happens if it goes wrong at 2 AM. And when you can't answer that, the safe move — the move that doesn't get your name on an outage — is to leave it alone.
That's worth sitting with, because it means the thing blocking most infrastructure and security work isn't fear of change. It's fear of unrecoverable change. Those are different problems, and only one of them is actually about the change itself.
If you can always get back to the last known-good state, "act" stops being the scary option. The downside of a wrong change shrinks from "we're in an outage and I'm reconstructing what the config used to look like from memory" down to "we undid it, we're back, let's figure out what went sideways." That's a completely different risk calculus, and it's the one that lets people move.
This is why I think a real rollback guarantee is the linchpin. Not a feature next to the others — the thing everything else depends on. Delegation is safe when a mistake is reversible. Security teams can execute instead of just filing tickets when the executor can undo itself. And you can let an AI propose a change, because the human approving it knows the floor: worst case, we roll it back. Take reversibility away and every one of those falls apart. You're back to being scared to touch anything.
Here's the catch. "Rollback" is one of the most oversold words in this industry. Almost every tool has some notion of it. Very few give you a guarantee, and the gap between the two is where people get hurt.
A real guarantee, as far as I can tell, needs three things, and all three are load-bearing:
The exact before-state, captured at execution time. Not what your IaC config said last week. Not what the audit log recorded about the change after the fact. The actual state of the specific resources you're about to touch, captured in the moment right before you touch them. For a credential rotation that's the exact configuration, expiration, and attached permissions. For a firewall change it's the exact rule set. If you didn't capture it then, you don't have it.
Bound to the specific change. That before-state has to live with the change that caused it — not in a separate log you have to go correlate under pressure. Any operator should be able to pull up the change and find exactly what it's the undo for.
A tested, typed restore. Rolling back a credential rotation is not the same operation as re-enabling a disabled account. Each undo has to know how to restore its specific kind of change, and it has to have been run before you needed it. An untested rollback is a hope, not a guarantee.
The reason so many tools claim rollback without delivering the guarantee is that re-applying config, restoring a backup, or reading the audit trail all look like rollback and aren't. They move you toward a state that resembles the one you had; they don't restore the exact prior state, bound to the change, with a tested undo. I wrote a whole piece on where the common tools break down, so I won't relitigate it here. The short version: reconstructing the before-state after the fact, during an incident, is a forensics exercise, not a rollback. Those are not the same thing, and finding out which one you have at 2 AM is the worst possible time.
So here's the bar I hold us to, and I think it's the right one for anyone building in this space.
Rollback should be designed into every change type and enforced at plan time — before a change runs, the system should already know how it comes back. A representative set of high-risk changes should be verified against live infrastructure, and that set should keep expanding.
And when a change genuinely can't capture and restore its before-state — some operations are like that — the honest move is to say so, plainly, before it runs. Tell the operator "this one isn't cleanly reversible, here's what recovery would take," and let them decide with their eyes open. That's worth more than the word "rollback" on a slide that quietly means "re-apply the config and pray."