Skip to content
Post Mortem

Playbook ยท Act in the first 5 minutes

My API key leaked

A leaked key lets a stranger spend your money. Cancel it first, investigate second.

CRITICALTake a breath. Do these in order.

How you might know

  • A surprise bill or spending alert from an AI, cloud or payments provider.
  • An email from GitHub, a cloud provider or an AI company saying a key was found in public.
  • You pasted a key into a chat, screenshot, AI prompt, public repo, gist, forum post, or code that runs in the browser.
  • Usage you do not recognize: odd hours, odd regions, odd volume.

Do this now

  1. Work out which key and which account (about 30 seconds)

    Look at the first characters of the key and find it in the provider's key list. Do not stop to investigate further. You just need to know where the cancel button is.

  2. Cancel it. Now.

    Disable or delete the key at the provider. Yes, this may break your app for a few minutes. A broken app is an annoyance. A stolen key is a bill and possibly a breach. On AWS, deactivate first (you can switch it back on if something critical breaks), then delete it once things are stable.

  3. Make a smaller replacement

    Make a new key that can do as little as possible while still working. Where the service allows it, limit where it can be used and set a spending cap. Store it in the safe place your hosting service provides for keys, never inside your code. Use one key per tool, so cancelling one never breaks everything.

  4. Find out what the key could do, and did do

    Check what the key was allowed to do, then look at the record of its use from when it was exposed until you cancelled it. Questions to answer: did it read data? Create things? Send messages? Spend money? (On AWS, the record is called CloudTrail.)

  5. Hunt for backdoors

    When attackers get a powerful key, they often leave a spare. Look at everything created while the key was exposed: new users, new keys, connected apps, scheduled jobs, servers and storage. On hosting accounts, check every region, including ones you never use. Remove what you did not create, and replace the keys involved too.

  6. Close the leak at the source

    Find where it was exposed (your code and its history, screenshots, chat messages, records of what your app did, or the website itself) and remove it. Removing it does not make the old key safe again, which is why step 2 comes first.

  7. Tell the right people

    Tell the provider's support (many will review fraudulent charges, though refunds are never guaranteed), your team, and customers if their data or accounts were reachable. If personal data may have been exposed, check what your local law requires and how fast.

Do not do this

  • Do not wait for proof of misuse before cancelling. Cancelling is cheap and reversible, because you can always make a new key.
  • Do not just delete the file or rewrite history and call it done.
  • Do not paste the replacement key into chat, tickets or a prompt.
  • Do not delete logs or accounts while "cleaning up". You need them to see what happened.
  • Do not reuse the leaked key's permissions blindly. Take the chance to shrink them.

Who to tell

  • Whoever owns the provider account
  • Your team lead or whoever is in charge
  • Provider support or abuse team
  • Customers, if their data or accounts were reachable

What to save before you clean up

  • The service's usage, billing and activity records for the time the key was exposed
  • Your code history, access records and where the key appeared
  • A written timeline: when it was exposed, found, cancelled

In the next 24 hours

  • Turn on secret scanning and push protection in your code host so keys are blocked before they are pushed.
  • Set spending caps and alerts on every provider.
  • Prefer short-lived credentials (for example, roles instead of long-term keys on AWS) over keys that live forever.
  • Write down where every key lives and who can cancel it.

Official help for the platforms involved

Links open each provider's own documentation, so you do not have to search for it.

What to write in the post mortem afterwards

  • How the key was exposed
  • How long it was exposed before it was cancelled
  • Why cancelling took as long as it did
  • Which guardrail prevents a repeat: scanning, scoping, caps, short-lived credentials

References

Finished this one?

Add it to What I learned and earn 30 Trust. You can open your list from the badge at the bottom of the page.