Content Deep Dive

Petsem: Sometimes Dead Is Better

Blog post from Fly.io

Post Details
Company
Date Published
Author
Matt Braun
Word Count
1,300
Company Posts That Month
4
Language
English
Hacker News Points
-
Post removed?
No
Source URL
Summary

As AI-related credentials become increasingly attractive targets, Fly.io describes how it replaced HashiCorp Vault as the primary store for customer application secrets after its single-region, Raft-based setup caused slow reads and made app startup depend on one region. Its in-house system, Petsem, keeps a write master in one region and distributes read replicas across others, reducing p99 secret-read latency from about 10 seconds to 300 milliseconds while accepting that reads may briefly return stale values. Petsem also uses macaroon-based authorization: a worker must present both its own credentials and a separate token confirming it is authorized to access a particular app’s secrets, limiting the impact of a compromised worker and logging denied requests for alerts. The migration began in 2024 and was completed for customer secret reads in July 2025, though Vault remains in use elsewhere in Fly.io’s infrastructure. Petsem cannot protect secrets already exposed in a compromised host’s memory, but it can constrain access to secrets for apps running there; Fly KMS further reduces exposure by allowing apps to perform operations such as signing or encryption without receiving the underlying secret.

Trends Found in this Post
Trend Post Mentions Total Month Mentions Posts Companies MoM
Secrets Management 33 2,876 539 148 -7%
LLM 1 3,674 823 199 -44%
Observability 1 2,390 523 156 -23%
Use This Data

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.