RFC-004: BYOK Secret Boundaries

Status

Accepted (v1, March 2026). This RFC intentionally describes boundaries and properties rather than specific cryptographic mechanisms or storage implementations, both because those details will evolve and because this document is meant to be readable by people evaluating the design rather than auditing the implementation.

Context

Several design partners require BYOK, connecting their own LLM provider accounts rather than using pooled credentials we provide. This means customer credentials for paid third-party accounts will flow through our infrastructure, which is a meaningfully different risk profile than anything we'd built for previously.

Problem

We need an architecture where customer-supplied provider credentials can be stored, used to execute workflow steps, and rotated or revoked, while bounding the impact of any plausible failure mode (a compromised application server, a logging misconfiguration, a code review miss, an insider mistake) to the smallest possible blast radius.

Goals

• A credential, once stored, is never displayed in full again to anyone, including our own staff.
• The code path that uses a credential is narrow, audited, and distinct from any path that could export or display it.
• No credential ever reaches a log, including error logs and anything forwarded to third-party observability tools.
• Revocation is immediate and verifiable by the customer.
• A compromise of one component does not trivially expose credentials belonging to other customers.

Non-goals

• Specifying the exact cryptographic primitives or key management hierarchy (covered in internal documentation, not this RFC).
• Supporting credential sharing across workspaces, each workspace's credentials are scoped to that workspace only.
• Solving for credentials belonging to systems other than LLM providers in this iteration (e.g., customer database credentials for future integrations would need their own review).

Proposed design

Establish a narrow "secret access" interface that is the only code path permitted to decrypt and use a stored credential, and require that every call site using customer credentials go through it, enforced via code review checklist and, where feasible, lint rules that flag direct access to underlying storage. Apply active log scrubbing at the boundary of this interface so that even a programming mistake that tries to log a credential produces a redacted placeholder instead. Build a "test connection" action that validates a credential against the provider without ever returning it to the caller in a displayable form. Make rotation/removal take effect by invalidating cached decrypted material immediately, not on a TTL.

Alternatives considered

Store credentials encrypted with application-managed keys, decrypt on read wherever needed. Simpler to build, but means many code paths can decrypt credentials, which makes "what's the blast radius of a compromised server" a much larger and harder question to answer confidently. Rejected in favor of funneling all access through one narrow, auditable interface.

Don't support BYOK; offer cost pass-through on pooled keys instead. This avoids the entire problem category. Rejected because it doesn't satisfy the compliance and data-handling requirements that several design partners described as non-negotiable, for them, "your infrastructure never touches our provider relationship in a way we don't control" is the actual requirement, and pass-through billing doesn't address it.

Tradeoffs

A narrow, funneled access pattern is more upfront engineering effort than "decrypt wherever needed," and it constrains how features can be built, anyone adding a feature that needs to use a customer credential has to go through the narrow interface, which can feel like friction in the moment. We accept this friction deliberately: it's the mechanism that keeps the list of "things that can go wrong with a customer's credentials" short enough to actually reason about.

Open questions

• How do we extend this model if we add support for credential types beyond LLM provider keys (e.g., customer webhook secrets, integration tokens) without re-deriving these boundaries from scratch each time?
• What's the right way to give customers visibility into when their credentials were used (which runs, which steps) without that visibility itself becoming a new surface to secure?
• Should we support time-boxed or scoped credentials (e.g., a key that's only valid for a specific workflow or time window), and if providers don't support that natively, is it worth building an internal proxy layer to approximate it?