Designing for Audit From the Start

What you actually need to know

• Audit events live in a separate store from application logs.
• The store is append only and tamper evident.
• Sensitive actions emit events from a controlled vocabulary.
• Retention matches the regulatory regime.
• The cost at design time is small. The retrofit cost is large.

The core argument

Audit is one of those system properties that you either design in from the start or retrofit painfully later. The retrofit is painful because audit cuts across every code path. Every sensitive action needs to emit an event. Every event needs to be in the right shape. The store needs to exist with the right properties. Doing this work after the fact across a large existing codebase is brutal.

The design at the start is small. A separate audit store. A controlled vocabulary of event names. A library that emits events. A storage choice with append only properties. A retention policy. The total is roughly two weeks of engineering work spread across the project setup.

The teams that design for audit from the start ship systems that survive the first compliance audit and the security questionnaires that come with enterprise deals. The teams that retrofit audit later spend a quarter doing it under deadline pressure, usually because a customer or auditor forced the question.

The other benefit is that an auditable system is easier to debug, easier to support customers with, and easier to investigate incidents in. The audit trail is the same data that helps you answer when a customer asks what happened to their account on a specific day. The investment pays back across many uses, not just compliance.

The architecture

How much does this cost

Features the audit system must have

• A separate store from the application database.
• Append only storage characteristics.
• A controlled vocabulary of action names.
• Structured event records.
• Tenant scoping on every event.
• Customer accessible query API where applicable.
• Tiered retention with documented policy.
• Anomaly alerting.
• A regular review cadence.

Expert opinion

The audit work is the kind of investment that earns nothing visible at week one and prevents large pain at year three. The teams that designed for audit from the start ship systems that survive compliance and enterprise security review. The teams that retrofit pay a multiplied cost under deadline pressure. The math is consistently favorable for the upfront investment.

>

Yashveer Singh, founder of Yashveer Labs

How this played out on a real project

A client SaaS was preparing for SOC 2 Type II. Their audit logging was minimal. The auditor was going to fail them on the control. We had eight weeks before fieldwork.

We designed and built the audit system in three weeks. Append only store in a separate account. Controlled vocabulary of fifty event names. Library to emit events from every sensitive action. Customer query API. Documentation.

The retrofit across the existing codebase took another four weeks. Every sensitive action was identified and instrumented. The audit covered the prior six months by replaying application logs into the new format where possible.

The fieldwork ran successfully. The auditor noted the audit logging as a strength. The cost of doing it under pressure was real. The cost of having designed it from the start would have been a fraction.

For more on the related work, see audit logs that pass real audits and audit trails for sensitive actions the pattern that earns trust.

Common mistakes teams make

1. Treating audit as solved by application logs.
2. No separate audit store.
3. No controlled vocabulary. Free text event names.
4. No tenant scoping on events.
5. No retention policy.
6. No customer query API.
7. No tamper evidence.
8. Treating audit design as out of scope for the MVP.

A two week design plan

1. Days one to three. Pick the storage. Cross account append only.
2. Days four to six. Define the controlled vocabulary. List the events.
3. Days seven to nine. Build the emitter library and the storage pipeline.
4. Days ten to twelve. Build the query API. Test the retrieval.
5. Days thirteen and fourteen. Documentation. Policy. Trust portal entry.

For more on the related work, read audit logs that pass real audits and building a security program from zero a twelve month plan. On the broader compliance side, SOC 2 Type I vs Type II is the natural next read.