Your Compliance Evidence Is Probably Fiction. Here's How to Know.

"We generated your SOC 2 report" is not the same as "we verified your security controls."

A recent investigation blew the lid off a compliance automation vendor that was generating hundreds of identical audit reports — pre-written auditor conclusions, fabricated board meeting minutes, device security evidence for employees who never completed onboarding. The auditor conclusions weren't even written by auditors. The platform vendor wrote them.

And nobody caught it. For years.

This isn't surprising. It's the logical endpoint of an industry that optimized for producing documents instead of producing evidence.

But this problem isn't limited to bad actors. Even with great teams and honest intentions, compliance evidence routinely lags months or years behind what's actually deployed. Your infrastructure changed 400 times since your last audit. Your compliance documentation reflects maybe three of those changes.

And it's about to get worse. AI coding agents are accelerating the rate of change in production systems by orders of magnitude. Compliance evidence was already stale at human speed. At agent speed, the gap between "what's documented" and "what's deployed" becomes a canyon.

The Compliance Industry Has an Evidence Problem

When a compliance platform says "all data in transit is encrypted using TLS 1.2 or higher," there are two possibilities. Either the platform examined your actual infrastructure and found TLS 1.2 configuration. Or it generated a sentence that sounds right.

One is evidence. The other is creative writing. And right now, most compliance automation is creative writing.

Evidence Needs Provenance

The failure isn't AI or automation. The failure is that the evidence has no provenance. You can't trace a compliance claim back to the observable fact that supports it.

We built TestifySec around a different idea: every piece of evidence is an attestation. A signed, timestamped, verifiable statement about something that actually happened. Not a screenshot. Not a checkbox.

Concretely:

• Signed with a Fulcio certificate tied to the identity that created it — you know who
• Timestamped by an RFC 3161 Timestamp Authority — you know when
• Stored in Archivista with a SHA-256 subject hash — you know it hasn't been tampered with
• Queryable by artifact digest — you can trace any claim back to the exact evidence

An auditor can independently verify that evidence was created by a specific identity at a specific time, and that nobody has touched it since. Good luck doing that with a screenshot.

How We Actually Collect Evidence

The investigation revealed that one vendor's "integrations" were literally containers for manual screenshots. This is what the alternative looks like:

CI/CD pipeline step executes
    → witness wraps the step
    → captures environment, materials, products, execution trace
    → signs with Fulcio (keyless, OIDC identity)
    → timestamps via TSA (RFC 3161)
    → stores in Archivista (SHA-256 indexed)
    → maps to relevant NIST 800-53 controls

No screenshots. No manual uploads. No human in the loop fabricating anything. The evidence is a byproduct of your actual development workflow.

Vulnerability scan runs? Signed attestation. Build pipeline produces a container image? The provenance chain from source commit to final artifact, captured as linked attestations. Policy engine checks a release? Pass or fail, signed and timestamped.

In the platform, it looks like this:

Your SSP Should Come from Your Infrastructure, Not a Template

The vendor in question used identical policies and controls for every single customer. 259 SOC 2 Type II reports with the same boilerplate. The only differences? Customer signature, system description, a diagram, and an org chart.

Your System Security Plan implementation statements should come from scanning your actual repositories, Dockerfiles, Kubernetes manifests, and CI/CD configurations. When the platform says "the organization employs AES-256 encryption for data at rest," it should be because it found spec.encryption.keyManagementService in your Kubernetes StorageClass.

Our AI scans your connected repos and generates implementation statements grounded in what it actually finds. If your infrastructure doesn't implement a control, the platform creates a gap — not a fiction. Your compliance team refines AI output. They don't start from a template that every other customer also received.

Documents Should Map to Controls, Not Fabricate Them

The investigation showed pre-populated board meeting minutes and fabricated employee documentation. We took the opposite approach: upload your actual vendor questionnaire responses, and the platform maps them to the controls they satisfy.

The pipeline:

1. Upload — document ingested, SHA-256 hashed for integrity
2. Extraction — content parsed into structured sections
3. Classification — each section classified by type and intent
4. Control Mapping — AI maps sections to NIST 800-53 controls with confidence scoring

A vendor questionnaire response about TLS encryption gets mapped to SC-8 with an 85% confidence score and a source reference. The vendor actually answered the question. The platform just connects it to the right controls.

Compliance Should Update When Your Code Does

Your infrastructure changes every day. Engineers push code. Dependencies update. Configurations drift. A compliance report from six months ago reflects your security posture on the day someone took the screenshots, not today.

We run continuous assessment workflows that re-evaluate controls against your latest code. When a change affects a control's implementation, a new gap surfaces. When a vulnerability scan finds a new issue, RA-5 and SI-2 controls update with signed evidence. Every assessment produces a new signed attestation.

This Sounds Hard. That's Where AI Comes In.

If you're reading this and thinking "cryptographic attestations for every pipeline step, mapped to NIST controls, with continuous reassessment... that sounds like a lot of work" - you're right. It would be, if humans had to do it.

We use AI agents to find the evidence, not create it. That's the difference. Our AI scans your repos, reads your Kubernetes manifests, crawls your CI/CD configs, and identifies which controls your infrastructure already satisfies. It maps vendor documents to the controls they address. It detects when code changes affect control satisfaction.

The AI doesn't write fiction about your security posture. It goes and looks at your actual infrastructure and reports what it finds. If a control isn't satisfied, it says so. If evidence exists, it finds it and links it.

That's the difference between GRC and GRC on autopilot. One generates documents. The other finds evidence.

Everything Maps to Controls. Everything.

The reason this works across any tech stack, any evidence type, and any compliance framework is that we built the platform around a single architectural decision: everything maps to controls.

Code repository? The platform reads your configs, your manifests, your pipeline definitions, and maps what it finds to the NIST 800-53 controls they satisfy. Vendor questionnaire? Uploaded, parsed, and each section mapped to controls with confidence scores. Vulnerability scan output? Wrapped as an attestation and mapped to RA-5. Container image provenance? Mapped to SR-4. STIG check results? Mapped to the relevant CM and SC controls.

It doesn't matter if you're running GitHub Actions or GitLab CI. Kubernetes or bare metal. Terraform or CloudFormation. Go, Python, or Rust. The compliance mapping engine doesn't care about your stack. It cares about what your stack proves about your security controls.

Today we support the full NIST 800-53 catalog with FedRAMP Low, Moderate, and High baselines. CMMC, SOC 2, ISO 27001, and FedRAMP 20x support are on our roadmap as those frameworks finalize. But the architecture doesn't change. Once evidence is mapped to controls, you project it onto whatever framework your auditor needs. The evidence is the same. The mapping is what changes.

Most compliance tools are built around a specific framework and work backwards to evidence. We built around evidence and work forwards to any framework. That's a fundamentally different architecture, and it's why we can support use cases that template-based platforms can't touch.

And when the mapping engine finds a control that your infrastructure doesn't satisfy? It doesn't fabricate evidence. It creates a gap. A real, trackable, prioritized gap with the specific control reference, what's missing, and what needs to happen to close it.

1,223 gaps across 48 work streams. 21 high risk. Each one traced back to a specific control, prioritized, and organized into a remediation plan your team can actually execute. That's what honest compliance automation looks like. Not "everything's green." A clear-eyed view of where you stand and what you need to fix.

We Helped Write the Standard

We didn't arrive at this by reading a whitepaper. TestifySec Director of R&D Frederick Kautz co-authored NIST SP 800-204D — the federal guidance for integrating software supply chain security into DevSecOps CI/CD pipelines. The DOD CIO references it as the definition for secure software supply chain. The document defines exactly what we're talking about: attestation, provenance, SBOM, and SLSA as the building blocks of verifiable compliance.

We also contributed to the Software Supply Chain Best Practices paper.

The point: the people building this platform are the same people who defined what good looks like at the federal standard level. We didn't read the spec and build a product. We wrote the spec, then built the product that implements it.

No Black Boxes

The core evidence infrastructure, Witness and Archivista, are open source projects. Audit the code yourself. Verify attestations are created correctly. Run your own instance if you want.

If your compliance platform is a black box, you're trusting the vendor. Recent events show where that gets you.

Five Questions to Ask Your Compliance Platform

Next time you're evaluating a compliance platform, or renewing the one you have, ask these:

Where does the evidence come from? If "our AI generates it" is the whole answer, and nobody's looking at your actual infrastructure, you're paying for creative writing.

Is it cryptographically signed? Unsigned evidence can be fabricated or backdated. Full stop.

Can someone outside your organization verify it? Black box platforms require you to trust the vendor. We've seen how that goes.

Does it update when you ship code? If not, your compliance posture is a snapshot. Probably from months ago.

Can you audit the attestation system itself? If the code isn't open source or independently audited, you're trusting the trust layer. That's circular.

The bar shouldn't be "does this report look professional?" It should be "can someone independently verify that this evidence reflects what's actually running?"

Compliance theater has real consequences. For the companies that rely on it, for the customers whose data they're supposed to protect, and for everyone trying to do this honestly.