Audit System

Immutable audit logging with cryptographic integrity verification

Executive Summary

RayRay implements an append-only audit logging system designed to satisfy NIST 800-53 audit control family requirements. Each event is cryptographically chained using SHA-256 hashes, creating a tamper-evident log that supports forensic analysis and compliance reporting.

Architecture

Hash Chain Integrity

Each audit event includes a hash computed over the event content plus the previous event's hash. This creates a blockchain-style chain where modifying any historical event would break all subsequent hashes.

Event Structure:
┌─────────────────────────────────────┐
│ _id: "evt_abc123"                   │
│ _timestamp: "2026-03-12T15:30:00Z"  │
│ _prev_hash: "sha256:7f8a..." ◀──────┼── Previous event's hash
│ _hash: "sha256:3b2c..."             │   (computed with prev_hash)
│ event_type: "review_approve"        │
│ user_id: 42                         │
│ resource_type: "extraction"         │
│ resource_id: 123                    │
│ outcome: "success"                  │
└─────────────────────────────────────┘

Storage Backend

Environment	Backend	Configuration
Development	FileAuditBackend	`.security/audit/` directory
Production	S3/Azure Blob	Pluggable storage interface

NIST Control Mappings

Control	Requirement	Implementation
`AU-2`	Audit Events	Comprehensive CUD operation logging
`AU-3`	Content of Audit Records	Timestamp, user, IP, resource, action, outcome
`AU-4`	Audit Storage Capacity	Daily rotation, configurable retention
`AU-5`	Response to Audit Failures	Alerting on write failures
`AU-6`	Audit Review	Query API for analysts and auditors
`AU-9`	Protection of Audit Information	SHA-256 hash chain prevents tampering
`AU-11`	Audit Record Retention	7-year default (configurable)

Event Types

Event Type	Description	Logged Data
`document_upload`	Document submitted	filename, file_size, user_id
`extraction_start`	AI extraction initiated	document_id, ai_model, ai_provider
`extraction_complete`	AI extraction finished	extraction_count, confidence, processing_time
`review_approve`	Human approved extractions	workflow_run_id, extraction_count, notes
`review_modify`	Human modified extractions	workflow_run_id, modified_fields
`review_reject`	Human rejected extractions	workflow_run_id, rejection_reason
`export_download`	Report exported	format, document_count, comparison_id

API Endpoints

Query Audit Events

GET /api/audit/events?start_time=2026-03-01&end_time=2026-03-12&event_types=review_approve,review_reject

Response:
{
  "events": [
    {
      "_id": "evt_abc123",
      "_timestamp": "2026-03-12T15:30:00Z",
      "event_type": "review_approve",
      "user_email": "reviewer@agency.gov",
      "resource_type": "extraction",
      "outcome": "success"
    },
    ...
  ],
  "total": 147,
  "has_more": true
}

Verify Integrity

GET /api/audit/verify

Response:
{
  "valid": true,
  "events_checked": 1847,
  "date_range": {
    "start": "2026-01-15",
    "end": "2026-03-12"
  },
  "first_failure": null,
  "errors": []
}

Export for External Audit

GET /api/audit/export?format=json

Response: application/jsonl
(event log as JSON Lines format)

Integrity Verification Procedure

The verification endpoint recomputes all hashes and validates the chain integrity. This procedure should be run:

Before any compliance audit
After system maintenance
As part of continuous monitoring (automated daily)

# Automated verification (cron)
0 6 * * * curl -s https://rayray.example.com/api/audit/verify | grep -q '"valid":true' || alert-team.sh

Retention Policy

Event Category	Retention Period	Rationale
Security Events	7 years	FedRAMP/NIST requirement
Review Events	7 years	M-25-21 audit trail
Export Events	3 years	Operational tracking

Compliance — NIST control framework
API Reference — Full endpoint documentation