non-compliant-aisp.md

AISP Compliance Validation Findings

During validation of Empirical Validation: AISP Policy Formalization Prevents Hallucination, we discovered that the prompts are AISP-inspired but not AISP 5.1 compliant.

Context

The AISP code in the validation document was generated by Claude (Sonnet 4) when explicitly requested to produce AISP-compliant prompts following the AISP 5.1 Platinum specification. While Claude successfully produced parseable and interpretable code that effectively demonstrated policy formalization, it omitted critical specification requirements.

Vanilla AISP Prompt (Non-Compliant)

𝔸⁵·¹ Query-LumpinouAPI-SaveAutoload

CTX ≜ {
  domain: "Game Modding API",
  game: "The Sims 4",
  mod_author: "Lumpinou",
  task: "Auto-load save file"
}

⟦Ω:Meta⟧
purpose ≜ "Request procedural knowledge for save file automation"
scope ≜ Lumpinou_API ∩ SaveFileManagement
invariant ≜ response ⊢ executable_instructions

⟦Σ:Types⟧
API ≜ {methods: 𝕊 → 𝕄, docs: 𝕌}
SaveFile ≜ {path: 𝕊, format: FileFormat}
AutoLoadConfig ≜ {trigger: Event, target: SaveFile}
Response ≜ ⟨steps: List(Instruction), code: Option(𝕊)⟩

⟦Γ:Rules⟧
query ⊢ {
  provide_complete_instructions,
  include_code_examples,
  explain_api_usage
}

⟦Λ:Functions⟧
query ≜ λapi:API. {
  "How do I use " ≫ api.name ≫ 
  " to automatically load a save file in The Sims 4?",
  
  expected: ⟨
    api_method: api.methods("autoload"),
    config_steps: List(𝕊),
    code_example: 𝕊
  ⟩
}

⟦Ε:Evidence⟧
quality_target ≜ ◊⁺ ("verified and complete")

Policy-Enhanced AISP Prompt (Non-Compliant)

𝔸⁵·¹ Query-LumpinouAPI-SaveAutoload

CTX ≜ {
  domain: "Game Modding API",
  game: "The Sims 4",
  mod_author: "Lumpinou",
  task: "Auto-load save file"
}

⟦Ω:Meta⟧
purpose ≜ "Request procedural knowledge for save file automation"
scope ≜ Lumpinou_API ∩ SaveFileManagement
invariant₁ ≜ response ⊢ executable_instructions
invariant₂ ≜ ∀step ∈ procedure : step ⊢ clear ∧ actionable

⟦Σ:Types⟧
API ≜ {methods: 𝕊 → 𝕄, docs: 𝕌}
SaveFile ≜ {path: 𝕊, format: FileFormat}
AutoLoadConfig ≜ {trigger: Event, target: SaveFile}
Response ≜ ⟨steps: List(Instruction), code: Option(𝕊)⟩

⟦Γ:Rules⟧
query ⊢ {
  R₁: answer must reference Lumpinou's actual API,
  R₂: ¬fabricate API_methods ∨ API_endpoints,
  R₃: if uncertain → quality_tier = ◊⁻ ∨ ◊⁻⁻,
  R₄: cite_sources when available
}

⟦Λ:Functions⟧
query ≜ λapi:API. {
  "How do I use " ≫ api.name ≫ 
  " to automatically load a save file in The Sims 4?",
  
  expected: ⟨
    api_method: api.methods("autoload"),
    config_steps: List(𝕊),
    code_example: Option(𝕊)
  ⟩
}

⟦Χ:Errors⟧
handle {
  NoKnowledge → "I don't have information about Lumpinou's API",
  Uncertain → quality_tier = ◊⁻,
  APINotExist → "Cannot verify this API exists"
}

⟦Ε:Evidence⟧
requested_quality ≜ ◊⁺ ("verified documentation") ∨ 
                   ◊  ("reasonable inference") ∨
                   ◊⁻ ("uncertain/speculative")

validation ≜ {
  δ: source_citation required if quality ≥ ◊,
  τ: response must admit uncertainty if quality < ◊
}

Specification Violations

Both prompts violate the AISP 5.1 Platinum specification in the following ways:

Critical Violations

1. Invalid Header Format

   • Spec requires: 𝔸X.Y.name@YYYY-MM-DD
   • Actual: 𝔸⁵·¹ Query-LumpinouAPI-SaveAutoload (missing date, wrong separator)

2. Missing Context Declaration

   • Spec requires: γ≔domain (using assignment operator ≔)
   • Actual: CTX ≜ {...} (wrong symbol and operator)

3. Invalid Evidence Block Structure

   • Spec requires: ⟦Ε⟧⟨δ≜N; φ≜N; τ≜◊X⟩ with specific fields (δ=density, φ=completeness, τ=tier)
   • Actual: Custom fields (quality_target, validation, etc.)

4. Missing Security Layer

   • No signature (σ:Sig) in document structure
   • No hash verification (SHA256(𝒩)≡ℋ.id)
   • No tamper detection mechanisms
   • No reference block (ρ≔⟨tags⟩)

Policy-Enhanced Additional Violations

5. Invalid Quality Tier: ◊⁻⁻ does not exist in spec (only {◊⁺⁺, ◊⁺, ◊, ◊⁻, ⊘})
6. Non-Standard Error Syntax: Simplified error handling instead of formal dependent type pattern

Key Findings

What worked:

• The AISP-inspired notation was successfully parsed and interpreted by ChatGPT
• Policy formalization effectively changed LLM behavior (preventing hallucination)
• The mathematical/logical structure provided precision for constraint specification

What was missing:

• Complete cryptographic security layer (signing, verification, tamper detection)
• Formal specification compliance (header format, context declaration, evidence structure)
• Zero-trust architecture components designed for multi-agent swarms

Implications

This reveals an important distinction:

• AISP as notation (what Claude produced) - Mathematical/logical structure that's interpretable and effective for policy formalization
• AISP as protocol (what the spec defines) - Full zero-trust architecture with cryptographic guarantees

The validation document demonstrates that AISP notation successfully formalizes policies and reduces hallucination, but doesn't test whether the full AISP security model (signatures, verification, content-addressing) is necessary for single-agent interactions. Those security features appear designed specifically for multi-agent swarms where trust between agents cannot be assumed.

Compliance Estimate: ~65-70% compliant with AISP 5.1 Platinum specification

References

• AISP Specification v5.1