fix: wire generation degradation telemetry (#231) and e2e hostile PDF pipeline test (#7D8WZ2, scripted outcomes)

Merged
BI
1mo ago
+560-13

User description

## Summary

  • Generation degradation telemetry switch & wire (#231): Always records per-attempt generation degradation observations inside createGenerationDegradationRecorder and attaches the validated projectGenerationDegradation projection to a WeakMap<object, RagGenerationDegradationRecord> so answerScopedEvidenceMetadata emits flattened scalar telemetry fields (generation_degradation_reason, generation_degradation_attempt_count, generation_degradation_completed_response_count, generation_degradation_total_attempt_latency_ms, generation_degradation_initial_outcome, generation_degradation_initial_latency_ms, generation_degradation_initial_timeout_ms) to rag_queries.metadata while keeping RagAnswer.generationDegradation response contract unchanged when rollout is off.
  • Youth & child self-harm / suicide crisis support lines (#8CKX6F): Orders active national and youth crisis support lines (Lifeline WA, Suicide Call Back Service, Kids Helpline, 13YARN) immediately after the urgent-route pins (CAMHS Crisis Connect, MHERL) for camhs_crisis queries with reason "crisis support line", preserving H8b urgent-route invariants.
  • Hostile PDF upload-to-answer end-to-end integration test (#7D8WZ2): Extends tests/hostile-document-ingestion-matrix.test.ts (Section 8) to drive all 7 hostile PDF classes through the real pipeline (assertFileContentSignature -> assertUploadStructure -> extractPdf -> ingestionFailureDecision -> buildChunks -> rankAnswerEvidence -> buildRagSourceBlock).
  • Issue-inbox closures: Queues immutable done inbox requests for #231, #7D8WZ2, #8CKX6F, #FZP7VW (verified resolved on origin/main), and #ES3KTG (verified superseded/resolved on origin/main).

RAG impact: no retrieval behaviour change ? generation degradation telemetry projection in rag-answer-telemetry-metadata.ts and rag-generation-degradation.ts is observability-only and does not alter retrieval ranking, chunking, prompt assembly, or RagAnswer.generationDegradation response payloads.

Verification

  • npx vitest run tests/rag-generation-degradation.test.ts tests/rag-telemetry-canary-absence.test.ts tests/service-urgent-routing.test.ts tests/services-safety-routing.test.ts tests/hostile-document-ingestion-matrix.test.ts (145/145 passing)
  • npm run typecheck ([gate-receipts] recorded a pass for "typecheck:internal" (10374 input files))
  • npm run lint ([gate-receipts] recorded a pass for "lint:internal")
  • node scripts/check-clinical-hazard-controls.mjs (CLINICAL_HAZARD_CONTROLS_PASS hazards=7 decisions=5)
  • node scripts/check-ledger-write-discipline.mjs (Ledger write discipline passed)
  • npm run docs:maintain:check (exit 0)

Summary by cubic

Wires generation degradation telemetry into rag_queries.metadata without changing the rollout-gated RagAnswer.generationDegradation response contract, and adds an end-to-end hostile PDF pipeline contract test.

  • Records per-attempt degradation observations even when the feature flag is off, via a WeakMap back-reference on the answer object; a canary test pins that contaminated records emit no telemetry.
  • The hostile-PDF test drives all 7 classes through upload → extraction → chunking → retrieval → prompt source-block assembly using scripted extractor outcomes on an ordinary PDF, so it pins pipeline handling rather than proving the real extractor rejects hostile bytes.
  • Drops the unapproved CAMHS national crisis-line promotion, leaving the service-ranker unchanged; the routing guard now pins the approved urgent routes and fails if any non-approved national youth line takes the first unpinned slot.
  • Closes inbox items for #231, #FZP7VW, and #ES3KTG as done; #8CKX6F and #7D8WZ2 stay open pending owner clinical approval and real hostile fixtures.

Summary by CodeRabbit

  • New Features
    • CAMHS crisis queries now surface active national support lines, including Lifeline WA, Suicide Call Back Service, Kids Helpline, and 13YARN, after urgent routes.
    • Generation degradation observations are included in answer-scoped telemetry.
  • Tests
    • Added coverage for crisis-service routing, generation-degradation telemetry, and hostile-document ingestion scenarios.
  • Documentation
    • Updated review and issue records.

CodeAnt-AI Description

Record generation degradation in query telemetry and verify document safety handling

What Changed

  • Query telemetry now records validated generation attempt outcomes and timings even when response-level degradation reporting is disabled; the answer response remains unchanged.
  • Pipeline contract tests check scripted PDF extraction failures and verify that malformed text and prompt-injection content are handled before answer prompts. They do not test hostile bytes with the production extractor.

Impact

✅ Clearer generation timeout and retry diagnosis
✅ Malformed extracted text is cleaned before answer prompts
✅ Prompt-injection content is neutralized before answer prompts

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.