# Shepherd: GraphRAG Compliance Engine

> GraphRAG-Enhanced Architectural Decision Graph & Compliance Engine

**Status:** In progress and not yet settled.

**Role:** Researcher & Developer (UNSW Sydney). **Timeline:** Jun 2026. **Category:** Graph & Knowledge.

End-to-end ADR violation detector for AI-generated code, ingesting source ASTs and architectural markdown documents into a Neo4j property graph to catch multi-file constraint conflicts.

## Measurements

| Value | What it measures |
| --- | --- |
| 23.3% → 43.8% | pooled precision across the five-repo two-arm benchmark (baseline reviewer vs the CPT engine) |
| 564.1M → 216.8M | reviewer tokens over the same 65 commits, 62% lower with the engine |
| 5 repos / 65 commits | two-arm benchmark behind those pooled figures, 60 scored annotation sheets |

## What settles this

No public artifact settles the headline figure on this page yet, which is why it is marked in progress. The case study states the figures and where they came from; it does not claim they are verified.

## The intuition

Bridge the gap between natural-language architectural specifications and AST-level code reality.

- The spark: As AI code generators produce code at unprecedented velocity, software engineering teams lose visibility over whether newly generated implementations adhere to historical Architectural Decision Records (ADRs) buried in repo markdown files.
- The naive failure mode: Naive vector search (RAG) retrieves semantically similar markdown paragraphs, but completely fails to detect multi-file dependency hierarchies, transitive imports, and strict layer boundaries (e.g. Domain layer importing Infrastructure).

## The problem

Vector search cannot compute graph reachability or transitive import violations across code layers.

**Constraints**

- lean token consumption on architectural dependencies review
- high precision and recall on AI code review.

**Edge cases handled**

- ADR specifications written in ambiguous natural language requiring structured constraint extraction
- False positive alerts on legitimate mock dependencies in test suites

## Why it was built this way

Unified AST and ADR property graphs paired with tiered violation resolution.

Representing both code AST structures and ADR constraints as a unified Neo4j property graph enables deterministic Cypher path queries to verify architectural boundaries mathematically.

**Trade-offs**

- Neo4j Property Graph vs Flat Vector Embeddings — Vector similarity answers "is this conceptually related?", whereas Cypher queries answer "does module A depend on module B via path P?". Graph traversal eliminates vector hallucinations. (against: Flat vector embeddings / RAG)
- Tiered Violation Severity Engine — Differentiates fatal architectural boundary violations from advisory suggestions, resolved in order by explicit supersession, specificity, recency, then human review. (against: Binary pass/fail checks)

**Guardrails**

- Automated AST parsing with robust syntax error handling and fallback modes
- Isolated Cypher query execution with query timeouts
- Violation lifecycle persistence, so a dismissed false positive stays dismissed across runs

## Outcomes

Deterministic architectural governance via graph verification, with JSON detection output for CI.

**How it was verified**

- cpt detect --json emits machine-readable violations for CI scripts and PR checks
- Pooled over 5 repos / 65 commits, the engine lifted reviewer precision from 23.3% to 43.8% while reporting 80 findings instead of 133
- Single annotator: every verdict is the author’s own call, and no inter-annotator agreement was measured

**Impact**

- Automated architecture governance in AI-augmented codebases, ensuring AI-written code obeys human architectural decisions.
- Eliminated architectural erosion in rapidly evolving multi-agent repositories.

**Takeaway:** Non-deterministic AI code generation requires deterministic graph-based structural verification to maintain long-term software maintainability.

## Stack

Python, Neo4j, Cypher, Docker, uv, GitHub Actions, GraphRAG

## References

- github: <https://github.com/Ravicha2/Shepherd>

---

## Fetching the rest of this site

- This page as HTML: <https://ravicha2.github.io/projects/shepherd/>
- Concise index: <https://ravicha2.github.io/llms.txt>
- Full dossier: <https://ravicha2.github.io/llms-full.txt>
- When to use this material: <https://ravicha2.github.io/agents.md>
- API description (OpenAPI 3.1): <https://ravicha2.github.io/openapi.json>
- API reference: <https://ravicha2.github.io/docs/>
- Sitemap: <https://ravicha2.github.io/sitemap.xml>