Your AI Agent Configs Are Probably Broken (and You Don't Know It)

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • MyrinNew
    Senior Member
    • Feb 2024
    • 5168

    #1

    Your AI Agent Configs Are Probably Broken (and You Don't Know It)

    I've spent the last year wiring up AI coding tools: Claude Code, Cursor, Copilot, Cline, Codex CLI, and friends. Skills, hooks, memory files, MCP servers, agent defs, plugin manifests — the whole stack.


    Here's the annoying truth: a huge chunk of these configs are silently broken. And the tools mostly don't tell you.


    The problem nobody talks about

    If you misconfigure ESLint, it screams. If you misconfigure a SKILL.md, nothing happens. The skill just… never triggers. No error. No warning. It's like the file doesn't exist.


    Vercel even measured this: skills invoke at 0% without correct syntax. Not "less often". Zero. One wrong frontmatter field and your carefully written skill is invisible to the agent.


    And then you get the classic "almost right" output. Stack Overflow's 2025 developer survey has 66% of devs calling "almost right" their biggest AI frustration — and honestly, misconfigured agents produce exactly that. You think you gave the model the right rules/tools, but you didn't (or you did, just in the wrong format).


    It gets worse if you use multiple tools. Cursor for editing, Claude Code for terminal work, maybe Copilot for inline completions. Now you're maintaining parallel configs in different formats that are supposed to stay consistent. A rule that works in .cursor/rules/testing.mdc might contradict what you put in CLAUDE.md. Nobody catches it.


    What actually goes wrong in the real world

    After digging through official specs, research, and a lot of "why is this not working?" debugging, the failure modes repeat:


    Skills that never trigger:
    • YAML frontmatter is missing / malformed (most common)
    • Name is PascalCase instead of kebab-case
    • No trigger phrases → agent has no way to discover it
    • Invalid values in model/context fields


    Skills that trigger when they shouldn't:
    • A deploy/publish/delete skill without disable-model-invocation: true — Claude can auto-trigger it without you asking


    Hooks that quietly fail:
    • Event name is wrong (PreToolExecution instead of PreToolUse)
    • command missing (so the hook is basically a comment)
    • Script path points to nothing
    • Dangerous commands sneaking in (rm -rf in a hook with no guard… yeah)


    Memory files that make the agent worse:
    • Generic fluff like "be helpful and accurate" (wastes context; the model already knows)
    • Important rules buried mid-file (primacy/recency applies to LLM context too)
    • Files that are too long for the tool to reliably respect — Windsurf caps rules files at 12,000 chars, Copilot global instructions work best under ~4,000 — past those ceilings the tool either truncates or deprioritizes your instructions


    Cross-tool conflicts:
    • Your CLAUDE.md says npm test but your AGENTS.md says pnpm test — one agent runs tests correctly, the other doesn't
    • Cursor rules allow unrestricted Bash, but your CLAUDE.md disallows it — the agent's permissions depend on which tool you're using
    • Multiple instruction layers with contradictions (and good luck knowing which one actually takes precedence)


    MCP servers with protocol violations:
    • Missing required fields in tool defs
    • Invalid transport config
    • Schema mismatches that lead to tools "existing" but not actually working


    So I built a linter for this

    I couldn't find a tool that just tells you, plainly, across tools: "this config won't work".


    So I made one.


    agnix is a linter for AI agent configurations. It validates skills, hooks, memory files, plugins, MCP configs, and agent definitions across Claude Code, Cursor, GitHub Copilot, Cline, Codex CLI, OpenCode, and Gemini CLI.


    It's currently 156 rules. Every rule links to its source — an official spec, vendor docs, or research paper — with RFC 2119 severity levels (MUST vs SHOULD vs BEST_PRACTICE) so you know what's a hard requirement vs a recommendation. It's not "trust me bro".


    What it looks like:






    $ npx agnix .

    CLAUDE.md:15:1 warning: Generic instruction 'Be helpful and accurate' [fixable]
    help: Remove generic instructions. Claude already knows this.

    .claude/skills/review/SKILL.md:3:1 error: Invalid name 'Review-Code' [fixable]
    help: Use lowercase letters and hyphens only (e.g., 'code-review')

    .claude/settings.json:12:5 error: Script file not found: './scripts/lint.sh'
    help: Create the file or fix the path

    .cursor/rules/testing.mdc:1:1 error: Missing required frontmatter
    help: Add YAML frontmatter with description, globs, and alwaysApply fields

    Found 3 errors, 1 warning
    2 issues are automatically fixable

    hint: Run with --fix to apply fixes







    It also auto-fixes.
    • agnix --fix . rewrites configs to comply with specs
    • agnix --fix-safe . applies only high-certainty fixes (things like normalizing a skill name to kebab-case — not changes that might alter semantic meaning)


    What it covers

    Claude Code:
    • CLAUDE.md, hooks, agents, plugins, skills
    • Frontmatter errors, invalid models, broken script paths, generic instructions, dangerous auto-invocation, manifest issues


    Cursor:
    • .cursor/rules/*.mdc, .cursorrules
    • Missing frontmatter, invalid globs, deprecated formats, boolean-vs-string alwaysApply


    GitHub Copilot:
    • .github/copilot-instructions.md, .github/instructions/*.instructions.md
    • Empty files, missing applyTo frontmatter in scoped instructions, invalid glob patterns, file length limits


    MCP:
    • *.mcp.json
    • Protocol violations, schema errors, transport config


    AGENTS.md:
    • AGENTS.md, AGENTS.local.md
    • Tool-specific size limits, platform-specific features without guards, nesting issues


    Cline:
    • .clinerules, .clinerules/*.md
    • Structure + validation


    Codex CLI:
    • .codex/config.toml
    • Config validation


    OpenCode:
    • opencode.json
    • Schema validation


    Gemini CLI:
    • GEMINI.md
    • Format validation


    There are also cross-platform rules (XP-*) that detect conflicts between tools, and prompt-engineering rules (PE-*) that catch patterns that consistently degrade behavior.


    How it works (quick)

    It's written in Rust.

    1. Walk the repo (respect .gitignore)
    2. Detect file type by path (SKILL.md, .mdc, settings.json, etc.)
    3. Validate in parallel across CPU cores
    4. Emit diagnostics + suggested fixes


    Typical performance: single file under 10ms, 100-file project ~200ms, 1000-file project ~2s.


    The rules stay current — a CI workflow monitors upstream specs weekly and flags when vendor documentation drifts from what agnix expects.


    Not just a CLI

    agnix also runs as an LSP, so you get real-time validation in your editor:

    There's also an MCP server (agents can lint their own configs), plus a GitHub Action for CI:






    - name: Validate agent configs
    uses: avifenesh/agnix@v0
    with:
    target: 'claude-code'







    Try it





    npx agnix .







    That's it. Run it on your repo. In my experience, almost every repo with agent configs has a few issues — and they're usually the silent ones that have been dragging quality down for weeks.


    If you want a "real" install:






    npm install -g agnix # npm
    brew tap avifenesh/agnix && brew install agnix # Homebrew
    cargo install agnix-cli # Cargo







    MIT/Apache-2.0, open source: github.com/avifenesh/agnix


    Why I think this matters

    The ecosystem is fragmenting fast. Every tool invented its own format, its own conventions, and its own special ways to fail quietly.


    We have linters for code. For configs. For IaC. But the layer that tells AI agents how to behave — the stuff sitting between you and basically every AI interaction — had nothing.


    agnix doesn't tell you what to write. It tells you when what you wrote won't work.





    agnix is open source and free. GitHub | Docs | npm




    More...
Working...