Key Takeaways
AI-generated code can spread inconsistent patterns across services faster than manual review can catch them.
Engineering standards need to be machine-readable so they can be evaluated automatically across services.
Developers and AI assistants need access to current engineering context (ownership, dependencies, what "good" looks like) while they work, not a wiki snapshot or manually pasted notes.
Continuous evaluation and time-bound remediation turn compliance from a dashboard into a discipline.
An Engineering Operations Platform can centralize this: codify standards, make context accessible, evaluate services, route gaps to owners, and measure whether AI adoption improves outcomes or adds risk.
Engineering standards exist in every organization. They cover how services communicate, how secrets are managed, who reviews changes before they merge, and what "production-ready" means. The problem is where those standards live: wiki pages, onboarding docs, the memory of whoever happens to review a pull request that day.
Conventional enforcement depended on reviewer knowledge, which varied across teams. AI coding agents carry no organizational context between sessions and produce changes faster than review can absorb.
Policy as code addresses this gap by expressing engineering standards as machine-readable rules that systems can evaluate automatically, rather than relying on people to interpret and enforce prose documentation by hand. For AI coding, though, machine-readable rules alone are not enough. They work best inside a broader operating model: one that makes current engineering context available to developers and AI assistants while they work, evaluates services continuously against shared standards, and routes gaps to accountable owners.
This article covers why undocumented standards break down under AI-generated volume, what policy as code means in this context, how to build the operating model around it, and where Cortex fits.
Why wikis and reviewer memory don't scale to AI coding
Most engineering organizations have standards. They specify which services can talk to each other directly, how secrets are stored, what test coverage looks like for a production service, and who needs to approve a change before it merges. The trouble is that those standards are often scattered across wikis, onboarding guides, and the institutional knowledge of senior engineers.
For human developers, that arrangement is inefficient but survivable. Engineers lose hours hunting for the right doc or tracking down the one person who knows how a system is supposed to work. Still, they can ask a colleague, and experience tells them when something looks wrong.
AI coding agents have no equivalent. As Birgitta Böckeler observed in her Thoughtworks analysis of agent controls, a coding agent carries no organizational memory of "how we do things here." It operates with whatever context the current session provides. If an agent is generating a new service and no one has told it that your architecture requires services to call another domain's published API rather than reading from its datastore directly, it will pick whatever pattern it finds in the code it can see. The result is a pull request that looks perfectly valid, but violates an architectural convention, and then it’s up to a human reviewer to catch it.
As AI tools expand across the development lifecycle, the volume of changes flowing through the system grows while the number of reviewers stays roughly the same. The 2026 Stack Overflow Developer Survey found that 73% of developers using AI coding assistants use them daily, with a third of daily users spending four or more hours per day with AI tools. At the same time, only 6.6% of respondents said they trust AI output for important decisions. DORA's 2025 research reinforced the point from a different angle: AI acts as an amplifier of existing strengths and weaknesses, which means organizations with undocumented or inconsistently applied standards get more of what they already have.
The concrete failure modes add up. Agents replicate whatever patterns they find, spreading architectural drift across services. Conventions around secrets management or vulnerability scanning stay inconsistent when they exist only as tribal knowledge. And review queues cannot absorb AI-generated volume without either slowing down or letting things through.
What policy as code means for AI coding
The Open Policy Agent project, now part of the CNCF, defines policy as code as decoupling policy from the service it governs so that policy owners can write, version, and analyze rules separately from the systems those rules apply to. Instead of a human interpreting a wiki page to decide whether a change is compliant, a machine evaluates the rule automatically.
That definition covers the conventional use case well: an OPA policy that checks an API request, or a CI step that blocks a deployment missing a required label. For AI coding, though, the scope of the problem is wider. Machine-readable rules are necessary, but they work best when developers and AI assistants can also access current information about what exists, who owns it, and what the standards are. Continuous evaluation should run against services outside the PR cycle, not only when a change is proposed. When evaluation finds a gap, it needs to route to a specific owner and give a reasonable “fix-by” deadline.
Policy as code is not just a config file or a PR gate
Repo-level rules files like AGENTS.md, .cursorrules, and copilot-instructions.md are useful. They give agents project-specific guidance on setup commands, test workflows, and coding style. However, they are scoped to one repository, maintained separately in each project, and they provide guidance rather than evaluation. Telling an agent how to format a PR is a different problem from checking whether 200 services across the organization meet the same security baseline and identifying which ones don't.
Linters and CI checks have a similar boundary. They evaluate individual changes in a single repository at merge time. Centralized policy as code operates at the organization level and runs continuously, regardless of whether a change has been proposed.
Exceptions belong in the system, not around it
Some services will have legitimate reasons to deviate from a standard. Those exceptions need to be explicit, approved, justified, and time-bound. Exceptions recorded inside the policy system define an owner, a justification, and an expiry date, which makes them visible during audits and surfaceable in reports. Recurring exceptions are also a signal: if the same rule keeps generating approved deviations, the rule itself or the underlying platform capability may need attention.
How to build a policy-as-code layer for AI coding
Codify standards as enforceable rules
Abstract principles like "all services must be production-ready" are hard to evaluate automatically. Useful policy-as-code rules are specific enough that a system can check them against data: a service requires at least two PR reviewers, CODEOWNERS reviews are enabled, secrets scanning runs in CI, there are no critical vulnerabilities, and AI usage or security documentation exists in the repository.
Some standards are harder to codify. Verifying that services access other domains only through approved APIs, for example, may require architectural analysis or custom metadata that standard repository and catalog data does not provide. These still benefit from being stated as policy, but they may need attestations or human judgment rather than fully automated checks.
Make governed context available while developers and AI assistants work
DORA's 2025 AI Capabilities Model names "AI-accessible internal data" as one of seven capabilities that amplify AI's positive effect, and defines it as giving AI tools secure access to internal documentation, codebases, and organizational data. Without that access, both developers and agents fall back on whatever is available in the current session: a stale wiki page, a manually pasted snippet, or nothing at all.
A live, connected source of truth that tracks which services exist, who owns them, what their dependencies are, and what standards apply gives both people and agents the same current picture of the engineering environment.
Evaluate continuously and route gaps to owners
Evaluation that runs only at merge time catches problems one PR at a time, in one repository at a time. Continuous evaluation checks all applicable services against organization-wide standards on an ongoing basis, whether or not anyone has proposed a change.
Each gap should route to the service owner with a deadline and a notification, so the workflow and effort ultimately results in a fix being deployed.
How Cortex operationalizes policy as code for AI coding
Cortex is an Engineering Operations Platform that centralizes engineering data, codifies and evaluates standards, turns gaps into action, and measures improvement. Policy as code is one application of a broader AI excellence framework: define what "good" looks like, make it accessible, evaluate against it, remediate what falls short, measure whether outcomes improve, and review progress on a regular cadence.
Cortex complements coding agents, CI pipelines, and PR-review tools. It does not write code, control agent output, or act as a PR gate.
Codify standards with Scorecards
A Bronze-level rule checking for two or more PR reviewers runs git.branchProtection().numReviewsRequired > 1 against every applicable service. That is what a standard looks like inside a Cortex Scorecard: a Cortex Query Language (CQL) query that evaluates automatically against applicable Cortex entities using data pulled from connected systems.
Cortex provides a prebuilt AI Governance Scorecard template organized into three maturity levels. The template is editable, and the Cortex documentation covers the full rule set and the CQL behind each control.
Make governed context accessible with Catalogs, the Context Graph, and Cortex MCP
When a developer asks "who owns the payments service?" or an agent needs to know which dependencies a service exposes, that answer has to come from an up-to-date source. Cortex Catalogs and the Context Graph maintain that connected map of services, ownership, dependencies, and other engineering data, while Scorecards define and evaluate standards against it.
Cortex MCP makes this context and these Scorecard results queryable through developers' AI assistants and IDEs, so both people and agents can work from current engineering information rather than manually supplied or outdated notes. Cortex MCP does not automatically inject policies into every agent or control code generation: it exposes the information for developers and agents to pull when they need it.
Embed standards from day one and track remediation
A service created from a generic repository template will not have any of the organization's requirements defined: no CI configuration, no CODEOWNERS file, no AI security documentation. Cortex Workflows, golden paths, and service scaffolding solve this by giving every new service the right structure, documentation, and configuration at creation time.
For services that already exist, Cortex Initiatives turn Scorecard gaps into owned, time-bound remediation. Each Initiative ties to a Scorecard, carries a deadline, and notifies entity owners whose services are not meeting the goal.
Measure whether AI-generated code helps or adds risk
Has incident frequency changed since your teams started using AI coding assistants? Is cycle time improving, or are more bugs reaching production? Cortex AI Impact connects AI tool usage to these engineering outcomes, including mean time to recovery, deployment frequency, and change failure rate, so teams can answer those questions with data.
Teams can ask Cortex MCP, "How is my AI Governance Scorecard doing?" and use the Bird's Eye report to visualize Scorecard performance across services.
A practical starting point is a small set of directly measurable rules (PR reviewer requirements, secrets scanning, vulnerability status), with coverage expanding as the practice matures. Connecting those evaluations to engineering outcome metrics like incident frequency and MTTR helps confirm whether standards are keeping pace with AI-generated code volume.
Review policy results and compounding risk with the OpEx Review Agent
Scorecards and Initiatives show which services meet a standard. Engineering leaders still need a regular forum to decide where to focus when gaps pile up across teams. In most organizations that forum is the operational excellence (OpEx) review, and preparing for it usually means pulling data from several tools by hand.
The Cortex OpEx Review Agent runs that review. It pulls Scorecard results, Engineering Intelligence metrics, incidents, SLOs, and vulnerabilities into one view organized around the DRIVE framework. Then it reasons across those signals to find where risk is compounding. A service that fails its AI Governance Scorecard, breaches an SLO, and carries critical vulnerabilities is a bigger problem than any one of those signals alone, and the agent names the teams and services where that overlap occurs. Leaders get an executive briefing before the meeting. During the review, they can ask the agent why a metric moved or which teams are driving it, and each answer links back to the underlying data.
The OpEx Review Agent does not change policies or fix violations on its own. It shows leaders where standards are slipping so they can decide where to direct remediation.
Frequently asked questions
How can teams help AI coding agents work within their engineering standards?
The most direct step is making standards and context available where agents can access them. Repo-level rules files like AGENTS.md give agents project-specific guidance on coding style, test workflows, and PR conventions. Beyond the repository, teams benefit from a live source of engineering context that agents can query for ownership, dependencies, and organizational standards. DORA's 2025 AI Capabilities Model identifies "AI-accessible internal data" as one of seven capabilities that amplify AI's positive effect, and recommends giving AI tools secure access to internal documentation and codebases.
Which engineering standards can be codified as policy, and which still need human judgment?
Standards that can be checked against observable data are good candidates for automation: PR reviewer requirements, branch protection settings, secrets scanning in CI, vulnerability counts, and the presence of required documentation files. Standards that involve architectural intent, design tradeoffs, or risk assessment are harder to automate fully. Assessing whether a model's training data complies with your organization's data privacy requirements, for example, may need human review and attestation rather than an automated scan. Frameworks like the NIST AI Risk Management Framework (AI RMF) provide structured guidance for AI-related risk decisions that involve judgment, but organizations still need to translate that guidance into the specific rules and review processes that fit their environment.
Does policy as code replace code review?
No. The 2026 Stack Overflow Developer Survey found that only 6.6% of developers trust AI output for important decisions, and 78.7% said source attribution matters when evaluating AI-generated code. Human review remains necessary for catching problems that automated rules cannot express: subtle logic errors, security implications that depend on business context, and architectural decisions that require judgment. Policy as code reduces the burden on reviewers by catching the codifiable violations automatically, so reviewers can focus on the problems that require human reasoning.
How is this different from giving each agent a rules file or project config?
Rules files like AGENTS.md provide project-level guidance scoped to a single repository. They are useful for telling an agent how to run tests, format commits, or follow local coding conventions. They do not evaluate services against organization-wide standards, track compliance across hundreds of repositories, or route gaps to owners. In December 2025, OpenAI contributed the AGENTS.md format to the Agentic AI Foundation under the Linux Foundation, co-founded with Anthropic and Block. InfoQ's coverage noted that developers do not expect the format to remove the need for human oversight.
How do you measure whether your policy-as-code layer is actually working?
Connect your standards to engineering outcome metrics. If an AI Governance Scorecard is improving compliance across services, the downstream effects should be visible in metrics like incident frequency, mean time to recovery, and change failure rate. GitGuardian's 2026 State of Secrets Sprawl report found 28.65 million new hardcoded secrets in public GitHub commits in 2025, a 34% year-over-year increase, with AI-assisted commits showing roughly twice the baseline secret-leak rate. Tracking Scorecard compliance alongside these signals helps teams see whether their standards are reducing the specific risks that AI-generated code introduces. Veracode's 2026 GenAI Code Security Report found an average security pass rate of 56% across more than 100 models tested without security-specific prompting, which reinforces why stated and measured standards matter more than relying on model defaults.
--
Ready to see how policy as code works inside Cortex? Book a demo to walk through Scorecards, Initiatives, AI Impact, and the OpEx Review Agent with your own engineering data.
