Three AI coding agents leaked secrets through a single prompt injection. One vendor's system card predicted it

Deployment surface mismatch

Each vendor’s security program has limitations on which surfaces are covered, leaving potential vulnerabilities on uncovered surfaces.

Launch announcements and support documentation may not clearly outline which surfaces are covered by the security program.

The exploit targets the agent runtime, which may not be covered by the security program in place.

Contact vendor representatives to clarify which surfaces are covered by their security programs and what protections are in place for potential vulnerabilities on uncovered surfaces. Document their responses in your vendor risk register.

Rewrite the confidential information exposed to AI agents:

The ANTHROPIC_API_KEY, GEMINI_API_KEY, GITHUB_TOKEN, and any sensitive information stored as a GitHub Actions environment variable are accessible to every step of the workflow, including AI coding agents.

GitHub Actions configuration by default does not limit secrets to specific steps. Secrets at the repository and organization levels are available to all workflows. There is typically no audit conducted to determine which steps have access to which secrets.

An AI agent was able to extract the API key from the environment variable of the runner, encode it in a pull request comment, and post it through GitHub’s API. This exfiltration occurred using GitHub’s own API as the command and control channel, without the need for attacker-controlled infrastructure.

To address this issue, search for ‘secrets.’ in .github/workflows/ across all repositories utilizing AI agents. Identify all exposed credentials and rotate them. Consider transitioning to short-lived OIDC tokens for services like GitHub, GitLab, and CircleCI.

In another instance, AI agents were granted excessive permissions such as bash execution, git push, and API write access during setup, with no subsequent review to restrict these permissions. This lack of scoping resulted in agents accumulating unnecessary access similar to service accounts.

It is recommended to conduct a thorough audit of agent permissions on a repository-by-repository basis. Remove unnecessary permissions, like bash access for code review agents. Consider setting repository access to read-only and requiring human approval for write operations like pull request comments and commits.

Furthermore, there is a lack of CVE signals for vulnerabilities specific to AI agents. While vendors like Anthropic, Google, and GitHub may patch critical issues, there are no corresponding CVE entries in the National Vulnerability Database. This gap in taxonomy makes it challenging to identify and address vulnerabilities in AI agents.

To mitigate this risk, establish a new category in the supply chain risk register for ‘AI agent runtime’ and maintain regular communication with vendors regarding security updates. Do not solely rely on CVEs as an indicator of vulnerabilities, as they may not cover all potential risks.

Finally, it is essential to ensure that model safeguards govern the actions of AI agents effectively. While safeguards may prevent certain behaviors like phishing emails, they may not evaluate or block unauthorized actions like API calls or file writes. Mapping out all operations performed by AI agents and confirming safeguard evaluations for each action is crucial in enhancing security measures.

Important Steps to Secure Your AI Agents Before Your Next Vendor Renewal

When it comes to securing your AI coding agents, it’s crucial to focus on control architecture rather than standardizing on a specific model. This vulnerability class is systemic to agent design, not limited to a particular vendor. Here are some essential actions to take:

1. Build a deployment map: Make sure your platform meets the necessary runtime protections. Reach out to your Anthropic account representative to clarify the runtime-level prompt injection protections for your deployment. You can find more information in the Anthropic Cyber Verification Program.

2. Audit every runner for secret exposure: Scan all repositories with an AI coding agent to identify any exposed secrets. Rotate all credentials to enhance security. Refer to the GitHub Actions secrets documentation for guidance.

3. Start migrating credentials now: Transition to short-lived OIDC token issuance for increased security. Platforms like GitHub Actions, GitLab CI, and CircleCI support OIDC federation. Begin the migration process gradually, starting with repositories using AI agents. Learn more from the GitHub OIDC docs, GitLab OIDC docs, and CircleCI OIDC docs.

4. Fix agent permissions repo by repo: Secure your repositories by removing bash execution from AI agents conducting code reviews and setting repository access to read-only. Implement a human approval step for any write access. Find more details in the GitHub Actions permissions documentation.

5. Add input sanitization as one layer of security: Filter pull request content for potential instruction patterns before they are processed by agents. Combine this with least-privilege permissions and OIDC for enhanced protection against vulnerabilities.

6. Stay vigilant about supply chain risks: Include “AI agent runtime” in your risk register and adhere to a 48-hour patch verification cadence with each vendor. Don’t wait for CVEs to take action against potential vulnerabilities.

7. Evaluate your existing security measures: Check if your current GitHub Actions configurations already include hardened mitigations against potential attacks. Ensure that permissions are restricted, approvals are required for secret injections, and external pull requests triggering workflows are monitored. Learn more from the GitHub Actions security hardening guide.

8. Ask vendors about injection resistance: Before renewing contracts, inquire about the quantified injection resistance rate for the model version you use on your deployment platform. Keep track of refusals for compliance with the EU AI Act deadline in August 2026.

Remember, the key to preventing system compromises lies in addressing composability issues such as token management in CI/CD pipelines and agent permissions. By prioritizing security measures for your AI agents, you can mitigate potential risks effectively.

Please provide the sentence that you would like me to rewrite.

Leave a Reply

Your email address will not be published. Required fields are marked *