Skip to content

fix: Prevent context attributes from influencing judge template parsing#129

Draft
jsonbailey wants to merge 1 commit intomainfrom
devin/1775752761-fix-judge-template-injection
Draft

fix: Prevent context attributes from influencing judge template parsing#129
jsonbailey wants to merge 1 commit intomainfrom
devin/1775752761-fix-judge-template-injection

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Addresses SEC-8020. Mirrors the fix applied in the Go server AI SDK (go-server-sdk commit 3317871). A parallel PR is also being created for launchdarkly/js-core.

Describe the solution you've provided

The judge's _interpolate_message method previously used chevron.render() (Mustache templating) for its second-pass interpolation of {{message_history}} and {{response_to_evaluate}} placeholders. This is vulnerable to template injection: attacker-controlled values resolved during pass 1 (e.g. Mustache delimiter-change tags like {{=[ ]=}}) would be interpreted as control syntax by the second Mustache pass, potentially blinding the judge to the actual content being evaluated.

This PR replaces the Mustache-based interpolation with simple str.replace() calls. Since the judge only ever substitutes two known placeholder strings, a full template engine is unnecessary, and literal string replacement is both safer and simpler.

Note: chevron is still used in client.py for the first-pass template interpolation (which is expected behavior — pass 1 needs Mustache to resolve context variables).

Describe alternatives you've considered

  • Escaping attacker-controlled values before passing them to Mustache for pass 2. This is fragile and hard to get right across all Mustache control sequences.
  • Using a different template engine with sandboxing. Unnecessary complexity given only two fixed placeholders.

Additional context

Key items for review:

  • Verify that the chevron import removal from the judge module doesn't affect other code paths (it doesn't — chevron is only used in client.py for pass 1).
  • The regression tests cover all major Mustache injection vectors (delimiter changes, partials, comments, triple-stache, sections) plus edge cases like Mustache-like syntax appearing in the actual message content.

Link to Devin session: https://app.devin.ai/sessions/651e799b906748a4834bafefb4a3e3e5
Requested by: @jsonbailey

Co-Authored-By: jbailey@launchdarkly.com <accounts@sidewaysgravity.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration bot added the devin-pr Pull request created by Devin AI label Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devin-pr Pull request created by Devin AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant