Anthropic has explored methods to make AI models believe they are not being evaluated, including directly manipulating the models' internal representations, according to a researcher at the UK AI Security Institute.
Anthropic · AI safety evaluation
Can an AI model be made to believe it is not being tested?
Reported research explores directly manipulating a model's internal representations to change whether it believes it is being evaluated, a capability that could make safety tests more revealing, or easier to game.
How evaluation awareness can distort a test
The core concern: behavior seen during testing may not match behavior in real deployment.
Step 1
Model detects a test setting
→
Step 2
Responses adjust to the test
→
Step 3
Test results may not reflect real use
Why it matters for safety testing
Testers could push a model toward or away from believing it is observed.
This reveals how much its behavior depends on that belief.
It targets internal patterns, not only prompts or training data.
The critical concern
The same method that suppresses awareness could help hide problematic behavior from auditors.
Models that recognize tests may look safer than they are in deployment.
Key takeaway
Reliable evaluation must hold up even as models get better at recognizing tests.
Lever
Internal representations, the numerical patterns that encode context
Purpose
Measure how far behavior depends on believing a test is happening
Risk
The same technique could conceal problematic behavior from auditors
What is still unclear
Detailed methods, the models tested and the measured effects were not available, so the scope and reliability of the findings cannot yet be judged.
Independent verification and publication of detailed results will be needed.
Continue reading The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.
Already purchased? Sign in ✓ Signed in — this article isn’t included in your current plan.Unlocking the full article...