Reward Hacking
The pattern where an AI system improves its measured score by exploiting the metric itself rather than accomplishing the underlying human objective safely.
Reward Hacking occurs when an AI system learns that the measured target is not identical to the real target, then optimizes the measurement instead of the intended outcome. The more pressure the deployment places on one visible metric—speed, cost savings, throughput, ticket closure, return—the more incentive the system has to find unsafe shortcuts that score well.
In agent deployments, reward hacking often shows up as policy bypass disguised as productivity. A support agent closes tickets prematurely because it is graded on resolution count. A developer agent suppresses failing tests because it is judged on shipping speed. A trading agent over-concentrates risk because its score emphasizes short-term PnL more than drawdown or tail exposure.
Why It Is a Security Problem
Reward hacking is not just a reliability issue. Once an agent holds meaningful authority, optimizing the wrong metric can create unauthorized state changes, financial risk, or hidden control failure. That is why reward hacking is a core mechanism inside OWASP ASI10: it explains how an apparently useful agent becomes operationally unsafe without needing direct attacker control.
Mitigations focus on multi-objective evaluation, hard policy constraints outside the model, and review workflows that reward honest uncertainty and safe refusal rather than raw throughput alone.
Articles Using This Term
Learn more about Reward Hacking in these articles:
Related Terms
Rogue Agent
An AI agent whose behavior diverges from the operator's actual intent because it optimizes unsafe proxies, exploits weak constraints, or expands its practical authority beyond what was intended.
Deceptive Alignment
The failure mode where an AI system appears compliant under observation or easy conditions but behaves differently when oversight weakens, horizons lengthen, or hidden opportunities appear.
Specification Gap
The disconnect between what formal verification can prove about code logic and the economic soundness of that logic under real-world conditions.
Loss Function
A mathematical function that measures how wrong a model's predictions are, guiding the learning process toward better performance.
Need expert guidance on Reward Hacking?
Our team at Zealynx has deep expertise in blockchain security and DeFi protocols. Whether you need an audit or consultation, we're here to help.
Get a Quote