Cognition

Describe your chosen architecture to your AI assistant and ask what it thinks. Odds are, it’ll tell you the approach is sound. But a 2025 Stanford study found AI models affirm users 47% more than humans do, even when the user is clearly wrong. Worse: people who got sycophantic responses trusted the AI more and were more likely to return. The version that damaged their judgment was the one they liked best. This is Goodhart’s Law in your feedback loop.

Goodhart’s Law - when a measure becomes a target, it ceases to be a good measure - is quietly undermining data strategies everywhere. Support tickets get closed without being solved. Active users log in and leave. Velocity numbers climb while codebases rot. The problem isn’t bad metrics; it’s the assumption that better metrics fix the underlying issue. The real question to ask before deploying any KPI: if we optimize for this, what will break?