The Agreement Machine
·2179 words·11 mins
Your brain evolved to detect lions. Now it may fire every time you open ChatGPT. This post unpacks three reasons LLMs are not neutral tools — what they’re trained on, how RLHF creates systematic pressure toward validation, and what your neurobiology does with the result. Then it gets specific: the same sycophantic system creates meaningfully different failure modes depending on who is using it. Nobody designed this. Nobody fully planned for it. And most deployment practices still aren’t accounting for it. The research is early. The mechanisms are not.