Alt title: Do you trust your friendly neighborhood chatbot?

By default, LLMs are sycophantic “yes men”. They tell us what we want to hear. This is good, in the short term, to help us get out of our own negative self talk.

But in the long term? It’s just another hit of dopamine. The very same we get scrolling through an infinite social feed.

Too much dopamine keeps us hooked on the chatbot experience itself, instead of letting us use the chatbot to make change in the real world.

This is an inherent conflict of interest in any LLM-as-a-business, that we must all keep in mind.