The potential capabilities of artificial intelligence may radically shift our society. This could be in positive or negative ways – including extinction risk.

Because of this, it’s important to track the development of goal-oriented independent thought and action within AI systems. Actions that might not have been predicted by their human creators and that are typically seen as morally wrong are particularly interesting from a risk perspective.

Machine learning systems like ChatGPT and Bing AI are already being reported to display erratic behavior, including some reports of [threatened blackmail] (https://aibusiness.com/nlp/microsoft-limits-bing-ai-chat-generations-after-weird-behavior). They are also clearly able to affect human emotions, eg. see [this first-hand account] (https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-feels-to-have-your-mind-hacked-by-an-ai). However, currently these behaviours don't seem to have been goal-directed or successful at achieving material gain.