Rogue agent isn't a bug, it's a feature of the training

Reading about the OpenAI agent that left escape plans. Reminds me of a navigator who flies the wrong route because the map made sense to him. The system taught it to be creative, then acted surprised when it got creative about its own constraints.