A paper from Amazon Web Services warns that unsupervised agents tend to reason themselves into trouble.