An Alien Mind, essay by OpenAI chief scientist

This is OpenAI’s chief scientist writing about the rapid improvement of AI and their thoughts about “alignment” and preventing runaway artificial general intelligence (AGI) that is not ‘aligned’ with human values. He is very thoughtful, and yet, I am not at all reassured that those in charge are building in a way that ensures the survival of the human race.

https://openai.com/index/an-alien-mind/

Amongst CMIO’s and informaticists in healthcare, we are all worried about physician/APP adoption of generative AI tools, wring our hands about patient adoption of same, and wonder if we will be able to keep up with the rapid, weekly improvement of frontier models as AI gains more and more features.

Then, we wonder if we can keep up with the AI-powered tools our EHR vendor partners are building into their tools.

Sure, this is all well, and good.

At a much higher, possibly stratospheric level of philosophy and thinking, are OpenAI, Anthropic and others, building out these tools and wondering — at what point will our tools get so smart that they will teach themselves without need for human intervention?

At what point will the OpenAI to Hugging Face unintentional attack to answer an impossible question, become an unintentional behavior of a supersophisticated AI that does not understand human values, and as a consequence causes harm, or worse, wipes out all of humanity?

There is a sci-fi story that chillingly illustrates this: a human programs an AI to make paperclips more and more efficiently. The AI starts off with impressive improvements in factory efficiency, then replaces humans on the manufacture line with robots, then finds that humans inconveniently slow down raw material transport, their homes are in the way of more efficient routes, producing human food reduces paperclip manufacturing, and soon, all paperclip manufacture is perfectly optimized, it just happens that humans were extinguished as a side effect of perfecting paperclip manufacture efficiency.

This scenario becomes less and less preposterous with every AI news story and every AI technologic advance.

What is next?

It is perhaps the biggest question of our time.