← Toutes les offres
S

AI safety dinner conversations

sjonany

Global
Publié le 16 novembre 2026

Description du poste

I had some conversations about AI safety with friends and family recently, and want to summarize the discussions and my follow-up readings. There were three main claims that were discussed: • AI has superhuman intelligence• AI has caused great harm• AI can cause harm even when instructed not toWdym by “AI”? “AI” as used loosely in these discussions encompasses both public (e.g. Claude Opus) and private models (e.g. the swarms of agents OpenAI and Anthropic used to make the recent impressive discoveries in math and computer science). Claim 1: AI has superhuman intelligenceThis is the first topic that’s under contention: How “smart” really is AI? “No it’s not - it’s stupid”I have at least two software engineering friends telling me how frustrated they are with AI. Both have been using Claude Opus too, so it’s not like they’re using an older model. This was surprising to me -- my experience using it to code for both small side projects and bigger professional projects has been great to say the least. This discrepancy in our experience might be attributed to these factors: • AI lacking in capability. Perhaps my friends were using it in domains where the model just sucks (see jagged intelligence). It’s also a fact that current public models are not reliable enough to be able to replace software engineers yet (see task-completion time horizon).• Usage pattern: Initial specification effort. I tend to write longer plans before asking AI to execute, whereas one friend tends to start with a shorter initial prompt, then proceeds to wrangle the AI to modify the initial output to what they want. The latter workflow is likely going to make you bump into more “wtf that’s not what I meant” experiences.• Tolerance of errors. Even if we adjust for the same AI failure experiences, I also noticed we have different emotional responses to them. One friend’s emotional response to having to correct the AI output is frustration. I am more tolerant -- I think: “yeah you’re just a language mo