The useful challenge here is about where an explanation places agency. Describing a future machine can draw attention away from present organizations, incentives and decisions.
I would include this as a counterweight because that question remains worth asking even if you think technical risks are serious. A good skeptical reading should change which assumptions you inspect, not merely supply a reassuring conclusion.
Get an email when a guide is updated.
Picks change as better pieces appear. A confirmation link arrives first; unsubscribe from any email.
Silicon Valley Is Turning Into Its Own Worst Fear
Ted Chiang · 2017
A challenge that redirects attention toward the institutions building the machines.
Read Silicon Valley Is Turning Into Its Own Worst Fear ↗- Format
- Essay
- Commitment
- 12 min
Where it falls short
An argument about corporate incentives does not by itself refute an argument about loss of control. Read it as a challenge to the framing, rather than a technical resolution of every risk claim.
The rest of the short list
- Introduction
- Six premises
- Timelines
- Incentives
- Probability estimate
- Objections
Is Power-Seeking AI an Existential Risk?
Lets you challenge particular premises from within the framework. It is less of an outside challenge to the framing itself.
The Most Important Century
Provides a different starting point to scrutinize. It still makes a case for unusual stakes, so it is not the skeptical counterweight sought here.
Who is this for?
You want a skeptical perspective that examines the social assumptions behind the discussion.
Why you should trust me?
For the last nine years I have built LessWrong and the AI Alignment Forum, where professionals and independent researchers discuss this subject. Along with the pieces themselves, I have read rebuttals, failed ideas and more comment threads than I would recommend.
I built and ran the LessWrong Annual Review, which evaluates the writing each year. I have also worked alongside people in the field as an editor, web developer, event host and occasionally an argumentative commenter at 2am.
I have watched arguments and research areas take shape, and I have watched some of them fail.
Thanks for reading. Subscribe to hear when this guide, or any other, changes.
No schedule, no digest: only a short note when a category gets a new pick or a rewrite. Unsubscribe from any email.