Skip to content

The Best Plans for What Governments Should Actually Do About AI

Frontier AI Regulation and supporting policy readings

A list of desirable outcomes is easy to agree with. The harder questions concern institutions: what information they need, what authority they have and how they respond when a developer falls short.

I would use this paper as the starting document for a policy discussion because it organizes the problem around governance tools. It gives a reader a way to move from a concern about frontier systems to a proposal that can be examined.

Where it falls short

A framework is not a ready-to-pass bill. Legal authority, institutional capacity and the state of the technology need a fresh check before turning any recommendation into action.

The rest of the short list

Let's think about slowing down AI
by Katja Grace · 2022
EssaySource facsimile
For the case that restraint is possible

Let's think about slowing down AI

Katja Grace · 35 min

Addresses an earlier question: whether slowing development deserves serious consideration. It offers less of the institutional design this category calls for.

Research report
Is Power-Seeking AI an Existential Risk?
Joe Carlsmith · 2022
  1. Introduction
  2. Six premises
  3. Timelines
  4. Incentives
  5. Probability estimate
  6. Objections
ReportSource facsimile
For the underlying risk argument

Is power-seeking AI an existential risk?

Joe Carlsmith · 3 hr

Helps identify what a policy is supposed to prevent. It is an input to choosing interventions, rather than a playbook for implementing them.

Who is this for?

You want to move from “someone should do something” to concrete questions about oversight.

Why you should trust me?

For the last nine years I have built LessWrong and the AI Alignment Forum, where professionals and independent researchers discuss this subject. Along with the pieces themselves, I have read rebuttals, failed ideas and more comment threads than I would recommend.

I built and ran the LessWrong Annual Review, which evaluates the writing each year. I have also worked alongside people in the field as an editor, web developer, event host and occasionally an argumentative commenter at 2am.

I have watched arguments and research areas take shape, and I have watched some of them fail.