They don't make buttons large enough for multiple people to press. It's always going to be someone in the end.

Yes they do?

There's also plenty of buttons that can't be pressed unless unlocked by multiple keys which cannot be turned by a single person.

Exactly. AGI is something that will significantly affect all of humanity. It should be treated like nuclear weapons.

Effectively kept secret and in the shadows by those working on it, until a world-altering public display makes it a hot politically charged issue, unaltering even 80 years later?

Edit: Honestly, I bet that "Altman", directed by Nolan's simulacrum and starring a de-aged Cillian Murphy (with or without his consent), will in fact deservedly win a few oscars in 2069.

International co-operation to control development and usage. The decision to unleash AGI can only be made once. Making such a decision should not be done hastily, and definitely not for the reason of pursuing private profit. Such a decision needs input from all of humanity.

> International co-operation to control development and usage.

Non-starter. Why would you trust your adversary to "stay within the lanes". The rational thing to do is to extend your lead as much as possible to be at the top of the pyramid. The arms race is on.

With these things, the distrust is a feature, not a flaw. The distrust ensures you are keeping a close eye on each other. The collaboration means you're also physically close and intellectually close. Secrets are still possible, but it's just harder to keep them because it's easier for things to slip

It's rational only if you don't consider the risks of an actual superhuman AGI. Geopolitical issues won't matter if such a thing is released without making sure it can be controlled. These competition based systems of ours will be the death of us. Nothing can be done in a wise manner when competiton forces our hand.

And quickly proliferated around the world to other superpowers and rogue states…

Remember the soviets got the nuke so quick because they just exfiltrated the US plans

> It should be treated like nuclear weapons.

Seeing how currently nuclear weapon holders are elected, that would be a disaster

The disaster will happen if AGI is created and let loose too quickly, without considering all the effects it will have. That's less likely to happen when the number of people with that power is limited.

>It should be treated like nuclear weapons.

Either you get it or you're screwed?

It's already being treated like nuclear and that's a problem

  - highly centralized
  - lots of misinformation
    - lots of fear mongerng
  - arms race between most powerful countries
    - who can't stop because if the other gets a significant lead it could be used to destroy the other
  - potentially world changing
    - potential to cause unprecedented levels of harm
    - potential to cause unprecedented levels of prosperity

Sometimes things are just done better with your enemy than in direct competition with them. "Keep your enemies closer" kinda thing.

As a parallel, look at medicine and gain of function research. It has a lot of benefits but can walk the line of bioweapons development. A mistake causes a global event. So its best to work together. Everyone benefits from any progress by anyone. Everyone is harmed by mistakes by any one actor. That's regardless of working together or not. But working together means you keep an eye on one another, helping prevent mistakes. Often ones that are small and subtle. The adversarial nature is (or can be) beneficial in this case

Regardless of who invents AGI, it affects the entire world.

Regardless of who invents AGI, you can't put the genie back on the bottle (or rather it's a great struggle to that's extremely costly, if even possible)

Regardless of who invents AGI, the other will have it in a matter of months

Good point. I like George Hotz’ philosophy on this - paraphrasing badly - if everyone has AI, nobody can be subjugated by AI.

That's all fine and dandy if we skip the development phase and AI is already invented.

But this doesn't work during the transition. During the development. "The button" here is for AGI. As in, when it's created and released.

its a button everyone should have finger on. if you can get it to do something good, go for it?