What follows is an unstructured blob of my thoughts last weekend.
Say I identify myself through my intelligence and job. What if, due to AI, the former no longer holds any value and the latter becomes obsolete?
- Complete commoditization of all intellectual work, e.g. software.
How did nations come to an agreement during the nuclear arms race of the cold war? Will we recognize it before it is too late with super intelligence (SI) just like we did then? Or will this be a switch that is silently flipped?
In light of the future, the only outcomes worth thinking about are (since unsafe SI, by definition, does not leave us any meaningful actions):
We don’t developer SI in the short-term
We get SI that is “safe”
SI will be created by one of the highly subsidized parties (including nations). Whoever creates it first will use it to remain the only one so it has total global dominance.
This is obvious thus once other parties think someone else is close they’ll have to nuke them, which means everyone gets nuked (because of “if you nuke me, then I’ll nuke you” thinking). Similar to the cold war, hopefully, that means we’ll create contracts that enforce the development of “safe” AI (under the assumption that AI is not orders of magnitude more powerful than nuclear bombs).
What does “safe” even mean in the context of safe AI? What is the vision of companies like Super Intelligence Inc.?
In my view “safe” is such a broad term. In a way everything can be exploited to gain an advantage in some way and those advantages become exponentially large due to the nature of SI. That leaves only one possible outcome that is not total annihilation, which is a dystopian end result where some AI will make all decisions for us (likely without us realizing) such that we can’t exploit it (and thereby making it safe for us to use). Does that make us dogs who magically obey their masters?
Are we already being controlled? Are we in a simulation?
Possibly SI needs us because we can survive things it can’t and it needs us to “switch it back on”.
Why has no alien society taken over? Do they occasionally comes for resources? What even are the resources from their point of view?
The entire world would unite against this common enemy (yes, this thinking is inspired by the snake episode of Rick & Morty).
They don’t see the benefit of invading and imply live at peace with what they have. What is the rational benefit of space travel? The universe is finite.
By definition, we don’t know what SI would do. Although I do believe it would still be bound by logic and thus giving us room to guess.
Whoever control the narrative, controls the society.
What if fossil fuels run out before we have reached SI?
What can we do? I’d say just experience and enjoy your life as you know it. Don’t go against the paradigm shift and enjoy the ride.