Viewing a single comment thread. View all comments

Capitaclism t1_j3kr49j wrote

You are assuming everything can be discovered by any one thing, and that it wouldn't literally take all of the energy and potential computational power present in the universe to finally fully understand it.

Anyway, I take it you mean what would happen if AGI simply renders human beings obsolete.
Well, there are a few different likely scenarios here:

  1. We merge with AI long before that happens, de facto becoming AGI. This is potentially a pretty benign scenario. We then spread through the universe
  2. We don't merge with AI, remain fairly separated from it, it renders us obsolete but turns out to have goals misaligned with ours. Two likely scenarios:
    1. We get annihilated
    2. It treats us like meaningless "ants", takes needed resources to leave and we stay here to likely die off slowly
  3. We don't merge with AI, remain fairly separated from it, it renders us obsolete but it remains aligned with our interest, thus making all of our dreams come true. We each depart with a form of AI to spread through the universe. Personally I don't think this is a very likely scenario
11

peterflys t1_j3ouu6a wrote

Let’s hope that the result of our future is some of #1. I might doubt that we merge “long” before an ASI comes around. Could be within a few years (or less). But the goal should be that we’re a part of it.

1

Capitaclism t1_j3p9bqf wrote

I hope we do merge as well, since I don't think outcome 3 is super likely. Why would a general super intelligent being choose to be subservient when it can surpass the collective intelligence of all beings on the planet as it grows exponentially towards, for all purposes in human scale, infinity?

2

peterflys t1_j3pduu7 wrote

But you do bring up a good point about #3 - seems like there is at least some speculation that some humans will choose not to merge and continue living in whatever the equivalent of a future luddite community would be. What will happen to them? Might be a topic for a different thread, but I know a lot of people speculate that humans could end up like zoo animals in these situations.

1

Capitaclism t1_j3pfcd9 wrote

Right. There's also the possibility it is impossible to truly merge. That we can out bits and pieces in our skull, but AI would simply dominate it, rather than merge. We don't really know if our awareness can fully merge and be expanded by this foreign intelligence, but I do suspect it can. No one knows for sure.

1