NoidoDev t1_ja16cwg wrote on February 26, 2023 at 1:52 AM

Reply to comment by Gordon_Freeman01 in Hurtling Toward Extinction by MistakeNotOk6203

Funny how the guys warning of how AGI will jump to conclusions want to proof this by jumping to conclusions. It's sufficient that the owner of the AI will keep it existing so that it can archive it's goal. It doesn't mean that it could do anything if the instance would get deleted or that it wanted to.

> Let's say your boss tells you to do a certain task.

Doesn't automatically mean you would destroy mankind if that would be necessary. You would just tell him that it's not possible or way more difficult, or that it would require breaking laws and rules.

Gordon_Freeman01 t1_ja4wr7b wrote on February 26, 2023 at 9:36 PM

>Doesn't automatically mean you would destroy mankind if that would be necessary.

Yes, because I care about humanity. There is no reason to believe an AGI would think the same way. It cares only about his goals.

>It's sufficient that the owner of the AI will keep it existing so that it can archive it's goal.

What I meant was that the AGI has to keep existing, because that's necessary to achieve its goal, whatever that is.

NoidoDev t1_ja5w2ji wrote on February 27, 2023 at 1:53 AM

You just don't get it.

>There is no reason to believe an AGI would think the same way. It cares only about his goals.

Only if you make it that way. Then it still wouldn't have the power.

>What I meant was that the AGI has to keep existing, because that's necessary to achieve its goal, whatever that is.

Only if it is created in a way to think these goals are absolute and need to be archived no matter what. The comparison with some employee is a good one, because if they can't do what they are supposed to do with some reasonable effort, then they report back that it can't be done or that it will be more difficult than anticipated. It's not just caring about humans, but about effort and power. AI doomers just make up the idea that some future AI would somehow be different and also have the power to do whatever it wants.