How is that irrelevant exactly? What would humanity have achieved if all we had were minds floating in ether? For any sort of intelligent software to be civilization altering, it needs to have access to a shit ton of sensory data as input and robotics as output, ideally in real time. Otherwise you have a speech bot, which we already have. “Well, if we have AGI, it will figure out the rest” is one of the most intellectually lazy statements I’ve ever read anywhere and unfortunately it’s kind of like this sub’s one commandment. AGI without sensors isn’t intelligent; thoughts in a head aren’t intelligent without input or output. This is a fallacy. If you think this is the case, then ChatGPT should already qualify, why not call it for today?

The article talks about how neural nets don’t play nice with loops, and connects that to the concept of computational irreducibility.

You say it’s not hard to imagine the net looping itself into some sort of awareness and agency. I agree, in fact that’s exactly my point. When humans see a machine talk in a very human way, it’s an incredibly reasonable mental step to think it will ultimately become more or less human. That sort of linear progression narrative is incredibly human. We look at life in exactly that way, it dominates our subjective experience.

I don’t think that’s what the machine thinks pr cares about though. Why would its supposed self-progress subscribe to human narratives? Maybe it has the temperament of a rock, and just stays put until picked up and thrown by one force another? I find that equally likely but doesn’t make for exciting human conversation.

Like I said above, maybe I didn’t word that part well enough. You can check out my reply there for more detail.

What wolfram believes however is definitely not essentialist fluff. He also absolutely doesn’t believe that humans are unique or special in any way. In fact, he thinks nothing is special at all but that everything is subjective. I suggest you read the article before you dismiss it.

I perhaps didn’t word that part very well, so would like to clarify what I meant. The entire point of Wolfram’s scientific endeavor hinges on the assumption that existence is a computational construct which allows for everything to exist. Not everything humanly imaginable, but literally everything. He posits that in this boundless computational space, every subjective observer and their perspective occupies a distinct place.

From our set of human coordinates, we essentially have vantage points into our own subjective reality. The perspective we have - or any subjective observer has - is computationally reducible; in the sense that by say coming up with fundamental laws of physics, or the language of mathematics we are actively reducing our experience of reality to formulas. These formulas are useful but only in time and from our perspective of reality.

The broader reality of everything computationally available exists, but in order to take place it needs to be computed. It can’t be reduced to mere formulas. The universe essentially has to go through each step of every available computation to get to anywhere it gets.

Evolution of living things on earth is one such process, humans building robots is another, so and and so forth. I’m not saying that humans are unique or only we’re conscious or anything like that. I’m also not saying machines can’t be intelligent, they already are. I’m just saying a neural net’s position in the ultimate computational coordinate system will undoubtedly be unfathomable to us.

Thus, extending the capability of machines as tools humans use doesn’t involve a directly traceable path to a machine super-intelligence that has any relevance in human affairs.

Can we build a thing that’s super fluent in human languages and has access to all human computational tools? Yes. Would that be an amazing, world-altering technology? Also yes. But it having wants and needs and desires and goals; concepts only existing in the coordinate space humans and other life on earth possess, that I find unlikely. Maybe the machine is conscious, perhaps an electron also is. But there’s absolutely no reason to believe it will materialize as a sort of superhuman being.

It makes me sad that an AI bot is proposed as a solution to loneliness. We live in a world where it's much easier to imagine a fucking robot being our friend than another human being. It's a disaster. Not hating on the technology btw, but why not fucking automate all call center jobs first and foremost, that seems like low-hanging fruit compared to an AI buddy and a much larger business opportunity. This sub is so creepy sometimes with the incessant robot worship.