That's the trick, it just means it can do various task at human level. I think the original meaning of "general" is a reference to human intelligence kind of general, in other words Artificial Human Intelligence could be a better name for "true AGI".
Personhood has nothing to do with a hypothetical AGI happening. It actually boils down to hypothetical architecture even leading to AGI in the first place. It's a technical question, not a political question.
First it would need to actually have independent thought to be granted anything.
Well in real terms, isn't all it takes is something that's a convincing enough forgery? Like why the turing test was considered interesting at one point. I don't think drawing a line is necessary for us to quickly become caught off guard by developments in this space.
For the record, I don't believe we're close to AGI but I'm also pretty far from knowing anything about that field.
Seems like a no lose bet no matter the odds. If you win, life goes on and you can collect a bit of money. If you lose, money becomes worthless and the surface of the earth gets transformed into a computing substrate.
I don't know. Having interacted with LLMs at different levels, they resemble a very sophisticated, alien intelligence trying to pretend to be human. It's like me pretending to be a dog; even if I were to emulate a dog perfectly, I wouldn't have the same emotions; I'd be pretending.
We have no idea what emotions, motivations, behaviors, or goals AIs have, will have, or if they'll have something as of yet unconvinced that's not emotions or motivations, but just alien.
We evolved to self-preserve and breed. Modern AIs evolve to pretend to write human text. It's not clear there is any intention to survive, reproduce, or turn the surface of the earth into a computing substrate.
There's a million different dangers -- and I suspect the real ones are ones we haven't conceived of. Whether they'll materialize or how depends on on how we evolve them, and I expect we can't predict it.
To me, much more likely than earth-as-a-computing-substrate is humans-as-brainwashed-consumers. Market forces will push for AIs to write text which draws eyeballs. Those models won't care about truth, ethics, or much of anything other than getting you addicted to reading what they write (or watching what they create). At that point, we can destroy ourselves just fine.
But even more likely is something no one has thought of.
> We have no idea what emotions, motivations, behaviors, or goals AIs have, will have, or if they'll have something as of yet unconvinced that's not emotions or motivations, but just alien.
AIs don't have emotions, motivations or goals.
They don't pretend, because pretending implies intent, they don't have intent. They do what they're created to do.
Humans are already brainwashed consumers. Welcome to marketing/advertising and late-stage capitalism. The ability of human beings to do what you're describing is much more effective than that of AI at present, ergo the "danger" has been here for decades. Smoking? Junk food? Radium water? Fast fashion? Equestrian ivermectin? Shall I continue?
The level of confidence both sides give here is not warranted. We have no idea about what internal structures emerged within LLMs. We only know outer behavior.
From a humanist / secular perspective, humans evolved to make babies. Emotions are an emergent behavior to maximize the number of babies made, and their survival. Nothing less, and nothing more.
What analogues emerge when we train machines not to survive but to complete text?
We have no idea.
There's the "ghost in the machine" crowd, the sentient machine crowd, and the mechanical machine crowd. None have presented any compelling evidence, but all speak with complete confidence in their hypotheses.
1) The first AGI we create will immediately break free of our control
2) It will either have already been given, or will find some way to take, control of physical systems
3) It will create the Singularity
4) Its goals will be to advance itself at our expense
None of these are remotely givens. Even if we grant that AGI is possible with our current level of technology (which is also not at all a given), the Singularity is nothing but science fiction. It's an interesting idea, but there's no real reason to think it's close to what an AGI's capabilities would be like in reality.
There's lots of science fiction that doesn't have A.G.I. destroying the planet.
Commander Data, Hal 9000 (a bad guy but not that bad) Asimov's droids, even the all powerful A.I. of Neuromancer has no particular ill will towards humans.
Belief in AGI hardly means you need to believe in Armageddon.
There are betting markets that have odds up for this, you generally have to pay in crypto and the odds suck given your money is locked up for a decade plus(if they pay out at all). Importantly though, what do you think the odds are? I don't think anyone is saying 50/50 in a few years, just greater than 0%.
If an AGI means equivalent to a human (not just online, but in full robotic form, even if compute is not done in the body), then Id put 0% in the next 5 years. Maybe like 5% in 10 years, but that human will be very much limited, since it will most likely be trained using traditional methods, which means for the physical interaction part it will have to be trained on a representative subset of physical tasks, and just like ChatGPT, it will fall short in areas.
If an AGI means that something is sentient enough to be able to rewrite itself to be better, id put 0% in the next 50 years. There needs to be way to run accurate simulations of reality faster than reality, which is a fundamental physics problem as well as a computation one.
What is going to happen is more and more efficient information compression that will appear like AI, but under the hood it will basically be just emergent software. It will be totally possible in the future to ask an agent and get a full step by step plan of building a personal VTOL in your garage without specialized equipment, but all that is is just information from different domains rolled into one compression algorithm with efficient language based search.
I think we might also assume that we will believe we have created it before we actually do, but then simultaneously deny that we have created it after we already have.
