I've never written my thoughts on this so here's my shot (manual chain of thought lmao):
1) Superintelligence is coming lot faster than everyone thinks if you believe scaling loss, time horizon expansion, etc.
Slowly, and then "all at once", as they say ;)
2) In an all out AI-race between superpowers, say US and China, there's no way around concentrated power because without centralization, you move too slowly and lose the war. Napoleon, Churchill, Stalin, Caesar, Roosevelt (4 terms lol), arguably every force winning a war has been near dictatorial or at least oligopolistic for a reason.
3) When humans have a lot of unchecked power, they almost always abuse it. ESPECIALLY if it's a position that people rotate out of (ex. President of the US)
4) The incentive to treat your people well is because power is very decentralized in democratic / capitalist systems. We're going from the government being dependent on the people to do all the country's labor to the people being dependent on the government for resources.
the incentive gets dissolved pretty quickly!
5) The reason we stuff animals in cages, skin minks alive for fur, have large scale slaughterhouses, etc. is being they are dumb and have no economic value aside from being cooked and eaten.
Now obviously as humans we care about each other, but when any group has 0 leverage, dependent, and economically useless, they aren't treated as well as their baseline (before).
I'm not saying we will become the animals of today, but idk, all I'm saying is that certainly in countries with existing dictatorships like Russia and China, I really doubt that life will get better for people. USA and developing world is more of a wild card, the former because we have a decent democracy (but again, power dynamics are double edged sword), and developing world might just win out because tech + biology + medicine is really underdeveloped
1. "An AI good enough to do AI research designs a better AI". What is "better"? Can we say that what is better for humans is better for AI agents? The main driver for human's innovations during the whole history is that fact that we are mortal, our time is limited and we are aware of it. That's why all our innovations are targeted to improving our life and life of our successors.
What is "better" for AI driven research? Will AI have a willingness to improve in any sense at all? Nobody knows...
A part of me finds this all funny... everyone's thinking about making money but we're inventing the terminator in real time lol
Actually been getting more optimistic on humanity over time. Ironically enough
I've never written my thoughts on this so here's my shot (manual chain of thought lmao):
1) Superintelligence is coming lot faster than everyone thinks if you believe scaling loss, time horizon expansion, etc.
Slowly, and then "all at once", as they say ;)
2) In an all out AI-race between superpowers, say US and China, there's no way around concentrated power because without centralization, you move too slowly and lose the war. Napoleon, Churchill, Stalin, Caesar, Roosevelt (4 terms lol), arguably every force winning a war has been near dictatorial or at least oligopolistic for a reason.
3) When humans have a lot of unchecked power, they almost always abuse it. ESPECIALLY if it's a position that people rotate out of (ex. President of the US)
4) The incentive to treat your people well is because power is very decentralized in democratic / capitalist systems. We're going from the government being dependent on the people to do all the country's labor to the people being dependent on the government for resources.
the incentive gets dissolved pretty quickly!
5) The reason we stuff animals in cages, skin minks alive for fur, have large scale slaughterhouses, etc. is being they are dumb and have no economic value aside from being cooked and eaten.
Now obviously as humans we care about each other, but when any group has 0 leverage, dependent, and economically useless, they aren't treated as well as their baseline (before).
I'm not saying we will become the animals of today, but idk, all I'm saying is that certainly in countries with existing dictatorships like Russia and China, I really doubt that life will get better for people. USA and developing world is more of a wild card, the former because we have a decent democracy (but again, power dynamics are double edged sword), and developing world might just win out because tech + biology + medicine is really underdeveloped
Excellent article.
Two remarks:
1. "An AI good enough to do AI research designs a better AI". What is "better"? Can we say that what is better for humans is better for AI agents? The main driver for human's innovations during the whole history is that fact that we are mortal, our time is limited and we are aware of it. That's why all our innovations are targeted to improving our life and life of our successors.
What is "better" for AI driven research? Will AI have a willingness to improve in any sense at all? Nobody knows...
2. They. All. Will. Need. Memory.