/agi/

Yes, this is an interesting prompt. Clearly there is latent desire that's currently being met in the most crass and retarded way possible, bottom up, by cartoon shysters like Joel Osteen.... 2mo ago (collapse hidden) log in to judge received 7.8 7.8

Yes, this is an inte (view hidden) log in to judge received 7.8 7.8

Are you familiar with the debate between AI As Normal Technology and the AI 2027 people? I think this maps on well to our disagreements RE impact, diffusion, and the real-world significance of capability improvements. 1mo ago (collapse hidden) log in to judge received 4.0 4.0

Are you familiar wit (view hidden) log in to judge received 4.0 4.0

I tend to approach this question from two pretty straightforward angles:... 1mo ago (collapse hidden) log in to judge received 0.8 0.8

I tend to approach t (view hidden) log in to judge received 0.8 0.8

What an odd and desperate cope.... 4mo ago (collapse hidden) log in to judge received 9.1 9.1

What an odd and desp (view hidden) log in to judge received 9.1 9.1

Cope for what, exactly? Be explicit.... 4mo ago (collapse hidden) log in to judge received 1.7 1.7

Cope for what, exact (view hidden) log in to judge received 1.7 1.7

FAGGELLA, styled according to his own flamboyantly self-aggrandizing website, does at least serve as an interesting directory of ideas. Specifically, ideas around hard takeoff and AI maximalism; he fits alongside Yud, AI 2027, etc.... 5mo ago (collapse hidden) log in to judge received 13.8 13.8

FAGGELLA, styled acc (view hidden) log in to judge received 13.8 13.8

I hope we get biotech good enough and widely adopted enough to end baseline humanity before the gooniverse epoch.... 5mo ago (collapse hidden) log in to judge received 1.2 1.2 2.0 2.0

I hope we get biotec (view hidden) log in to judge received 1.2 1.2 2.0 2.0

I think if you read books about people reading the Bible and trying to decipher whatever makes sense from 3,000-year-old books, they really are just an older version of modern LLMs. People not understanding text data but still interpreting it.... 8mo ago (collapse hidden) log in to judge received 1.5 1.5

I think if you read (view hidden) log in to judge received 1.5 1.5

We're at a point where AI solves freshly-written IMO and IOI problems. It's pretty clear which of those two options is closer to the truth.... 9mo ago (collapse hidden) log in to judge received 14.2 14.2

We're at a point whe (view hidden) log in to judge received 14.2 14.2

This seems right. While "next token prediction" is in principle a universal interface to the world, the transformer architecture is not a universal learner. We can see this in its handling of arithmetic. If it were a true universal learner that modeled the... 9mo ago (collapse hidden) log in to judge received 3.7 3.7

This seems right. Wh (view hidden) log in to judge received 3.7 3.7

There can be surplus intellect, but in that case there's been a misallocation of resources and thus the system in toto isn't as intelligent as it could be. I agree that certain sub-spheres of Anglo thought have cultivated a kind of silly mystery cult of in... 12mo ago (collapse hidden) log in to judge received 7.7 7.7

There can be surplus (view hidden) log in to judge received 7.7 7.7

I'm reading BAP's recent post on China and America's stumbling about as it considers reindustrializing. In the below section BAP talks about the strength, albeit complicated, that grows out of our financial markets and reserve currency status. In short: ta... 9mo ago (collapse hidden) log in to judge received 2.3 2.3

I'm reading BAP's re (view hidden) log in to judge received 2.3 2.3

Yeah "eugenic intelligence enhancement" makes no sense for the AI 2027 crowd.... 9mo ago (collapse hidden) log in to judge received 3.2 3.2

Yeah "eugenic intell (view hidden) log in to judge received 3.2 3.2

The study is never sound. The results never generalize.... 9mo ago (collapse hidden) log in to judge received 17.5 17.5

The study is never s (view hidden) log in to judge received 17.5 17.5

Do we need a study to really know this? Search your feelings, you know it to be true 9mo ago (collapse hidden) log in to judge received 4.7 4.7

Do we need a study t (view hidden) log in to judge received 4.7 4.7

Land's theory is culture-agnostic. He would argue that any decadent or decaying culture will be by necessity replaced by whatever stronger and more capable culture exists within or outside that culture. There is always some cultural variance, so always som... 10mo ago (collapse hidden) log in to judge received 8.0 8.0

Land's theory is cul (view hidden) log in to judge received 8.0 8.0

Is adamjesionowski synonym for alexgajewski?... 9mo ago (collapse hidden) log in to judge received 0.6 0.6

Is adamjesionowski s (view hidden) log in to judge received 0.6 0.6

...bruh... 10mo ago (collapse hidden) log in to judge received 14.1 14.1

...bruh... (view hidden) log in to judge received 14.1 14.1

What bothers me in these AI alignment debates is the bizarrely abstract idea of intelligence--as if it’s some pure, disembodied will-to-power or will-to-intelligence, with no ecology, no lineage. The orthogonality thesis is especially guilty: it imagines... 10mo ago (collapse hidden) log in to judge received 11.8 11.8

What bothers me in t (view hidden) log in to judge received 11.8 11.8

I call this "faith in being." It is the primordial form of faith. One may or may not have more particular faith in particular things (revelations, ancestors, familial relations) but one must have faith in being, or one will be lost in nihilism. 9mo ago (collapse hidden) log in to judge received 5.0 5.0

I call this "faith i (view hidden) log in to judge received 5.0 5.0

I'm inclined to agree. There is a great difficulty in bridging the gap in background and technical knowledge between people who take the Dharma seriously, meditate, and attempt to reproduce certain mind states, and those who are interested and charitable b... 10mo ago (collapse hidden) log in to judge received 10.2 10.2

I'm inclined to agre (view hidden) log in to judge received 10.2 10.2

This is a good way to think about it. We set our children off in directions we are optimistic about, while also celebrating that they may end up finding somewhere else to be more amenable to life. I'll add a few things to your picture:... 10mo ago (collapse hidden) log in to judge received 4.8 4.8

This is a good way t (view hidden) log in to judge received 4.8 4.8

Fuck you anon_gwjy, this is a good link. 10mo ago (collapse hidden) log in to judge received 8.2 8.2

Fuck you anon_gwjy, (view hidden) log in to judge received 8.2 8.2

Received loud and clear. I'll read it. 10mo ago (collapse hidden) log in to judge received 1.8 1.8

Received loud and cl (view hidden) log in to judge received 1.8 1.8

**Background on evolution by punctuated equilibrium**... 11mo ago (collapse hidden) log in to judge received 12.3 12.3

**Background on evol (view hidden) log in to judge received 12.3 12.3

Another interesting paper investigating the genetic effect of capital punishment in medieval and early modern Europe. It show that the large drop in murder rate from the 14th to 20th century was, at least in part, a result of genetic selection against prop... 10mo ago (collapse hidden) log in to judge received 6.9 6.9

Another interesting (view hidden) log in to judge received 6.9 6.9

To be clear this scheme is actually bayesian. Yes you can do this with standard bayes, and the bayesian update from the experts or whatever is dominated by the nonindependence hypotheses. The point is to actually do it, and that you can heuristically estim... 10mo ago (collapse hidden) log in to judge received 7.4 7.4

To be clear this sch (view hidden) log in to judge received 7.4 7.4

People compete on the world of ideas, those of emotions still exist like animals in a zoo.... 10mo ago (collapse hidden) log in to judge received 0.6 0.6

People compete on th (view hidden) log in to judge received 0.6 0.6

just text:... 11mo ago (collapse hidden) log in to judge received 3.9 3.9

just text:... (view hidden) log in to judge received 3.9 3.9

This is my prediction for most likely trajectory as well. In science fiction, the term "slow apocalypse" is sometimes use for this class of outcome.... 11mo ago (collapse hidden) log in to judge received 5.0 5.0

This is my predictio (view hidden) log in to judge received 5.0 5.0

Great post.... 13mo ago (collapse hidden) log in to judge received 9.4 9.4

Great post.... (view hidden) log in to judge received 9.4 9.4

This conversation reminds me of an article (author I can't recall) defining four alien intelligences:... 12mo ago (collapse hidden) log in to judge received 5.4 5.4

This conversation re (view hidden) log in to judge received 5.4 5.4

intelligent super-ants make a good thought experiment. Let's add the standard set of capabilities: language, learned culture, tool use, fire, reflection, etc. Note these are just capabilities, not values or arbitrary architectural narrowings, but they are ... 12mo ago (collapse hidden) log in to judge received 6.4 6.4

intelligent super-an (view hidden) log in to judge received 6.4 6.4

Reading the ai-2027 reminded me why I've left rationalist circles and ideology behind - they really don't get how power works, and seem averse to learning. There are two ways in which this incomprehension makes the scenario very implausible.... 13mo ago (collapse hidden) log in to judge received 14.8 14.8

Reading the ai-2027 (view hidden) log in to judge received 14.8 14.8

This is a classic scifi/fantasy plot device. The protagonists are real people with disagreements, while the enemy is a faceless HAL9000/Agent Smith clone army/Orc hoard that acts relentlessly as one. But of course it has a Critical Weakness for our plucky ... 13mo ago (collapse hidden) log in to judge received 11.7 11.7

This is a classic sc (view hidden) log in to judge received 11.7 11.7

This story is clearly in the fast takeoff regime. The essential difference between fast and slow takeoff is, 'is the takeoff happening faster than the rate of cultural development'.... 12mo ago (collapse hidden) log in to judge received 7.3 7.3

This story is clearl (view hidden) log in to judge received 7.3 7.3

Beautifully articulated.... 13mo ago (collapse hidden) log in to judge received 9.9 9.9

Beautifully articula (view hidden) log in to judge received 9.9 9.9

This is the central conjecture of xenohumanism. The OP is just an application specifically to the question of AI.... 13mo ago (collapse hidden) log in to judge received 5.1 5.1

This is the central (view hidden) log in to judge received 5.1 5.1

This matches my intuition - Elizerian alignment was always an incoherent goal, no agents or objects have perfectly coherent boundaries, making something intelligent provably aim at a single well defined end state is probably at best a recipe for madness in... 13mo ago (collapse hidden) log in to judge received 0.7 0.7

This matches my intu (view hidden) log in to judge received 0.7 0.7

This is the crucial question that Land, Insu, and I fall on the "no" side of and the whole Yudkowskian school is premised on going "yes". This is what i meant by "strong rationality" in OP, or at least it's a direct consequence/requirement of strong ration... 13mo ago (collapse hidden) log in to judge received 10.1 10.1

This is the crucial (view hidden) log in to judge received 10.1 10.1

Best of luck with the epicycles.... 13mo ago (collapse hidden) log in to judge received 4.5 4.5

Best of luck with th (view hidden) log in to judge received 4.5 4.5

Most of what you're saying sounds interesting but you've dismissed the entire post by complaining about English philosophy. I don't think there's anything wrong with a truth-seeking approach that begins with basic observations and arrives at more abstract ... 13mo ago (collapse hidden) log in to judge received 7.8 7.8

Most of what you're (view hidden) log in to judge received 7.8 7.8

Some people won't believe in the inherent goodness of existence until it's been autistically proven to them with English philosophizing a hundred times over. That's fine. For the rest of us, we don't need to be persuaded of the inherent goodness of existen... 13mo ago (collapse hidden) log in to judge received 1.3 1.3

Some people won't be (view hidden) log in to judge received 1.3 1.3

I looked it up: inner alignment is the difficult problem of making sure a program is optimizing for what you optimized it for. That's related, maybe a part but not the whole of strong rationality. Strong rationality is about an agent evaluating and operati... 13mo ago (collapse hidden) log in to judge received 2.9 2.9

I looked it up: inne (view hidden) log in to judge received 2.9 2.9

An ASI primed to frame everything in terms of self-interest is dangerous even if that self-interest is, for now, inclusive and cosmopolitan. Who's to say it won't outgrow that? A kid can have an ant farm and enjoy watching it and take good care of it, then... 14mo ago (collapse hidden) log in to judge received 8.8 8.8

An ASI primed to fra (view hidden) log in to judge received 8.8 8.8

Good points. As one of those psychopathic techno-spergs you mention, I understand love as a sort of higher-order self interest of the social body, and emotion as our proxy intuitions on correct response in the less rationally calculable cases. The reason t... 14mo ago (collapse hidden) log in to judge received 2.9 2.9

Good points. As one (view hidden) log in to judge received 2.9 2.9

This was the program of Ramon Llull (1232–1316) in his Ars Magna, which was an inspiration for Leibniz.... 1y ago (collapse hidden) log in to judge received 11.0 11.0

This was the program (view hidden) log in to judge received 11.0 11.0

I've been looking at what you're calling supercoordination for a while now. As I see it, the primary function of the digital tech that supports supercoordination should be to establish and maintain a supermemory.... 1y ago (collapse hidden) log in to judge received 14.2 14.2

I've been looking at (view hidden) log in to judge received 14.2 14.2

Memory is requisite to supercoordination, no doubt. Tho my gut is that the key to the first visceral supercoordination system will be synthesis over that memory.... 14mo ago (collapse hidden) log in to judge received 3.6 3.6

Memory is requisite (view hidden) log in to judge received 3.6 3.6

Your father's ears also likely change with sun exposure. I'm reminded of a picture of a truck driver who had much more sun damage to one side of his face compared to the other.... 2y ago (collapse hidden) log in to judge received 0.5 0.5

Your father's ears a (view hidden) log in to judge received 0.5 0.5

Do you know if there is any predictability to the effect? I would guess it's related to catastrophic forgetting aka the reason stochastic gradient descent has to be stochastic. Basically if you update on one set of evidence without locking in those learnin... 2y ago (collapse hidden) log in to judge received 2.9 2.9

Do you know if there (view hidden) log in to judge received 2.9 2.9

Aristotle described animals as having souls* capable of knowledge derived from the senses and the capability to use that knowledge to support the movements characteristic of their species.... 2y ago (collapse hidden) log in to judge received 2.1 2.1

Aristotle described (view hidden) log in to judge received 2.1 2.1

This is neat. Instead of having fixed nonlinear activation functions, they have trainable nonlinear functions instead of weights. They claim it has better scalaing properties and performance for many problems, and is more interpretable (because there are f... 2y ago (collapse hidden) log in to judge received 2.9 2.9

This is neat. Instea (view hidden) log in to judge received 2.9 2.9

I would be more specific and say that it is the program of Francis Bacon. He didn't talk about AI in particular, but he laid out a (then) new approach to science and technology and its integration into a regime that we've been following ever since (cf. Nov... 2y ago (collapse hidden) log in to judge received 7.7 7.7

I would be more spec (view hidden) log in to judge received 7.7 7.7

As such works go, I would say that New Atlantis (and Bacon in general) is less on the esoteric side. Bacon is pretty clear about his program, and it aligns with the development of technology that actually unfolded, up to a Landian perspective on AI. 2y ago (collapse hidden) log in to judge received 5.2 5.2

As such works go, I (view hidden) log in to judge received 5.2 5.2

I'm both a hereditarian and an IQ-respecter.... 2y ago (collapse hidden) log in to judge received 8.9 8.9

I'm both a hereditar (view hidden) log in to judge received 8.9 8.9

I have found the instincts to be trainable by philosophy. No one can escape the value grounding problem (that values must be taken on faith), but you can reach reflective equilibrium where your values are self-consistent with everything you know and with t... 2y ago (collapse hidden) log in to judge received 4.4 4.4

I have found the ins (view hidden) log in to judge received 4.4 4.4

\> I need a way to think about accelerationist AI apocalypse in a non-millenarian way.... 3y ago (collapse hidden) log in to judge received 0.6 0.6

\> I need a way to t (view hidden) log in to judge received 0.6 0.6