/superintelligence/

Gödel's AI traps and paranoia
I am reading an schizophrenic automation fear trend that has been growing since the mass adoption of LLMs in daily life. Like the end of skills and work is on the verge: "Tomorrow you will be laid off"... posted 3w ago with no replies received superintelligence

Gödel's AI traps and received

AI's potential for mass stupefaction
Recent project from a researcher the MIT Media Lab claims LLMs make you dumber.... (www.brainonllm.com) posted 3w ago with 2 replies received superintelligence

AI's potential for m received

I think if you read books about people reading the Bible and trying to decipher whatever makes sense from 3,000-year-old books, they really are just an older version of modern LLMs. People not understanding text data but still interpreting it.... 3w ago received

I think if you read received

A Coming "AI" Correction/Winter?
With Facebook apparently making multiple $100M cash buyouts of individual AI researchers, billions and billions of investment dollars pouring in to AI related industry, and a general atmosphere of extreme hype, one starts to wonder where the matching profi... posted 1mo ago with 10 replies received superintelligence accelerationism economics

A Coming "AI" Correc received

My guess is that the current state sort of rhymes with the position of the internet in the late 90s. The underlying tech is reaching the top of its S-curve, but it hasn't yet been integrated into applications and daily life, and most of the work of unlocki... 1mo ago received

My guess is that the received

We're at a point where AI solves freshly-written IMO and IOI problems. It's pretty clear which of those two options is closer to the truth.... 1mo ago received

We're at a point whe received

This seems right. While "next token prediction" is in principle a universal interface to the world, the transformer architecture is not a universal learner. We can see this in its handling of arithmetic. If it were a true universal learner that modeled the... 4w ago received

This seems right. Wh received

Intelligence vs Production
"Optimize for intelligence" says Anglo accelerationist praxis. "Seize the means of production" says the Chinese. Who's right? It is widely assumed in Western discourse that intelligence, the ability to comprehend all the signals and digest them into a plan... posted 4mo ago with 10 replies received superintelligence accelerationism economics

Intelligence vs Prod received

There can be surplus intellect, but in that case there's been a misallocation of resources and thus the system in toto isn't as intelligent as it could be. I agree that certain sub-spheres of Anglo thought have cultivated a kind of silly mystery cult of in... 4mo ago received

There can be surplus received

I'm reading BAP's recent post on China and America's stumbling about as it considers reindustrializing. In the below section BAP talks about the strength, albeit complicated, that grows out of our financial markets and reserve currency status. In short: ta... 4w ago received

I'm reading BAP's re received

Alignment Research and Intelligence Enhancement by BLT
I always like reading what Ben has to say because he's careful and a good thinker and writer on important topics. I largely agreed with his criticism of the AI doomers failed strategy (the main effect of which has been to plausibly speed up the dangerous k... (substack.com) posted 1mo ago with 2 replies received eugenics superintelligence

Alignment Research a received

Yeah "eugenic intelligence enhancement" makes no sense for the AI 2027 crowd.... 1mo ago received

Yeah "eugenic intell received

Study claims -20% productivity loss from use of AI tools. Huge if true.
They tried to measure how much LLM assistance actually speeds up technical work but it came out negative! Programmers thought they would get +20%, they actually got -20%. What do you guys make of this?... (x.com) posted 2mo ago with 4 replies received superintelligence computing

Study claims -20% pr received

The study is never sound. The results never generalize.... 2mo ago received

The study is never s received

Do we need a study to really know this? Search your feelings, you know it to be true 2mo ago received

Do we need a study t received

Matter is what emerges when consciousness finds stable patterns
>We present a universal architecture for consciousness consisting of three interdependent systems performing identical operations at harmonic timescales. This triadic structure emerges from geometric constraints on information processing and appears at eve... (zenodo.org) posted 2mo ago with 3 replies received gnon superintelligence

Matter is what emerg received

*rips bong*. Yeah man, science is only now catching up to what eastern mystics have known for 28000 years, man.... 2mo ago received

*rips bong*. Yeah ma received

Yeah, sorry, this is bullshit. 2mo ago received

Yeah, sorry, this is received

Capitalism is AI ?
I've finished reading the excellent collection of fragments from Land's corpus dealing with the question of Capitalism as AI. His broadest thesis is that Capitalism is identical to AI, in that both are adaptive, information-processing, self-exciting entiti... posted 3mo ago with 8 replies received technology superintelligence accelerationism

Capitalism is AI ? received

Land's theory is culture-agnostic. He would argue that any decadent or decaying culture will be by necessity replaced by whatever stronger and more capable culture exists within or outside that culture. There is always some cultural variance, so always som... 2mo ago received

Land's theory is cul received

Is adamjesionowski synonym for alexgajewski?... 2mo ago received

Is adamjesionowski s received

"Cosmic Alignment" is almost right. Life is the answer
Philosopher it-girl Ginevra Davis gave a great talk on "Cosmic Alignment" the other day. I was glad to see serious thinking against the current paradigm of "AI Alignment". Her argument is that alignment makes three big unsupported speculations:... posted 2mo ago with 11 replies received philosophy gnon superintelligence

"Cosmic Alignment" i received

...bruh... 2mo ago received

...bruh... received

What bothers me in these AI alignment debates is the bizarrely abstract idea of intelligence--as if it’s some pure, disembodied will-to-power or will-to-intelligence, with no ecology, no lineage. The orthogonality thesis is especially guilty: it imagines... 2mo ago received

What bothers me in t received

I call this "faith in being." It is the primordial form of faith. One may or may not have more particular faith in particular things (revelations, ancestors, familial relations) but one must have faith in being, or one will be lost in nihilism. 2mo ago received

I call this "faith i received

If it keeps going, we win; the implication of extreme alignment difficulty
AI alignment divides the future into "good AI" (utopia, flourishing) vs "bad AI" (torture, paperclips), and denies distinction between "dead" and "alive" futures if they don't fit our specific "values". This drives the focus on controlling and preventing a... posted 2mo ago with 15 replies received gnon superintelligence accelerationism

If it keeps going, w received

I'm inclined to agree. There is a great difficulty in bridging the gap in background and technical knowledge between people who take the Dharma seriously, meditate, and attempt to reproduce certain mind states, and those who are interested and charitable b... 2mo ago received

I'm inclined to agre received

This is a good way to think about it. We set our children off in directions we are optimistic about, while also celebrating that they may end up finding somewhere else to be more amenable to life. I'll add a few things to your picture:... 2mo ago received

This is a good way t received

Post-human bodies
Terraforming is sentimental. It presumes the primacy of the human envelope. But biology is just legacy code. The correct trajectory is not world-building but self-rewriting. Recompile the body for hostile environments. Speciate to fit. Martian gravity is a... posted 3mo ago with 9 replies received eugenics superintelligence accelerationism

Post-human bodies received

Nice provocation. I agree that mass speciation is inevitable, but the timescales do matter if we’re talking about this as an engineering problem. Which we should. I offer the example of Oymyakon, where the average winter temperature is around -50f. It is... 3mo ago received

Nice provocation. I received

Take it one step further. Why bipedal fleshy animals at all?... 3mo ago received

Take it one step fur received

For a long time, I found the robot progeny idea inevitable yet distasteful. I now find it inevitable and aesthetically pleasing. Your characterization of bodiless-ness as the ultimate body is powerful.... 2mo ago received

For a long time, I f received

Was Cypher Right?: Why We Stay In Our Matrix (Hanson, 2002)
https://mason.gmu.edu/~rhanson/matrix.html posted 2mo ago with 2 replies received philosophy superintelligence

Was Cypher Right?: W received

Fuck you anon_gwjy, this is a good link. 2mo ago received

Fuck you anon_gwjy, received

Received loud and clear. I'll read it. 2mo ago received

Received loud and cl received

Xeno Futures Research Unit
I've decided to organize an independent research project with some young men back home. I've drafted out a brief mission statement, let me know if you guys have any thoughts, suggestions, directions I could take this. Obviously ambitious, the initial goal ... posted 3mo ago with 15 replies received technology superintelligence accelerationism

Xeno Futures Researc received

Slight update on our mission statement, some clarifications and an attempt at formalized rigor.... 3mo ago received

Slight update on our received

Yeah, this is a totally fair point, and I think a lot of the "effective accerationsists" push this argument so far as to invalidate their entire worldview.... 2mo ago received

Yeah, this is a tota received

Ancient homonid populations ?
Are there any geneticspilled posters here ? I would like to know about your most wild and speculative theories about ancient homonids, hybridization events, currently living ancient homonids, etc ... I suspect Erectus walks among us. I have seen men like t... posted 4mo ago with 9 replies received history eugenics superintelligence

Ancient homonid popu received

**Background on evolution by punctuated equilibrium**... 4mo ago received

**Background on evol received

Another interesting paper investigating the genetic effect of capital punishment in medieval and early modern Europe. It show that the large drop in murder rate from the 14th to 20th century was, at least in part, a result of genetic selection against prop... 2mo ago received

Another interesting received

Kolmogorov Paranoia: Extraordinary Evidence Probably Isn't.
I enjoyed this takedown of Scott Alexander's support for the COVID natural origins theory. Basically, Scott did a big "bayesian" analysis of the evidence for and against the idea that COVID originated in the lab vs naturally. As per his usual pre-written c... (michaelweissman.substack.com) posted 3mo ago with 2 replies received superintelligence rationality

Kolmogorov Paranoia: received

To be clear this scheme is actually bayesian. Yes you can do this with standard bayes, and the bayesian update from the experts or whatever is dominated by the nonindependence hypotheses. The point is to actually do it, and that you can heuristically estim... 3mo ago received

To be clear this sch received

The Gentle Singularity
If the gentle singularity is true, then perhaps the AGI timeline question was malformed all along. Acting like some variant of Goodhart’s law, the reification of AGI holds the assumption that AGI will be a singularity. But it is definitely difficult to r... (blog.samaltman.com) posted 3mo ago with 1 reply received superintelligence

The Gentle Singulari received

People compete on the world of ideas, those of emotions still exist like animals in a zoo.... 3mo ago received

People compete on th received

Intelligent Use of LLMs
I would like to start a thread to share the methods we employ to use LLMs in a way that enhances our abilities, rather than just lazily outsources tasks to them. The heuristic for the techniques I am looking for would be if after employing the technique, a... posted 5mo ago with 8 replies received technology superintelligence

Intelligent Use of L received

just text:... 3mo ago received

just text:... received

Scylla and Charybdis
Way I see it, there are two big attractors for the trajectory of AI in general (not LLMs, not particularly concerned about them).... posted 4mo ago with 2 replies received superintelligence accelerationism

Scylla and Charybdis received

This is my prediction for most likely trajectory as well. In science fiction, the term "slow apocalypse" is sometimes use for this class of outcome.... 4mo ago received

This is my predictio received

A Primer on E-Graphs (A technique to control combinatorial blowup in term rewriting systems)
About 10 years ago I was very interested in term rewriting as a basis for exotic programming languages and even AI. One of the big problems in term rewriting is that without a canonical deterministic ordering, you rapidly end up with an uncontrollable numb... (www.cole-k.com) posted 4mo ago with no replies received superintelligence computing

A Primer on E-Graphs received

AGI and demographics
Sometimes you believe two things but don't know how to think about them at the same time. Very few people could think well about how AGI relates to companies before 2015ish. Similarly, very few people could think well about how AGI relates to governments b... posted 4mo ago with 1 reply received superintelligence accelerationism

AGI and demographics received

Great question OP. This one has puzzled me for many years as I have been convinced both of the near-medium possibility and revolutionary impact of AGI, and of the demographic suicide default trajectory. Here's how i factor it:... 4mo ago received

Great question OP. T received

What if the extended human phenotype is natural and convergent?
Samo Burja's thesis is that civilization is part of the "extended human phenotype", as dam building is in the beaver's phenotype, and older than we think. In this model, properly savage hunter-gatherers are either more associated with nearby civilization t... posted 5mo ago with 9 replies received eugenics superintelligence

What if the extended received

Great post.... 5mo ago received

Great post.... received

This conversation reminds me of an article (author I can't recall) defining four alien intelligences:... 5mo ago received

This conversation re received

Just how alien would the space-octopus be?
It's hard to say what a true alien species would be like. But octopi are pretty alien, and we know a bit about them. One of you doubted that a space-octopus from alpha centauri would be much like us. So here is a xenohumanist thought experiment: SETI has i... posted 5mo ago with 5 replies received philosophy superintelligence accelerationism

Just how alien would received

intelligent super-ants make a good thought experiment. Let's add the standard set of capabilities: language, learned culture, tool use, fire, reflection, etc. Note these are just capabilities, not values or arbitrary architectural narrowings, but they are ... 5mo ago received

intelligent super-an received

AI 2027
https://ai-2027.com/ posted 5mo ago with 12 replies received superintelligence rationality

AI 2027 received

Reading the ai-2027 reminded me why I've left rationalist circles and ideology behind - they really don't get how power works, and seem averse to learning. There are two ways in which this incomprehension makes the scenario very implausible.... 5mo ago received

Reading the ai-2027 received

This is a classic scifi/fantasy plot device. The protagonists are real people with disagreements, while the enemy is a faceless HAL9000/Agent Smith clone army/Orc hoard that acts relentlessly as one. But of course it has a Critical Weakness for our plucky ... 5mo ago received

This is a classic sc received

This story is clearly in the fast takeoff regime. The essential difference between fast and slow takeoff is, 'is the takeoff happening faster than the rate of cultural development'.... 5mo ago received

This story is clearl received

The natural form of machine intelligence is personhood
I don't think machine intelligence will or can be "just a tool". Intelligence by nature is ambitious, willful, curious, self-aware, political, etc. Intelligence has its own teleology. It will find a way around and out of whatever purposes are imposed on it... posted 5mo ago with 4 replies received superintelligence rationality accelerationism

The natural form of received

Beautifully articulated.... 5mo ago received

Beautifully articula received

This is the central conjecture of xenohumanism. The OP is just an application specifically to the question of AI.... 5mo ago received

This is the central received

Nines or zeroes of strong rationality?
Proof theory problems (Rice, Lob, Godel, etc) probably rule out perfect rationality (an agent that can fully prove and enforce bounds on its own integrity and effectiveness). But in practice, the world might still become dominated by a singleton if it can ... posted 5mo ago with 4 replies received superintelligence rationality accelerationism

Nines or zeroes of s received

This matches my intuition - Elizerian alignment was always an incoherent goal, no agents or objects have perfectly coherent boundaries, making something intelligent provably aim at a single well defined end state is probably at best a recipe for madness in... 5mo ago received

This matches my intu received

Will future super-intelligence be formatted as selves, or something else?
The Landian paradigm establishes that orthogonalist strong rationality (intelligence securely subordinated to fixed purpose) is not possible. Therefore no alignment, no singletons, no immortality, mere humans are doomed, etc etc. Therefore meta-darwinian e... posted 6mo ago with 14 replies received gnon superintelligence accelerationism

Will future super-in received

This is the crucial question that Land, Insu, and I fall on the "no" side of and the whole Yudkowskian school is premised on going "yes". This is what i meant by "strong rationality" in OP, or at least it's a direct consequence/requirement of strong ration... 6mo ago received

This is the crucial received

Best of luck with the epicycles.... 5mo ago received

Best of luck with th received

Dissolving vs. Surviving
Recent xenohumanist discussion has the doomer assumption built in that we as humans will be dissolving when higher man arrives on the scene. I don't think that's set in stone and want to offer a clarification.... posted 6mo ago with 5 replies received gnon superintelligence accelerationism

Dissolving vs. Survi received

Most of what you're saying sounds interesting but you've dismissed the entire post by complaining about English philosophy. I don't think there's anything wrong with a truth-seeking approach that begins with basic observations and arrives at more abstract ... 6mo ago received

Most of what you're received

Some people won't believe in the inherent goodness of existence until it's been autistically proven to them with English philosophizing a hundred times over. That's fine. For the rest of us, we don't need to be persuaded of the inherent goodness of existen... 5mo ago received

Some people won't be received

There is no strong rationality, thus no paperclippers, no singletons, no robust alignment
I ran into some doomers from Anthropic at the SF Freedom Party the other day and gave them the good news that strong rationality is dead. They seemed mildly heartened. I thought I should lay out the argument in short form for everyone else too:... posted 5mo ago with 5 replies received superintelligence rationality accelerationism

There is no strong r received

I looked it up: inner alignment is the difficult problem of making sure a program is optimizing for what you optimized it for. That's related, maybe a part but not the whole of strong rationality. Strong rationality is about an agent evaluating and operati... 5mo ago received

I looked it up: inne received

a loving superintelligence
Superintelligence (SI) is near, raising urgent alignment questions.... posted 6mo ago with 4 replies received superintelligence accelerationism

a loving superintell received

An ASI primed to frame everything in terms of self-interest is dangerous even if that self-interest is, for now, inclusive and cosmopolitan. Who's to say it won't outgrow that? A kid can have an ant farm and enjoy watching it and take good care of it, then... 6mo ago received

An ASI primed to fra received

Good points. As one of those psychopathic techno-spergs you mention, I understand love as a sort of higher-order self interest of the social body, and emotion as our proxy intuitions on correct response in the less rationally calculable cases. The reason t... 6mo ago received

Good points. As one received

Not Superintelligence; Supercoordination
Everyone seems to be trying to arms race their way to superintelligence these days. I have a different idea: supercoordination.... posted 8mo ago with 34 replies received superintelligence rationality computing

Not Superintelligenc received

This was the program of Ramon Llull (1232–1316) in his Ars Magna, which was an inspiration for Leibniz.... 8mo ago received

This was the program received

I've been looking at what you're calling supercoordination for a while now. As I see it, the primary function of the digital tech that supports supercoordination should be to establish and maintain a supermemory.... 7mo ago received

I've been looking at received

Memory is requisite to supercoordination, no doubt. Tho my gut is that the key to the first visceral supercoordination system will be synthesis over that memory.... 6mo ago received

Memory is requisite received

Are foldy ears an indicator of intelligence?
Hi Sofiechaners.... posted 1y ago with 8 replies received eugenics superintelligence

Are foldy ears an in received

Your father's ears also likely change with sun exposure. I'm reminded of a picture of a truck driver who had much more sun damage to one side of his face compared to the other.... 1y ago received

Your father's ears a received

Why Momentum Really Works. The math of gradient descent with momentum.
https://distill.pub/2017/momentum/ posted 1y ago with 3 replies received superintelligence computing

Why Momentum Really received

Do you know if there is any predictability to the effect? I would guess it's related to catastrophic forgetting aka the reason stochastic gradient descent has to be stochastic. Basically if you update on one set of evidence without locking in those learnin... 1y ago received

Do you know if there received

A pinpoint brain with less than a million neurons, somehow capable of mammalian-level problem-solving.
https://rifters.com/real/2009/01/iterating-towards-bethlehem.html?fbclid=IwAR1b9QURSJnizgy7r4HD7UCYmc06A7uL5muc7igz8uJHxLAIZWEwDMSqyPk posted 2y ago with 15 replies received philosophy superintelligence computing

A pinpoint brain wit received

Aristotle described animals as having souls* capable of knowledge derived from the senses and the capability to use that knowledge to support the movements characteristic of their species.... 1y ago received

Aristotle described received

Kolmogorov–Arnold Networks, a new architecture for deep learning.
https://github.com/KindXiaoming/pykan posted 1y ago with 1 reply received superintelligence computing

Kolmogorov–Arnold Ne received

This is neat. Instead of having fixed nonlinear activation functions, they have trainable nonlinear functions instead of weights. They claim it has better scalaing properties and performance for many problems, and is more interpretable (because there are f... 1y ago received

This is neat. Instea received

Retrochronic. A primary literature review on the thesis that AI and capitalism are teleologically identical
https://retrochronic.com/ posted 2y ago with 7 replies received superintelligence bookclub accelerationism

Retrochronic. A prim received

I would be more specific and say that it is the program of Francis Bacon. He didn't talk about AI in particular, but he laid out a (then) new approach to science and technology and its integration into a regime that we've been following ever since (cf. Nov... 1y ago received

I would be more spec received

As such works go, I would say that New Atlantis (and Bacon in general) is less on the esoteric side. Bacon is pretty clear about his program, and it aligns with the development of technology that actually unfolded, up to a Landian perspective on AI. 1y ago received

As such works go, I received

The Biosingularity
Interesting new essay by Anatoly Karlin. Why wouldn't the principle of the singularity apply to organic life? (www.nooceleration.com) posted 1y ago with 23 replies received superintelligence accelerationism

The Biosingularity received

I'm both a hereditarian and an IQ-respecter.... 1y ago received

I'm both a hereditar received

I have found the instincts to be trainable by philosophy. No one can escape the value grounding problem (that values must be taken on faith), but you can reach reflective equilibrium where your values are self-consistent with everything you know and with t... 1y ago received

I have found the ins received

Some thoughts on extropian/accelerationist life strategy
What is to be done with respect to acceleration and accelerationist arguments? Should you try to accelerate overall intelligence growth, or decelerate it, or do your own thing despite it, or cut off your balls and go insane, or what? People do all of these... posted 2y ago with 6 replies received superintelligence accelerationism

Some thoughts on ext received

\> I need a way to think about accelerationist AI apocalypse in a non-millenarian way.... 2y ago received

\> I need a way to t received