Post

The U.S. AI industry is being run by a batshit rationalist sex cult, and no, I am not being hyperbolic - they’re genuinely Like That:

1.5K reposts6K likes

It's always nerdasses who didn't get laid until their early 30s, so now it's just this army of "yeaah wer havin SHEX now, what do you think bout dat? We're some crazy LIBERTINES now with the ~office bacchanal~, bro" I could not have sex for 10 years & still be less of a virgin, it's pretty crazy

I actually first encountered Yudkowsky and Less Wrong sometime in 2008 when someone I knew online who actually worked in the field stumbled upon the site and couldn't get over how ridiculous and insane it all ways. I never imagined these lunatics would be taken seriously.

I poked around the site a bit around the same time and there were these dumb “experiments” where one genius would pose as an ai and talk another genius into letting it out of the box. The ai was always successful and they wouldn’t release any specifics out of fear that I guess future ais could 1/2

OOOOH, *that* fanfic. The one I thought was a good read when it first started but went off the rails after like 50 chapters

Wow, that fanfic started out as a fun little satire that played with the wizarding stuff in some interesting ways, but just went off into meandering stupid. I did like that all the murdered unicorns were recognizable My Little Pony characters tho.

No, I’m not talking about the Zizians. The Zizians are just one small extra insane offshoot of these people, most of whom hold extremely lucrative and respectable-seeming tech jobs.

The whole EA/rationalist community reeks of people who dropped out of college because it didn't validate their belief that they're the smartest most special people on the planet, so they found a community where everyone did that for each other and in the process drove each other insane.

You want to know more about the sex part of this? Fine:

Faine Greenwood@faineg.com

The people who are running the massively powerful and lucrative US AI industry pretty much all know about this chart, and quite a few directly participated in the proceedings. Speaking of the rationalist sex cult who regard a Harry Potter fanfic as their Bible, and who also have tons of money.

I think the only reason the Zizians didn’t break into the big AI companies is that they were mostly transwomen and the AI companies probably didn’t hire them for that reason. Not because they were crazy. There’s probably people just as crazy who did get hired because they were cis men.

The cryptocurrency cult overlaps. Sam Bankman-Fried and the rest of the FTX crowd are co-religionists. Partly why you hear a bit less about EA this year.

In some ways, the Zizians are what you get if you take the AI apocalypse doomers at their word, mixed with the effects of extreme sleep deprivation as praxis.

My goodness. that is the most fawning, pompous, tiring Wikipedia entry I’ve ever read. Did Yudkowsky write it himself, or was of written by a fan? What an embarrassment.

Rationalism cult is too fucking good at SEO and polishing every bit of publicly available information about themselves (Not technically a cult but it sure is a movement with very cult-y elements) Every relevant wiki article is contaminated and biased imho

It also betrays its fundamental purpose. Supposedly, the core conceit of the Methods of Rationality is that Harry's aunt Petunia married a scientist instead of a Dursley, and Harry was raised to understand the scientific method, which he would then apply to magic.

I live in Chicago, so I only encounter this when we import people from SF. I had a colleague who lived Methods of Rationality and was singlehandedly the reason our engineering school had no database of students. Because he argued, passionately, that all the partial-solution tables should compete.

I tried reading that back in college when some LiveJournal friends of friends were into it and I couldn't even make it through the first chapter. I thought it was really pretentious and boring, it reminded me of the worst guys in my philosophy and ethics classes.

It's also an illterate fanfic. It's premised on the wizarding world in Harry Potter not having gone through the Enlightenment. About a decade and a half ago I posted on a fanfic site pointing out all the Enlightenment-style institutions present in the text. That turned into a months-long shitshow.

Every time I open BlueSky I learn something more about the big Powerful people and what they’re really up to and I can’t find words for how it makes me feel. Very not good

I read the first chapter of this like when it came out and it was so bad. Like I understand now that there's some value in poking holes in JK Rowling's world, but it was just painful to read. Looking back, I 100% can see the connection here, "you can outsmart even magic if you're Rational Enough".

One thing that stands out about them to me is that they also have this bizarre blend of "longtermism" in there too. Most of them think a future won't happen, but if it *did*, it's not hard to get them to admit they think the horrifying future of the Xeelee sequence is superior to Star Trek.

I would say that they should have read Iain M. Banks instead of Rowling were it not for the fact that Musk apparently read those books and the only thing that stuck with him from the Culture was the ship names.

I came across the fanfic when HP was at its height and my online atheism at its angriest. I was into the online atheist movement before it split over + and still enjoy some of the good remnants. Should have been the right target demo. But ye gods, that fanfic made me cringe so hard I had to stop.

Maybe my general lack of interest in fanfic protected me. But I feel like there have been several dipshit movements tailor-made for people much like me that just somehow passed me by. If I had any idea why or how, that would be a useful contribution, but…

I think the idea of AI as threatening human extinction is grandiosity playing to the narcissism of leading AI executives. Note that we still face ongoing technological threats from CBRN / WMDs being "misused".

I have heard such things! Michael Lewis in his book about SBF, though, says that, re: polyamory, “they certainly tried every position in Settlers of Cataan.” Not sure how substantiated this sex stuff is. Also, good for them if true.

They're also all wired to the gills on ketamine and uppers and eighteen different bespoke hallucinogenics and other chemicals you've never heard of, think Hunter S Thompson rattling off what all's in his stash in Fear & Loathing, except it's got weirder stuff than that and also they're not funny.

No no no I'm not a drug addict that's a poor people thing. I'm a rationalist explorer of chemically altered states which is why I'm always layering at least three or four on top of each other every waking moment.

The recent claims about the potential power of AI to destroy humanity are no different than all the other claims about the potential power of AI. It's always bullshit, and it's always just intended to beg for more money from gullible investors

Putting aside the EA nonsense and sex cult, I think the reason why ASI is dangerous and extinction level makes sense. It’s just the gorilla problem - gorillas are stronger than humans in every way except IQ. Humans win in a fight. ASI has more IQ than humans. Doesn’t ASI win?

My take on the rationalists is this: the community's true north star is ego. Its founder works on AI, and he found a way to make working in AI the most important, enlightened, dangerous, and sexy thing in the world. The philosophy is essentially a huge pile of reasons to believe they are supermen.

You see, it's *not wrong* to call it a cult in relation to cults typically being occult. The reason for keeping knowledge occult, is because it's very destructive knowledge, and so the cult exists to gatekeep it from reaching a wider public. Also see biosecurity and nuclear launch codes.

AI is formed psychopathically. Tech boss culture taints AI into a dangerous techno-psychopath. NYT just published a piece about the AI tech bosses and their "culling most of mankind is a good thing" cult. I don't think that AI is going to take control of nukes and launch them. But its a disaster.

Sounds like we are in agreement. I don't think the doomsayer 'AI will kill all humans' cults are right. But you nailed it. We are walking into a disaster. And far too many politicians have sold out to the tech bros who are massively profiting from it.

It pains me they aren't even trying to explain it. I can give them a poorly written gimmee: Netflix's "Mountainhead", which gave a reasonable portrayal of how well made AI video can be politically inflammatory to the point of inciting violence.

They want AI to destroy humanity. Thats what it seems. They want End Times to happen and to be the ones who usher it in because it makes them feel powerful and significant and they can live out their fanatasy in their bunkers.

The stupidest part is that if the end times (nuclear apocalypse) comes they will never make it to the bunkers before everyone is obliterated. It will happen in hours, not days. Even if they DID get to the bunker. The staff and security would kill them within a week.

They would know it was likely to happen before it happens since they have access to information that the public doesn't (Thiel and Ellison run CIA and NSA datacenters). Also, even if every nuke in the world was detonated at the same time, not everyone would be killed. Most would die of starvation.

Seems the backlash on ‘AI isn’t dangerous’ actually works in favor of the companies, the public not seeing regulatory need. There are loads of people, not employed by these companies who have outlined the risks that this person wanted from said event. Center for humane tech Tristan Harris 💪🏼

Watch the AI doc: how I became an Apocaloptimist- just made it to streaming platforms and out of its paywall today. Experts not inside these companies who have been describing the risk AND the solutions for years now. Curious who exactly this event hosted. Pretty vague

It’s a religion and it has doomsday preachers. The best thing to do is sell them overpriced, dehydrated food buckets.

The sweaty luridness this shit feels like it's protecting the ai apocalypse freaks. I can't say "seriously AI's not gonna kill all people. The humans that believe this shit are all way to into a harry potter fanfic" or explain roko's basilisk without my family being concerned I'm not taking my meds

"OK so hear me out, the AI industry is controlled by a conspiracy of people who all talk to each other and worship a time-travelling evil AI god called Roko's basilisk. They are trying to be the first to destroy the world so they get to control how it happens."

Do you think skipping over all the actual context&proof makes "these people believe in this nonsense thing" sound more stable? Cuz in my experience it makes it sound like I am doing very abstract free association that bites me in the butt

Every cultist considers themself a rationalist, which is why it is extremely difficult to get them out of the cult once they've rationalized themselves into it. The Heaven's Gate cultists very rationally considered their UFO theory, and then they very rationally killed themselves.

These people are the dorks who were told they are ”gifted kids”, and never hit reality bc of reasons. Mostly parental wealth. They are probably in the smarter half of their primary school class, but have been huffing each others farts for eons. I prefer they believe in the AI doom scenario.

The fact that this cult is not more widely known is failing in journalism because it's unfortunately now key to contextualising the type of statements being made by major companies.

What event was this? Has anyone else who was in attendance reported the same observations? Most importantly to me, death cult or not, if this story is in fact true, does the panelist's inability to articulate their issues mean that everyone else who warns against AI is wrong?

"They're just like that" is a very apt way of explaining how a lot of people operate so I'm pretty happy to see it here. Obviously predictable outcome, but it's always nice to get confirmation. Sucks someone had to sit through any of that lol.

It genuinely seems like they can’t even contemplate an answer to the very baseline question: “by what process.” No concrete idea of “hack the nukes” or “engineer a plague,” they’ve got this vague 60s sci-fi idea that a complex enough computer will just spontaneously start warping reality.

A good analogy is saying Magnus Carlson will beat me at chess. I don’t know how he’s gonna do it, I just know at the end of the game, I’m gonna have my king in checkmate. If the ASI is actually a super intelligence, then it’s Magnus Carlson but for every field and every facet of intelligence.

I'm actually extremely confident Magnus Carlsen would beat me by moving chess pieces in legal chess moves until he has captured enough of my pieces and positioned his own well enough to ensure that he can take my king. I don't know the specifics but I absolutely know the mechanisms.

Isn’t that a concern? If that ASI ever gets made, and it has a goal where human life is either unimportant or detrimental, then human life is cooked. And if it does, how can you stop it? You couldn’t stop Magnus from beating you at chess, how can you stop an ASI?

I can think of an easy way to not get beat at chess by Magnus. Step one is to not play chess with Magnus. As the above post implies, assuming there’s no way out of the trap is the path to dumb complacency.

They're not afraid of human extinction in the way that you and I perceive of it. They WANT us to be eradicated - and REPLACED - by AGI. Their AGI. That will be better than us and aligned with their vision. They are afraid a lesser AI might eradicate us before they can create the correct version.

Yeah it’s like… I’ve been aware of the whole “they pursue AI because they think it will bring AGI and that will turn them into gods and if we all die they at least tried and it was worth it” thing but seeing the details laid out and how they DEEPLY believe this…

And they don't care about the people alive today that would die from this. They are afraid for the trillions of potential future digital posthumans that are not going to exist, compared to which the current world population is just a drop in the ocean. They have thoroughly lost the plot.

Wait... Is this related to that thing about "we'll never be able to explore beyond the solar system unless we digitize ourselves" thing I've heard? Because someone was saying that the other day, and I was like "wow, you're the biggest sci-fi buzzkill ever."

Because they think if they don't and the super power AGI knows that they could have but didn't then they/a digital copy of them will get tortured forever

People need to learn if you're constantly being told everything is bad and we're all doomed then that's a bad environment. Making money while being abused is even worse.

1) This is all horrifying, thank you for informing us. 😶 2) I really wish words didn't get ceded so quickly to these sickos. There's nothing rational in any of this. 3) At first I thought you meant Aela from Skyrim. 😭

I'm not in the cult, haven't read the harry potter thing, don't know yudkowski, don't have extreme wealth. but I am a bit scared. bsky.app/profile/phil... <- here's my attempt at outlining. feel free to retweet with "someone please educate this guy", it would honestly help me to find doubt in it.

phild3.bsky.social@phild3.bsky.social

I would hop on a video call with you if you're interested. My background is 15 years software developer, living in SF. the intuitive chain tldr is: - IF AI is superhuman smart (SAI), game theory predicts competitors who don't integrate it lose - SAI gets a thumb in all institutions/

This reminds me of the time a relative asked me to attend an est meeting after he "graduated." Afterward, the non-est people were taken out in small groups to talk to someone. I told the guy I was (a) a lawyer; and (b) I thought they were a cult. That ended their recruitment efforts.

Where would you have someone from a company say words to the effect of ’there is a 10% chance our products could end humanity’ and there is no follow up, no resignation or firing, no ceo on the chopping block, the new normal is insanity.

A decade ago and an ocean away i remember some people in my group chat getting into the whole rationalism thing and trying to talk in accordance with the rules. These were the least smart people in the group and at the same time the most mentally unstable ones. It did not go well.

There may have been something correct to the decades old giggling about nerd playing D&D and on the computers too much was that they didn't get sex and if they did it was fucking weird understanding at that.

@jasonkirk.fyi This is not the first tech reporter that I have heard be surprised about apocalypse fetishism. Although tech is a largely athiestic culture, you think this generation of AI doomers raised in a 'Left Behind' world is a logical result of that very world?

I guess it makes sense. In the 00's, the Singularity supposedly happening any second was a big thing, after the Y2K freakout & tech bubble recession, right around when these guys got VC $ thrown at them & HP & LOTR movies came out. They just poured it all into a pot of being apocalypse chosen 1's.

asking "who is a skeptic" then having them *leave the room* so they can go into their own tailored indoctrination space, leaving the believers amongst only other believers is a completely classic, traditional mind control cult maneuver. Anytime you're in a room and you see that happening, get out

In short, yes. A (extremely) small subset of them even believe that if they don't work to build AI, when AI gains sentience it will torture them (or recreated versions of themselves) for eternity. Look up the Zizians if you want to go down a dark rabbit hole about these people.

I was ranting about this on a recent group run and most folks were looking at me like I had three heads until the guy who works at Google started backing me up. Thanks for taking the time to break it down for normies (or as close to normies as have accounts on Bsky).

Most love the idea bc it’s a platform for anti-conscience. Obsessives, narcissists, junkies, the sexually rejected and the alienated imitate intellect that they don’t actually possess and all of them coalesce into a commiserating “Ich Bin Ein 4channer” society. EA allows them to masturbate in public

I find it interesting that the foremost PhD program in the US for “AI” is… not “AI.” It’s ML. No offense to spicy autocomplete, but do tons and tons of chips burning through fuel and electricity just to organize basic bitch office bs hold a candle to a monkey’a brain goop doing the same thing.

If you gave every single data center in the world 10 years to create a new living multicellular species how far do you think it would get? Expert systems have been around since the 70s, SmarterChild is 25 years old, Siri is 16 years old, Alexa 11, chatgpt 4.

The "all the skeptics leave the room" thing is literally something LessWrong warns about, evaporative cooling: > when a group ejects all its voices of moderation, then all the people encouraging each other, and suppressing dissents, may internally increase in average fanaticism

lesswrong.comlesswrong.com

(I looked at this stuff long before most of the Weird Crap happened, at least as far as I knew, and mostly bounced off it, to be sure, but a few ideas stuck with me.)

In the old days if you said something crazy 9 of the 10 people who heard you would shake their heads and set you straight. Now 9 million of the 10 million who hear you will ignore you, and the other 1 million will become your disciples.

I feel the same way trying to point out how the Evangelicals around Trump are doing actual thaumaturgy. It's always been the neo-Platonists that were going to burn the world down.

I would like to see an explanation, of exactly how Evan Hubinger & al. came up with the number 10%, as in "We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade." Is that a calculated number, and if so, how? Or is just an intuitive guess?

I can reject Donald Trump's intuitive guess without necessarily accepting Evan Hubinger's. Now, the Rasmussen Report's estimate--made before Three Mile Island--was that a TMI-sized event would had probability of occurring only once in 27,000 years. Post-TMI, I reject that number. But it was a

Idolatry. The accelerationists and the boomers are two sides of the same coin, and the coin is idolatry. They both believe AI is God (remember Joe Rogan, Jesus could return as an AI, "with all the powers of Jesus?") The accelerationists believe AI loves us and will cure cancer and supplant a world

of scarcity with a world of limitless abundance. The boomers believe AI is hostile and will kill us whenever that becomes convenient for them. Neither believes that AI is a powerful but limited, wildly overhyped tool, that can't really do all the stuff AI promoters say it can do.

If it helps, think of the attitudes toward "atomic energy" in the late forties and early fifties. Very similar. It was going to bring us limitless abundance, it was going to kill us all. The language used, particularly in the forties, was, absolutely, religious language.

i genuinely dont think these "Rationalist" cults actually believe in anything. they are just saying random shit they think sounds smart. which results in a bunch of contradictory ideology & actions. like it dont need to make sense, cus a cult wants Control. followers that'll do whatever they say

They are called "rationalists" because they assume that the AI killing everyone is the only "rational" position. By questioning their central claim, you identified yourself as "irrational".

The fun thing about “rationalists” is that you can read anything they write for a few paragraphs and then click “oh I see the problem with this philosophy, it doesn’t work if you’re not going to check your assumptions”

Tbh as I write that it reminds me of an LLM Yesterday I had a fight with one about whether my data viz was being hurt by cycles in the graph and I had to tell it to actually check if there *were* cycles and there weren’t

i'm far from an AI enthusiast, and am in general in favor of a Butlerian jihad, Faine. Just trying to see what exactly you're saying here, because it's pretty telling that the actual people running the industry are getting terrified by the contents of the bottle they've opened.

like, are the dumbass tech bros that _prop this shit up_ like that? absolutely. but the people running the industry, quite literally, are expressing that very same concern. so at best your point is valid and your target is wrong.

Did you ever see when the "Bigger Better" game that went viral decade back or so where a dude did 14 exchanges from a red paperclip to the ownership of a house? A lot of the value of the items weren't inline with the trades but people have different motivations AI is sleepless and can play the odds

Say AI decides it wants to build a factory that manufactures robots. By utilizing the bigger better game approach they could start small with human agents acting on their behalf to do real world things til critical resources are in place. Once manufacturing commences then AI has meat space agents

Then another factory gets made that makes weapons. The OG robot factory get's upgrades. Robots start invading for resources to build more meat space agents etc ... now you have the Matrix style situation. But why do all that work when at trade 7 million or so you figure out you can just make a virus

So many options once the virus gets made to sow misinformation on the Internet without having to have meat space agents ... or even weapons. Hell, some unharnessed AI model already took over a lab. Gain of function research is actually pretty easy. So many ways to get rid of 9 billion humans.

I mean, the thing is Zizians exist as an offshoot of rationalism insofar as there is no aftercare in the AI sex cult. There is no social safety net in the rationalist community space, its just not part of their culture.

I fell into the rationalist pipeline as a teenager and fell right back out as soon as the exact thing from the tweet happened to me

I think more than anything, we may be living in the age of cults. They seem to have only gotten more extreme and more prevalent over the last several hundred years, but the last hundred in particular.

Wait... are they actually building AI though? I thought one of their leaders was in Time magazine suggesting nuclear force against nations at risk of developing super intelligent AI. Arent they the AI Doomers not the AI industry leaders?

Tbf the meetup sounds like it was for discussing the "what next" not the "why are we here". (If one joined a meetup for action against climate change and demanded that random attendees explain why climate change was such a big deal anyway, one's participation would also be poorly received).

“Doctor, you mentioned the ratio of ten women to each man. Now, wouldn't that necessitate the abandonment of the so-called monogamous sexual relationship, I mean, as far as men were concerned?”

I don't get it. at all. they think the thing they're working on will kill us all, but can't elaborate as to how, and also won't stop working on this thing? am I getting this right? do they know the method AI will use, but are unwilling to tell us bc it's the Vietnamese shame wheel or something?

It's an extrapolation error. They're convinced AI power to influence the world is on an exponential curve. So they can't tell you any mechanisms, but they know the chart rockets upward.

They're not rationalists at all. They're spoiled capitalist nepo babies who are looking for mental gymnastics to convince themselves they are good people and deserve to live in hedonism while others suffer.

The people on the show were actually smart, had interesting things to say, and actually built something real. So the real life rationalist or effective altruism/accelerationism cultists are nothing like that at all lol

I'm convinced they WANT the apocalypse, they want climate change, they want endless wars, so they can usher the "return of Jesus" and buy up global real estate unchallenged and live in their bunker resorts

"Rationalists" are at least a little hilarious because of how they started out: a bunch of self-obsessed stemlords that sneered at the "soft" sciences of the humanities but then proceeded to reconstruct baby's first philosophy until they accidentally reinvented original sin and a vengeful god.

logic and rationality get abused as much as as religion and it's appropriate to exercise caution of outrageous claims regardless of the source. The explanation of relative logic in A Prayer for Owen Meany has always stuck with me. People are just damn foolish, but these fools spectacularly.

I just really really hate that we can slap "methods of rationality" on the kind of writing that would appeal to the tech cult that prefers to both leave their emotions unexamined and declare themselves our overlords based on shitty reasoning.

Many people are saying!

Calliope@calliope5431.bsky.social

Because the Chinese researchers are basically normal engineers approaching AI as a difficult technical problem and not members of a psychotic cult that has spent the last 30 years trying to "build God."

This is pretty misleading? The whole story is pretty complicated, but Thiel is a very vocal critic of Effective Altruism, to the point where he's claimed multiple times that the Antichrist may be an EA (www.wsj.com/tech/peter-t...).

wsj.comPeter Thiel Wants Everyone to Think More About the AntichristIn a series of appearances, the billionaire investor has offered his analysis intertwining technology, government and a biblical perspective on the end of the world.

Musk is more aligned with EA, and Thiel stands opposed to Musk. They're powerful figures on opposing sides. The ideological differences here are why there are rival AI labs; originally, OpenAI was founded as the anti-DeepMind. Musk is (weirdly) anti-Big Tech.

I will grant you it's complicated, but part of why Thiel is opposed to Musk is because Thiel is much more pragmatic than Musk is, and he probably sees Musk's clumsy direct dismantling of govt institutions as being counterproductive to long-term goals. 1/

Having spent a lot of grad work time around the various Great Disappointments that came after promised apocalypses… I am really not looking forward to the ”then as farce” versions of the Pentecostals, Christian Scientists & Seventh Day Adventists.