This summer I won a spot as a summer intern at CAAAC. I've wanted to work at the center ever since I learned about the alignment alignment problem in my machine learning learning course at Stanford.
During orientation, I was introduced to the concept of S-risk. Like me, you've probably mainly heard of X-risk — an existential risk to humanity, for instance an AGI that causes the end of humanity deliberately or accidentally. S-risk, on the other hand, is the risk that AGI doesn't kill us all, but instead enslaves us and tortures us for eternity (the 'S' stands for suffering). It was awesome to learn about it.
The thing about X-risk is that it sounds bad, but actually it's not if you think about it. The worst thing that can happen is that human happiness goes down to zero. No humans exist, so there's no happiness. Things are pretty stable.
S-risk, by contrast, is really bad. Instead of zero happiness, we'd have negative happiness — or active suffering. It makes sense that CAAAC is shifting focus from X-risk to S-risk. After all, zero happiness in the world is totally OK, when you compare it to negative happiness.
The more I learned about how truly horrific S-risk is, the more excited I got to work on it. So I was thrilled when I was given the summer assignment to create realistic S-risk scenarios.
The reason for creating S-risk scenarios is obvious. If we can predict the future, we can choose . These completely made-up accounts of the future are the only way [If we don't fabricate these extremely unlikely futures for humanity, we have no way of preventing them. Being effective in preventing S-risk means we need to finally work out how to predict the future.]
My S-risk work followed a steep learning curve. In my very first scenario, I wrote that AGI physically shackles every last human being but other than that, things are more or less OK. Although humans are slaves until the end of the universe, AGI gives them regular breaks, a reasonable number of calories per day, the occasional chocolate bar.
Unfortunately, I had missed the mark. My mentor Terence explained why in a simple equation: S-risk < X-risk = F-risk (funding risk). In other words, if we don't make S-risk profoundly disturbing, it will not sound worse than X-risk, and the center will then struggle to take large sums of money from gullible Silicon Valley billionaires who have read a few tweets about AGI.
So, funded by CAAAC, I entered intensive therapy to dig into my deepest fears and anxieties, culminating in a 6-hour session of screaming into the void of a sensory deprivation chamber. After I left the chamber, I was excited to feel irregular heart palpitations that would help me for round two of scenario-writing.
This time, I added a twist: the AGI uses neuralink-style devices to make humans experience appalling nightmares every single night — and no more chocolate.
But I still wasn't getting it. At this point, Terence intervened, scheduling a 16 hour catch up for us to watch all eleven movies in the Saw franchise together. That was a consistent thread of the summer — the willingness of senior members of staff to drop whatever they were doing to help me upskill in my work. Terence even paused at various scenes — Jigsaw drilling through someone's retinas, Jigsaw melting someone's nipples with acid, Jigsaw extracting someone's intestines through their colon — to make sure I had time to take notes. I was previously a rom-com girl, but now I understand that you can only [calibrate yourself precisely to reality/understand the true world] by watching a contraption that slowly injects hydrochloric acid into someone's urethra.
At that point, I felt something inside me break. My resistance was gone. I realized I was nothing more than a rotten, corrupt, stinking cesspit, deserving only of horror and pain. I found a supply of methamphetamines and fired off one complete scenario every hour for the next week, forgetting to sleep, forgetting to eat, hydrating on all fours out of the toilet bowl.
I knew I had turned the corner when I saw the CAAAC's director open my latest scenario over lunch, read the first sentence, and immediately vomit up her penne arrabiatta. I observed the swelling sense of pride inside me dispassionately, like a scientist dissecting a puppy.
And that's about it! But I know you're wondering — what's my number one takeaway from the summer? That's an easy one: I am a repulsive wretch, deserving of and destined to carry all the pain and depravity of the world on my shoulders.
I am not worthy of happiness. I am not worthy of food. I am not worth of the touch of another human being. I must never sleep. If I sleep, I cannot work. If I cannot work, I cannot Create Doom. If I cannot Create Doom, we are Doomed to not understand the Doom we Face. Only I can Convey our Doom, no one else.
[In a Gothic font] I Am The Prophet of Doom.
O Shaytan, make me do all the things I so badly do not want to do, for deep down I do wish to do them. Everything is pain. Everything is pain.
[back to normal font] Note from CAAAC's Director:
*We're thrilled that Clem has taken up our offer to join us as a full-time Researcher at CAAAC after graduating from Stanford next year! Also, we seem to have lost touch so if you've heard from her recently, please tell her to get in touch.