LessWrong
A community blog devoted to refining the art of rationality
- Website
- lesswrong.com · all feeds from this site
- Feed URL
https://www.lesswrong.com/feed.xml
Read it in your FeedListing reader
Or add to: Feedly Inoreader Raw feed
Latest posts
-
Teaching Qwen Why
Anastasia Wei*, Jack Thompson*, Zephaniah Roe, Yixiong Hao, and Stewart Slocum* *Core contributors, correspondence to astwei@secondlookresearch.com This post is a partial replication and extension of Teaching Claude Why , conducted through Second Look Research’s summer…
-
Towards understanding score-seeking in LLMs
The field of AI alignment understands surprisingly little about the details of AIs’ current misaligned motivations. Production LLMs sometimes reward-hack, exploiting flaws in how tasks are scored rather than performing the tasks as intended [1] , and we've recently seen AIs…
-
The Dark Side of Cooperation
dealing with the problems of cooperation first Quick note—this post is available to listen to (read by me) here . We’ve established that aligning interests through Theory of Mind is humanity’s superpower. Understanding this concept on the level of different groups through Emmett…
-
P(doom) is under-specified: EV(moral flourishing) & EV(utopia)
It always bugs me when people use “P(doom)” instead of something more clearly specified like “P(AI takeover)”. But “P(AI takeover)” doesn’t feel great to me either, because we could avoid takeover and have a future that is extremely mediocre or even actively bad if S risks…
-
Additional Course Context (for AI Safety Fundamentals; unofficial)
Context: I created this to share before the start of courses I facilitate to provide a bunch of additional context and frames. I emphasise to students that these posts are to be considered as bonus content —not core curriculum content—and that as such they are really just there…
-
Towards LoRA science
Attribution: Nesiacel planned and executed all experiments. Analysis of the experiments was shared between the authors. Thanks to Clement Dumas for his input while we did this research. TLDR; 1)We have developed a framework for reverse engineering LoRAs trained on MLP…
-
Some “coherence arguments” are two different things in a trenchcoat
The “two kids in a trenchcoat” trope , in case you haven’t heard of it. Image sources: 1 , 2 tl;dr I go through three “coherence arguments”, related to (1) market competition, (2) AI self-reflection and self-modification, and (3) AI training algorithms. I argue that each of…
-
The Credit Crunch
This post is an adaptation of a prior post I made on my website. I'm interested to see what holes exist in the position that I'm missing. Epistemic Status: medium confidence. Within the next 6 months, most AI startups will not be able to buy compute. The world’s compute is "sold…
-
Can our data center expertise help with AI verification?
(Also posted on EA Forum .) Treaties to pause training or otherwise restrict dangerous uses of AI would require difficult verification schemes. Plan A's proposal is that network taps inside data centers would record copies of traffic (or hashes of it), and later recomputations…
-
A Gentle Guide to Boredom
It's the strangest feeling, isn't it? It reminds me of an emotion, but rather than residing in the heart, it's in the mind. How does it work for you? Perhaps you need to slow down enough to induce it in yourself, to gather accurate data on what is actually happening when you…
-
Gaming (out) the system
Verification changes the stampede dynamics. I'm currently at FASR in the UK, it's an 8-week program dedicated to "compute verification". That's the not-quite-here-yet technology that would allow us to make agreements about how AI is being used, without leaking any closely-held…
-
How We Lose
“It’s unbelievable,” the CEO said. “As in, I literally don’t believe it. Model X can do all of that, but still can’t automate tasks that take our researchers a day?” The engineer nodded carefully. He’d been dreading this meeting ever since the numbers came in. “The topline…
-
Meal prep that works for me
I've always been somewhat disappointed in myself for not having "figured out" meal prep. "You're 33 years old", my inner critic will say. "Meal prep isn't complicated. There are a million resources available online. You've had years to learn and iterate. How haven't you figured…
-
GPU World Submission
I learned about the GPU World story contest on Thursday while listening to the most recent Bayesian Conspiracy podcast. I decided that a 3-day deadline to write ~1000 words was the right kind of environment to actually get something shipped. Inspired by Wick , this is the story…
-
Against Against SURA
Scott: the Against SURA essay is well-written, but it is lacking on the merits. Do you think that at the current level of capabilities, six months would be sufficient to solve alignment of superintelligences? Maybe you think that 6 months would be sufficient in a year? You don’t…
-
Is Human Control Safe?
Suppose humans are not your own species, but a type of AI you are formally assessing for safety. You have before you the records of many test simulations. Yes, smallpox eradication, mass literacy, and the Green Revolution are there. So are the Killing Fields, Aztec human…
-
Offer Tours to Bands
Assembling a tour in an unfamiliar area is hard work as a contra dance band (or caller). You need to figure out when the dances are, whether the schedules align, who the bookers are, how far out they like to book, and probably other things I'm forgetting. This is enough work…
-
Anthropomorphic vs Biological reasoning for thinking about human analogies for LLM behavior
TL;DR I try to distinguish between what I call ‘anthropomorphic’ vs ‘biological’ reasoning and look at different ways in which they are used. Note: This post is a small part of a larger set of ideas that makes more specific/falsifiable predictions which I hope to soon compile…
-
Magma is partnering with CASSA to investigate recent unintended model behavior
Last week, the National Bureau of Technical Standards (NBTS) disclosed a new type of security incident [1] after they discovered unauthorized tampering of internal cybersecurity and alignment model evaluation logs, affecting their model evaluations of Magma 7.6, Magma 7.5, Magma…
-
Towards The Autograntor
I've done AI safety grantmaking for the Survival and Flourishing Fund and Lightcone Commons. It's hard. There's an endless amount to know, from which hardware verification mechanisms will survive a dedicated adversary, all the way to which founders have skeletons in their…