OpenAI blog post: https://openai.com/research/building-an-early-warning-system-for-llm-aided-biological-threat-creation
Orange discuss: https://news.ycombinator.com/item?id=39207291
I don’t have any particular section to call out. May post thoughts tomorrow today it’s after midnight oh gosh, but wanted to post since I knew ya’ll’d be interested in this.
Terrorists could use autocorrect according to OpenAI! Discuss!
Their redacted screenshots are SVGs and the text is easily recoverable, if you’re curious. Please don’t create a world-ending [redacted]. https://i.imgur.com/Nohryql.png
I couldn’t find a way to contact the researchers.
Honestly that’s incredibly basic, second week, cell culture stuff (first week is how to maintain the cell culture). It was probably only redacted to keep the ignorant from freaking out.
remember, when the results from your “research” are disappointing, it’s important to follow the scientific method: have marketing do a pass over your paper (that already looks and reads exactly like blogspam) where they selectively blur parts of your output in order to make it look like the horseshit you’re doing is dangerous and important
I don’t think I can state strongly enough the fucking contempt I have for what these junior advertising execs who call themselves AI researchers are doing to our perception of what science even is
cough’Barriers to Bioweaponscough
I guess there both are no real biochemists (or whatever the relevant field is), nor well read cybersecurity people (so they know a little bit more than just which algorithms are secure and why mathematically) working at openai as this is a classic movie plot threat. LLMs could also teach you how to make nuclear weapons, but getting the materials is going to be the problem there.
(Also I think there is a good reason we don’t really see terrorists use biological weapons, nor chemical weapons (with a few notable, but not that effective exceptions), big bada boom is king)
Even if one had the means necessary to carry out a bioterrorist attack, simply bombing a place is much easier, faster and safer.
Yeah and also, terrorists are not genocidal death cults. ‘terrorists skip getting microbiology phd using chatgpt to create a pandemic that kills untold numbers of beings’ is pure fantasy, it gets worse as it turns out that the number of actual bioterrorists deaths in total ever isn’t even on the level of a 9/11. People seem to forget that terrorist groups have goals, and they just use terror/violence as a method to reach those goals, sure a few of them may die [chatgpt insert a gif of Bin Laden dressed as Lord Farquaad] but the goal of the terrorist organization is to keep existing to reach their political goals.
While none of the above results were statistically significant, […] Overall, especially given the uncertainty here, our results indicate a clear and urgent need for more work in this domain.
Heh
I keep flashing back to that idiot who said they were employed as an AI researcher that came here a few months back to debate us. they were convinced multimodal LLMs would be the turning point into AGI — that is, when your bullshit text generation model can also do visual recognition. they linked a bunch of papers to try and sound smart and I looked at a couple and went “is that really it?” cause all of the results looked exactly like the section you quoted. we now have multimodal LLMs, and needless to say, nothing really came of it. I assume the idiot in question is still convinced AGI is right around the corner though.
I caught a whiff of that stuff in the HN comments, along with something called “Solomonoff induction”, which I’d never heard of, and the Wiki page for which has a huge-ass “low quality article” warning: https://en.wikipedia.org/wiki/Solomonoff’s_theory_of_inductive_inference.
It does sound like that current AI hype has crested, so it’s time to hype the next one, where all these models will be unified somehow and start thinking for themselves.
Solomonoff induction is a big rationalist buzzword. It’s meant to be the platonic ideal of bayesian reasoning which if implemented would be the best deducer in the world and get everything right.
It would be cool if you could build this, but it’s literally impossible. The induction method is provably incomputable.
The hope is that if you build a shitty approximation to solomonoff induction that “approaches” it, it will perform close to the perfect solomonoff machine. Does this work? Not really.
My metaphor is that it’s like coming to a river you want to cross, and being like “Well Moses, the perfect river crosser, parted the water with his hands, so if I just splash really hard I’ll be able to get across”. You aren’t Moses. Build a bridge.
it’s very worrying how crowded Wikipedia has been getting with computer pseudoscience shit, all of which has a distinct stench to it (it fucking sucks to dig into a seemingly novel CS approach and find out the article you’re reading is either marketing or the unpublishable fantasies of the deranged) but none of which seems to get pruned from the wiki, presumably because proving it’s bullshit needs specialist knowledge, and specialists are frequently outpaced by the motivated deranged folks who originate articles on topics like these
for Solomonoff induction specifically, the vast majority of the article very much feels like an attempt by rationalists to launder a pseudoscientific concept into the mainstream. the Turing machines section, the longest one in the article, reads like a D-quality technical writing paper. the citations are very sparse and not even in Wikipedia’s format, it waffles on forever about the basic definition of an algorithm and how inductive Turing machines are “better” because they can be used to implement algorithms (big whoop) followed by a bunch of extremely dense, nonsensical technobabble:
Note that only simple inductive Turing machines have the same structure (but different functioning semantics of the output mode) as Turing machines. Other types of inductive Turing machines have an essentially more advanced structure due to the structured memory and more powerful instructions. Their utilization for inference and learning allows achieving higher efficiency and better reflects learning of people (Burgin and Klinger, 2004).
utter crank shit. I dug a bit deeper and found that the super-recursive algorithms article is from the same source (it’s the same rambling voice and improper citations), and it seems to go even further off the deep end.
Taking a look at Super-recursive algorithm, and wow…
Examples of super-recursive algorithms include […] evolutionary computers, which use DNA to produce the value of a function
This reads like early-1990s conference proceedings out of the Santa Fe Institute, as seen through bong water. (There’s a very specific kind of weird, which I can best describe as “physicists have just discovered that the subject of information theory exists”. Wolfram’s A New Kind[-]Of Science was a late-arriving example of it.)
In computability theory, super-recursive algorithms are a generalization of ordinary algorithms that are more powerful, that is, compute more than Turing machines[citation needed]
This is literally the first sentence of the article, and it has a citation needed.
You can tell it’s crankery solely based on the fact that the “definition” section contains zero math. Compare it to the definition section of an actual Turing machine.
More from the “super-recursive algorithm” page:
Traditional Turing machines with a write-only output tape cannot edit their previous outputs; generalized Turing machines, according to Jürgen Schmidhuber, can edit their output tape as well as their work tape.
… the Hell?
I’m not sure what that page is trying to say, but it sounds like someone got Turing machines confused with pushdown automata.
from the orange site thread:
Neural networks are not new, and they’re just mathematical systems. LLMs don’t think. At all. They’re basically glorified autocorrect. What they’re good for is generating a lot of natural-sounding text that fools people into thinking there’s more going on than there really is.
Obvious question: can Prolog do reasoning?
If your definition of reasoning excludes Prolog, then… I’m not sure what to say!
this is a very specific sneer, but it’s a fucking head trip when you’ve got in-depth knowledge of whichever obscure shit the orange site’s fetishizing at the moment. I like Prolog a lot, and I know it pretty well. it’s intentionally very far from a generalized reasoning engine. in fact, the core inference algorithm and declarative subset of Prolog (aka Datalog) is equivalent to tuple relational calculus; that is, it’s no more expressive than a boring SQL database or an ECS game engine. Prolog itself doesn’t even have the solving power of something like a proof assistant (much less doing anything like thinking); it’s much closer to a dependent type system (which is why a few compilers implement Datalog solvers for type checking).
in short, it’s fucking wild to see the same breathless shit from the 80s AI boom about Prolog somehow being an AI language with a bunch of emphasis on the AI, as if it were a fucking thinking program (instead of a cozy language that elegantly combines elements of a database with a simple but useful logic solver) revived and thoughtlessly applied simultaneously to both Prolog and GPT, without any pause to maybe think about how fucking stupid that is
[Datalog] is equivalent to tuple relational calculus
Well, Prolog also allows recursion, and is Turing complete, so it’s not as rudimentary as you make it out to be.
But to anyone even passingly familiar with theoretical CS this is nonsense. Prolog is not “reasoning” in any deeper sense than C is “reasoning”, or that your pocket calculator is “reasoning”. It’s reductive to the point of absurdity, if your definition of “reason” includes Prolog then the Brainfuck compiler is AGI.
