Well, the LLM was prompted to find the odd one. Which I consider a (relatively) easy one. Reading the headline, I thought that the LLM was able to point this out by itself, like “Excuse me, but you had one sentence about pizza toppings in your text about programming. Was that intended to be there for some reason, or just a mistaken CTRL-V?”
me when the machine specifically designed to pass the turing test passes the turing test
If you can design a model that spits out self-aware-sounding things after not having been trained on a large corpus of human text, then I’ll bite. Until then, it’s crazy that anybody who knows anything about how current models are trained accepts the idea that it’s anything other than a stochastic parrot.
Glad that the article included a good amount of dissenting opinion, highlighting this one from Margaret Mitchell: “I think we can agree that systems that can manipulate shouldn’t be designed to present themselves as having feelings, goals, dreams, aspirations.”
Cool tech. We should probably set it on fire.
I agree, except with the first sentence.
- I don’t think a computer program has passed the Turing test without interpreting the rules in a very lax way and heavily stacking the deck in the bot’s favor.
- I’d be impressed if a machine does something hard even if the machine is specifically designed to do that. Something like proving the Riemann hypothesis or actually passing an honest version of Turing test.
The Turing test doesn’t say any of that. Which is why it was first passed in the 60s, and is a bad test.
Despite the hype, from my admittedly limited experience I haven’t seen a chatbot that is anywhere near passing the turing test. It can seemingly fool people who want to be fooled but throw some non-sequiturs or anything cryptic and context-dependent at it and it will fail miserably.
As somebody said, and im loosely paraphrasing here, most of the intelligent work done by ai is done by the person interpreting what the ai actually said.
A bit like a tarot reading. (but even those have quite a bit of structure).
Which bothers me a bit is that people look at this and go ‘it is testing me’ and never seem to notice that LLMs don’t really seem to ask questions, sure sometimes there are related questions to the setup of the LLM, like the ‘why do you want to buy a gpu from me YudAi’ thing. But it never seems curious in the other side as a person. Hell, it won’t even ask you about the relationship with your mother like earlier AIs would. But they do see signs of meta progression where the AI is doing 4d level chess style things.
As somebody said, and im loosely paraphrasing here, most of the intelligent work done by ai is done by the person interpreting what the ai actually said.
This is an absolutely profound take that I hadn’t seen before; thank you.
I’m confused how this is even supposed to demonstrating “metacognition” or whatever? It’s not discussing its own thought process or demonstrating awareness of its own internal state, it just said “this sentence might have been added to see if I was paying attention.” Am I missing something here? Is it just that it said “I… paying attention”?
This is a thing humans already do sometimes in real life and discuss – when I was in middle school, I’d sometimes put the word “banana” randomly into the middle of my essays to see if the teacher noticed – so pardon me if I assume the LLM is doing this by the same means it does literally everything else, i.e. mimicking a human phrasing about a situation that occurred, rather than suddenly developing radical new capabilities that it has never demonstrated before even in situations where those would be useful.
