Hmmmm

Some people were talking about AI agents and whether LLMs can strategise.

So I asked ChatGPT to play chess with me

It tried to make an illegal move after 3 moves

chatgpt.com/share/68ba54...

So I think the answer is still no. It can't

For comparison, this is what the older ChatGPT models responded with. It certainly failed at this completely trivial reasoning riddle. I can't remember when I took this screenshot, but it must've been a year or 2 ago

So I asked it the same question, but without using chain of thought. It seemed to now solve this problem in one step, which is a tad odd. Especially considering in the last response, it said "people often answer playfully", implying this is might be in its training data. So it might be memorising

So, GPT5 is out. I had to ask it t he same question, what's the shortest 4 letter word?

It seems it can solve it, but the chain of thought it employed seemed to make absolutely zero sense. It looks more like it's trying to regenerate the same prompt over and over again and got stuck

Nice try Gemini lol. It's been like 2 years since I first came up with this trick question and some models still struggle to spot it it on first attempt

Kraut, that's not codfish liver. That's a really common Chinese fish thing.

It's a sort of fried fish in oil mixed with salted fermented soybeans.

I absolutely loved it as a child, and I still kind of like it. It's somewhat expensive here in the UK.

Here's me holding my can of it