People keep telling me ChatGPT is smart. Is it really?
I once again put it to the test. It was very good at some things, but not others. It was very good at giving me bullshit, my dog-ate-my-homework excuses, offering me a bar graph after I asked for a map, falsely claiming that it didn’t know how to make maps.
A minute later, as I turned to a different question, I discovered that it turns out ChatGPT does know how to draw maps. Just not very well.
The main test I asked for was a map of states with (major) ports and above average income. Here’s what I got the first time I asked.
To which I asked, “Why did you highlight Illinois”, which wasn’t in the table of raw data ChatGPT had shown me a moment earlier.
Ok, sure, I said. ChatGPT came back with this:
Now I am confused. How come you highlighted Florida the first time but not the second? (And what happened to Texas?)
ChatGPT replies with a mixture of mumbo jumbo (about stricter income criteria) and obsequiousness (“you are sharp to notice that”) — and another offer to redraw the infographic. This time it disappeared the whole west coast (and for good measure changed the color scheme):
Um.
Wild!
Ok, let’s try once more.
Vermont?
No shit.
Overall, five maps, all different. None of them are quite right. (Illinois comes and goes. S do Florida, Washington, Texas, California, etc.).
How are you suppose do data analysis with “intelligent” software that can’t nail something so basic? Surely this is not what we always meant by AGI.
§
The recent vals.ai financial benchmarks showed comparable failures reading basic financial reports. (On questions like “Which Geographic Region has Airbnb (NASDAQ: ABNB) experienced the most revenue growth from 2022 to 2024?” performance was near zero.)
It’s true that newer models get more and more questions right, in large part as a function of being trained on more data. But their ability to reliably follow instructions still seems lacking. It is a fantasy to think that we can get to safe AI without addressing the problem head on.
Gary Marcus is certain that neurosymbolic AI can help.
