Computer Science, Off Course! Episode 4 - What Computers Can't Do

There is a new episode, again! This time about the book What Computers Can't Do from Hubert Dreyfus, a philosopher who in the 1960s worked at MIT while AI research was going on there (by Minsky, for example, who we might one day also cover on the pod, as I read a lot of his stuff for my upcoming Dutch book).

In this period he wrote a first version of this book (a paper called Alchemy and AI), and it stirred a lot of controversy. As Paula McCorduck writes in her book Machines who thing:

Unsurprisingly, Dreyfus’s work was not greeted with cheers by AI workers. They objected that he was basically ignorant of what computers could already do, that his ignorance led him to misunderstand anything but the simplest empirical evidence [...], that he could not grasp current work or its implications and assumed that if a program wasn’t already doing something, then it never would. Some who have read him argue that he is simply a man trying to promote his own particular brand of philosophy and using artificial intelligence as a scapegoat.

Something like today's AI-fans might also say of critics!

Seymour Papert penned a lengthy reply called "The Artificial Intelligence of Hubert L. Dreyfus: A Budget of Fallacies" in which he called Dreyfus's work "technical nonsense".

Is it? It is very interesting to read the book in 2026, as so many things did come true and still hold, and so many things didn't.

What AI?

What should be noted of course, is that this book dates from (or maybe one could say: helped cause) the first AI winter, in which people are gradually coming to the conclusion that symbolic AI—AI that works by capturing complex systems in rules and executing those rules—can never work. Herbert Simon predicted in the mid 1950 that computers would win the world championship in chess in 10 years, clearly by the time that Dreyfus's book comes out in 1972, that prediction did not come true at all. Dreyfus writes:

Rather, it has turned out that, for the time being at least, the research program based on the assumption that human beings produce intelligence using facts and rules has reached a dead end, and there is no reason to think it could ever succeed.

And yeah, that is sort of true, still! LLMs might be able to do a lot, much much more than Dreyfus ever predicted, but indeed not "using facts and rules". Dreyfus has a few very cool examples like what gift to bring to a party (which he takes from Bourdieu).

[K]nowing how to give an appropriate gift at the appropriate time and in the appropriate way requires cultural savoir faire. So knowing what a gift is is not a bit of factual knowledge, separate from the skill or know-how for giving one. The distinction between what a gift is and what counts as a gift, which seems to distinguish facts from skills, is an illusion fostered by the philosophical belief in a nonpragmatic ontology

So many cultural rules and social norms that prescribe what is ok, but at the same time, so much freedom. Dreyfus writes:

Bourdieu comments: The active presence of past experiences . . . deposited in each organism in the form of schemes of perception, thought, and action, tend to guarantee the 'correctness' of practices and their constancy over time, more reliably than all formal rules and explicit norms.

Hanna added here: where does the robot get ideas for gifts? Because she (and I am sure many other people also) get them while showering!

Concrete failures versus theoretical failures

One thing that Dreyfus sadly does not do very well in the book is to point at things that computers can never do, such as Hanna's lovely example of tasting food (which I am hinting at in my drawing for this episode). This is not so much a matter of a computer can't do it—I am sure people would argue that a computer equipped with a mass spectrometer can analyze food and thus "taste" it–but more of a thing a computer does not have the capacity for, as it does not experience the world, as Dreyfus calls this "it is not in a situation".

Sometimes he hints at differences, but not clearly (enough):

This enables us to see the fundamental difference between human and machine intelligence. Artificial intelligence must begin at the level of objectivity and rationality where the facts have already been produced. It abstracts these facts from the situation in which they are organized and attempts to use the results to simulate intelligent behavior. But these facts taken out of context are an unwieldy mass of neutral data with which artificial intelligence workers have thus far been unable to cope. All programs so far "bog down inexorably as the information files grow."

Drawing on philosophers

Dreyfus is so fun to read for me, because he keeps citing philosophers, but not all of their broadness, which would simply be to much for me to all read and take in, but specifically for their application to AI. He for example extensively builds upon the work of Searle, who explained in his book that there are metaphors that work, without them directly relying on comparisons, like when you say: this person is just like a block of ice. It has meaning, it can be understood (to mean they are not very gentle, caring people) but why? Dreyfus says (with Searle):

don't know any better way to describe these abilities than to say that they are nonrepresentational mental capacities.

Yes! There are certain things that your brain "just does", nonrepresentationally. He explains this early in the book, with a quote that I see as the basis on which AI and even broader, computer science, are build:

[There is a]n epistemological assumption that all knowledge can be formalized, that is, that whatever can be understood can be expressed in terms of logical relations, more exactly in terms of Boolean functions, the logical calculus which governs the way the bits are related according to rules.

And as I said, he was so helpful to get me some footing in philosophers thinking about AI (as Turing, famously, neglected) Wittgenstein, for example:

[Wittgenstein says:] We are unable clearly to circumscribe the concepts we use; not because we don't know their real definition, but because there is no real "definition" to them. To suppose that there must be would be like supposing that whenever children play with a ball they play a game according to strict rules.

Fringe consciousness

Dreyfus also gives us nice concepts to think about, like fringe consciousness:

Fringe consciousness takes account of cues in the context, and probably some possible parsings and meanings, all of which would have to be made explicit in the output of a machine. Our sense of the situation then allows us to exclude most of these possibilities without explicit consideration. We shall call the ability to narrow down the spectrum of possible meanings as much as the situation requires "ambiguity tolerance."

Let's leave the final quote to Weizenbaum, who we will do soon also:

What is wrong, I think, is that we have permitted technological metaphors, (...) and technique itself to so thoroughly pervade our thought processes that we have finally abdicated to technology the very duty to formulate questions. (...) Where a simple man might ask: "Do we need these things?", technology asks "what electronic wizardry will make them safe?" Where a simple man will ask "is it good?", technology asks "will it work?" Thus science, even wisdom, becomes what technology and most of all computers can handle

Final note: Dreyfus's thinking inspired me a great deal this year. Not only for this episode, I also wrote a Dutch piece in Volkskrant on his puzzle on "the box is in the pen".

Homework

In the episode I suggested to read this paper, but actually reading it, I don't think it is really good. Teaches me (hopefully? maybe?) to not do a search while recording.

What is a great homework, I think, is to write down for yourself (here in the comments, if you want to):

1. What computers still can't do now?
2. What computers should never do (like, eating ice cream)