Computer Science, Off Course! Episode 5 - Plans and Situated Action

We are at already at episode 5 (so halfway the season, as we will do 10 episodes, maybe with a bonus one at the end).

This one is on the book "Plans and Situated Action" by 1985 by ethnographer Lucy Suchman, based partly on her work at Xerox PARC that she did in the 1970. In her most famous study, she puts videos in a room with a new photocopier in it, and observed people (smart computer people) struggle with the interface. You can see the video below:

The video is extra funny if you know one of the people struggling is Turing Award winner Alan Newell! So we can't say that these are 'dumb' people not knowledgeable about computers (in theory, at least).

The video also made me think of the question of Brenda Laurel of episode 2: What is the performance the software is giving? What performance is the copier given?

This type of study, ethnography applied to digital interfaces, at the time was very new, and it clearly shows how needed it was (and is!) to study how people actually use software. This I think is done way way too little, and it is done, it is often done by people not directly involved in making software (in business testers or QA people, in research HCI people). And while that might add something, it sadly means that people making software are entirely removed from ever seeing such struggles.

Situated action

So what is the book about? One core concept of the book is "Situated action". How I understand the book is that at that time, cognitive science (which was then much more intertwined with computer science and AI) took the position that when people do things, they have a plan in mind and simply execute the plan. Such a plan, she says, can "prescribe action at whatever level of detail one might imagine."

In the book, she argues that there is a second view [[1]] of what people do, which she calls a situated action:

Situated action as such comprises necessarily ad hoc responses to the actions of others and to the contingencies of particular situations. Rather than depend upon the reliable recognition of intent, successful interaction consists in the collaborative production of intelligibility through mutual access to situation resources, and through the detection, repair or exploitation of differences in understanding.

This made me think of the work of Hanna's colleague Mark Dingemanse, who worked on the repair that happens when people communicate (paper) and about the use of the word "huh" (video)? I wrote about his work for NRC last year too, in Dutch.

When do we have AI?

One of the things she explores in the book is the Turing Test, although she does not say it with so many words:

That is, if one builds a device whose behavior is indistinguishable from that of an intelligent actor, is that device intelligent?

This I think you have to see in context too, she lives in the same world as Hubert Dreyfus, surrounded by people trying to get to "AI" (like Newell who we saw in the video). I am not entirely sure of their connection, but she thanks him in the foreword, so at least they know each other a bit! Her goal, like the work of Dreyfus is to understand the issues they run into, although she frames this as a way not just of critiquing the program of AI, but of understanding what interaction means:

In any case, my purpose here is to clarify some of the existing troubles in the project of constructing interactive machines, as a way of contributing to our understanding of interaction. (emphasis ours)

What is a brain? Suchman tries to capture what Cognitive Scientists argue:

[...] that mind is best viewed as neither substance nor as insubstantial, but as an abstractable structure implementable in any number of possible physical substrates.

Impact of Suchman's work

It is an interesting question what the legacy of Suchman is. I found an article in MIT technology review from the 90s raving about her work:

Suchman questioned the computer scientists’ assumption that office work was so straightforward and procedural that it should be tailor-made for computerization. To prove her point, she began studying the most seemingly procedural group she could find: accounting.

It is interesting to see how hew work not seems to have made it mainstream, because sadly we still have today have computer scientists like Michiel Bakker saying (in Dutch) that all knowledge work is computer work.

However in the 90s apparently, it looked like she was having an impact. The MIT tech review piece explains how her pioneering work made more of the same type of work possible:

Today, from a lone practitioner initially focused simply on observing work practices, such “ethnographic” efforts have swollen to about a dozen anthropologists, artificial intelligence experts, and computer scientists striving to create technology based on the nature of real workgroups in the real world.

With real world impact:

Their growing understanding of the nature of these jobs has allowed them to write scientific papers on the often-overlooked but important ways knowledge is informally created and shared in the office, while also providing fodder for design of novel technologies to make work easier. Already these exotic and seemingly fuzzy pursuits have paid millions in demonstrable benefits to Xerox and its customers, an indication that parc has a unique approach to innovation that foretells even bigger dividends down the road.

How they see their own work is also interesting:

“Our goal is not first and foremost to create fundamental knowledge,” he relates. “Our goal is to crack real problems that really make sense, but crack them by going to the root of those problems. In the process I believe very profound fundamental knowledge gets produced.”

This says something about what fundamental is defined as!

Issues with the linguistic nature of AI

In a part of the book that is most interesting maybe to today's AI discourse, Suchman talks about the "increasingly linguistic, rather than mechanistic" communication that people have with machines. It is very interesting to see how that was observed then, I imagine it was talking more about programming with newer languages like COBOL or FORTRAN versus assembly language, not about chatbots, of course. But much of what she says is still very true!

She sites work from the mid 1980s describing some differences:

[T]he ability to interact gracefully depends on a number of relatively independent skills: skills involved in parsing elliptical, fragmented, and otherwise ungrammatical input; in ensuring that communication is robust (ensuring that the intended meaning has been conveyed); in explaining abilities and limitations, actions and the motives behind them; in keeping track of the focus of attention of a dialogue; in identifying things from descriptions, even if ambiguous or unsatisfiable; and in describing things in terms appropriate for the context. (Hayes and Reddy. p. 232).

A lot of these things are still so true, an LLM cannot, for example, ensure that the meaning of what it outputs has been conveyed, as it itself does not grasp the meaning, nor can it accurately understand if you as a user understand it. There is such a huge difference between pressing a button and chatting with an LLM in terms of communication, lots of grounds to explore for ethnographers of computers (if only they would be listened to...).

What did not come true

Some things of course, like with the work of Dreyfus, which we covered last episode, no longer really holds up:

Simulation of natural language understanding, even when the language is written rather than spoken is proving to be a profoundly difficult problem that is inseparable from the problem of simulating intelligence as such.

We are now in a time where we absolutely have simulation of understanding, but not, I would say, intelligence. They were not so closely related after all, understanding a box can be in a pen is doable if you have enough data, but no situated world model. The above mentioned Hayes and Reddy are cited again in explaining some of the difficulties that AI was facing then:

Until a solution is found to the problems of organizing and using the range of world knowledge possessed by a human, practical systems will only be able to comprehend a small amount of input, typically within a specific domain of expertise.

Vagueness!

As I say in the episode as well, it is so nice to see my feelings about the issues with the abstract and mechanical way of thinking of computer science vocalized by others! Suchman writes:

Researchers interested in machine intelligence and in human-machine communication have embraced the traditional philosophical and scientific view that purposeful action is planned action. Yet even casual observation of purposeful action indicates that, as common sense formulations of intent, plans are inherently vague. To the Cognitive Scientist [these are the AI researchers of the day], this vagueness is a fault to be remedied, insofar as in any event a plan is the prerequisite for purposeful action, and the details of action are derivative from the completion and modification of the plan.

This is so so descriptive of how computer science people still think (and how we are taught to think!) that vagueness is bad and must be fixed. Whereas vagueness can be an essential part of software as well, which we talked about a bit before in the episode on Peter Naur (and we will talk about that more also in the upcoming episode on Weizenbaum!).

On LLMs

And of course there is also something to chew on regarding LLMs, where she says:

[E]very occasion of human communication is embedded in, and makes use of, a taken for granted but mutually accessible world.

Clearly LLMs does not have this, they will never be able to access the shared world, at least not in the same way as people do.

Closing it of with one quote for the radicals among us. Suchman cites Erving Goffman about the possibilities of language:

Thus the whole framework of conversational constraints ... can become something to honor, to invert, or to disregard, depending as the mood strikes. (Erving Goffman 1975, p. 311).

That again reminded me of the description of a complex, human, social acitivity like gift giving as Dreyfus described.

Homework

There is a lot of fun things to do after this one!

• There is a set of papers between Lucy Suchman and Herbert Simon from the 1990s you could read. It starts with this one by Vera and Simon and a response by Suchman and a reply again by Vera and Simon. They all appeared together in a special issue of Cognitive Science.
  
• Think of a computer system when you use it, and try to reflect on what your objective or plan is, and what actions are you taking (and for bonus points: what is the conversation you are having with the computer)
• When you can: Observe users of your software and see what happens! You can of course also do this for something you have not build!

You can also check out Hanna's homework.

[[1]]: She is careful not to call her views opposing or alternative, but complementary!