Computer Science, Off Course! Episode 2 - Computers as Theatre

Listen to our second episode now!

What a fun a weird book this is! The core idea of the book is explained immediately, it is about:

the notion that when people were using computers they were interacting in representational worlds, much more like plays in which they were characters than computers on which they ran programs. What a funny idea—at once both obvious and strange.

I feel the same, this idea is obvious and strange, but it is mostly something we never really consider in Computer Science. We assume, in the phrasing of user stories for example, that the users wants something that can be obtained from the computer. The users wants to upload a receipt. No, this books explains, the user does not want this, he has to play the role of a person that does.

I find the comparison so fun and interesting because it sheds new light on what making software is about:

Both theatrical design and interface design are aimed at creating representations of worlds that are like reality, only different.

Laurel also does a great job at analyzing different types of software through this lens, both games—in which of course the role playing is very clear—and office applications. But both are, for sure, representing worlds slightly different from our own.

Here a comparison to Naur is relevant, because he was talking about how programming was trying to represent the world in code. Where he focuses on the similarities between the real world and the digital one, Laurel focuses on where they are not the same. Laurel also cites another author here:

In his book The Elements of Friendly Software Design (1982), Paul Heckel remarks, “When I design a product, I think of my program as giving a performance for its user.”

What a lovely way to think of software!!

What is a user interface?(The quote for Hanna)

This quote made me think a lot about what software is!

Consider the following question: What is being represented by a human-computer interface?

1. A way for a person to communicate with a computer.
2. A way for a computer to communicate with a person.
3. A surface through which humans and computers can communicate.
4. A way for humans and computers to construct actions together

What a fresh new way to think!! It is, of course also applicable to AI. What are you constructing, for example, when you are chatting with ChatGPT? Are you constructing that conversation, or something outside of it, like a program? Or are you constructing knowledge?

When you (like me) only learn a little bit about UI design (I got 1 course on it) you never really consider what it is for. It really got me thinking about Hedy, I had never thought about it in this way. Is Hedy communicating something to you, or are you communicating something to it?

There is another related quote, that for me also struck a cord, because it fits nicely with Peter Naur of last week's episode <>

Art is the external representation of things that happen in the head of the artist. Art forms differ in terms of the materials they employ, the way the representations are created, what they purport to represent, and how they are manifested in the world. Different forms have different powers—to engage, to provide pleasure and information, to evoke response.

This also holds for UI elements, and made me think of Tinder. They could have simply chosen a button for a match, but a swipe is very different action, it represents a different feeling.

Screenplays as programs

In addition to the idea that you play a role when using software, and that software play a role, there is one more core idea on the book. And that idea is that a screenplay is like a program, in that it still needs to be executed to do something:

A program is a set of instructions that defines the potential actions that
make up a human-computer activity and their representations. These actions and representations may change as the result of ongoing action (for instance, as the result of capturing or inferring people’s preferences). A program also defines the environment for action and the other objects that inhabit that environment, including their representations and capabilities

This is such a shift in register from how we normally think of running a program. What I think of as running a program is simply the execution of it by the computer. And there the metaphor does not work, as there are no potential actions. The code is just transformed into bytecode. But her scope is wider. She sees the execution of the program as including all actions, all potential actions, that a user might do in/on/with it!

What I think I realized through this, is how much of thinking about programming and computing is still tied to the very very old notion of writing code to do a calculation, on the command line. Execution might take some input, and prodce output but it does not encompass it.

What a gift, I also thought, this theatre person is giving me in terms of thinking about computers and programming! (or: what a loss that I did not get it before, that too).

The text [of a play] therefore, is a combination of explicit and implicit notational systems which have as their initial purpose the enablement of an event in which performers and audience can share as partners. (Hilton 1991) [[1]]

Well, what about AI?

She talks about it directly:

Can computers think? There is an easy way out of it; computer-based agents, like dramatic characters, don’t have to think, they simply have to provide a representation from which thought may be inferred.

Note to self, and to fresh readers…! When we release our episode on Sherry Turkle, we can link that one here.

This simplistic notion of conversation led many early interface specialists to develop a model of interaction that treats human and computer as two distinct parties whose “conversation” is mediated by the screen. [...] Dialogue is not just linearized turn-taking in which I say something, you go think about it, then you say something, I go think about it, and so on.

This is however, and we of course can't hold this against Laurel, exactly how we do conversations with AI now!

People working together:

And to coordinate on process, they need to update, or revise, their common ground moment by moment. All collective actions are built on common ground and its accumulation.

Here I wrote down that with current LLMs, only one party does this: the person. A bit like Ted Chiang described in his beautiful essay:

The computer-science professor Murray Shanahan suggests that we think of this as role-play; the data scientist Colin Fraser describes it as a person “collaboratively authoring a document with an LLM.” Some users might not understand that they are role-playing or co-authoring a document, and others who do understand nonetheless forget, because of how engrossing the interaction is. Either way, the companies selling LLMs typically encourage this misunderstanding.

Homework exercise

1. For two software systems that you use (or have made, even beter), what is the performance that it is giving?
2. When using a piece of software, write down when you are updating or revising the common ground between you and the software?

Hanna’s Homework The Computer as a Gym

Dear Felienne,

I had promised you a shorter homework for this week, and I think I did it: it's a tiny little bit shorter (based on my own estimate). But also, it isn't really exactly the homework you suggested. I did something else instead. I guess I went off course... but in some way, I hope that's the point. So what did I do?

What the Stair Climber, Slack, Ellipticals, and Emails Have in Common

As a first step, I reflected on how I see the computer. In our episode, we discussed that Brenda Laurel sees it as a theatre. I see it more as a gym. That's because I find working with the computer super exhausting. But there are a few more parallels I want to draw. For instance, think about all these emails that come in and that need to be answered. This is really like the stair climber in the fitness centre, where the stairs just keep coming and coming... an endless flow of stairs, similar to the endless flow of emails. There is no destination we will reach, but they keep us feeling busy and productive.

Then there's Microsoft Teams, where we have all these online group meetings during which everyone is dying on the inside (at least I assume everyone else feels this way, too?). To me, that's just like a spinning class. So again, you spend all this energy and effort, but in the end, you really don’t get anywhere. You're still in the same room, even though on the big screen everyone stares at, it seems like you’re moving forward. And this is, of course, what these online group meetings often do. You spend a lot of energy and get nowhere, but on the shared screen, it still seems as if you’re covering some distance or solving real problems.

There are also applications like Slack and Metamost that are meant for communication and collaboration. I love using those tools and the nice dopamine kicks that a thumbs up on these platforms provides. Yet, I feel like hanging out on these platforms is a bit like using an elliptical in the gym while you're on the phone with your best friend or reading a magazine. If you were honest with yourself, you’d realise that you're just pretending to work out. That’s procrastination at its best: it leaves you guilt-free, or even feeling proud. Computers are just so great for that!

And finally, computers always have these processes running in the background. And this is really like the music and the gym that's just always there and running in the background. And sometimes you notice it, but there's not really much you can do about it... But somehow it is still interwoven with what you do and how you move. I am sure there are plenty of computer processes running in the background that affect me without me being aware of this.

The Missing Option: Interfaces for Human-Human Interaction

I also thought about the multiple-choice question you asked me during our podcast episode. To remind our readers, you asked me, “What is being represented by a human-computer interface?” And you gave me those four options from Brenda Laurel's book:

1. A way for a person to communicate with a computer.
2. A way for a computer to communicate with a person.
3. A surface through which humans and computers can communicate.
4. A way for humans and computers to construct actions together.

I really hate multiple-choice questions. I always think of something that is not captured by the provided options. And when I think about my own relation to computers, I really miss a fifth option here, too. That's because I use the computer mostly as an interface to communicate with other humans. Answering emails in Thunderbird, writing collaborative scientific papers on Overleaf, playing a game against another person, putting podcasts online, reading the documents my students send me, or posting on LinkedIn: it’s all a way to connect with others, a way to co-create materials, a way to exchange human attention and ideas. In such cases, I believe the computer fades into the background!

So, to answer the question: a human-computer interface is also a surface through which humans communicate with other humans. It's a means for humans and other humans, remote humans, to construct actions together. While it is not one of Laurel's answers, it still somewhat fits her theatre theme, where actors interact with each other on stage. I don't think this contradicts her work at all, but it places more emphasis on the various humans involved.

Slaying Metaphorical Dragons

In our episode, we have touched upon Laurel's idea of a shared performance between a user, or as Brenda Laurel would say, an interactor and a computer. This resonated with me because it is something I can relate to, for instance, when using my to-do app. Whenever I add a to-do to my to-do app, I feel like there’s a problem to solve. And then I get to be the hero who solves that problem with the computer. Typically, I need to open all kinds of computer programs, and with those applications, I have to slay many, many metaphorical dragons. But then, when I'm done, I can put this little checkmark next to the task, and there’s a sense of relief, and sometimes I'm even a bit proud. It comes with a feeling of closure that I also really get from good stories. So, in that sense, I think my to-do app is great. Completing tasks with a computer has this whole drama and performance built in. Just now, for instance, I used my favourite markdown editor to complete this homework assignment, and I am already looking forward to marking the “podcast homework” task with a little checkmark in my to-do app. It will feel great :)

Cheers,
Hanna

Footnotes

[[1]]: She cites 'Hilton 1991' in the running text of the book, but it is this paper from 1993 <> hallucinations <>