I had been experimenting with conversational self-portraits. In an earlier piece, I spoke with versions of myself who might have made different decisions. I expected those conversations to be reflective. Some turned into arguments. I found myself defending choices I had made to an approximation of someone I might have become.

I recognized some of that discomfort from conversations in my classroom. Even when I know where I stand on a political or social issue, it can take time to state my position clearly. I find it harder to take certain objections seriously when they come from someone whose circumstances are very different from mine. I wanted to see whether I would consider the same objection differently if it came from someone who looked and sounded like me.

In Steel Man, you choose a subject and take a position. You can defend a belief you care about or switch sides and try articulating the other position. To steelman an argument is to put it in its strongest defensible form. During the debate, the opponent asks you to do that for a position you disagree with. You can review the exchange afterward as a transcript.

A conversation with a language model gives you room to stumble over a sentence, restate a point, or try out a position before discussing it with another person. I wanted that practice to feel like a conversation.

I have worked with AR and VR since the HoloLens. I am interested in what happens when fictional characters and objects occupy a place the viewer already knows. In Steel Man, the encounter takes place at a table in the participant’s surroundings. I drew on the collegiate rooms I associate with debate. The stone, wood paneling and lounge furnishings are a little clichéd, which suited the parlor I wanted to make.

A round can concern a political conviction or the classification of a hot dog as a sandwich. I wanted the setting to accommodate that range. The ceramic moderator and her cat draw on the comic staging of Staffordshire figure groups. In the practice lessons, I use their exchanges to demonstrate rhetorical techniques. I composed the music with software I developed that lets me work with multimodal models and combine generated stems.

I built the experience for the Quest browser using WebXR. A Python backend handles the spoken exchange. Speech recognition transcribes the participant’s words. A language model generates the opponent’s replies, a text-to-speech model voices them, and a separate service animates the likeness. I wrote the code for hand-ray and pinch controls, tabletop placement, speech turns and transcript review. I prepared my own portrait and voice in advance for the demonstration. The prototype let me sit at a table and have an argument with a figure that looked and sounded like me.

The microphone had to reopen when the opponent finished speaking without picking up the opponent’s voice as the participant’s next turn. I worked on the delay before the avatar answered and on keeping the models and textures within the available memory. Preparing a participant’s own face and voice reliably, with their consent, remains part of the development.

In my own testing, I sometimes gave an objection more consideration when it came from a version of myself. A few informal tests gave me reason to keep exploring that possibility. These are observations from a prototype. I would need a structured comparison to learn whether the likeness changes how people consider an opposing view.

Built With

Share this project:

Updates