Mapping the sound of a memory into a sculpture - a malleable device which can store audio clips based on its shape.

Work is ongoing. Intent to submit paper to CHI 2020.


August 2018 - Present

Team members:

Sebastian Schloesser

Siyu Hou

Chris Lucyzwek

My role:

Designer, programmer, prototyper

Tools used:

-Fabrication tools such as laser cutter, wood shop tools



-Adobe Suite


Created for INFO 262: Tangible User Interfaces



During desk research, we found previous work which showed nostalgia generates positive affect, increases self-esteem, fosters social connectedness, and alleviates existential threat. 

But how much information is needed to evoke the process of nostalgia? Artists, sculptors, songwriters know that emphasizing just the most important parts of a memory is sometimes more impacting than trying to create a perfect recreation of that event. In other words, a painting that captures the bright, emotional essence of a flower is more evocative than a high-definition photo of that same flower (see painting of flowers below). Artists highlight the concept, not the exact image.

From artist Helen Dealtry


Our group was particularly struck by the haptic act of sculpting, of storing pieces of a memory into an abstracted, tangible form.

We propose a haptic memory-recall device to help trigger a stronger sense of nostalgia through physically embedding memories into a dynamic, changing sculpture that replays audio from past events based on its shape




Focus for prototype 1: Getting sensors working (Arudino + Processing), test silicone as a "sculpt-able" material for the interface since it somewhat approximates the feeling of clay

Gained experiences with sensors                      Confirmed they worked as intended



Need to reconsider form and how it acts as an interaction interface.



Need to figure out how to embed flex sensors in silicone.



Device is not able to retain shape


Focus for prototype 2: Getting sensors embedded into silicone

Made progress on cohesive code.                          Successfully had sensors working inside silicone.



Gained experience in communicating over Serial to interface between Processing and Arduino.


Due to embedded lead strand, the prototype could now somewhat retain its shape to achieve a "sculptural" effect... but it wasn't working flawlessly

Because of it's elongated form and silicone materiality, users who informally tested the device would be very aggressive and twist the device, breaking the flex sensors and LEDs encased in the silicone. That's a major problem - how do we restrict the axis of movement in a direction that the sensors can work with? Time to reconsider the choice of silicone as a material...


Remaining two major challenges.

1) How do we make a device that better retains its shape?

2) How do we restrict the user's movement so that they can't twist in all directions, risking breaking the delicate sensors?

The Solution

Laser-cut wood arms

Friction fit dowel allows wood arms to hold position around dowel

By creating a mechanical hinge, we limit movement to around only one axis

Now we can approximate shapes into a sculptural device...

House shape

River/wave shape

Waterfall shape

Flex Sensors

Arduino + Processing handles audio

LED Strip

USE-CASE SCENARIO - Recording sounds from frog

"This is incredibly poetic."

Copyright © 2019, Julia Park