04｜5-in-5: Day2 - Language Game

5-in-5: Day 2

When thinking about non-verbal communication, I find it unavoidable to approach the topic from a linguistic perspective and examine how language operates. On the second day of my 5-in-5 challenge, I built upon Wittgenstein’s later concept of language games to explore how I could create an interactive installation that visualizes non-verbal communication through body movement tracking and 3D holographic projection.

‍

In Philosophical Investigations, Wittgenstein discusses the public and private aspects of language. He argues that a purely private language—one that belongs to a single individual and is entirely inaccessible to others—does not exist. Moreover, he asserts that language does not function based on a singular, absolute “essence.” Instead, its meaning and usage are rooted in people’s forms of life (Lebensform in German)—that is, the diverse ways people live, act, and interact. The plural form forms already indicates that human activities are varied, meaning that in everyday life, people engage in different modes of communication and interaction. The more diverse these activities become, the more varied and dynamic the corresponding language games will be.

For this 5-in-5 prototype test, I used a laser cutter to create a transparent, pyramid-shaped acrylic model to experiment with the 3D hologram effect. My goal is to use p5.js and ml5.js to track and display movements, turning the real-time motion data captured by the camera into a 3D hologram. By layering the body movements of different participants, I aim to highlight the shared nature of language—one that is continuously interpreted and reinterpreted, forming an ever-changing and organic model of communication. Building on this idea, I propose that if language itself is inherently plural—shaped by human activities, contexts, and interactions—then non-verbal communication should be even more fluid and multifaceted, resembling an assemblage of ever-changing expressions.

During my initial primary research, I observed that when people are restricted from using spoken language, their interactions largely rely on body language. This led me to consider using motion tracking to capture participants' body movements in real time when they communicate in a non-verbal setting. To enhance engagement, a nearby screen provides prompts such as, “How do you say ‘Have a good day’ without words?”, encouraging participants to communicate solely through body language. By layering the movement data of multiple participants, the collective body shapes are visualized as a floating 3D image inside a transparent acrylic pyramid, creating a shared visual experience.

‍

Through this approach, we can not only demonstrate the idea of a common, wordless form of communication but also bring to life Wittgenstein’s concept of public language—a collective “non-language” constructed through individual actions and interactions.

‍

‍