This is an interactive immersive installation design based on emotion visualization. It aims to develop immersive interactive space that can communicate our real-time emotional state. This will not only help us in becoming more self-aware but also reflect on our affective state. This is a slow experience of self-examination. Each viewer can see the particle landscape constructed by himself and his emotions at the same time. This depiction and reproduction of emotions keeps us away from our own subjective experience and encourages us to think rationally when narrating what happened to us. I create a kind of "imaginary reason" for our own feelings, aesthetic experience and moral practice in order to understand them. The audience will experience not only the emotion at the moment, but more importantly, through the interaction of facial expressions, to achieve psychological repair. This work was placed in the community of Wuhan, hoping to repair people who were psychologically affected by the epidemic, and to repair the city.

Emo is a new form of full-sensory human–computer interaction in an immersive space. In this space, abstract and subtle emotions can be transformed and expressed in the form of vision and experience through data visualization. Emo is a response to the emergent social crisis of COVID-19 that has holistically transformed human life. The installation attempts to represent human emotion and it promises to facilitate social connections while simultaneously differing us from others and the outside world. Emo is a social experiment that explores how immersive technology and facial recognition can help connect people through their common attributes – emotion. As participants enter the installation, they are tracked by a webcam, which transforms their facial emotion into a water flow of particles on the projection screens, forming a three-dimensional representation of their inner world. The aim is to demonstrate both the importance of this embodied aspect of the social experience and the capacity of immersive space to activate bodies and alter social norms to facilitate connection. There are two unique features of this design. First, it is a multi-sensory interactive system with a focus on emotion detection and feedback. The sensing system enables the environment to be responsive to the participant’s reactions, creating dialogues between humans and space, and extending the human’s inner world to a wider medium. Second, this system provides a space where the participants can become immersed in a world constructed by his or her own feelings, thus creating opportunities for meditation, reflection, and psychological intervention. Through the implementation and observation of this design, it is observed that media plays an important role in extending the human sensation. The raised awareness of emotion demonstrated that the context of the installation transcended the boundaries of human bodies, extending the emotion to a tangible scale. “Imaginary rationality” is brought to the participants through objectifying subjective experience in this interactive process. Emo also explores the potential of inviting objective recognition of emotion in both personal and interpersonal perspectives by exceeding human boundaries with an integrated aesthetic. The abstract particle landscape was intended to give participants a coordinated aesthetic to support a feeling of united personality. . This integrated aesthetic broke the boundaries of personal subjective recognition and served as a bridge for participants’ understanding. Especially in the pandemic period, participants were spiritually connected through mutual experience and empathy construction in the particle landscape. Through this design and implementation, the functions of mediation and psychological intervention in public art were also explored. It has proposed a promising way to construct a public art with social awareness and responsibility.

https://www.youtube.com/watch?v=WHAZjurogYs

https://mp.weixin.qq.com/s/53IpFeokyRWvcVJE_5Fj7A

Details

Team members : Xiying Bao

Supervisor : Prof. Zhao Huasen

Institution : Tongji University

Partners : Tongji University

Descriptions

Technical Concept : Facial expression recognition refers to the extraction of a specific expression state from a given static image or dynamic video sequence to determine the psychological emotions of the target person. The interaction in this work was realized with processing implemented using the OpenCV library. The Dlib library was used to implement the facial landmarks. The pre-trained facial landmark detector provided by the Dlib library is used to estimate the location of 68 (x, y) coordinates that map to facial structures on the face. In Emo, the steps to use OpenCV in processing to obtain face landmarks are as follows: (1) Load the face detector; (2) Create the facemark class; (4) Detect the face; (5) Run the facial landmark detector; (6) Collect the facial landmark data; (7) Map the landmark data to the number, velocity, and RBG color of the particles used in the visualization.

Visual Concept : The visualization of emotion is supposed to be a trial to make visible a process that is in large part invisible and only partially perceived by the one experiencing it. It must reflect the narrative quality of our personal history as a visible metaphor. To do that, emotion visualization must be engaging. The work done for this project through emotional experience helps us establish our identity. . “Flow,” a continuously changing state embodied by natural mountains and rivers, is the ultimate essence of nature and is the key visual language utilized in the visualization of this work. The measurement of emotion in this work refers to the tri-dimensional theory of feeling of Wilhelm Wundt. Control corresponds to frequency. Valence corresponds to particle color. Arousal corresponds to the velocity of the particles. Intensity corresponds to the number of particles.

Credits

Xiying Bao

Xiying Bao

Xiying Bao

Xiying Bao

Xiying Bao

Xiying Bao

Xiying Bao