Speculative Design: Face

Our mission is to empower people to conquer the stigmas around sharing and outwardly expressing certain emotions. We are innovating the ways to equip people with resources to communicate effectively by forecasting the potential of future interaction designs and holistic experiences at the systems level.

This project began with an investigation into the facial expressions recorded in college senior photos. The data sample was taken from three University of Arkansas Razorback yearbooks—each forty years apart—from 1930, 1970, and 2010.

However, we realized that by looking exclusively at male photos, our data was incomplete and not showing the whole story. We cannot understand male emotional patterns completely without understanding the patterns in female emotional expression as well.

This realization sent our research in a new direction. Once we gathered the same data for women’s photos, we shifted the focus of our project from exclusively look at displays of masculinity to a broader inspection of how people struggle with emotional expression at a larger scale.

After our research led us to an interest in emotional expression, we began to extract the core problems related to this issue. We also identified our two potential target audiences—men and women—since they face these issues in different ways. We then brainstormed methods of bot intervention and how each idea related to the problems and target audiences.

Men and women are both affected by many common social influences, but male emotion is more restricted in interactions that are intimate while females struggle more in the work environment.

“Mirror” → Personal workout assistant or buddy which responds to user in real-time.
Amazon Alexa → Amazon’s virtual assistant that can verbally respond to users or control other devices.
Akinator → Online game that utilizes machine learning to play a 20-question style game with users.

Once we understood our users, the problems that they face in emotional communication, the stakeholders that influence them, and the potential ways that a bot could intervene in a user’s life, we began to brainstorm ways that a bot or personal assistant could help our target users.

Concept 1 → Mood Bot / Mood-Tracking App
This mobile portal and app add-on aids users in tracking mood & energy levels to better understand their changes in emotion over time and on a daily basis.

Concept 2 → Audacia / In-Ear Personal Assistant
This personal assistant earpiece technology speaks to users through mock conversations so that users can practice being assertive or emotional before being face-to-face with another human being. She keeps a database of conversations in order to learn how to respond to users in different types of situations, coaching them to express themselves better.

Concept 3 → MoodMirror / Pocket Mirror with Embedded Virtual Assistant
The MoodMirror makes you aware of your everyday facial expressions. When you flip the compact mirror open, it will be able to identify which of the 7 universal emotions you are showing. The compact will aid you through analysis of your expression to track how often you are experiencing each one and give you advice to improve unpleasant facial expression habits.

Using chatbot technology we were able to explore a specific way in which a user might interact with our proposed personal assistant. By doing this we were also able to get an idea of how machine learning and AI would work through our concept of Face.

This personal bot is named Face due to its function of helping users “face” difficult emotional situations while simultaneously analyzing the user’s face. When a user is talking to Face, they are externally looking at their own face, as well as, internally reflecting on themselves and how they “face” the world.

This personal assistant helps users achieve their desired outcome when going into an emotionally charged interaction. The assistant begins by prompting the user to explain and rationalize their communication goals for a specific scenario, responding specifically to previously-collected data regarding the user’s communication style and facial expressions. Then, the assistant facilitates mock conversations so that users can use this advice to practice the conversation before the actual interaction. Finally, Face provides feedback based on the user’s ability to successfully communicate their emotions verbally and through facial expressions.

We created a mock chatbot on the website chatfuel to put our scenario and script to the test. We were able to create a fully functioning prototype that users were able to try out for themselves. During this process we learned there was a significant difference between a chatbot and a virtual assistant. Because of this, it was necessary to change our script many times to fit the prototype of a chatbot instead of a virtual assistant.

Before a user entered into the chatbot interaction…they were prompted with a brief introduction:

“Hello! Face is a mirror-embedded personal assistant which analyzes your emotions through your vocal tone, inflection, and facial expression in order to provide guidance in emotionally vulnerable interactions.

You are taking on the role of an employee who lacks confidence when interacting with their work superiors. You will be asking your boss for a pay increase. You’ve been working with your boss for about a year but you aren’t very close so you still feel awkward making conversation with them. Face will prompt you to have a practice conversation and analyze your voice and facial expression to give you feedback.”

“Hey Nicole, what are we facing today?”

“I’m asking for a raise”

“Who will you be talking to?”

”My boss”

“What is your relationship with them like?”

“Not very close”

“Do you have trouble being assertive when you talk to them?”

“Yes”

“Okay—let’s practice how your conversation might go.
I’ll start the conversation when you say “ready”.

“Ready”

Below is a small sample of the feedback we received from our user testing. While the chatbot format does not directly translate into a mirror-embedded personal assistant, this allowed us to mock up the ways in which a potential user might be inclined to interact with Face.

Describe your experience with the bot. What happened?

→ “It allowed me as the user to practice what I hypothetically would’ve said before I tried to say it.”

Describe the bot’s personality.

→ “I liked how placid the bot was, it had a soothing voice and was super easy to interact with.”

What was effective? Was there a “wow” moment?

→ “Really neat speculative-ness with voice input—I think if you guys had an opportunity to flesh this out in the interface you wanted, you’d be able to make it work really well.”
→ “I really enjoyed the bot speaking and the fact that it’d playback the user’s voice—it really helped make the user realize what they were saying (more effective).”
→ “The voice messaging from the bot was great, it helped the bot feel more personal and would make verbal practice more effective.”

Face is meant to be impersonal yet empowering. This bot is designed to appeal to those who struggle with difficult conversations and particularly vulnerable interactions in their everyday lives. The overall tone of the bot is meant to be direct, formal, and unemotional. This is intentional to alleviate any excess stress for the user before having to practice difficult situations.