Everyone has a different sense of humor. What is funny varies depending on the person. That being said, it seems impossible to teach a robot, which should cater to all, how to laugh.
However, researchers at Kyoto University in Japan are trying to do just that through the creation of a funny bone in Erica, the Japanese android. The researchers would like to improve conversations between people and robots by creating an artificial intelligence system with an understanding of the human nuances of humor. The research is published in the journal Frontiers in Robotics and AI.
“Spoken dialogue systems must be able to express empathy to achieve natural interaction with human users,” wrote Dr. Koji Inoue, lead author of the study and assistant professor in the Department of Intelligence Science and Technology within the Graduate School of Informatics at Kyoto University.
“However, laughter generation requires a high level of dialogue understanding. Thus, implementing laughter in existing systems, such as in conversational robots, has been challenging.”
"Laughter generation requires a high level of dialogue understanding. Thus, implementing laughter in existing systems, such as in conversational robots, has been challenging."
Dr. Koji Inoue
Despite the challenge, the researchers hoped to design artificial intelligence that will take its cues from a shared laughter system.
“We think that one of the important functions of conversational AI is empathy,” explained Inoue. “Conversation is, of course, multimodal, not just responding correctly. So we decided that one way a robot can empathize with users is to share their laughter, which you cannot do with a text-based chatbot.”
How did humans share their laughter with the robots?
The shared-laughter model is designed by creating three subsystems. The robot must detect laughter, decide whether to laugh or not and choose an appropriate laugh. In this way, once a human laugh the artificial intelligence system can give an appropriate and empathetic response.
Erica was put through speed dating.
In order to collect data and analyze shared laughter, the researchers collected shared laughter samples using speed dating dialogue. The speed dating contained dialogue between Erica, who was controlled by one of four actresses speaking into a microphone in a separate room, and the subject. Erica's non-verbal behaviors were controlled by the operator as well. A total of 82 sessions were conducted, and each lasted between 10 and 15 minutes.
“The dialogue task was speed dating, where the purpose of the dialogue is to get to know the other participant. Therefore, participants try to make themselves friendly, and we expect to observe many laughs,” writes Inoue.
How was the data analyzed?
“Our biggest challenge in this work was identifying the actual cases of shared laughter, which isn’t easy because as you know, most laughter is actually not shared at all,” Inoue said. “We had to carefully categorize exactly which laughs we could use for our analysis and not just assume that any laugh can be responded to.”
The laughter samples were labeled as “vocal laughs,” which means that the users showed more than just a smile. This included “solo laughs,” which is a simple “ha ha,” and “speech laughs,” laughs while the user is speaking.
“We only include samples uttered on the user’s turn because we want the system’s laughter to be a response to user dialogue,” writes Inoue.
Then, the researchers used the “vocal laughs” to analyze where shared laughter occurred. They were annotated as shared laughter if Erica responded to the user’s laugh in a timely manner. The researchers referred to existing studies on classifying laughter types and decided to use two simple classifications for Erica’s response laughs: mirthful and social.
Inoue writes, “Mirthful laughs are likely to be elicited by positive moods and expressed toward the dialogue itself, whereas social laughs tend to be used to augment and “fill” the conversation although humor is not involved.”
How did they test it?
All factors considered, the team tested the shared laughter system, as well as tested two baseline models for comparison. Erica never laughs in the first model and in the second, Erica utters a social laugh whenever she detects a human laugh without filtering proper context or response.
These models were tested through four short dialogues between a person and Erica. The researchers hoped to test Erica’s new sense of humor through these two to three-minute conversations. Each scenario testing the shared-laughter model proved to be different. During the first exchange, Erica only used social laughter. In the second and third, she only uttered mirthful laughs. Lastly, in the fourth, she used both types of laughter combined.
The team got over 130 people to listen to each scenario within the three models. The people listening were to evaluate the interactions based on how natural, empathetic, and understandable the conversations were. Based on the evaluations, the shared-laughter system proved to work the best.
“The most significant result of this paper is that we have shown how we can combine all three of these tasks into one robot,” said Inoue. “We believe that this type of combined system is necessary for proper laughing behavior, not simply just detecting a laugh and responding to it.”
“We believe that this type of combined system is necessary for proper laughing behavior, not simply just detecting a laugh and responding to it.”
Dr. Koji Inoue
There are still many more laughing styles Erica could learn. The team has not yet attempted to model unshared laughter which is one of the most common.
Even so, laughter is only one type of natural human interaction.
“Robots should actually have a distinct character, and we think that they can show this through their conversational behaviors, such as laughing, eye gaze, gestures and speaking style,” Inoue says. “We do not think this is an easy problem at all, and it may well take more than 10 to 20 years before we can finally have a casual chat with a robot like we would with a friend.”