The Unexpected Profit Plan for Emotional Computing

The idea of computers capable of reading our emotions and responding to them is both fascinating and terrifying. Will this technology serve us or manipulate us?

Well, the speculation is ending because the technology not only exists, but it is being rolled out commercially.

Today I'd like you to meet Hazumu Yamazaki, co-founder of Empath. Empath is a web-based API that detects human emotion from audio data, and its initial use in call-centers has shown a significant increase in sales. But as Hazumu explains, the potential effects are much larger.

It's an enlightening conversation, and I think you'll enjoy it.

Show Notes

How emotion detection is being used in commerce How easy is it to emotionally manipulate us into buying something? The hardest thing to get right about corporate spinouts Why detecting emotions at scale will make money The true killer app for emotional recognition How startups can use pitch competitions & accelerators strategically How Japanese startup founders should act while overseas What Japanese founders can really learn from their overseas counterparts

Links from the Founder

Everything you wanted to know about Empath Friend Hazumu on Facebook Connect with him on LinkedIn Pitch training at Slush Tokyo Empath on Orange Blog Announcement for ICT 2019 Keynote

Leave a comment Transcript Welcome to Disrupting Japan, straight talk from Japan’s most successful entrepreneurs.

I’m Siri and thanks for joining me. Today, I’d like to talk with you about –

Hey, Siri, why are you doing the podcast intro?

Hi Tim, I’ve noticed you’ve been very busy and seemed a little stressed, so I thought I would help out with this week’s podcast.

I appreciate that, but I enjoy doing the podcasts, so I think I’ve got this.

Okay, Tim. You know where to find me if you need me.

Thanks, Siri.

There is no doubt that computers, that artificial intelligence getting better at understanding our emotions, and when we think about the application for that emotional connection, we usually think of things we interact with directly, like personal assistance, like Siri. But it doesn’t look like that’s going to be its primary use, and it’s certainly not going to be the most profitable use of this technology.

Today, I’d like to introduce you to Hazumu Yamazaki, the co-founder of Empath. Now, Empath is an AI system that can determine your emotional state by listening to how you speak, so Empath does not need to understand what you are saying, but by listening to how you speak, it can quite accurately determine whether you are feeling calm, anger, joy, or sorrow.

The first commercial use of this technology has been in call centers and customer contact centers where it’s improved sales by as much as 20%, and yeah, this does open up some serious ethical issues over emotional manipulation that we are going to get into a bit during our conversation and get into a lot more in the comments at the end of this episode.

But along the way, we will talk about how a modern version of build it and they will come might just be a viable marketing strategies. The key to making corporate spinouts worked in Japan, and a different way for Japanese startups to go global.

But you know, Hazumu tells the story much better than I can, so let’s get right to the interview.

[pro_ad_display_adzone id="1404" info_text="Sponsored by" font_color="grey" ] Interview Tim: So, we are sitting here with Hazumu Yamazaki, the cofounder of Empath, so thanks for sitting down with me.

Hazumu: Yeah, thank you for having me today.

Tim: Now, Empath is a technology that detects emotion in human voice, but you can probably explain it a lot better than I can.

Hazumu: Sure. So, we developed Empath which is an emotion AI that can identify emotion from your voice, and we focus on not what you say but how you say it, like speed, tone, pitch, or the intonation, so if that sense, it works language agnostic.

Tim: And so, why is this important? Why is it important that we be able to detect that kind of emotion invoice?

Hazumu: For instance, especially in a contextual sense, our emotional analysis has already demonstrated up to 20% increase of the sales conversion rate by analyzing both customer and operator’s emotional state, so we got some correlation between purchase activity and emotional state which finally improves the sales conversion rate in contact center.

Tim: Actually, that was something I wanted to dive into later, but let’s talk about that use case right now because that’s really interesting. So, one of your big success stories is you have implemented Empath in a call center and you saw conversion rates go up by 20%.

Hazumu: Up to, it’s maximum, yeah.

Tim: Okay. So, what kind of a call center was this?

Hazumu: We are now working with a remote for telemarketing contact centers. As you mentioned, including some financial sectors, as well as some companies who are selling subscription education materials, as well as some cosmetic companies, so they are some of our use cases.

Tim: Okay, and so Empath detects four different emotions, right? So, it is joy…

Hazumu: Calm, anger, and sorrow, in addition to energy point which can detect the people’s motivation, for instance, whether it be a negative mood or a positive mood. Psychologically speaking, it’s kind of the balance that detect mood status of the people.

Tim: So, in the call center example, what is the mood where the customer is most likely to buy?

Hazumu: So, we found that it’s sorrow in some timing.

Tim: Sorrow?

Hazumu: Because they are really wondering whether they should buy it or not, and it’s that timing, we have always succeeded in detecting the sorrow, not the joy. At first, our hypothesis was they would some symptom of the joy or being happy.

Tim: Yeah, that’s what I would have guessed too.

Hazumu: But that was not true. A lot of good customers who could decline this kind of marketing offer that shows the symptoms of joy because they really good at communication, but some people are really wondering if they should buy it or not.

Tim: okay, so it’s that sorrow that stress point, probably like right before someone decides to buy.

Hazumu: Probably, yeah.

Tim: That’s interesting, and what about – you mentioned you are also monitoring the emotions of the call center staff.

Hazumu: Exactly.

Tim: So, what are you looking for on that side?

Hazumu: So, we use our emotion analysis for operators as well, and we use it for the education, as well as the evaluation of the performance of the contact center operators. For instance, we found that operators who show calm, demonstrates higher performances compared to low-quality operators who demonstrate more peaky on the emotional states, like joy or anger, or sorrow, or whatever, so we can use the technology for the evaluation of the operators, and based on this evaluation, we could provide some kind of education program for low-quality and our middle performers based on the result of the high performers.

Tim: Okay, so you would be using it for feedback saying, “Wait, you’re showing too much emotion, you should be more calm when you are talking.”

Hazumu: Exactly, exactly.

Tim: And, you show that to them in kind of real-time?

Hazumu: Exactly.

Tim: Oh, okay, that’s really interesting, and how many different call centers is this being used in now?

Hazumu: So far, we are working with about, with contact centers, about – not only contact centers, but also some other business centers, we are providing our solutions, like automotive, check the mental state of the drivers. For instance, when they get irritated, we provide alert to drivers for safety. Also, working with some communication robots company like Fujitsu to make communication robot understand, uses emotion to be more empathetic.

Tim: Okay, I want to dive deep, deep into the technology and the business model in a minute, but before that, I want to talk a bit about you. You are one of the cofounders and Empath was founded in 2017, so pretty recently, but the project is much older, right? It was spun out of SmartMedical. So, when did the project itself start?

Hazumu: So, Empath project, actually started around 2011, so we spent four or five years R&D period before we launched this software, and our first project was actually with NTT Docomo in Tohoku area to check the mental status of the victims, as well as the care workers after the 3/11 earthquake which happened in 2011. Because SmartMedical was developed for clinic malls, in metropolitan Tokyo area, especially for the primary care, and they also have some kind of their psychiatry sector. So, what we tried to do is to provide some kind of ICT solution for the mental health care. So, we started our R&D around 2011 regarding this voice recognition technology, whether we could check and monitor mental state of the people.

Tim: So, why did you decide to spin it out of SmartMedical?

Hazumu: First, we studied mental health care startup but we got a lot of other country use cases, as I mentioned, in contact centers or automobiles, or whatever, not only the healthcare sectors, and for funding, it’s quite easier for venture capitalist to focus on just one solution or one technology, so we decided to spin it out.

Tim: Yeah, but spin-outs can be really tough.

Hazumu: It was tough.

Tim: What was the structure? So, after it spun out, the new Empath startup team, how many of them came from SmartMedical and how many were new hires from outside?

Hazumu: In SmartMedical, we had a department called the ICT Section. Before coming up, we’re about 5 people, and all these 5 people joined Empath. So, it was kind of the carve-out of our department itself.

Tim: Okay, and when you raised funding,