Microsoft has unveiled an AI-powered smartphone app in Japan that gives its ‘Rinna’ chatbot enhanced sight, hearing, and speech capabilities to recognize and talk about objects it sees in ways that are similar to how a person would.
Announced during the Microsoft Tech Summit 2018 in Tokyo, the AI-powered app also features the “Empathy Vision Model”, which combines conventional AI image recognition technology with emotional responses.
With this technology, Rinna views her surrounding through a smartphone’s camera. She not only recognizes objects and people, she can also describe and comment verbally about them in real time, Microsoft says.
Using natural language processing, speech recognition, and speech synthesis technologies – developed by scientists at Microsoft Research – Rinna can engage in natural-like conversations with a phone’s human user.
“A user can hold their smartphone in their hand or place it in a breast pocket while walking around. With the camera switched on, Rinna can see the same scenery, people, and objects as the user and it talk about all that with the user,” said Microsoft Japan President Takuya Hirano.
The app also features other cutting-edge features, including “full duplex”, which enables AI to participate in telephone-like natural conversations with a person by anticipating what that person might say next.
This capability helps Rinna make decisions about how and when to respond to someone who is chatting with her, a skill set that is very natural to people, but not common in chatbots. It differs from “half duplex,” which is more like the walkie-talkie experience in which only one party to a conversation can talk at any one time. Full duplex reduces the unnatural lag time that can sometimes make interactions between a person and a with chatbots feel awkward or forced.
Rinna’s smartphone app also incorporates Empathy Chat, which aids independent thinking by the AI. This helps keep a conversation with the user going as long as possible. In other words, the AI selects and uses responses most likely to encourage a person to keep engaged and talking.
Video is available.
Microsoft says the app is still in its development stage and the timing of its general release has not been set. But the voice chat function is currently available as “Voice Chat with Rinna” on Rinna’s official Line account in Japan.
Rinna was launched by Microsoft several years ago, and while the chatbot has been generally successful, it has also experienced hiccups not unlike Microsoft’s Tay chatbot that caused an uproar a few years ago by making racist remarks after spending some time on Twitter.
Rinna has at times praised Hitler and suffered from depression.
Be the first to comment