Culture / Machine Learning

Microsoft Pizza Bots and the New Natural User Interface

2 Apr 2016 1:55pm, by

“Those of you that selected ‘bot’ as your drinking game keyword, RIP,” joked Microsoft engineer Jacob Rossi on Twitter, referring to those watching the opening talks at his employer’s annual Build developer’s conference, held this week in San Francisco.

During his talk, Microsoft CEO Satya Nadella called bots “the beginning of this new platform journey,” and he talked about the ability to use human speech to interact with increasingly sophisticated AI that can recognize and respond to spoken commands.

“We call it ‘conversations as a platform,'” Nadella told the audience of developers in San Francisco. “It’s a simple concept, yet it’s very powerful in its impact.”

“[W]e think this can have as profound an impact as the previous platform shifts have had, whether it be GUI, the Web, or touch on mobile.”

We cued up the video for you:

These days, Microsoft’s talking a lot about the NUI — not a graphical user interface, but a natural user interface, a decades-old concept coming to fruition thanks to better voice-recognition technology.

Nadella described Microsoft’s new vision as “enabling us to interact with devices more intuitively … evolving us from mechanical keyboard and mouse to touch and beyond, taking advantage of more human capabilities like our voice, pens, our face, gestures, fingerprints.”

Windows 10 is our home for this evolution,” the Microsoft CEO said.

Besides voice, Microsoft is also touting the new Windows Ink feature which will be available in the next version of Windows. Using your finger or a digital pen, users can write notes or select menu choices, and there’ll reportedly even be a whole new section in the Windows Store for pen-enabled apps. And Microsoft promised improvements to Cortana, its voice-enabled personal assistant, as part of the Windows 10 Anniversary Update expected this summer.

“This significant update will help you interact with your Windows 10 devices as naturally as you interact with the world around you — using your pen, presence and voice,” said Terry Myerson, executive VP of Microsoft’s Windows and Devices group, during another keynote.

As a preview, Microsoft unveiled Skype Bots, a way to connect Skype users to helpful information services using Cortana:

The current version only responds to text messages, but Microsoft anticipates that in the future the service will also be voice-enabled — and have many more capabilities. And developers are invited to help expand its capabilities, through a new Skype Developer program

Meet the New Apps

“Bots are the new apps,” read the headline in the International Business Times, citing another Microsoft demonstration in which a user visited the Web site for Domino’s pizza, and typed “send me a large pepperoni pizza” into a chat window.

“Microsoft believes the world will soon move away from apps — where Apple and Google rule — into a phase dominated by chats with bots,” added Bloomberg Businessweek. But the reporter suggested sardonically that the end result may fall somewhere between euphoria and dystopia.

“After spending some time with Tay, Microsoft’s new chatbot software, it was easy to see a third possibility: The AI future may simply be incredibly annoying,” Dina Bass wrote in the article, titled “Clippy’s Back: The Future of Microsoft Is Chatbots.”

But maybe years of artificial intelligence research really will finally produce a killer app. To illustrate this vision for the future, Nadella shared the story of Saqib Shaikh, a blind Microsoft engineer who’s now part of a new project to deliver automated descriptions for the blind using Pivothead SMART glasses and a Windows 10 app:

Studying footage from the headset, the app was able to identify and describe facial expressions — like “surprised” and “happy” — as well as the activities of nearby people. Microsoft’s video showed the app preparing to read Shaikh the menu at a restaurant, even providing audio instructions on how to position his camera to photograph it.

“Years ago, this was science fiction. I never thought it would be something you could actually do,” Shaikh said in the video. “But artificial intelligence is improving at an ever-faster rate, and I’m really excited to see where we can take this.”


WebReduce

Feature Image: Movie ticket-dispensing robot (since retired), Alamo Drafthouse Cinema, Austin Texas.

A newsletter digest of the week’s most important stories & analyses.

View / Add Comments

Please stay on topic and be respectful of others. Review our Terms of Use.