Every year, Microsoft holds a developer event called "Build." And recently, those events have gone from snoozers to exciting showcases. Microsoft has a winner with Windows 10 (as long as you ignore the phones), a robust personal assistant in Cortana (that works just fine on a laptop), and a wild holographic future to plan with HoloLens. It's a lot to take in, and at this year's Build Microsoft we got updates on all of it. And a few surprises.
Going in, on one was totally sure what would be coming next for Windows 10, but it turns out there’s a lot that Microsoft has planned. It’s not just that there are new apps, there are also new bots, which will help people handle all sorts of small tasks. In fact, those bots and Microsoft’s vision of how they should work stole the entire show. Windows, Xbox: you’re cool, but the future is bots.
Microsoft has really big software ambitions, and it laid them all out at Build.
A new way to think about Bots. Bots Everywhere!
The big idea from Microsoft today was making conversation the next platform for computing. At the heart of that plan are intelligent bots. These little AI bots will respond to your words and provide you assistance. Satya Nadella noted that Microsoft had erred with its launch of Tay, a social chatbot that users quickly taught to be a racist. "We want to build technology so it gets the best of humanity, not the worst," said Nadella. "We quickly realized it was not up to this mark. So we're back to the drawing board."
In the future, Nadalla says human language will be the UI layer, bots will be the new apps, and digital assistants will be the new "meta apps." Artificial intelligence will be infused into all of this. Cortana is the best example of an intelligent bot Microsoft has been developing for a while. But Microsoft also wants to get bots into some of its popular services, like Skype. Cortana is now hanging out inside Skype, waiting for queries and highlighting key bits of text. And third-party bots from restaurants or hotels can chat with Cortana, who mediates the conversation for you inside of Skype.
A whole new Bot framework for developers
Seriously, Microsoft waited until near the end of its keynote to talk about the most exciting and interesting stuff. It’s making it way easier for Bots to talk to any chat app — from Skype to Slack to whatever. We also got a sneak peek at intelligent tools that will be coming to bots in the near future. Microsoft showed how to create a Domino’s Pizza bot with Microsoft’s tools, but it could work in multiple apps and supports natural language.
This is all run off a Cortana Intelligence Suite, which runs of something called the Microsoft bot framework. It’s a platform of cognitive services like computer vision and natural language that developers can use as they create smart bots to interact with Cortana or their customers. There are 22 APIs available starting today that cover years of machine learning and AI research.
Cortana gets smarter
She’s coming to the Xbox as your personal assistant, helping to find new titles and sharing tips and tricks on gameplay. But Cortana is also getting a huge update in the Anniversary Edition of Windows 10. Cortana will more intelligently understand time, for example. She can identify the PowerPoint you were working on last night, or the toy store you visited last year, and suggest calendar appointments based on emails and texts you receive.
Cortana can also work with third party apps more deeply — developers can set up actions that do things with their app. You can have Cortana order food with Just Eat, for example.
The next generation of Skype will talk to bots
Microsoft keeps pushing Skype, and now it’s making it a lot smarter. Basically, Skype will be able to talk to Cortana, which "brokers" a conversation with a third-party bot. Cortana is smarter than the bot (she understands context), but the bot is what does things like book a hotel room.
During a Skype demo, a bot from Cups and Cakes asked for an address to complete a delivery. Cortana brokered the conversation and gave the user a map with updates on arrival time. When asked to add an appointment to a calendar, Cortana automatically engaged with a bot from the Weston hotel, filling in location and date, and allowed the user to book the hotel room with a few clicks. Finally, she suggested chatting with a friend who lived in the same area, and produced an automatic message in the mode of Google’s smart Inbox replies.
All this intelligence can even work in real time, inside a video chat. Skype users can just add "video bots" to their chat. We’re not totally sure what that means, but it sounds wild.
The next update for Windows 10 comes this summer
Microsoft is calling it the "Anniversary Update" for Windows, and it has a bunch of new features will make Windows 10 better. It will support biometric authentication, and will even bring this feature into the browser. It works using a fingerprint sensor right next to the trackpad. Onstage, Microsoft logged into the USAA website using just a fingerprint.
Windows talked about the "Universal Windows Platform," a system it described as open and — most importantly — a platform that works across all different kinds of devices — from PCs to phones to HoloLens to wearables. And there are a ton of developers tools too: a Linux command line, a tool to help convert legacy Windows apps to the new platform, and much more.
Finally, Microsoft spent a lot of time talking about the new features coming to Ink, as well — more on that below.
Windows Ink - Draw everywhere
To highlight its stylus Microsoft showed off a new "Ink Workspace" that features a bunch of pen-powered apps. When you write yourself a reminder with Stick Notes, Cortana can now recognize dates and set automatic reminders for tasks like "Call mom tomorrow."
If you draw two points of interest on a map, it will tell you the distance and offer directions, and if you drag it across a section of text it will generate automatic highlights. Smart routes you trace on a map will even adapt if you shift to to a three dimensional view. Microsoft also showed off virtual stencils and rulers that let users more precisely shape what they draw on the screen.
New universal apps for Windows 10
We knew that Facebook, Instagram, and Twitter were committed to "Universal Windows Platform Apps," but we got a news that more are coming: Starbucks is the big one. But you can also expect Uber, Vine, Bank of America, WWE, and a ton of games.
Microsoft showed off a few graphical flourishes developers can now add to Windows apps, including a motion blur and highlight effect you can activate with trackpad gestures. They also dove into some code that lets developers easily add Ink abilities into their apps. A virtual ruler was involved. Microsoft also showed off an "Electronic Flight Bag," which is basically a Surface tablet that pilots can use to track all the information they need to fly. Boeing is going all in with it.
But a bigger deal is that Microsoft is providing a desktop tool to developers so they can convert their current Windows apps to the new universal platform. Microsoft says it can work on 16 million current apps.
The Linux command line comes to Windows
The Bash shell is coming to Windows. Even if that sentence is gibberish to you, it’s still meaningful. You can think of it as Windows supporting Linux — native Ubuntu binaries can run on Windows, and Microsoft partnered with Canonical to do it. Bash stand for "Bourne Again Shell," and it means that developers can fire up a terminal window and code the way that many of them are used to: with Linux commands. Yes, it sounds nerdy, but it’s a super big deal. Developers used to have to use third party tools to do it, now it’s going to just be built in.
Microsoft integrates gaming across Windows and Xbox
The gaming portion of the keynote started out with an announcement that Forza is going to be created as a universal app going forward — so the same one will work on both Xbox One and a Windows PC. Quantum Break and Killer Instinct will also come to Windows 10.
And that desktop app converter that we mentioned earlier? It will also work on classic win32-based games. Phil Spencer showed that Microsoft could just download Age of Empires II HD from Steam and convert it to a universal app. And it even worked onThe Witcher 3. That means that all the Windows 10 features like Live Tiles and Notifications will work with these games.
The Xbox One? It’s also a development kit now. Developers can take any standard, retail Xbox One and use it to develop games and apps. A demo showed off a desktop app running on the Xbox, and the software automatically adapted to give a sense of how their app would work on the big screen in the living room. A preview of developer mode is available starting today.
Last but certainly not least, Microsoft is finally going to create a unified store that works across both the PC and Xbox.
HoloLens is shipping to developers
Microsoft announced that the developer edition is shipping out today. It’s expensive, so it’s not really meant for consumers to buy, but it’s still an important milestone. It was a big moment for the HoloLens team, and the co-creators shared a hug onstage, whispering audibly, "I love you man."
The first HoloLens app shown on stage was Galaxy Explorer, which seemed to be holograms of planets and galaxies you can manipulate with gestures. Microsoft is releasing both the app and the source code today, giving HoloLens developers something they can study and build from. And NASA showed off a HoloLens called OnSight and Destination Mars, which lets you view the red planet through the perspective of the agency’s scientists.
Case Western Reserve University and The Cleveland Clinic showed off medical apps they built for HoloLens. There was a holographic anatomy program that projected a life size body with transparent skin and the organs clearly visible inside. A professor patched in remotely from Cleveland as an avatar and walked students through lesson on a giant holographic brain. The avatar was a bit wonky, a frozen head, hand and screen name floating alongside the students onstage. But the professor was able to accurately point out small structural details inside the holographic brain as he worked through the lesson from across the country.
