Microsoft One-ups Google with Copilot Stack for Developers
No prizes for guessing the focus of this year’s Microsoft Build developer conference. Of course, it was AI — just as it had been at Google I/O earlier this month. But whereas Google’s AI announcements seemed disorganised and all over the map, Microsoft came up with a cohesive framework to entice developers: its new “Copilot stack.”
In the opening keynote, Satya Nadella positioned the ChatGPT-inspired AI era as the latest installment in society’s pursuit of the “dream machine” (referencing M. Mitchell Waldrop’s 2001 book about J.C.R. Licklider).
The main focus of his presentation, though, was Copilot. While he started out referencing GitHub Copilot, the first Microsoft collaboration with OpenAI, it was only a matter of time before Windows was inserted into proceedings. “Next, we are bringing Copilot to the biggest canvas of all, Windows,” he announced to the live Build audience. That got the biggest cheer of the day.
Nadella then introduced a video about chat AI functionality that began with the magic words, “integrated into all of Windows.” Later in the opening keynote, other forms of Copilot were introduced — including Microsoft 365 Copilot, for office workers.
Technical Details about Copilot Stack
In the second keynote, Microsoft Chief Technology Officer Kevin Scott got into more detail about the new copilot initiatives, from a developer perspective. He began by highlighting Microsoft’s partnership with OpenAI, putting the success of the relationship down to Microsoft having “an end-to-end platform for building AI applications.” He then positioned Azure as “the cloud for AI” and Windows as “the best client for AI development.”
Scott then brought on stage Greg Brockman, the president and co-founder of OpenAI. The discussion between the two around ChatGPT plugins was particularly enthusiastic, with Brockman encouraging developers to “really go into specific domains and figure out how to make this technology work there.” He used the example of plugins in the legal domain, where developers are “getting expertise and talking to lots of lawyers and understanding what their pain points are with this technology.”
Then came the big news. Scott said that Microsoft has built a “Copilot technology stack” for developers, enabling them to add AI functionality to any software — in other words, a “copilot.”
Commenting on the frontend layer, Scott said that with a copilot, “it’s going to be way less of that fiddling around mapping user interface elements to little chunks of code than you’re accustomed to.”
For the orchestration layer, Scott described it as “the business logic of your copilot.” In LLM terms, it’s where the prompting happens. Microsoft’s orchestration mechanism to help build its apps is called Semantic Kernel (see my separate writeup about that), which has been open sourced. Scott added that there are other open source alternatives for orchestration too — he gave a special shoutout to LangChain. In addition, Microsoft has a new tool called Prompt Flow, which Scott said was “another orchestration mechanism that actually unifies LangChain and Semantic Kernel.”
Part of Microsoft’s orchestration layer is the “meta prompt,” which Scott described as a “standing set of instructions that you give to your copilot that get passed down to the model on every turn of conversation.” He added that it’s “where a bunch of your safety tuning is going to happen.”
“Grounding” is where we get into things like vector databases and “Retrieval Augmented Generation” (RAG), both of which I discussed in my recent interview with Pinecone. Scott described grounding as “all about adding additional context to the prompt that may be useful for helping the model respond to the prompt that’s flowing down.”
Finally, at the bottom of the stack are foundation models and infrastructure. “We give you a bunch of choices for how to use foundation models in this copilot platform, on Azure and on Windows,” Scott said.
Azure AI Studio
The third keynote featured Scott Guthrie, EVP Cloud + AI, and several of his Microsoft colleagues. Part of what he was promoting was Microsoft’s development platform for creating a ChatGPT plugin. “We’re embracing an open plugin standard that provides plugin interoperability across ChatGPT and all of the Microsoft Copilot offerings,” he said.
One of the more interesting products Guthrie referenced was the Azure AI Studio, which he said “makes it incredibly easy to ground your AI models with your own data and to build Retrieval Augmented Generation (or RAG) based solutions.” He added that it enables you to “build your own copilot experiences that are specific to your apps and organizations.”
For prompt engineering, which Guthrie reiterated is a part of the orchestration layer of the copilot stack, Microsoft is introducing a new framework called Prompt Flow.
“Prompt Flow provides [an] end-to-end AI development tooling that supports prompt construction, orchestration, testing, evaluation and deployment,” he explained. As Kevin Scott had hinted at earlier, Prompt Flow can leverage Semantic Kernel and LangChain as well.
Finally, Guthrie announced Microsoft Fabric, “a unified platform for data analytics, really designed for the era of AI.” He added that “it’s lake centric and has an open data architecture, and it has deep integration with Microsoft 365.”
Overall, this felt like a much more cohesive set of AI announcements than what Google announced recently. The Copilot stack in particular will surely resonate with Microsoft’s enterprise-focused developer community.
