Microsoft’s Road to Open Agile Development

One of the oldest rules of thumb in software development is Conway’s Law, which suggests that “organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.” Or, as it’s often paraphrased, “don’t ship your org chart.” It’s closely followed by its more recent corollary, the Reverse Conway’s Law, which points out that you can change organizations by designing software for the organization you want to have.

That’s what is happening with Visual Studio Online (VSO) and Microsoft.

Initially released four years ago as an online version of Visual Studio’s Team Foundation Server, VSO has become a hub of Microsoft’s effort to establish an internal Open Source ecosystem. At the same time, it’s providing Redmond’s development teams with the tools they need to use to move from monolithic applications to continuous delivery of modern cross-platform service architectures, and to build the internal open source culture espoused by CEO Satya Nadella. These two roles may seem far apart, but they are now driving Microsoft’s collaborative development tools in the same direction — and making Microsoft a more open company at the same time.

We recently spoke with Microsoft’s Brian Harry and Nicole Herskowitz about what Microsoft is doing with its enterprise development platform, and how it was evolving beyond its on-premises team roots. Herskowitz explains that every company is now a software company, but “it’s how they differentiate” that is the point. But the increasing importance of software to business means that software development methodologies need to change. “It’s critical that they become more agile, and more efficient. They need to be faster, iterating on their applications based on end-user feedback,” says Herskowitz. Tools like VSO are key to this, providing an alternative to email-heavy processes.

It’s not just development, either. It’s all part of the broader move to DevOps, changing the way teams work — even at Microsoft. “Our team has changed,” Herskowitz notes. “We moved from shipping waterfall-developed boxed software every couple of years to shipping a service every three weeks.” That’s led to the VSO team rethinking their tools: “We had to make that change, now our customers are changing too. So we understand the needs they have, and it really influences the tools we build to help them solve these problems.”

That’s a change that reflects in how users can build VSO into their existing development toolchain. Microsoft used to be very much an ‘our way or the highway’ company. If you used Microsoft development tools, you used its project management tools, its source code repositories, its test and build chain. That’s all changing, as its customers move to using a wide mix of tools and technologies — and as Microsoft does the same.

The process of opening up VSO as an application lifecycle management chassis has meant that it needed to be a tool that you could plug other tools into, and that could inter-operate with other tools and services. Harry explained how the team approached it: “The first step was to support REST and OAuth, with service hooks to support events across services. We could then build out integration with a bunch of different tools like Zendesk for ticket management, Trello for managing task boards. There’s also a multiplexing agent with hundreds of additional things like Asana for project management.”

Microsoft’s realization that it is unable to be all things to all developers means that VSO is going to continue to be an important part of the company’s cross-platform and open developer strategy — with future releases adding support for working with the popular open source code repository Github. With Microsoft moving its own open source code from CodePlex to Github, it’s a logical move.

Harry points out that it’s not the only change: “Add to that our investments in the next generation of our build tooling, with support for cross platform and cross technology.” That new build tool is going to be very important for open development at Microsoft; coming out-the-box, it’ll be able to support builds on everything from Xcode to Node.js to Java to .NET. “We’ve got agents that run on Mac and on Linux to support delivery there,” Harry says. There’s also the option of using the new build tools and VSO together to integrate with other build systems, with the .NET team using them to work with a Jenkins server.

There are significant pieces missing, with Harry noting that VSO is still missing governance tools. “It’s going to depend on how sophisticated an organization is, and how legally sensitive its code.” For companies with complex demands that could mean integration with Black Duck’s or  Palamida’s tooling, and with their consulting services. Less sophisticated organizations will be able to work with tools like JFrog’s Artifactory. He’s clear that this won’t remain a gap for long, telling us, “As part of our push we’ll be making sure there is a solution there for customers.”

VSO’s changes aren’t just a matter of supporting its customers. They’re also a reflection of change inside Microsoft, following the open-sourcing of key technologies. As Harry says, “Some of the GitHub integration is because it’s a centre of gravity in the open source space and because of that, Microsoft hosts a lot of its own open source projects there. So one reason we’re looking at doing GitHub integration is driven by our own needs and our desire to host development efforts in the open, using our VSO capabilities.”

Bringing Microsoft’s own development teams onto VSO has affected development. As Harry explains, “We certainly have scale requirements we hadn’t hit anywhere else.” With over 20,000 developers currently using VSO inside Microsoft, it’s by and large the biggest single group of users employing the tools. “There were things like the scale of Active Directory — the number of people brought to VSO that pushed us to go make improvements for large teams,” he told us.

It’s a two way street, states Harry: “One of reason I was excited about bringing the broader Microsoft to VSO is that it’s created a really good avenue to harness energy in Microsoft to build tools to help make VSO better.” Harry points out a recent development: “Two or three sprints ago we announced a new wiki experience that improved the VSO welcome page. It was built by another team in Microsoft and contributed to VSO.”

Similarly, the new build system is benefiting from working with other teams inside Microsoft. “The team in Office [that is] doing a lot of Android and iOS development contributed a bunch of stuff to the build system to handle code signing and other key features they need in Android and iOS — and those are going to be part of the commercial release,” he says. Harry is pleased how things are working, commenting, “This is exactly the kind of internal open source ecosystem within companies we’re trying to enable.”

As Herskowitz told us, “It’s a broader example of how we’re supporting an open platform, connecting various different services. We’re changing as a company, to a very open development approach – embracing the community to help us further extend the development of our technologies and tools. We’re seeing a lot of that with .NET.” That change is reflected in the company’s tooling, as development moves away from small teams and large blocks of code, built using traditional waterfall processes. “The community is taking small work units, branching the code, and taking it to new places.”

Building tools to foster openness is key to being a more open company, and VSO is key to changing Microsoft’s development culture. The source code to VSO is available to anyone at Microsoft, and anyone can check out code, and submit pull requests to the team. With VSO and Yammer short-circuiting Microsoft’s old processes, and with its org chart dramatically changed, maybe it’s time for you to stop shipping your org chart too.

Feature image via Flickr Creative Commons.