Oxide Computer’s Bryan Cantrill on the Importance of Toolmaking
Tools made during development can serve a wider purpose than the immediate problem they address because if it’s a problem for one, it’s potentially a problem for many. So spend time making tools says Oxide Computer Company co-founder and Chief Technology Officer Bryan Cantrill, in his 2022 P99 Conference talk “Sharpening the Axe: The Primacy of Toolmaking.”
Yet, due to an “unseen, unknowable payoff,” Cantrill explained that some people continue comparing time spent on toolmaking to “stealing time.” Oh. Controversy.
The Argument at Hand
Even though toolmaking is wildly important, it isn’t, “explicitly encouraged. It can feel like a distraction.” He doesn’t mask his disappointment when explaining, “toolmaking has worked in the shadows. It’s been done on the sly serendipitously. sometimes, you know, without management’s knowledge or authorization.”
Toolmaking has a value, but it’s more of a delayed value whereas building features and closing JIRA tickets has more of an instantaneous value to a business. Cantrill agrees that, “It is absolutely hardest when the deadlines are tight and the opportunity costs teams high [but] the best time to develop a debugger is when debugging.”
To those business owners who think that spending time on toolmaking is going to capsize the company, Cantrill has some insight to share: “What’s going to capsize your company is building something that no one wants. When you’re building a tool that you yourself need, you are tautologically building something that someone wants.”
Whether or not that’s large enough to build a business on was not the purpose of Cantrill’s talk. Flickr founder and current Slack CEO Stuart Butterfield has pointed out both Slack and Flickr were both created as gaming apps but made their mark on the world after pivoting as the tools (photo sharing and chat, respectively) proved more valuable than the original ideas themselves.
So then why isn’t toolmaking encouraged? Well, “payoff is at a distance but the cost is immediate,” is how Cantrill describes the stealing time feeling toolmaking conjures. Tight deadlines, short runway, and the weight of, “The immediate cost that you had today [associated with] working on this tool [vs] working on the artifact that you’re responsible for delivering.” Toolmaking in a way can be compared to the other thing no one wants to address — technical debt.
But when looked at as an investment that pays in compounding interest, the upside of building your own tools “is often completely unbounded,” Cantrill said. Consider what the software landscape would look like if not for the following tools:
Unix, circa 1969
The software system is considered the birth of modernity born at AT&T Bell Labs in the late 60s, early 70s. At the heart of the Unix OS system is the kernel followed by other essential components including the shell, file system, and a core set of utilities.
The Unix team wanted to build an operating system but the best way to get it funded was to develop a system allowing them to better typeset their patent applications. There was a general purpose to this tool and Unix’s utility far outweighed the artifacts it was creating. Today, we have Linux, the world’s most popular open source operating system, as a direct result of this project.
NSELite, ca 1991
NSELite, developed by Larry McVoy because he needed a better way to develop software inside of Sun Microsystems in the late 1980s and early 90s, was the first embodiment of the bring over modify merge model.
McVoy later started BitKeeper, the first commercialized versos of NSELite. Due to a licensing dispute, Linus Torvalds invented git to continue developing the Linux kernel. NSELite is the direct grandparent of git and that happened all because McVoy wanted a better daily workflow.
DTrace, ca 2003
Cantrill was a direct creator of the DTrace debugging tool. After joining Sun Microsystems in 1996 to do Solaris kernel development, specifically to work on Solaris performance, he realized he had to understand the performance of the system. But before any of this could happen, he needed to instrument the running system. After about five years of work, the end result was DTrace.
“We knew it was important, but I think we had no idea how important it was going to be to be able to stitch together what was happening on the user level what was happening in the kernel, and find these huge performance problems up stack,” he said. He continued with, “DTrace played all these dividends that I didn’t really even appreciate at the time, because it allowed people to develop software better. That kind of ongoing dividend is something that’s very hard to anticipate as a tool maker.”
These are some incredibly high-profile examples and the tool doesn’t have to make an impact as large as these to matter. While they are all different, Cantrill says they share one important commonality, “These were toolmakers building the things that they themselves needed, and they did not these things that they were building did not have necessarily immediate tangible value in and of themselves.”
Cantrill says, “We change our environment to make things that will make other things. It is the core of who we are.” A woodsman was once asked, “What would you do if you had just five minutes to chop down a tree?” and he answered, “I would spend the first two and a half minutes sharpening my axe.” Why would engineers not spend the time to sharpen their axe?
