Why The New York Times is Dogfooding its External APIs
Print is dead, print will never die. There’s no doubt that a free and open press is one of the most important pillars democracy stands on. But because of the biggest symbol of free speech — the Internet — press has seen huge layoffs and paper shutdowns. And that’s because these folks that are the first to write about change and the newest shiny thing are often some of the last to use it. Even the stalwarts of award-winning journalism that were some of the first to create websites and utilize social media are finding themselves stagnated by technical debt.
But it doesn’t have to be this way. It’s just that, so far, no one’s talking about it yet. That’s why we were excited to sit down with The New York Times API architect Scott Feinberg to share this 164-year-old paper’s journey toward revamping its application programming interface program in an effort increase productivity, clarity and logical organization from the outside in.
Today in this two-part series, we’ll talk about where The Times is and how it’s planning to use external APIs as a way of dogfooding its internal ones, and tomorrow we’ll get into how APIs can reflect the pages of a newspaper and all the culture that comes from it all.
“The mission of The New York Times is to create and collect and distribute high-quality news and information. The APIs are very much that distribute part,” Feinberg said.
The New York Times is on the cusp of a structural change coming after working with the same not-so-great APIs for about six years now.
“It hasn’t been a priority. We haven’t spent a lot of resources on making our APIs great and it’s good enough and our data is valuable enough that people get around those things,” Feinberg said. “We don’t necessarily have standards or a single way of building those [internal APIs], nor how teams’ APIs talk to each other.”
Feinberg was brought in last May to improve the quality of both internal and external APIs and to put standards in place. “Now that we’ve recognized API standards and the use of APIs are important not just for external users and internal ones, we are reevaluating how we do that,” he said.
They hired Feinberg when he was working as developer advocate at crowdfunding payments provider WePay, an external API-based company. “My job was making the lives of the consumers of our external APIs good—how we handled documentation, API tooling, making the experience of our API really clean really nice.”
He was hired to kind of do the opposite—but in the same way—to The Times’ more than 150 internal APIs.
“The New York Times wants to take that approach and treat each team as an external API provider, taking this external and internal API problem and basically saying we’re going to treat these exactly the same,” Feinberg said. It plans to use its external APIs as a sort of testing ground to enable rapid improvement.
This may seem like an unusual plan when companies typically dogfood its APIs internally first before exposing them to the public, but for The Times it works better this way because it has a lot of free external APIs and the traffic is dramatically less for their external APIs than for their internal ones.
Feinberg admits that the current quality of The Times’ external APIs is less than ideal. “At The New York Times, we hold what we do publicly to a very high standard, and we haven’t always felt that way about our APIs. We don’t necessarily have the same requirements. We can make mistakes and people are relatively forgiving when you’re giving away something for free.”
On the other hand, not wanting to short-change its external API partners like IFTTT, LinkedIn, Flipboard, Google and Apple, Feinberg says that it makes everyone more committed to elevating the quality of the external APIs, “and then taking the same tools and techniques and bringing them internally.” By working with these external stakeholders, Feinberg and his team can more quickly uncover what are the data they actually need and what kind of experience they expect in return.
The Times also has a syndication bureau of paid API relationships, but Feinberg equates those APIs to “more like a feed and not so much an API in the traditional sense, but we are looking at better ways of sharing that with our partners especially when more savvy partners want more access to our data and our intelligence,” including varying formats like video. He says that even two years ago the simplicity of RSS-like feeds was acceptable, but that its partners are becoming more sophisticated and demanding API consumers.
“We’re seeing the publishing space which has really been behind in those sorts of things, with Facebook and Google and Apple creating its own ways of doing these things, [and] the space is getting more mature in how we share that data,” Feinberg said.
Currently, The Times has 13 external APIs serving tens of millions of calls a day, all of which are owned by Feinberg’s team. “So we can do whatever we want with them and move rapidly to make those changes.” Internally they have to persuade those respective teams to make changes.
“I view our external APIs as a way to find new ways to experiment and distribute our content. If the APIs are good, our information is strong and easy, we can rapidly test out new distribution platforms,” said Feinberg. For instance, the Times had one of the very first apps on the Apple Watch, even before the wearable’s launch.
The API team is focusing on going where readers are looking. “There’s going to be people who get introduced to The Times because they saw our content on an Amtrak train or on Facebook or in its Apple News app and that’s how we’re introducing a new generation” to The Times.
The New York Times has external APIs that are copies of its internal APIs, but Feinberg says they need a lot of clean up. “There is information that is useless for people that want to use [them.] We want to make sure our API interfaces are consistent but are the most useful interfaces for someone that’s consuming it.”
The question becomes: How can internal APIs be exposed externally and adapted to different use cases?
He says that just because an API works for you internally, that doesn’t mean it’ll be useful to someone else. “Those APIs are useful to you and they may provide similar concepts, but it’s not in the same domain of what someone is going to use your data for.” Its goal is that, once they’ve built an internal API with data that’s externally valuable, they can rework the API interface and iterate n API versions to accommodate different needs.
Check in tomorrow as we continue talking to The New York Times about its APIs, diving into why it’s so necessary that its internal APIs change, and how this evolution could affect the entire news industry as well as most other industries as well.
Feature Image via Pixabay.
