Technology

In New Journalism, The Data Is the Story

3 May 2014 1:12pm, by

In the next two weeks we will start to launch our data journalism projects. The first two will explore the contributions people make to open source efforts. The research is about developers, what they’re working on and what connections you can make when digging into the data.

Data journalism projects offer methods for us to write about developers by doing an ongoing analysis of data that we collect. It’s this narration of the data that, for us at least, will open new ways to cover the technology markets. This approach will provide us with a process to collect, analyze and report about trends in the developer community.

These data journalism efforts are new to me, but I am finding it is a lot like blogging, learning about RSS or posting a podcast. Like all these things, data journalism is a constant discovery process. It’s a narration, just like personal blogging is a narration of some aspect of your life.

That, to me, points to a different kind of reporting and why data journalism is gaining popularity.

I grew up in the post-Watergate era.  People like myself entered the field of journalism based on the view that we could change the world. I know I had dreams of uncovering corruption as an “investigative” journalist. The newspapers where I worked did do investigative journalism. But it was costly and it took resources from the newsroom. We still had to cover the cops beat, the courts, city hall and the parades. I covered a lot of parades.

But the day-to-day work was a lot of fun. In my first reporting job, I worked for The Augusta Chronicle. They assigned me to a one-man bureau in Thomson, Georgia. I received the attention of the mayor, who did not care for the issues I wrote about. About six months into the job, some out of state industrialists came to McDuffie County with plans for a toxic waste plant.

The community organized and the county commission turned the industrialists away. It was electric to cover stories in that atmosphere.  It was like theater. Everyone had a role on the stage. Our job was to narrate what these actors did.

Investigative reporting is a different form of journalism compared to daily coverage but they are connected. The investigative projects generate from the daily coverage and source development. The projects are often a story or series of stories that use data and research to chronicle one period of time, an issue, corruption, etc.

Now we have a new form of investigative reporting that is commonly referred to as data journalism. All kinds of projects are popping up. ProPublica, for example, does its own investigations, offers tools and has an ongoing collection of watchdog reporting that it aggregates.

But how does one get started? For me it meant working with developers like Chris Dawson and Sriram Surbramanian, the two developer leads on our first project. Both of these people understand how to collect, clean and organize data.

We are finding that the discovery process comes as the data gets collected. It’s very similar to reporting.  The story is there but it only reveals itself after collecting the information. Patterns form through the connections in the data. That process opens more questions which give hints about what new data to collect. After we collect and analyze the data, we should find patterns that will help us form questions to ask people who are pertinent to the story.

This is largely a homegrown process. We are developing our own tools to find connections in the data that we collect.

It’s a fun process. It has given us the chance to better understand graph databases and how they can be used to connect different data sources. It has helped us pinpoint shifting movements in specific developer communities.

If we can do this right, then we have the chance to create an organic process for collecting, analyzing and reporting what we find. We can find the stories in the data. We can discover who we should be talking to and what questions to ask.

None of this is perfect, but it does show me at least how this model can be applied to ongoing analysis of elected officials, government agencies and the role corporations play in public policy. These are not topics that we will cover directly. We’ll just look at the stacks people use, how they compare and the effects of that technology on the way we live and work.

It also gives me some hope that my dreams of becoming an investigative reporter are taking shape. I never thought it would take this form, but that’s just another example of what happens when you start to discover what the story really says.

Feature image courtesy of The Washington Post.

Participate in The New Stack surveys and be the first to receive the results of our original research.