Welcome, Charlie

I’m thrilled  to say that Charlie Szymanski, app developer and visualization pro, has agreed to join the Reporter’s Lab as our first developer. Charlie was one of the key players in the Sarasota Herald-Tribune’s prize-winning interactive IBISEYE hurricane tracker and produced the online components of Broken Trust and the Flip-Fraud investigation. He also programmed the front end for the National Journal‘s redesign and made wonky politics fascinating through features like a Congressional vote rating interactive.

Charlie knows what goes in to great investigative journalism and has the enthusiasm to create tools and delve into new endeavors. He’s just the right partner for the lab’s startup. Welcome, Charlie.

On the Media on data journalism and its cousins

Last week’s segment on data journalism on On the Media highlighted the Texas Tribune’s well publicized effort on data journalism, and had my take on the federal government’s open government initiative.

There’s a nuance I’m not sure came through on the subject — one that was probably impossible in a 5-minute piece.  Matt Stiles undersold the effort he makes on acquiring newsworthy data and making it public to people without the power to get it themselves. The prison records he mentioned should be published by the state — but they’re not. Lobbying and voting records should be easier to get. And the effort the Tribune makes in acquiring salaries from state and local governments might make a situation like the one uncovered by the Los Angeles Times in Bell, Calif. less possible.

Not all data journalism is alike — the Tribune’s effort focuses both visualizing databases that the state makes public voluntarily, and collecting and presenting important public data that’s much harder to get. That’s what makes it special. Of course, many other commercial and nonprofit news organizations are also making difficult data accessible, from the Washington Post’s Head Count of Obama administration officials to ProPublica’s work on Recovery Act tracker  and Dollars for Docs.

The nuance is that the news organizations aren’t always picking the low-hanging fruit of what the government wants to publicize. They’re putting money and time into getting new records and putting together others that are so difficult to use as to be effectively hidden.

The Reporter’s Lab

Welcome to the Reporter’s Lab, a site focused on tools, techniques and research for public affairs and investigative reporting.

The lab will address a vital precinct in the journalism production line that has sometimes been left behind by social media, data journalism and the accelerating pace of breaking news: the beat and investigative reporting that uncovers hidden, not public, stories. For professional and pro-am journalists who specialize in public affairs, the technological revolution passed them by sometime in the early millennium. Mining documents usually means reading and re-reading and manually cross-referencing. Mining data largely uses techniques popularized in the 1980s. Collecting records dispersed among dozens of websites involves a lot of clicking and saving.

I spent much of the last year interviewing reporters, editors, researchers, technologists and entrepreneurs and found a world of methods and tools for intelligence and security, information science, digital humanities and social science that could, if harnessed, transform reporting.  With few newsrooms investing in these advances, most reporters are still left on their own to unravel big document and data dumps, listen to hours of public meeting recordings and dig through government forms in dusty offices.

Enter the Reporter’s Lab. The lab will address the cost side of the investigative reporting equation, not by removing reporters from the mix but by removing some of the drudgery. It aims to do for modern reporting what photocopiers did in the 1970s, and e-mail, the Web, spreadsheets and databases did in the 1990s. It will go beyond the hype to test, create, commission or apply new methods to make the hard work of original reporting easier or more effective. It will guide reporters to the right tool for an immediate job, from sorting a handwritten government sign-in sheet to finding the contractor’s testimony in the school board meeting webcast.

If a tool doesn’t exist, we might create it. If it’s too hard to use, we’ll find a way to make it easier. And if someone already made it, we’ll test it on real documents to show you how it works in a newsroom. The lab is complementary and collaborative: we’ll work with Investigative Reporters and Editors, Sunlight Labs and other journalism and transparency organizations to test tools and help reporters use them.

We’re already working with researchers at leading universities including Harvard, Carnegie Mellon and Georgia Tech. Experts of all kinds care deeply about serious journalism and the lab will help connect them with practitioners and create even newer techniques and methods that we can apply in the future.The lab is also researching practices in public records administration to eliminate the need for technological workarounds.

If you have an idea for the lab, have records or documents to donate for research or want to use the lab as a sounding board, contact me.

Nordic Media Festival

It sometimes takes preparing for a new audience to focus on what’s new and what’s not in U.S. journalism. Last week, I went to Norway to talk about recent changes and technological advances in the American investigative scene. What I discovered in preparation was that the culture of investigative reporting has changed as much as the methods in the last few years. What really focused my attention were two recent projects based out of non-profit newsrooms.

The first is the prize-winning collaboration among  ProPublica, NPR’s Planet Money and This American Life, Wall Street Money Machine.  Would a news organization five years ago have produced “Bet Against the American Dream“, a takeoff on The Producers or “They Didn’t See it Coming,” from Auto Tune the News, to try to explain a complex financial story? Or a comic that walks through a financial deal?

California Watch’s recent  On Shaky Ground series tried other approaches, from an iPhone app showing seismic risks to a coloring book for kids who might find themselves in an earthquake.

Investigative reporting has often taken itself so, so seriously that it left the audience behind. These stories prove that it’s possible to reach our readers and viewers in an engaging way on serious public affairs stories without lecturing them — or even asking too much of their attention.

Here are the slides from the talk. Thanks much to the festival for inviting me — it was a great experience.

MAGIC and the next big thing

On April 12, Duke University and the National Archives brought together reporters, transparency advocates and watchdogs, and government officials to talk about barriers to better access to public information. Dubbed “MAGIC,” for Media Access to Government Information Conference, this wasn’t a typical celebration of open government. Instead, it focused on practical steps that the federal government and the public could take to improve access to key records. The video and transcript will be available soon on the MAGIC page and through Duke’s DeWitt Wallace Center for Media and Democracy.

One state official mentioned the strides made in public geographic data images and analysis that came from state and local governments working with the federal agencies to pool their resources. What, he asked, could be the Next Big Thing?

One answer might be work on entities, particularly organization names. Each agency and locality seems to keep track of organizations they do business with differently. The federal government uses DUNS records to classify companies, but that’s not public. Others use tax ids, and still others assign their own coding structure. The problem is that following organizations across state lines and across agencies is almost impossible, given the alternate spellings, duplicate names and complex relationships and subsidiary structures. Just think: it wouldn’t just be reporters who could suddenly figure out whether a contractor had been sanctioned or sued. So could local governments deciding whether to hire a labor law or pollution scofflaw as its contractor.