Last year we embarked on a process to define what impact means for a people-centered, community news lab.
By Darryl Holliday
Pullman Porter advertisement and portrait of a porter taken at Chicago Union Station. (Images: Wikimedia Commons)
As our young media start-up matures, remembering why we do this work is as important as building models for the future of local media. That’s why City Bureau’s first Impact Report begins with a story—a historical guiding star from our home on Chicago’s South Side—the story of the Pullman Porters.
At their peak, these 12,000 Black train workers serviced sleeping cars in the Jim Crow era, from the Midwest to the deep South and throughout the continental U.S. The Pullman Porters brought their families into the middle class, despite racist and severe working conditions; they formed the first Black labor union to receive a charter in the American Federation of Labor, laying the groundwork for the Civil Rights movement; and they helped fuel the Great Migration, the mass movement of 6 million African Americans from the rural South (which included my great grandmother’s move to Chicago).
They also embodied a novel kind of participatory journalism that City Bureau views as an inspirational ancestor. The Black press—including influential papers like The Chicago Defender—wouldn’t have been as successful without them serving as news gatherer, distributor, journalist, storyteller and “invisible grassroots organizer” while traveling the country, creating and carrying news and information from place to place, news outlet to news outlet and person to person.
That’s impact by any definition of the word. And more than 65 years later, as we work to measure the impact of City Bureau, we look to the Pullman Porters as a model for how relationships and information can create new forms of civic engagement.
The imperative embedded in this approach? We need to get many, many more people involved in the production and distribution of information—not just clicking our content.
What (Who) Is Journalism For?
I started thinking critically about journalistic impact in 2010, while working at my first professional newsroom, the Chicago Sun-Times. Like other reporters in traditional newsrooms, I became familiar with digital analytics tools like Chartbeat—each story broken down into its composite pageviews and uniques, valuing my work by the number of web ads that could be seen next to it. Setting aside all the economic issues with this model, City Bureau is a nonprofit organization and we don’t measure our work by the amount of profit it generates. Our mission is to bring people together to create more equitable, responsive and (hello!) impactful media.
So what does it mean to create impactful media? The journalism industry, at large, has failed to articulate a coherent theory of change for a long time. It’s not that news articles don’t lead to change. Every Pulitzer season is riddled with stories of legislation passed, politicians indicted and people literally freed from the bonds of slavery. These impacts are real, and the world is better for them.
But too often, this definition of impact is too narrow and focused on top-down systems of change. Police arrest a criminal. Government officials vote on how to regulate trade. It undervalues the contributions of grassroots movements, the importance of relationships, the power of everyday people and the incremental way in which communities make change when they are locked out of the halls of power.
These are the people who City Bureau was created to serve. We cannot measure our impact without acknowledging how they play a part in our collective work.
Our Process
City Bureau’s leadership team defined five strategic goals for 2019, the first of which was to “create and implement an evaluation structure,” including the development of programmatic theories of change; defining and tracking key performance indicators and collecting and analyzing the resulting data for our first-ever Impact Report.
Our organization has always been a response to harmful, inequitable coverage, which is too often upheld by myths of objectivity and “reach” at the expense of depth. Whether its a public workshop, a web app that enables participatory journalism at scale or a zine made for incarcerated men and women, our goal is not simply to reach the largest number of people but to understand and produce the information our communities need—in partnership with them. Our evaluation structure needs to be able to capture that work.
City Bureau’s leadership team spent the early months of 2019 drafting theories of change and logic models for each of our three programs: the Civic Reporting Fellowship, Documenters and the Public Newsroom. We formed an impact committee made up of staff and members of our board. We built metrics dashboards for each program using Airtable to test and monitor dozens of performance indicators. We wrote publicly about our process (“Inform, Engage, Equip” framework, our Community Engagement Guidelines and an information needs hierarchy). And we conducted a literature review, using 70 academic studies, white papers and organizational models to refine how we think about impact.
We emerged with a North Star for our work: 12 outcomes that fall into the categories of civic knowledge, generative relationships, information economy skills and information system resilience. Far more than clicks, ads sold or legislation passed, they paint a picture of the work City Bureau has done, and where we want to go with it.
Each of these outcomes is supported by, and assessed through, more than 20 key metrics routinely collected by City Bureau to serve as harbingers of programmatic success. Examples include the percentage of reporting fellows from under-represented identity groups in the media industry, the number of government agencies routinely tracked and ratio of first-time Public Newsroom attendees to returners. We know this is simply the beginning. But with our eye on these outcomes, we’re standing on the shoulders of those who came before us while drafting a blueprint for what comes next.
For years we have shaped our programs so that, taken together, they represent a long-term strategy for building media infrastructure that is both more democratic and more sustainable than existing models. These program outcomes provide a foundation for concrete measurement so we can ensure our work drives toward our vision: a future in which all people are equipped with the tools and knowledge to effect change in their communities.
Civic Knowledge
According to The Oxford Handbook of Civil Society, civic knowledge is “knowledge that some citizens need, knowledge that civil society generates and knowledge that people create, use and preserve when they act as members of a civil society.” Civic knowledge includes the information, skills, or dispositions that citizens should have to be full participants in a democracy.
Within the category of Civic Knowledge, City Bureau programs work toward three outcomes:
Knowledge of how the state works and your rights within it,
Knowledge of who wields power in the public sphere and how it is exerted and
Understanding of local information systems.
Generative Relationships
Information is not only shared by public institutions like newspapers, nonprofits and governments. At the core of any community are what we’ll call generative relationships: relationships that bring unforeseen, novel solutions to complex issues or problems. These relationships form bonds between individuals and their environment that lead to fuller participation in a democracy.
Notably, generative relationships tend to lead to process-oriented outcomes that are hard to predict at the outset of a project. These bonds help build community and center the lived experiences of individuals often marginalized by the political process. However, how these relationships and community are cultivated matters.
City Bureau measures Generative Relationships through the following outcomes:
Intersectional relationships are created, nurtured and maintained,
Relationships foster habits that generate culture and community and
Relationships enable collective action for common purpose and public good
Information Economy Skills
The Cambridge Dictionary defines the “information economy” as an economy in which knowledge, information, and services are more valuable than manufacturing. Information economy skills are necessary for individuals and communities to connect, create and thrive in today’s world. We also recognize the history of marginalization and economic inequity in Chicago (and beyond), so City Bureau prioritizes developing information economy skills explicitly in service of creating change for people who have been systematically oppressed.
City Bureau’s commitment to information economy skills can be measured through these three outcomes:
Ability to use relevant information economy tools and technology,
Ability to generate and distribute civic information and
Capacity to prioritize information to achieve justice-oriented ends, exert civic power and enact change.
Paid opportunities
In this category, which focuses on macro-level impact, we are especially cognizant of the long history of marginalization when it comes to the production and distribution of information. Not everyone has had equal access to tell their own stories or access information that could be used to tilt imbalances of power. That’s why these outcomes focus on equity by prioritizing people who have been left out of traditional approaches to journalism and civic education in the fulfillment of paid opportunities within City Bureau programming.
City Bureau programs aim to create the following outcomes in the category of Paid Opportunities:
Equitable access to the skills, knowledge and opportunities in the information system,
Equitable participation in information systems and
Opportunities to participate in civic work, community-building and employment.
*Note: A previous version of this article defined this category as Information System Resilience. We’ve revised the category to Paid Opportunities to reflect a key element within the overall goal of information system resilience.
Reimagine Impact
We’re sharing resources from our impact and evaluation process to continue the conversations held in our newsroom and in libraries across Chicago: If journalism is a public good, who is it accountable to? How do we measure our efforts beyond profits while working toward sustainability? What do metrics that prioritize participation and equity look like in an industry known for gatekeeping and exclusivity?
How can we reimagine impact?
Support City Bureau’s efforts to make more impactful, inclusive local media by becoming a City Bureau Press Club member today.
To get twice-monthly emails including Chicago news and events, sign up for City Bureau’s Chicago newsletter.