By Dr. Thomas Wolf and the WolfBrown team in the Cambridge Office
One of the pleasures of consulting is the opportunity to work alongside many different organizations, each wrestling with its own questions. And sometimes, when we step back from individual projects, we note certain themes, questions, and ideas across the work.
Two trends, in particular, we have noted here at the Cambridge Office:
- Clients are increasingly looking for data-driven work to help them plan, refine, and, at times, abandon programs. Measuring the impact of activities often requires sophisticated analytical models. But it also requires us to think carefully about what we measure, what different kinds of evidence can tell us, and how the process of evaluation itself can contribute to learning.
- Clients are looking for ways to expand the number and types of people they serve, and to understand not only whom they are reaching, but what those opportunities mean for the people who participate. The old concept of “outreach” has gradually been replaced by the idea of partnership—relationships where both sides have something to contribute and both have something to learn.
I asked team members to describe a project they are working on that follows these trends and what they are learning from it. Here is what I heard.
1. How do we learn from a program over time?
Bloomberg Arts Internship (BAI)
SREEHARSHA DAKKILI: I’m building a shared data infrastructure for BAI’s nine-city program that will include current and historical information on more than 2,500 interns including, very importantly, their activities after they complete the program.
This database allows my colleagues at WolfBrown, our client Bloomberg Philanthropies, and each local coordinating agency to review metrics for individual cities alongside national trends and numbers. It allows for solid information to refine the program going forward. Building the infrastructure is only half the work—we must make the data presentation as clear and simple as possible so that people are able to seamlessly interpret the numbers.
ANUSHA MOHAN: I have also been working on the BAI project developing a qualitative archive composed of our collection of materials from interns, mentors, and supervisors. The archive will consist of site observations, interview and focus group transcripts, work samples, and photos of the program in action. Having a rich history of our qualitative work across time is crucial, as it allows us to reference, reflect, and learn in the long term.
A rich qualitative repository is critical in several shifting landscapes: as the program and the BAI alumni network continue to expand, as both creative and non-creative workforce expectations continue to change in the age of AI, and as our thinking across evaluation frameworks and activities evolves.
VERONICA LI: This past summer, as I approached my final year of higher education, I worked at WolfBrown supporting the work described above. As someone who is interested in pursuing a career in arts consulting or arts management but only discovered this path in the latter half of my undergraduate studies, I know how important early exposure to the arts is for youth. I think bolstering opportunities for young students in this field is deeply important, and I am excited that I get to play a small part in doing so.
TOM WOLF: What strikes me about this work is that building the infrastructure is not simply a matter of collecting more data. We are asking how an organization can build knowledge over time. Quantitative data lets us look across thousands of participants, cities, and years; qualitative material preserves experiences and context that numbers alone cannot. And Veronica reminds us that behind all of this are actual young people encountering possible futures.
2. What does it mean for a program to make a difference?
Roundabout Theatre Company’s Education Programs
KATHLEEN HILL: At the start of this year, I began working with colleagues John Carwath and Asif Majid to complete an impact assessment of Roundabout Theatre Company’s Education programs. During their 2024–25 season, Education programming reached over 30,000 educators and students, early-career professionals, and community partners across New York City.
These programs embody the principles that ground our work in the Creative Opportunities practice area: removing access barriers to high-quality learning along the life cycle in the arts. Our collaboration over the next year will empower staff to make data-driven decisions and craft data-driven cases to ensure that Roundabout’s programming continues to be a pillar in communities across New York City.
Start With the Art, Arts for Learning Maryland
STEPHEN HOLOCHWOST: I am analyzing data from Start with the Art, a program of arts-integrated instruction that was offered to elementary-school students in the Prince George’s County Maryland School District by Arts for Learning Maryland. The program and its evaluation were funded through an Education Innovation and Research (EIR) grant from the United States Department of Education, and our study will allow us to conclude whether the program caused improvements in participating students’ academic and socioemotional skills.
TOM WOLF: These two projects approach a question we encounter again and again: What does it mean to say that a program “works”? Knowing how many people participated matters. But so does understanding what happened because they participated—and developing evidence that can help organizations make decisions about what comes next.
3. In the age of AI, are we still measuring the right things?
A Writing Rubric
JENA GU: I am working on a rubric for scoring student writing that captures qualities like creative thinking, originality, and lived experience to measure skill development in a way that withstands the influence of generative AI in writing.
The way students write is rapidly evolving with the rise of AI tools. In order to meaningfully assess learning, those who seek to measure learning must be willing to re-evaluate their existing assessment approaches and think creatively in order to adapt to this changing landscape.
TOM WOLF: I suspect this is a question we will be encountering far beyond student writing. When technology changes the way people accomplish a task, our measures may have to change too. It forces us to go back to a more fundamental question: What is it that we actually care about learning or understanding?
4. What makes responsible research possible?
Gaining Permission to Do Research with Students
CLAIRE PAVLIK PURGUS: In WolfBrown’s work as evaluators, we often need to work directly with young people. I assist my team with obtaining the approval of an Institutional Review Board (IRB), which sets legal and ethical standards for human subjects research.
IRB approval is essential because it verifies that WolfBrown’s human subjects research protocols adhere to IRB guidelines to protect participants, obtain informed consent, and protect vulnerable populations.
TOM WOLF: This is an aspect of evaluation that clients may not always see. Before we can analyze data, there is often substantial work involved in determining how that data can be gathered responsibly in the first place. Good research requires us to think not only about what we want to know, but about whom we are asking, what we are asking of them, and our responsibilities to the people whose experiences become our data.
5. What else can data do?
Carnegie Hall’s Big Note, Little Note
DENNIE PALMER WOLF: It is easy to think of data as the measures and metrics that tell us “how big,” “how many,” or “how heavy.” When it comes to the world of evaluation, this often translates to the arts and culture equivalents of “how many”: How much did the audience expand? How many visitors return? How many participants report greater life satisfaction, or exhibit changed heart rates?
But this greatly undersells what data can do.
Consider a project we are working on with Carnegie Hall: Big Note, Little Note, a multi-week program for families with young children that draws on the Hall’s now-global Lullaby Project, using weekly music-making as the setting for developing intimacy and joy in the early years. Just last year, we began working to capture the impact of the program on its youngest participants. Using the DRDP, the Desired Results Developmental Profile, we were able to demonstrate the growth in children’s foundational language, cognition, and social skills.
Mission accomplished—well, not quite.
The very act of collecting data—if used well—can:
- Create opportunity: Younger staff took on data collection, reporting findings and sharing implications, becoming experts, not “just staff.” More than toddlers changed.
- Force insight: In collecting data, we saw what had been invisible: Parents were changing as much as children, initiating new kinds of play with their children and others, becoming a complex social community. Now we have new measures of growth.
- Drive partnership: In Phase 2, we are looking at segments of free play compared to segments of musical play. That puts us in deeper-than-ever conversations with teaching artists, asking them, “So what shifts between play and musical play?”
In other words, data done well is way more than measuring.
TOM WOLF: I think Dennie’s last observation captures something I see running through all of these projects.
We often begin an evaluation with a question we want to answer. But the process of answering it can do something else as well. Building a database can raise questions about institutional memory. Studying access can lead us to ask what happens after someone gets through the door. Trying to measure writing in the age of AI forces us to reconsider what we mean by learning. Observing young children can lead us to notice changes in their parents. And collecting data can create new expertise and new conversations within an organization.
Perhaps that is one of the more interesting things about this work: good research doesn’t always leave us simply with an answer. Sometimes it leaves us with a better question.
Those are some of the questions emerging from our work right now. What questions are emerging in yours?
