This piece is part of a larger series examining more equitable practices in arts research and consulting. Our goal is to reflect openly and self-critically on the underlying processes, assumptions, and structures driving inequity and what would make the arts and culture sector more equitable.
“I want to thank all of the staff, coaches, and conductors of the ensembles who helped and believed in me and who enabled me to progress all the way from Training Ensemble to first violin of Academy Orchestra. I want to thank my fellow members of the orchestra who have made me feel like I belonged, inspired me, and helped me to grow.“– Gael B.1
When considering what outcomes are worthy of research for arts education programs that work with diverse populations of students, we as administrators must weigh both programmatic and pragmatic considerations. Pursuing research that will support and make a case for mission-driven programs – and evaluate outcomes for participants regardless of background – has implications for the broader field.
For the New Jersey Symphony, the research question has been inextricably linked to the evolution of our Saturday youth orchestra program from a single ensemble to a multi-level pathway for young musicians. As the program evolved, so too did our evaluation efforts – in collaboration with WolfBrown, our research partner since 2013. This post will examine this evolution, and the ultimate outcomes we investigated: musical growth, leadership skills, and social-emotional skills.
First piloted in 2015, the Training Ensemble of the Youth Orchestra represented a concerted effort by the New Jersey Symphony to remove barriers to participation for children in communities in and around Newark, New Jersey’s vibrant, largest city. Adapting many of the best practices from our then-concurrent after-school CHAMPS (Character, Achievement and Music Project)– which sought to harness the power of music to enhance students’ lives by fostering social and academic success – the Training Ensemble was imagined as an on-ramp for students from Greater Newark to the advanced ensembles of the Youth Orchestra.2 For all participants, a nominal registration fee would keep the opportunity affordable for families, and students without access to an instrument could rent one at a highly reduced rate.
The current program model supports intensive, curriculum-driven instruction, including group classes in technique, improvisation, theory and rhythm, ensemble sectional rehearsals, and semi-private lessons. Faculty include professional Teaching Artists and New Jersey Symphony musician-coaches. A peer mentor program encourages advanced students from the upper-level ensembles to assist, support, and inspire their younger, less experienced colleagues. Public performances provide regular opportunities to celebrate personal and collective excellence, sharing in the joy of ensemble music-making and the individual sense of achievement. Throughout the season, the student experience is elevated with complimentary tickets to New Jersey Symphony mainstage concerts.
The Training Ensemble – and indeed the Youth Orchestra program as a whole – shares many of the values of the El Sistema movement. Equity, excellence, impact, and community are core tenets. With increased and more equitable access to high-quality musical training, the Symphony has been serving children from increasingly diverse backgrounds.
Towards a Theory of Change
“My experience at New Jersey Symphony Youth Orchestra was filled with unexpected but welcome surprises – from exciting practices with wonderful friends I made along the way to learning how to compose entire scores from just three notes! I have learned so much about the magic of music during my time here and can’t wait to dive deeper in my future education.“Ashita B.
In WolfBrown’s retrospective evaluation of CHAMPS in 2019, it became clear that the next step for the Symphony – the expanded Training Ensemble – necessitated an approach from the perspective of design thinking. One powerful benefit of our continued research relationship was being able to further investigate this thinking in real-time as part of the program design and development.
Through ongoing discussions among researchers, staff, and artistic leadership, we agreed that musical growth, leadership skills, and social-emotional skills were the likely and measurable outcomes, based on learning from CHAMPS evaluations and observations from the early years of the Training Ensemble.
Therefore, our theory of change posits that by expanding access to high-quality musical training, the Training Ensemble promotes students’ musical advancement and achievement, as well as their development as people. From the theory of change, WolfBrown researchers proposed a schedule of evaluations to measure growth towards the ultimate outcomes. This included two established research tools (Dweck Mindset Instrument and Sense of Community Index) and a customized set of measurement tools — namely, writing prompts for students, and a growth assessment for each student to be completed by teaching artists.
We piloted these new tools in 2022-23 to assess students’ musical growth, ensemble behaviors, leadership skills, and social-emotional skills from the perspectives of both the students and the teaching artists.
From the short written responses to the prompts, the vast majority (81%) of students said that they would feel comfortable asking for help with an interpersonal problem at the program, and they knew who they would ask. This is indicative of a caring environment, which is essential for openness to learning and creativity.
Nearly all students (96%) were able to articulate what experiences in the program helped them improve their rhythm skills and/or in what specific ways their rhythmic skills improved; this demonstrates self-awareness, a key social-emotional skill.
Teaching artists were surveyed on how each student improved over the semester in four areas: technique & musicianship, ensemble behaviors, leadership, and social & emotional skills. Overall, the teaching artists indicated student growth in all categories, particularly in intonation (ability to play in tune), musical accuracy, and ability to integrate criticism.
Looking Back, Looking Forward
“The New Jersey Symphony Youth Orchestra has brought a lot of happiness and wonderful experiences. Playing classical music has always been enjoyable, but playing and connecting with others through music has made the experience much more amazing and special. I hope that future students here can experience the magic I did all these years.“– Caroleena F.-C.
The Training Ensemble – like CHAMPS before it – examines outcomes that are focused on individual growth. Our evaluation efforts continue to show that these outcomes are worthy of research because they help us engage and retain students from a diverse range of backgrounds.
Ultimately, we are optimistic that a program that is arts-centered and student-focused makes possible a supportive community where creativity thrives, musical traditions from all over the world are celebrated, and respect and understanding are nurtured.
Moreover, as children progress through the program, we encourage them to chart their own paths as future musicians, future leaders, future arts patrons, and future philanthropists. The implications range from the future of the arts workforce to the cultivation of audiences who are critical to the survival of the orchestral field.
“Since 5th grade, the New Jersey Symphony Youth Orchestra has given me so many opportunities to be inspired by music and love what I was capable of doing. I am who I am today because I was pushed to follow my heart and never accept any limits. The experience I had with Youth Orchestra is something I will carry with me for the rest of my life.“– Marisa O.
1 In their own words: Gael, Ashita, Caroleena, and Marisa began in the Training Ensemble in the first two seasons (2015/2016) and have since graduated from the upper levels of the Youth Orchestra in June 2023.
2 CHAMPS, which served students at two Newark charter schools between 2013-2019, was fully absorbed into the Training Ensemble in fall 2019 to serve more students and create an even more robust and sustainable support system for them. CHAMPS was annually evaluated by WolfBrown researchers, and also participated in the two-year study of the impact of El Sistema-inspired programs.
This is piece is part of our series on equity in the arts. You can read other posts here:
Researcher-Practitioner Partnerships in the Arts: Dr. Steven Holochwost, a WolfBrown Principal, writes about his ten year journey evaluating the New Jersey Symphony’s Character, Achievement, and Music Project (CHAMPS). He reflects on the nature of their research-practitioner partnership and insights that to a new program design.
Access to Evaluation Services – Finally, our colleague and collaborator Allison Russo shares how close to 100 arts education organizations in Newark are working together to gain access to quality evaluation services.