A National Collaborative Evaluation of Sistema-inspired Music Education

Funded by the Buck Family Foundation and the Andrew W. Mellon Foundation

systema2
Photos courtesy of Play on Philly! and Krull Photography

Contents

 

The Study: Evaluating the Effects of Sistema-Inspired Music Programs in United States Communities

A Joint Study by The Longy School of Music of Bard College & WolfBrown
Made possible by funding from The Buck Family Foundation and the Andrew W. Mellon Foundation


Overview of the Study

The past decade has witnessed a rapid proliferation of El Sistema-inspired music programs for children throughout the United States. This is not surprising, given the strong social mission of El Sistema and the musical and social effects reported from the program in Venezuela, its home country. Based on those reports as well as on preliminary findings from U.S.-based programs, there is much hope that El Sistema-inspired programs can build on the aspirations and assets of children and families, buffer the effects of risks like poverty, unsafe neighborhoods, and unequal opportunities to learn and develop, and build a wider understanding of the role that music can play in individual and community development.

But can this promise be realized? To do so requires that we build a compelling body of evidence about the quality of programs, the range of developmental outcomes they can foster, and the long-term outcomes that can be expected from different El Sistema-inspired models. This work is vital -- if the claims made for these programs outpace supporting evidence, the El Sistema movement risks losing credibility and, ultimately, the engagement of families and the support of communities. Recognizing this challenge, the Buck Family Foundation wisely called for a national and systemcatic study of the benefits to children associated with participation in El Sistema-inspired programs.

In response to this request for proposals, Longy School of Music of Bard College and the arts research firm WolfBrown formed a strategic partnership to carry out a two-year study of the benefits of El Sistema-inspired programs for the children they serve. The study was further supported by a grant from the Andrew W. Mellon Foundation, who is especially interested in El Sistema-inspired programs that are associated with professional and youth orchestras. The research team invited proposals from programs across the country and formed a consortium of sites that represent a range of geographical and program structure diversity. Over the first year (2014-15) of intensive planning, a core set of common measures was collaboratively designed and piloted. In 2015-16, we gathered our first set of yearlong data using these measures, and worked with sites to design additional measures tailored to their specific programs. In 2016-17, the team will work collaboratively to collect and analyze a second full year of data and share findings widely in the field of music education. Results will be shared publicly first within the U.S. El Sistema community at the August 2017 Take A Stand conference in Los Angeles, CA. The common measures will be made widely available to any El Sistema-inspired site who wishes to use them.


The Study

Rationale.
 The achievement gap has been an area of recognized policy importance in the United States since President Johnson's Great Society initiative. More recently scholars and policy-makers have begun to direct their attention to the opportunity gap as a catalyst for the achievement gap. One aspect of the opportunity gap is limited access to a "well-rounded education," which includes access to an education in the arts (Every Student Succeeds Act, 2015). Less affluent students are more likely to attend schools where the positions of full-time arts educators have been eliminated (Parsad & Spiegelman, 2012) and whose families are least likely to have the resources to pay for out-of-school arts instruction (Duncan & Murnane, 2015).

These facts are particularly troubling in light of correlational evidence that arts education in general (Catterall et al., 2012) and music education in particular (Cabanac et al., 2013; Gouzouasis et al., 2007; Vaughn, 2000) may benefit academic achievement. Among young children (ages 4 to 9 years), small-scale randomized-control studies have demonstrated that brief courses of music education can lead to increases in IQ (Kaviani et al., 2014; Portowitz et al., 2009; Schellenberg, 2004) though as others have noted (Schellenberg & Manakarious, 2012) these studies cannot speak to the effects of musical study that occurs over a period of years.

Moreover, while a number of researchers have argued that music education may catalyze neural plasticity (Schlaug, 2015), relatively few studies have examined the specific mechanisms through which music education might bolster academic achievement. There is some evidence that music education may foster aspects of socioemotional development that predict academic achievement, including school engagement (Eerola & Eerola, 2014) and persistence (Smith, 1992). However, evidence regarding music education's effects on other socioemotional domains is either mixed, as in the case of self-concept (Dege et al., 2014; Portowitz et al., 2009), or absent, as in the cases of goal orientation and growth mindset.

Our study focuses on how music education may foster change in a set of socioemotional domains – school engagement, persistence, achievement motivation (self-concept and goal orientation) and growth mindset – and how this change may drive improvements in students' academic achievement. In 2015-16 we collected data on socioemotional development and academic achievement from 280 students attending 10 El Sistema-inspired programs across the country and 225 comparison-group students matched by grade and school.[1] These programs feature high but variables levels of 'dosage,' ranging from 1 to 5 days of instruction per week throughout the academic year. Once data analysis is complete we will be able to assess whether rates of change in academic achievement and multiple potential mediators differ as a function of program enrollment. If positive rates of change for enrolled students exceed those for unenrolled students it would constitute promising initial evidence for the capacity of music education to benefit developmental domains that are known to support academic achievement. Because all of the programs included in our study disproportionately serve low-income students of color, the findings from our study will yield results that are generalizable to those students at the heart of the debate around inequality of opportunity, filling a gap in a research literature where most studies have been conducted with samples of majority-Caucasian middle- to high-income children.

Collaboratively with the core sites, we have also created a performance measure that looks at student growth in musical skills such as accurate playing of pitches and rhythms, phrasing and articulation, and tone and intonation.  Many sites are using these measures in tandem with jury processes they already employ – or in some cases, this measure is jump-starting a jury system.

While individual El Sistema-inspired sites have embarked upon various of their own research studies – to date – this is the first national cross-section of sample students using common measures.

Research Questions. 

The Study addresses the following questions:
  • What skills and abilities does participation in El Sistema-inspired programs build in participants that could lead to their thriving as people, learners, members of they community, and possibly as musicians ? (How might the orchestra-driven development of interpersonal or intrapersonal skills (focus, persistence, growth mindset, etc).,, lead to thriving in other domains?) 
  • How is the acquisition of these skills and abilities affected by program design (dosage, duration, quality)?
  • How does this process work for different children (of different ages, with different histories, in different schools and communities?)

[1] El Sistema is the Venezuelan program of music education that produced, among others, Gustavo Dudamel, Music Director of the Los Angeles Philharmonic.


Different Modes of Participation 

We work with a core set of approximately 10 sites that represent a range of program designs and locations. At the same time, we recognize the growing number of El Sistema-inspired programs with an interest in participating and experience to contribute. For that reason, we have established three different modes of participation:

Different Modes of Participation Brief Description
Core Sites  (approximately 10)
  Shared training on and discussion of common measures
Guidance in using common measures
Support to develop site-specific measures
Data analysis and reporting
Shared resources (e.g., evaluation strategies)
Collaborating Sites
  Common measures and guidance in using them
Shared resources (e.g., evaluation strategies)
Companion Sites
  Access to the project’s shared resources (tools, measures, website)


List of Core Sites

Conservatory Lab Charter School

Dorchester, MA

 

Incredible Children's Art Network (iCAN) (2014-16 only)

Santa Barbara, CA

 

Kalamazoo Kids In Tune

Kalamazoo, MI

 

KidzNotes

Durham, NC

 

New Jersey Symphony Orchestra – NJSO Champs

Newark, NJ

 

OrchKids – Balitmore Symphony Orchestra

Baltimore, MD

 

The People's Music School – Youth Orchestras

Chicago, IL

 

Play On Philly

Philadelphia, PA

 

Reno Philharmonic

Reno, NV

 

San Diego Youth Symphony Orchestra – Community Opus Project

San Diego, CA

 

Yakima Music en Accion (adding for 2016-17)

Yakima, WA


More about the Funders

The Buck Family Foundation is funding the pilot year and two-years of the study. Their investment grows out of a deep interest in music and music education. They see the focus on El Sistema-inspired programs as a way to make a field-wide contribution to understanding the outcomes of intensive and sustained involvement in music.

The Andrew W. Mellon Foundation contributed additional funds to support the study at sites connected to orchestral programs, and to include graduate students as research fellows and assistants to the project – thus developing a next generation of leaders in the field.



More about the Research Team

The evaluation study is a strategic partnership between the
 Longy School of Music of Bard College and WolfBrown an international arts consulting firm.

Longy School of Music of Bard College
 
Longy School of Music of Bard College is a small, world-class conservatory that is undergoing a process of growth and radical change unprecedented in its 98-year history. The merger with Bard, finalized in 2012, has enabled Longy to expand its educational offerings, community engagement and research agenda and to access musicians, scholars, philanthropists and world leaders around the country and around the globe. In 2009, Longy's Board of Governors made the strategic determination that its Conservatory program would drive the mission and focus of the School going forward. That mission is defined as empowering students to use music to make a difference in the world. Longy teaches its students to pursue a life time of community engagement wherever their musical paths take them. A key component of the curriculum is Longy's Teaching Artist Program. Through this required program, students lead innovative music projects while gaining valuable entrepreneurial and teaching experience in schools, community centers, prisons, hospitals and elderly care centers, as well as Sistema-inspired programs in the Boston area (See http://www.longy.edu/conservatory/cons_home.htm.) These values have led to a deep interest in and engagement with the leaders of FundaMusical, the umbrella organization led by Maestro Jose Antonio Abreu in Caracas, Venezuela.

Longy has actively supported the growth of Sistema nucleos around the country.  In 2012, the school formed a three-way partnership with the Los Angeles Philharmonic and Bard College called Take a Stand.  The first outcome of this partnership was the launching, in January, 2013, of a Masters of Arts in Teaching in Music (MAT), the only degree program in the world to offer graduate-level training in the principles of Sistema to musicians with the desire to respond to today's unique educational needs and participate in the growing Sistema movement in the United States.
  
WolfBrown

WolfBrown is an international consulting group focused on knowledge building in the fields of arts and culture with a particular emphasis on collaborative projects that build the unique strengths of cultural organizations and help them report on their public value. In the field of music and music education, the firm has worked with major and community-based organizations including symphony orchestras, community schools of music, national service organizations, school systems and city governments. Drawing on the experience of two researchers, Dennie Palmer Wolf and Steven Holochwost, the firm specializes in research and evaluation focused on youth and families, with a particular emphasis on the effects of high quality, long-term participation in arts and cultural education. Many of their evaluations underscore the role that organizations, their staff, as well as youth and family participants, can play in designing, collecting, and reflecting on research findings. In recent years, Wolf and Holochwost have worked with a number of music-focused organizations (Carnegie Hall, From the Top, Community Musicworks, and three nucleos: Play on Philly, YOSAL, and the New Jersey Symphony Orchestra). Throughout this work, they have tailored existing and developed original qualitative and quantitative measures designed to capture the consequences of intensive musical experiences. Throughout this work, there is a strong commitment to seeing youth and families as the seedbed for cultural vibrancy.

The Team

Dennie Wolf holds an Ed.D from Harvard University and has taught at Harvard Graduate School of Education and Brown University's Annenberg Institute. Dr. Wolf has grants from the Barr, Carnegie, and Spencer Foundations, as well as the National Endowment for the Arts. Wolf has collaborated with arts educators in communities as diverse as Boston, Chicago, Dallas, the state of Maine, the region surrounding Portland, OR on building city-wide and regional systems of arts learning. She has published extensively in the fields of symbolic and artistic development, arts education, and assessment in the arts and humanities. She is the 2013 national service to the field award winner from the National Guild of Community Arts Education.

Steven Holochwost holds Ph.D. degrees in Developmental Psychology from the University of North Carolina Chapel Hill and Music Composition and Theory from Rutgers University, where he studied with Charles Wuorinen.  He has received awards and fellowships from the National Science Foundation, the American Psychological Association, the Society for Research in Child Development, the American Society of Composers, Authors, and Publishers (ASCAP) and the Renee B. Fisher Foundation. Currently, the team is working on evaluations of three distinctive Sistema-inspired nucleos: an emerging program at the New Jersey Symphony, Youth Orchestra Salinas, and an intensive program in West Philadelphia, "Play on Philly."

Judith Bose holds a Ph.D. in Urban Education from the City University of New York Graduate Center and an MM in Vocal Performance from the New England Conservatory of Music. As the Director of Teacher Education and Educational Initiatives at Longy School of Music of Bard College she directed the innovative Teaching Artist Program in Cambridge and was the central architect of the curriculum for the MAT in Music at the Los Angeles campus. Having visited El Sistema sites in Venezuela, she is an active collaborator in the  Take a A Stand  partnership and professional development efforts, as well as served as a lead consultant with WNET and the Annenberg Foundation for an online professional development workshop series on El Sistema-inspired music education.


Archive of Newsletters

• Newsletter No. 10: March 2016
• Newsletter No. 9: January 2016
• Newsletter No. 8: November 2015
• Newsletter No. 7: October 2015
• Newsletter No. 6: September 2015
• Newsletter No. 5: June 2015
• Newsletter No. 4: April 2015
• Newsletter No. 3: March 2015
• Newsletter No. 2: January 2015


Other Downloads

• Advocacy Flyer (for sites to use with boards, in grant proposals, etc.) - November 2015 (.pdf)
Grant Announcement from the Andrew W. Mellon Foundation - May 27, 2014 (.pdf)
• Bard College Institutional Review Board Approval - October 24, 2014 (.pdf) 
• Consent/Assent Forms - English (.pdf) 
• Consent/Assent Forms - Spanish (.pdf)
• Headline Findings from the U.S. National El Sistema Study (October 17, 2016)