insights bar

Responsible Demographic Research

In WolfBrown’s Intrinsic Impact program we often talk about measuring what matters. We use impact indicators to align our surveying to arts organizations’ missions and our clients have affirmed the value of research that reflects their goals. There’s a flip side to measuring what matters – we don’t measure what doesn’t matter. As researchers, this serves our goals of keeping protocols short and analysis focused. When it comes to demographic questions, this principle is especially important.

Perhaps more than ever, demographic surveying is on the minds of many Americans. Next month the Supreme Court will address the constitutionality of demographic surveying when they hear challenges to including a citizenship question on the U.S. census. The same consideration is necessary when we design our survey questions: will members of certain demographic groups be less likely to answer this question and could it prevent them from finishing the survey? How will this bias our understanding of our audience? Collecting demographic information is essential to addressing the issues of equity, diversity, and inclusion that many organizations are grappling with. These questions are inherently personal though, and we need to address three concerns before we ask them:

1. Is there a use for the data the demographic question provides?

When surveying large groups, any personal question is liable to be sensitive to someone and should only be included with reason. For example, many arts organizations do not have initiatives that address gender. For them, it’s not only unnecessary to ask for the gender of survey respondents, but including this question often risks offending some segment of the audience. Not including space for non-binary respondents excludes a systemically oppressed group, but even including an “other” option can agitate respondents who believe in gender binarism. Either way, respondents may skip the question or not complete the survey.

2. Is the demographic question comprehensive of the identities it queries?

When a demographic question has been determined to be relevant, we still need to be conscious of how we ask it. Many surveys look for simplicity in analysis at the expense of acknowledging the intersectional realities of identity. Single-select questions are particularly in danger of doing this. The fastest growing racial demographic in the U.S. is multiracial individuals, yet many surveys erase their identity by requiring that respondents select one racial or ethnic identity without offering a write-in option. When we insist that respondents fit in a box, we dismiss their identity and damage the integrity of our answers. There are industry pressures for mutually exclusive demographic categories, so this conversation also needs to happen with stakeholders, funders, and service organizations.

3. What does the way demographic questions are asked communicate about the organization?

Demographic questions show how an organization views identity. In designing these questions an organization has an opportunity to practice cultural competence. For example, organizations can design a process that gives members of communities who are systemically excluded input in how the questions are asked. It’s not just about inclusive wording. The protocol format can also say volumes. When collecting data for aggregation why not be clear when it’s anonymous? If it’s not anonymous, can we communicate how we’ll be using it? When making demographic questions requirements for completing a survey, we need to consider if it tells people who are uncomfortable giving those answers their responses aren’t welcome.

Not addressing these concerns can damage the reliability of survey data and research. Questions sometimes need to be asked in ways that aren’t ideal, but consciously communicating why can mitigate these negative effects. Particularly when we survey about identities that are used as a basis of hate, discrimination, and oppression, we have a responsibility to ask questions sensitively, respectfully, and with purpose. If we do not, we risk invalidating the very communities we wish to serve.

Both I and WolfBrown are invested in inclusive questioning and hope to write more about it in the future. I invite our readers to share their own challenges or successes with us in the comments or by emailing me directly at alan.kline@wolfbrown.com.

This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*


× 5 = 35

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>