Newsletters > Immersed in The Future

WolfBrown’s new community of practice is designed to support nonprofit arts organizations as they navigate the complex programmatic space of augmented, immersive, and virtual experiences. We hope you find Immersed in the Future rewarding and invite you to join in!

Bridging Realities and the Future of Digital Arts Content

ISSUE 5 • May 2023

Bridging Realities and the Future “Depthcast” of Digital Performing Art Contents

By Marco E. Bruscoli, Scholar-in-Residence, WolfBrown

Coming out of the pandemic, nonprofit arts organizations are struggling to make sense of digital programming opportunities moving forward. Some have altogether abandoned digital initiatives undertaken during the pandemic. Others continue to search for a sustainable business model for digital content.

According to a June 2023 Audience Outlook Monitor survey of over 8,000 arts patrons across the US, approximately half say they have ‘little or no interest’ in watching ‘online programs offered by cultural organizations, such as live-streamed programs and video recordings of recent productions.’ Another 45% say they’ll selectively watch digital content if the spirit moves them. Just 5% say they actively seek out such content. Results do not vary by age.

Earlier research during the pandemic suggested that most people viewed digital arts content at home on laptops and other small devices with native audio systems (i.e., not a premium A/V experience). Other research suggests that less than 10% of arts audiences subscribe to any of the arts-specific streaming services such as BroadwayHD, Met Opera on Demand, medici.tv, IDAGIO, etc.

All we can conclude at this point is that the marketplace for filmed arts content for personal consumption, either live-streamed or recorded, has not congealed at any sort of scale. Those who continue to produce filmed content based on live performances face an uphill battle. They compete in a saturated market overflowing with commercially-funded audiovisual content and free content from nonprofits. Up against heavyweights like Netflix, Disney, and aggregators of arts-specific content, nonprofits often find themselves in a seemingly unwinnable situation.

Yet, change is afoot.

Paris Opera 3è Scene: “Clinamen” by Hugo Arcier © OnP / Les Films Pelléas

Trailblazing initiatives like the Met Opera’s streaming distribution and the Paris Opera’s 3rd Scene department are great examples of the drive towards digitalization. These pioneers deserve recognition for their forward-thinking investments. As the digital landscape evolves, however, so do user expectations. Digital programs that once seemed cutting-edge are starting to show their age.

In transforming the performing arts for the digital space, we’ve managed to broaden access. However, the quality and emotional depth we’re accustomed to in physical spaces doesn’t translate as well digitally. For example, think about the last time you streamed classical music. Did it provide an intense, uninterrupted emotional experience, or was it just background music? Were you listening intently, or dividing your attention between multiple activities? Unfortunately, the consumption of streamed content tends to be very passive and tends to allow for distraction. 

As we grapple with this challenge, two key questions arise for artistic institutions: How can we extend our digital offerings to provide a more enriching artistic experience? And, how can we strengthen our connection with digital natives?

To address these questions, we must consider the third evolution of the internet (Web 3.0). This evolution, currently underway, offers many paths towards elevated digital experiences. The digital world we live in is morphing into something more akin to the science fiction landscapes we’ve dreamed about – the “Metaverse” (a term borrowed from Neil Stephenson’s novel Snow Crash). Our virtual and physical lives are merging. According to a McKinsey study, “the Metaverse marks the transition from an attention-based internet economy to an experience-centered one.” In simpler terms, the internet is becoming a three-dimensional space, allowing for immersive experiences that go far beyond traditional streaming.

In simpler terms, the internet is evolving into a 3D domain, enabling immersive, multisensory experiences. Among many other things, these exceed the boundaries of conventional streaming. This evolution encompasses the full spectrum of extended reality (AR, MR, VR), and holds the promise of deepening audience engagement with digital content.

Streaming, broadcasting, webcasting, and all other methods of media distribution were not designed to hold or distribute  the massive amount of data required for deep immersive digital experiences. I like to envision the future of digital content as a complex transmission system, a ‘Depthcast.’ The future is synthesized through the integration and synchronization of various technologies currently on the market. These include spatial computing, haptic technologies, artificial intelligence (AI), and tokenomics. Combined with advanced methodologies like immersive storytelling, specific co-creational activities, and strategic user remuneration, these advances hold the promise of a richer sense of presence and more engaging interaction with digital art.

Imagine wearing a high-definition headset and finding yourself on stage during an opera. You would have the ability to choose your position on the stage enhancing the visual experience. Digital text inputs overlay the image, providing you with information regarding the plot and characters.

Or, envision the possibility of experiencing classical music through the eyes of a renowned conductor or a soloist or moving around a room following 3D sounds in different corners feeling on your body the vibration of the different instruments involved.

Or, picture yourself virtually navigating through an important art exhibition in a museum halfway around the world, examining artworks in breathtaking detail and interacting with docents or curators. Imagine running your hands over a sculpture through digital touch technology. These imaginative scenarios may seem unlikely, yet all the necessary “ingredients” are already in place, waiting to be fused together. (If you want a taste of what the future might look like, I recommend trying the Bigscreen application on MetaQuest 2. For the first time, I felt the vivid sensation of being in a movie theater, and my level of attention towards the film dramatically increased.)

Apple’s recent introduction of the Vision Pro shows us the evolution  of the user experience within the arts and entertainment. This XR interactive headset is set to further blur the line between the physical and digital environments. The headset liberates users from traditional interfaces. It also creates a multitude of opportunities for artists and producers to showcase their work in hyper-realistic 3D environments. Widespread adoption of this technology is still uncertain. Still, it clearly surpasses previous Oculus technologies offered by companies such as Meta, HTC, and Magic Leap. The high price point of the Vision Pro headset, at $3,499, is a major hurdle for now, especially with cheaper competitors on the market. However, less expensive models will almost certainly be available over the coming years.  

Image of Hong Kong Philharmonic Metaverse concert  showing immersive digital performing arts content possibilities.
Ka Lam Hong Kong Philharmonic Metaverse Concert

As with all new technologies widespread public adoption isn’t instantaneous, over the past year, the Metaverse (or whatever you want to call it) in the art field, has experienced slow growth, and not without its share of speculation and hype. At times, this has led to misplaced efforts and missed targets, like the underwhelming Hong Kong Philharmonic Metaverse Concert. But other institutions such as the Helsinki Opera Ballet or the Royal Shakespeare Company have been quick to explore the digital frontier, using the lockdown period to scratch the surface of the Metaverse, and now finding themselves in a leading position.

RSC website screenshot placed in computer clipart showing immersive digital performing arts content possibilities.
Stuart Martin/RSC/PA EM Williams and her onscreen Puck avatar in the Royal Shakespeare Company’s Dream

Live, IRL artistic experiences have always held transformative power, creating transcendent moments of connection through the strength of performance. As we continue to meld our physical and digital realities, Web 3.0 harbors real potential to democratize these moments. As this unified system of technologies, content, and social norms continues to evolve, performing arts institutions capable of conceiving and creating immersive experiences and spaces may revolutionize the digital experience, finally doing justice to the complexity of the artistic product. The Metaverse, extending beyond gaming and entertainment, represents the next step in the evolving relationship between humanity and culture. As we forge ahead, the challenge for us all is to pinpoint the right opportunities and the appropriate moment to embrace this – inevitable – digital transformation.

Projects We're Following

Highlights of immersive experiences that caught our eye

COSM image screenshot

Image: COSM

Shared Reality at Cosm

Cosm, a leading immersive tech and entertainment firm, collaborates with acclaimed new media artist Nancy Baker Cahill to debut an art experience in “Shared Reality,” blending virtual and physical realms. The immersive visual narrative piece, SEEK, will premiere at Cosm’s new immersive entertainment venue in Los Angeles in Spring 2024, featuring a 26.6-meter LED dome. Baker Cahill’s SEEK uses camouflage-themed visuals and sounds to explore abstract landscapes, inviting guests to perceive interconnected worlds. This pioneering partnership underscores Cosm’s commitment to pioneering immersive storytelling.

Sonic Sphere website screenshot placed in computer clipart

Image: The Shed

Sonic Sphere at The Shed

The Sonic Sphere is a remarkable development in the world of immersive sound experiences. Suspended in the 115-foot-tall space of The Shed, this spherical concert hall is truly an embodiment of groundbreaking musical and architectural synthesis. Drawing inspiration from Karlheinz Stockhausen’s pioneering ideas for the Kugel auditorium, the Sonic Sphere is designed to create an extraordinary sound experience for audiences.

dragon image from website screenshot placed in computer clipart

Image: Harry Potter Immersive

Immersive Harry Potter In NYC

A new Harry Potter experience in NYC offers an immersive journey into the wizarding world, transcending traditional displays of props and costumes. The exhibit incorporates dramatic lighting, set design, interactive technology, and unique scents to create authentic settings like Hagrid’s hut and the Great Hall. This exhibition by Imagine Exhibitions aims to transport visitors into the heart of Hogwarts, showcasing treasured moments from the films in a truly immersive and personalized manner.

immersive experience website screenshot placed in computer clipart

Image: The LUME

Connection at THE LUME Melbourne

“Connection” is an immersive experience crafted for THE LUME Melbourne by Grande Experiences with the engagement and blessing of both Aboriginal Melbourne and Wurundjeri country elders. The exhibition explores human relationships through light, sound, and technology. Segmented into four sections: Connection, Communication, Creativity, and Community, the goal is to encourage visitors to introspect on their methods of connecting with each other and the world, fostering a deeper understanding of their relationships.

A deep dive into the latest coverage of immersive art performance

Image of woman listening happily to music

Image: Wikimedia Commons

Neuroforecasting will be The Future of Hit Song Prediction

Researchers at Claremont Graduate University have developed a method using neurophysiological data and machine learning, achieving 97% accuracy in predicting hit songs. This new approach, known as neuroforecasting, could revolutionize entertainment content selection.

Adobe AI image website screenshot placed in computer clipart

Image: Violeta Ayala

Reclaiming Ancestry through AI

The Awichas project uses AI tools to reimagine the artist’s unknown female ancestors, challenging colonial visual narratives to produce dignified representations of Quechua elder women. An evolving NFT collection, it illustrates post-photography’s creative potential through AI’s unique image-processing capabilities.

NPR and Grimes website screenshot placed in computer clipart

Image: Theo Wargo/Getty Images

Grimes Offers AI Voice for Music Creations

Musician Grimes has offered AI-generated versions of her voice to creators for music-making, pledging to share 50% royalties on successful songs. She seeks to navigate the uncharted territory of AI’s use in music and copyright laws.

Immersive Tech Spotlight

Innovative developments in immersive technology

Verge website with AI generated image screenshot placed in computer clipart

Image: The Verge

Adobe Photoshop Firefly's AI Text Prompts

Adobe Photoshop introduces a new AI tool called Generative Fill, utilizing Adobe’s Firefly AI image generator. This innovative feature allows users to modify images extensively by extending, adding or removing elements using text prompts.

NerF website screenshot placed in computer clipart

Image: NeRF

NeRF

NeRF (Neural Radiance Fields) uses neural networks to transform 2D images into 3D models. Its applications in the performing arts could include creating detailed virtual sets, enhancing visual effects, and improving rehearsal techniques.

SHARE