How Freckled Sky invented and implemented jaw-dropping XR live interactive virtual performance
Freckled Sky gained notoriety as the first to create a show where live performers interact with holographic projection onto a water screen. The show aired live on America’s Got Talent at NBC in 2015 and gained the team popularity and favorable media coverage.
Being innovators and dreamers, a mix of artistic creators and technology experts, the team develops multimedia shows and interactive experiences, video content, and holographic effects within live concerts, tours, award shows, corporate events, theme parks, museums, trade shows, product launches, advertising. Freckled Sky has deep expertise in concept development, creative direction, lighting design, video production, and more.
Over and above the United States, Freckled Sky has produced and held shows across 4 continents: UK, Italy, France, Switzerland, Germany, Ukraine, Panama, UAE, India, Turkey, Egypt .
The company’s portfolio includes one-of-a-kind art for shows of varying format and scale including: P!nk's performance on Brit Awards 2019; Paula Abdul's North American tour 2018; Lil Yachty's underground concert in Atlanta; The first multimedia musical "Something Wicked This Way Come"; Ferrari launch roadshow in Europe; The Panama Canal opening ceremony, and many other Fortune 500 brands.
For America's Got Talent (season 10) auditions in 2015, the Freckled Sky team cast two dancers who performed in the artificial rain generated by 20 feet (6 m) water-screen. The audience and judges were stunned by the world's first interactive performance of this kind. In February 2019, Freckled Sky produced an interactive water projection performance for the Try portion of P!nk's mash-up at the 39th edition of Brit Awards, leaving "the audience in awe." The performance was hailed as one of the main highlights of the awards show.
Challenge
In uncertain times of global lockdown, the development of XR technologies in the live entertainment industry significantly influenced the market's initiative to test this solution at live concerts and performances. New technologies and unfinished workflows demonstrated technical challenges tough to tackle by most market professionals.
Solution
Remaining pioneers and experimenters, the idea of creating such production in Ukraine was born from the American multimedia show production company Freckled Sky together with the Ukrainian directors group TRI Direction, who have been working together for several years and creating unique entertainment projects around the world: multimedia shows for P!nk, James Arthur, Paula Abdul, Lil Yacty.
The team was ready to shoot a Live XR interactive virtual performance for Ingret with the song “Save My Planet“. Filming in this format at that time was realized for the first time in Ukraine and for the second time in the world. The first project of this format was Katy Perry's musical performance on American Idol.
To implement such a format Freckled Sky had to find a virtual studio with facilities at a total investment of about $500,000. It turned out that just at that time, a studio Platforma was opened in Kyiv, Ukraine, almost wholly built by Ukrainian technologies. Next, Freckled Sky needed specialists in virtual production. It turned out that in Ukraine, there are not just specialists but super-professionals – Pixela Labs, who invents technologies for the world leaders of the entertainment industry Epic Games. Without them, it simply would not have been possible to develop the last Disney prequel to Star Wars “The Mandalorian”, entirely created using virtual production technology. They also participated in the creation of the penultimate tour of Childish Gambino. The last missing link in this chain was OpenSET, which provided AR, live tracking, and Face MoCap. The leading Ukrainian production companies also supported the project: Flytek, Alight, Operatorskie technologii, To Be Production.To implement such a format Freckled Sky had to find a virtual studio with facilities at a total investment of about $500,000. It turned out that just at that time, a studio Platforma was opened in Kyiv, Ukraine, almost wholly built by Ukrainian technologies. Next, Freckled Sky needed specialists in virtual production. It turned out that in Ukraine, there are not just specialists but super-professionals – Pixela Labs, who invents technologies for the world leaders of the entertainment industry Epic Games. Without them, it simply would not have been possible to develop the last Disney prequel to Star Wars “The Mandalorian”, entirely created using virtual production technology. They also participated in the creation of the penultimate tour of Childish Gambino. The last missing link in this chain was OpenSET, which provided AR, live tracking, and Face MoCap. The leading Ukrainian production companies also supported the project: Flytek, Alight, Operatorskie technologii, To Be Production.
Among other things, the beauty of the sound of "Save My Planet" has a deep social meaning. For the creative implementation of the song idea and a vivid demonstration of the technology, the director's team proposed the artist's immersion and interaction with various environments: desert, space, underwater world…
Another unusual feature was the integrating the viewers directly inside the virtual environments - creating interactivity with viewers on a new level.
Results
Freckled Sky presented a unique project - Live XR Interactive Virtual Performance “Save My Planet” creating the virtual world around the artist INGRET. It’s a 1000-cubic-feet eXtended Reality environment and the second-in-the-world live interactive virtual performance using Mixed Reality Technology.
The effect is as grand as an ocean amidst the total immersive LED Screen experience with AR and life-tracking technologies. Thanks to the innovative solutions the project helps to dive into people’s emotions that are deeper than the seas of space, enabling the audience’s real-time “touch” with the artist and their connection across a long distance to reach, fascinate, and interact on a new level.
Talented and professional Freckled Sky’s team - artists, directors, designers and engineers - continues to develop virtual production in the world impressing audiences with the "mind-boggling, nothing before ever seen" shows. They already filmed 2 large-scale projects on the biggest XR studio in Las Vegas, USA in 2020-2021.
Contact: FRECKLED SKY hello@freckledsky.com
The project “Boosting local traditional industries with IT capacities” is implementing by TechUkraine in partnership with APPAU – Association of Industrial Automation of Ukraine, International Association Culture&Creativity Association with the financial support of the German government through GIZ that aims to increase the competitiveness of traditional industries in strong collaboration with Ukrainian IT companies.