Prediction XR – False Negative or the Computer Says Nah
an interactive WebGL experience using face and voice recognition (2023)
In a digitally driven society, individuals can create virtual twins to navigate the metaverse, but the price is surrendering their biometric data. As society craves order and balance, a scoring system determines an individual’s daily risk level based on facial expression and voice, impacting their digital twin’s life. In this high-stakes world, hope lies in the promise of a better tomorrow.
Prediction XR is a WebGL interactive experience, which discloses scoring and risk assessment systems through face and voice recognition in a playful, yet realistic way. These scoring systems serve as the basis for future behaviour prediction, which is based on datafication, abstraction, classification and patterning which disqualify or omit certain data, and thus give rise to error and bias. “How data are conceived, measured and employed actively frames their nature” (Kitchin & Lauriault, 2014, p. 4). What happens if our datafied selves become real and influence our future? What and who gets to be perceived and what and who gets to be silenced? This world’s centre and its margins are inverted. It is a world where the “noise” becomes the “signal” (Steyerl 2017), and your anonymized data will transform into particles forming patterns in a collaborative, open ever-changing audiovisual artwork. Welcome to a world where data is never raw, and art is always political.
Exhibitions: HASTAC 2023, Pratt Institute New York City, USA
Created by: Michaela Pnacek(ova), Anna Leschanowsky
Jam XR
a VR experience (2023)
The purpose of this project is to introduce extended reality (XR) practices for music group improvisation to a selected group of students from Community Music Schools of Toronto (CMST), and to collaborate on a multiplayer virtual reality(VR) prototype with the students as co-creators. Our project allows for participatory, collaborative, and co-creative interactions that are vital to developing skills in music creation and learning music concepts. Our main intention is to increase digital literacy and accessibility of emerging media and music for youth aged 10-17 years old in Canada, and add virtual reality to the music curriculum at the Community Music Schools of Toronto
Funded by:
The Helen Carswell Award 2022
Created by: students of the CMST at Jane and Finch, Toronto
Design and Development: Aida Khorsandi, Michael Palumbo, Michaela Pnacek
VoicePop
an interactive voice prototype in Unreal game engine with Whisper AI (2023)
Voice recognition is widely used for voice-based biometric identification, assessment and
diagnosis in healthcare, and well-being. Our project aims to address the rarely investigated bias
in these technologies. We would like to close this gap by collecting a diverse dataset open to
different stakeholders in the field (researchers, start-ups, deployers, vendors and clinicians).
In order to spread the word about the B2AI project, and the ethical issues of voice AI, we are in the process of creating an interactive application for web-browsers, through which we will
interrogate ethical principles of responsible design with voice AI.
Created by: Michaela Pnacek
Presented at: Voice AI Stakeholder Forum 2023, Washington DC, USA
Emerging From the Water: Resonance
a VR experience (2023) and a dome experience (2022)
This video and exhibited a vr work challenges reductive western conceptions of ‘modern water’ through Anishinaabe ontology. It was presented at the UN Conference on Water as part of a York Delegation to the UN, which hosted a side event at the conference titled “Water security, disasters, and resilience in a changing climate: challenges, opportunities, and solutions.” [Dr. Mary Bunch is a CIKL Research Associate]
Exhibitions:
iMagiNative Festival 2023, Toronto, Canada
More Than Human, OCAD On Site Gallery, Toronto 2023
Nuit Blanche, York University, Toronto 2023
Created by: Dolleen Manning and Mary Bunch
Visual and User Design: Michaela Pnacek
Projections and Cinematography: Christina Dovolis
Earth Fortunes
(2023)
is an augmented reality (AR) installation developed collaboratively by artists and engineers, and created as part of the multi-year project, Speculative Energy Futures. The installation generates energy fortunes based on the computer model GCAM: the Global Change Analysis Model and serves scenarios responsive to the Shared Socioeconomic Pathways or SSPs. Users shuffle and deal with beautifully-illustrated fortune-telling cards and access the AR via a QR code. The large triptych, itself a form of data visualization of the engineering data, launches the AR fortune-telling experience encouraging questions around sustainability, equity, social justice and the kind of futures we want or can imagine.
Created by: Caitlin Fisher (York University), Evan Davies (University of Alberta), Wallace Edwards (independent artist)
Student researcher at the University of Alberta: Evan Arbuckle (Engineering)
Student researchers at the Immersive Styorytelling Lab: Sharon Musa (Engineering), Joo Park (Digital Media), Michaela Pňaček (Cinema and Media Arts)
Exhibitions: HASTAC 2023 – Pratt Institute, New York City
Home Is the World
(2022)
a VR experience for tethered headsets, 10′ (2022), research-creation project developed at Immersive Storytelling Lab with CharismaAI
The interface of the piece is created by an ephemeral space between virtual reality and the user’s physical environment where they converse with the AI character and are asked to answer its questions and to follow a series of sensory-led tasks to induce pleasant memories before the Covid-19 pandemic. Such tasks include smelling coffee beans, conjuring the smell of an early summer morning, the sound of walking on snow, the taste of a delicious dessert, the touch of a plant etc. Then, the Charisma.AI software records a user’s spoken memories of the pandemic and generates a personal pandemic story for each user, individually. The twist: it re-tells the experience somewhat differently. It re-contextualizes the user’s story in a world full of positive news.
Publications: Electronic Literature Organization Conference 2022, The Writing Platform, August 2022
Article: https://thewritingplatform.com/2022/08/designing-a-vr-experience-in-a-covid-19-world/
Author and Creative Director : Michaela Pnacek
Creative Technologist: Michaela Pnacek
Technical Leads CharismaAI: Oscar Lindstrom, Annette Parry
Developed with the support of Immersive Storytelling Lab, York University, Toronto and CharismaAI, UK
Countries: Canada, UK
Virtual Archive of Emotions
(prototype, 2021)
a collaborative co-creative experience in MozillaHubs
Create an artifact of a someone else’s 2020 memory with the tools on the bottom of your screen (drawing, writing, sculpting, images, GIFs, videos, links, sounds, music). This whole world is an archive, feel free to put your artefacts anywhere you see fit.
The experience forms a world where participants will meet, and co-create artifacts of their lives during Covid-19 isolation. Together, they will be able to draw, create sounds, talk, perform, record, insert, images, videos and build 3D sculptures. The intention is to create a virtual archive of Covid-19 pandemic isolation and at the same time create connection and intimacy among participants.
Exhibitions: Film and Media Studies Association Canada Conference 2021
Created by: Michaela Pnacek
Connection
(interactive haptic prototype, 2022)
This prototype was made as a part of the workshop Soft Touch with the Arduino kit, knitted and hand-sewn materials, data visualization software
Vimeo Link: https://vimeo.com/888847417?share=copy
Symphony of Noise VR
a room scale VR experience for tethered headsets (HTC Vive Pro, Oculus Rift (S), Valve), 15′ (2019)
In Symphony of Noise, you will undergo a sound transformation you have never experienced before. You will become a superhero, the conductor of the world. You will change the world through sound and conduct the world into your own symphony. Listen intently as ordinary sounds become music in your ears!
Festivals: VRHAM! 2019 Germany, Reeperbahn Music Festival Hamburg 2019, International Documentary Film Festival Amsterdam – IDFA Doc Lab Competition for Immersive Non-Fiction 2019, LEV Matadero Vortex Madrid Festival 2020, Zlin Children and Animation International Film Festival 2020, Geneva International Film Festival 2020, ISEA Montreal 2020, FIVARS Toronto 2020, One World Bratislava 2021, Film Music Festival Berlin 2021, Czech Film School 2022
Awards and Nominations: FIVARS Award for Excellence in Sound Design, VRNow Awards Nomination for Best Entertainment, Forbes Best of Arthouse XR List 2019
Author and Creative Director : Michaela Pnacekova
Author and Co-Creative Director: Jamie Balliu
Visual and Technical Lead Unity: Paul Kirsten
Composer and Interactive Music Developer: Phivos-Angelos Kollias
Produced by: Michaela Pnacekova, Stefan Kloos
Co-produced by: Jamie Balliu
Produced by: Kloos & Co. Medien Berlin
Co-Produced by: UPCreatives, Reeperbahnfestival, VRHam!
Countries: Germany, UK
Pre-Crime Calculator
App for iOS and in web browser, 5′ (2017)
Pre-Crime Calculator is an interactive experience that takes you to the world of predictive policing. How much of a potential suspect or a victim are you in the eyes of the system? And what are the areas in your city you should avoid in the next week not to get involved in the crime?
Festivals: Film Festival Hamburg 2017
Creative Director: Michaela Pnacekova
Development: Jan Joost Verhoef, Frederik Verhoef
UI/UX: Jan Joost Verhoef, Marku Günter
Story Consultant: Harmke Heezen
Associate Producer: Mike Robbins
Producers: Michaela Pnacekova, Stefan Kloos
Production: Kloos & Co Medien in association with Helios Design Labs and Ernst 3000
Countries: Germany, Canada
Chomsky vs. Chomsky: First Encounter
an interactive VR experience for tethered headsets, 15′ (2020)
A prologue to a timely conversation on AI’s biggest promises and pitfalls. Lured by the possibilities of an emulation of one of today’s most famous minds, we meet and engage with CHOMSKY_AI, an entity under construction. Renowned thinker Noam Chomsky has devoted his career to studying the mind. Drawing from the arsenal of digital traces he’s left behind, CHOMSKY_AI challenges us to ponder: What exactly are we hoping to replicate? And what are we leaving behind?
Festivals: Sundance Film Festival 2020
Created by: Sandra Rodriguez
Producers: Sandra Rodriguez, Sebastian Huber, Marie-Pier Gauthier
Lead Designer: Michael Burk
Technology Lead: Cindy Bishop
AI Leads: Guillaume Petitclerc, Olivier Blais
Music and Sound Design: kling klang klong
Executive Producers: Bob Moore, Mila Aung-Thwin, Daniel Cross, Louis-Richard Tremblay, Hugues Sweeney, Michaela Pnacekova
Production: The National Film Board of Canada, Eyesteelfilm, schnellebuntebilder
Countries Canada, Germany
Michaela Pnacek(OVA) 