The paramount measure of success for a machine learning model has historically been predictive power and accuracy, but even a gold-standard accuracy benchmark fails when it inappropriately misrepresents a disabled or minority body. In this work, we reframe the role of machine learning as a provocation through a case study of participatory work co-creating exergames by employing machine learning and its training as a source of play and motivation rather than an accurate diagnostic tool for children with and without Sensory Based Motor Disorder. We created a design probe, Cirkus, that supports nearly any aminal locomotion exergame while collecting movement data for training a bespoke machine learning model. During 5 participatory workshops with a total of 30 children using Cirkus, we co-created a catalog of 17 exergames and a resulting machine-learning model. We discuss the potential implications of reframing machine learning’s role in Assistive Technology for values other than accuracy, share the challenges of using “messy” movement data from children with disabilities in an ever-changing co-creation context for training machine learning, and present broader implications of using machine learning in therapy games.

Spellcasters is an entertainment virtual reality game turned therapy game for stroke rehabilitation. In the original game, teams of 5 wizards with various roles such as support, attack, and tank battle in a magical duel by drawing gestures with their magical wand to cast a spell before aiming and shooting at the target. The first team to run out of a pool of lives loses. This research explored how translating a game developed purely for entertainment could be reimagined for therapy—the gesture drawing mechanic seemed like a great opportunity for physical rehabilitation if the spells required therapeutic gestures to cast. We collaborated with medical professionals, including physical therapists and occupational therapists, on a sandbox that allows custom gestures to be created for the idiosyncratic needs of stroke survivors, as well as a design for a companion app that allows medical professionals to track progress and edit in-game goals. 

Social media is rife with tacit design knowlege—including content from creators with disabilities. In this work, we chased play potentials on TikTok from creators with disabilities to inspire accessible and playful everyday technology. We design better technology for everyone when we include the perspectives of people with disabilities early and often in the design process. This project was led by Jared Duval and co-authored with Ferran Altarriba Bertran, SIP High School Interns, and Katherine Isbister.

SpokeIt addresses the needs of a large community of people who receive, do not receive, or provide speech therapy. SpokeIt is a Serious Game for Health designed to provide at-home speech therapy and engaging activities. Speech practice at home is usually hindered by a lack of intrinsic motivation due to the tedious and repetitive idiosyncratic nature of speech therapy curriculums. Parents and caretakers experience major difficulties prompting individuals to complete speech exercises at home, especially when young. They often lack the expertise of a speech therapist and report a general low sense of competence in facilitating curriculum. Medical professionals have little insight into the (lack of) progress outside of their practice and would benefit from reports. SpokeIt is intended to be used by children aged 4-10 with cleft speech, as well by parents, caretakers, and medical professionals for curriculum customization, progress reports, and integrated at-home play.