Spellcasters

A commercial VR wizard-dueling game reimagined as customizable stroke-rehabilitation therapy  with a clinician sandbox for authoring custom gestures and a telemetry dashboard

Overview

Spellcasters began as an entertainment VR game in which teams of five wizards duel by drawing gestures with a magic wand to cast spells. The team translated that gesture-drawing mechanic into a serious game for stroke rehabilitation, specifically targeting hemiparesis  spells now require therapeutic gestures to cast, and the curriculum is customizable per patient. Two companion artifacts extend the system: (1) a clinician-facing sandbox for authoring custom spell gestures matched to a patient's post-stroke motor profile, and (2) a companion clinician dashboard web application with co-designed data visualizations driven by the game's telemetry, so therapists can track progress, edit in-game goals between sessions, and interact with patients.

Problem & Motivation

Post-stroke rehabilitation requires high-volume, often tedious gesture practice for which intrinsic motivation is famously hard to sustain  the same dynamic the lab targets across its serious-games portfolio. At the same time, a therapy game built around one fixed set of "right" gestures can't accommodate the wide variability of post-stroke motor profiles. Spellcasters explores what happens when an engaging entertainment mechanic is re-authored for therapy, with clinicians retaining authoring power over the gesture set itself.

Approach, Team & Publications

Participatory design sessions and user studies were conducted with 14 physical and occupational therapists to develop the customizable-gesture sandbox and clinician dashboard. Earlier related stroke-rehabilitation work was conducted at UCSD (Cosman Lab) using bodystorming, Wizard-of-Oz, and object-theater methods. Lead Researcher: Jared Duval | Co-authors: Rutul Thakkar, Delong Du, Kassandra Chin, Sherry Luo, Aviv Elor, Magy Seif El-Nasr, Michael John | Publication: Duval, J., et al. Designing Spellcasters from Clinician Perspectives: A Customizable Gesture-Based Immersive VR Game for Stroke Rehabilitation. ACM Transactions on Accessible Computing (TACCESS) 15(3), Article 26, September 2022. https://doi.org/10.1145/3530820 | Demo video: https://youtu.be/zXLUNgFvpgg | Companion app Figma prototype: https://www.figma.com/file/0XGZRuvnULvt0BdOcQKvdl/Spellcasters-HighFi-Prototype