Sensorimotor & Sensory Stimulation for Individuals with Disabilities
Using Nature, Kung Fu, and AR Selfie Filters
Research and Report by Melody Hammer
Simulations by Felicia Tan
New York University
Developing Assistive Technology, Fall 2024
Overview
Our research team designed a unique approach to Sensory Stimulation Therapy (SST) using interactive multimedia for all levels of abilities ranging from non-disabled, to individuals with severe disabilities, classified levels provided by the Gross Motor Function Classification System (GMFCS) (Miller & Bachrach, 2006). We aimed to create an integrated experience that provided mental stimulation, movement, entertainment and relaxation for HeartShare’s day habilitation program community members, located in Woodside, Queens, New York.
The coordinators at HeartShare envisioned a recreational activity to enhance their day habilitation program: a specialized approach within Sensory Integration Therapy (SIT) called Sensorimotor and Sensory Stimulation Therapy (SST). The approach would incorporate an immersive sensory environment using emerging technologies and interactive elements to stimulate the mind and encourage movement for individuals of all abilities.
Goals of our Project
Our research team collaborated with the Hoffman Dayhab coordinators to brainstorm initial ideas. They emphasized the importance of promoting mental stimulation and physical movement while prioritizing the health and well-being of the residents. They requested that the interactive environment be portable and lightweight, allowing caregivers to easily move the experience throughout the center. Additionally, we considered specific needs: individuals requiring level 5 support are non-verbal and need hand-over-hand assistance, so the experience would need to be facilitated by a caregiver. For individuals with greater independence, tactile elements and additional interactivity could be incorporated into the design.
Sensory Therapy Research
Before developing our simulations, we explored the literature on Sensory Stimulation Therapy (SST), which enhances quality of life for individuals with disabilities by improving sensory processing in the brain, leading to better behaviors and therapeutic outcomes. An example of SST is a Snoezelen multi-sensory room.
Snoezelen multi-sensory room.
Dr. Jean Aryes, Sensory Integration Therapy (TM).
A pioneer in Sensory Integration Therapy (SIT), which also includes Sensorimotor stimulation was Dr. A. Jean Ayres. Her approach of SIT addresses sensory challenges, especially for individuals with autism and developmental disabilities. Her work remains a foundation for therapies that promote independence and well-being across diverse fields, including occupational therapy and neuroscience.
One idea from our brainstorming included an approach similar to EyeClick’s immersive projection systems, which turn ordinary surfaces into interactive, engaging environments (EyeClick, 2024). Hoffman Dayhab envisioned a similar technology that provides sensory stimulation through interactive visuals, music, and sounds, while ensuring safety and accessibility.
EyeClick’s immersive projection systems. There is a picture of EyeClick’s floor projections, with young kids dancing on top of the projection.
Claims
Diagram by Melody Hammer
Our approach focused on three key outcomes. First, our experiences are designed to promote relaxation for Hoffman Day Hab residents. This includes providing a calming effect, reducing anxiety, and enhancing emotional well-being. Second, we aim to encourage physical movement, helping residents utilize their motor skills to support overall physical health. Finally, our simulations foster engagement by encouraging residents to interact with their environment and others, creating opportunities for connection and meaningful participation.
1
Sensory stimulation will facilitate relaxation for residents of Hoffman Day Habilitation Program, providing a calming effect that reduces anxiety and enhances emotional well-being.
2
Sensory stimulation activities can promote physical movement for Hoffman Day Habilitation Program residents, helping them explore and use their motor skills in ways that support physical health.
3
Sensory stimulation will foster increased engagement among Hoffman Day Habilitation Program residents, encouraging interaction with their environment and with others.
Visiting our community Partners at HeartShare
We began our Ability Project research with an in-person site visit led by the Coordinators of Accessibility and Assistive Technology at HeartShare’s Hoffman Day Habilitation Program. The “dayhab” center provides assisted programs for adults with disabilities and overnight care. The dayhab program specializes in providing various levels of care, from Level 1 to Level 5, with Level 5 representing the highest level of assistance. Individuals at with level 5 abilities have limited voluntary control of movement and an inability to maintain an upright posture independently (Palisano, Rosenbaum, Bartlett, & Livingston, 2008). The center is equipped with dedicated rooms for physical therapy, arts and crafts, a dining hall, and a large room for games and entertainment.
During our tour, we were fortunate to meet with some of the members of the Hoffman Day Habilitation community, and we had the pleasure to interview one of the caregivers. When speaking with the caregiver, it was enlightening to hear about each individual’s daily routines, personality, their likes, dislikes and preferences. It gave our team the understanding that whatever we create should be customizable and based on each individual’s preferences.
They shared that some individuals loved to interact with lightweight musical instruments, like a tambourine. However, others did not like loud sounds, they would get agitated when the intercom made an announcement. Others liked listening to music and some liked watching television, while others loved to see themselves in a mirror and take selfies with the caregivers. Some may also struggle to express how they feel due to limited control over facial expressions and body movements. While these individuals may experience challenges with profound motor impairments, they are able to actively engage with the world through various senses, such as sight, hearing, and touch. Additionally, since some individuals have seizure disorders, we needed to ensure the environment avoided visual effects that might trigger an episode. Despite these limitations, the caregivers are still able to interact and communicate with the residents.
During our visit to Hoffman Dayhab, we had the privilege of meeting with some of our community partners and their caregivers. Our interactions with our partners informed our approach to designing three distinct sensory simulations tailored to individual preferences and needs.
Adaptive Approaches
Flexible Tripod and Mini Projector
To create the accessible sensory simulations for HeartShare, we used a versatile inexpensive mini projector, with a camera clamp, to attach to the back of a wheelchair, making it portable and accessible for various simulations.
Additional materials included a flexible tripod, smartphone, and apple watch for additional interactive elements.
Microsoft Directional D-Pad, Mircosoft 2024
For future iterations of the AR selfies for Hoffman DayHab I would like to consider incorporating switch controls like the Jelly Bean Switch or the Microsoft Adaptive Directional D-Pad Button, — which would make the experience more accessible for individuals with limited hand coordination.
These types of devices allow users to interact with the technology through simple, customizable inputs, enabling them to control features like capturing photos or selecting filters with ease. By integrating adaptive tools, the project would better meet the needs of individuals who require alternative interaction methods.
Jelly Bean Switch
Sensory and Sensorimotor Simulation TRials
Simulation #1 Nature
The first simulation draws from research on sensory stimulation and the therapeutic effects of nature sounds and visuals. This design promotes relaxation and helps individuals who may feel overwhelmed or overstimulated find calmness.
Benefits
Relaxation
Calming effect
Reduced overstimulation
Claim
Sensory stimulation will facilitate relaxation for residents of Hoffman Day Habilitation Program, providing a calming effect that reduces anxiety and enhances emotional well-being.
Production Visualizations by Felicia Tan, Tandon School of Engineering, PhD Student.
Simulation #2 Kung Fu
The second simulation was inspired by a man at HeartShare who enjoys watching kung fu movies. By incorporating sensorimotor stimulation, this design aims to engage mirror neurons to encourage physical movement in an interactive and enjoyable way.
Benefits
Promotes movement
Improves motor skills
Enhances cognitive engagement
Claim
Sensory stimulation activities can promote physical movement for Hoffman Day Habilitation Program residents, helping them explore and use their motor skills in ways that support physical health.
Production Visualizations by Felicia Tan, Tandon School of Engineering, PhD Student.
Sensorimotor Kung-Fu Simulation Trial, HeartShare Hoffman Dayhab, 2024
Simulation #3 AR Selfies
Our final simulation demonstrates how therapeutic sensory stimulation fosters engagement for residents of the Hoffman Day Habilitation Program, encouraging interaction with their environment and others.
One resident, a young woman enjoys dressing up in costumes and wigs and taking selfies—a process she finds both fun and stimulating. The AR Selfie Simulation allowed her to explore self-expression while also providing sensory stimulation and encouraging movement. This activity boosted all our moods, promoted engagement, and promoted inclusivity.
Benefits
Boosts Mood (it was fun!)
Fosters external engagement
Encourages Inclusivity
Claim
Sensory stimulation will foster increased engagement among Hoffman Day Habilitation Program residents, encouraging interaction with their environment and with others.
A community member at Hoffman Dayhab using our adaptive approach to taking AR Selfies from her chair.
The screen is projected for easier viewing.
A flexible tripod holds the device for hands-free interaction.
Apple Watch iOS Switch Control
While she did interact with the selfie app herself, we assisted her in taking pictures and videos using my Apple Watch – with the iOS switch control feature, ensuring ease of use. She truly was smiling the entire time, and we all had a great experience!
Our final considerations compared how simulations like Nature, Kung Fu, and AR Selfies can enhance relaxation, movement, and engagement for individuals with disabilities, supporting their physical and emotional well-being.
Moving forward, some considerations would be to explore technologies like eye-tracking to improve interactivity, to optimize lighting for sensory stimulation environments, and to create simplified interfaces with large touch targets to ensure accessibility for participants who can interact with the simulations using their hands.
Additionally, we hope to connect with more individuals at Heartshare, to continue identifying their unique interests and preferences for personalized simulations.
Partnering with HeartShare has been a true honor, aligning with our mission to create inclusive designs that make a meaningful impact on the lives of over 6,000 individuals each year.
About the NYU | Ability Project
The NYU Ability Project is an interdisciplinary research initiative based out of Tandon School of Engineering, in partnership with NYU’s Steinhardt School of Culture, Education, and Human Development and Tisch Interactive Telecommunications Program (ITP). The goal of the Ability Project is to focus on developing inclusive, accessible technologies and experiences for individuals of all abilities.
We were presented with diverse and individualized research projects, but one project that truly stood out was from the community at HeartShare Human Services of New York, who reached out to NYU for interdisciplinary support in creating a sensory stimulation for residents at the Hoffman Day Habilitation Program.