Tech support

Above: Blind pedestrian walking on tactile paving, Photo by zlikovec, iStock / Getty Images Plus

With the advancement of technology, people living with disabilities have more options than ever to assist them in their daily lives.

Several UC Santa Cruz faculty members are contributing to this effort by developing assistive technologies to make life easier for those with disabilities and other special needs, such as those with cleft, blindness, physical impairment, cognitive impairment, and other impairments caused by aging.

One of the university’s leading researchers in this effort is Sri Kurniawan, professor and chair of the Computational Media Department in the Baskin School of Engineering. She runs the ASSIST Lab (Assistive Sociotechnical  Solutions for Individuals with Special Needs using Technology).

Kurniawan works collaboratively with researchers and practitioners from UC Santa Cruz and beyond. In her project on developing a speech therapy game for children who have had reconstructive surgery for cleft lip or cleft palate, she works with Su-hua Wang, UC Santa Cruz professor of psychology, as well as a medical team at UC Davis Medical Center.  

The motivation for this project is the realization that while speech therapy is very important for these kids, some live far from speech therapists and aren’t able to get the therapy as often as they need. Sometimes parents’ work schedules prevent them from helping their kids practice speech exercises. 

When Kurniawan learned about the problem, she started brainstorming ways she could help, and came up with the idea of a game that kids could play themselves. Over time, she and her colleagues developed a speech therapy game for kids ages 2–11. The project was supported through a $548,000 grant from the National Science Foundation. Kurniawan said it’s heartwarming to see how the game is changing kids’ lives for the better. 

“Speech pathologists tell us, ‘This child was a quieter boy and never said much. You cannot believe how many words he was willing to say with the games!’” Kurniawan said.

Providing an assist

 

Kurniawan’s other projects include partnering with Santa Cruz nonprofit Hope Services to provide a virtual reality system that helps people with developmental disabilities develop socioemotional skills. They learn how to do everything from interviewing to picking up social cues by looking at others’ facial expressions.

She is working with Nancy Chen, UC Santa Cruz professor of anthropology, and nonprofit DigitalNEST (founded by alumnus and chancellor’s adviser Jacob Martinez) to develop an online platform where teens and elders can connect and share information. The young people can help the elders access online shopping and other services while in return the elders can share their knowledge about history, healthy cooking, or other subjects. 

“We don’t want the two cohorts to feel that they are giving something without getting anything back,” Kurniawan said. “It’s an exchange.”

Kurniawan has also collaborated with Roberto Manduchi, professor of computer science and engineering in the Baskin School of Engineering, on an iPhone app that helps blind people navigate public transit.

“Public transit can be difficult if you are blind because it is very visual,” Manduchi said. “There are lots of signs to follow. It’s very easy to get lost for everyone.”

Solving real problems

Manduchi also worked on an app to help blind people make their way around indoors. While GPS can help people outside, there hasn’t been something available to help move around indoors, Manduchi said. This is useful because blind people get disoriented very easily since they don’t have visual landmarks to help them keep track of where they are. 

In addition, Manduchi has worked on a free app that offers “optical character recognition”—in other words it can read text to the visually impaired. 

Through experience, Manduchi said he has learned the importance of always talking to blind people first before designing tools that can assist them. Years ago, he made a mistake by trying to make a replacement for the standard white cane. His idea was a flashlight that would shine laser lights with sensors that would measure distances of surfaces and would tell the blind person where obstacles and steps were. But once he talked to blind people about his design, he realized they didn’t like it because it needed batteries and could be expensive. 

He is proud that he is able to help raise awareness about the issues faced by people with disabilities when he teaches the class Universal Access. It is the only class at UC Santa Cruz devoted to disability. People with disabilities come to the class and talk about technology they use and relevant legislation. 

“We tend to forget diversity is not only racial, gender, etc.,” Manduchi said. “It is also ability, disability.” 

Innovative apps

While assistive technology can benefit people with disabilities, it can also help people of any ability, including doctors. Narges Norouzi, a UCSC teaching professor of computer science and engineering, is working on an app that would help doctors assisting COVID-19 patients using video call to get clinically useful information from the video.  

The app would record such info as blood oxygen saturation level, respiratory rate, and heart rate, all in real time. The app would get that information by interpreting the color changes in the patient’s face or fingertip. The data could then be used to determine what factors correlate with hospitalization or people getting better or worse. Researchers could also analyze what factors distinguish COVID-19 from other respiratory and flu-like illnesses. The app received a seed grant from the Center for Information Technology Research in the Interest of Society and the Banatao Institute, a systemwide UC initiative focused on creating IT solutions that generate social and economic benefit for all. 

Helping us help ourselves

Strictly speaking, Steve Whittaker, UCSC professor of computational media, doesn’t design assistive technology because his projects don’t focus on a specific population. Instead, his work is designed to tackle things people in general don’t deal with particularly well. 

For example, psychological research shows that people generally aren’t very good at interpreting or predicting their emotions, Whittaker said. If you ask people how they are likely to feel in the future, they usually make a prediction based on how they are feeling now. If they are feeling bad, they will likely say they expect to feel bad in the future. 

“We overemphasize the impact of recent negative events,” he said. 

To address this problem, Whittaker helped design an app called EmotiCal that serves as a virtual diary, prompting users a couple of times a day to think about how they are feeling and what they’ve been doing in the last hour or so. 

Using that data, the app can generate a model for users on their emotional profile. He compares what the app does to a Netflix recommendation of what program someone might want to watch.  

People can then use that info to make a plan for the next day, including their most positive activities. While there are other apps available that do something similar, he said his stands out because it is scientifically validated, while most are not.

“People are happier because now,” Whittaker said, “they have a method to understand and control their mood.”