Progress 08/01/19 to 03/31/20

Outputs
Target Audience:There are three target audiences: I. Rural older adults who wish to maintain their independence for as long as possible, but this independence is treatened by one or more elements of their health and wellness. They don't mind being monitored, if it reduces intrusion. II. Caregivers. These may be non-paid family members, or certified nursing assistants who work for an agency. They would prefer to lightly check in with their older adult clients, and only travel to their home if necessary. This is especially true during the coronavirus pandemic. III. Smart home companies that typically install a variety of sensors in an older adult's home. These sensors form an intelligent network to inform others on the resident's health and wellness. They frequently are confused by the data reported by the sensor network, and are seeking ways to easily and securely verify what is happening in the home. Changes/Problems:One of the biggest challenges for ALERT creating an application that could run all the time and be within reach of the user at home, while also maintaining a low profile and being unobtrusive while not actively in use. This challenge was overcome by designing the tablet application to leverage the aesthetic design of the host tablet device so that it could be turned into a digital picture frame when the user is not interacting with it; encouraging the user to prominently display it in a central area of their home. Previous design iterations proposed a tablet application that would send a notification to the user when a message arrived. Discovering the need for and the development of the picture-frame functionality required additional time, and additional code in the application as well as the server, for custom images to be delivered to the tablet App. This approach offered a much more compelling solution for the user, however, and proved a worthwhile use of time and resources. As we envision future refinements, we acknowledge the presence of some challenges to overcome with regards to the scalability of the ALERT server application. In its current state the server can manage multiple users quite easily. As that number increases, efforts will need to be made for running the server application across multiple hardware devices to spread the network and processing load, while still maintaining an easy approach for the tablet application to automatically use the most efficient server based on location and traffic. Challenges faced by Daiuto Sutter Some challenges were encountered both during the initial development process as well as after the application was deployed. While being developed, extra time was spent on choosing an email provider since our first choice was ultimately not viable. Gmail was initially chosen; however, it periodically required a human to fill out a CAPTCHA as it was not intended from programmatic use. Twilio Sendgrid was chosen as a replacement because it is an email service intended for the programmatic sending of emails. Once deployed, the University of South Florida (USF) requested an application that detects if a participant's tablet is on and the app is running. Since the initial application had already been developed and was running, we choose to develop a separate application (the app status page) that monitored log files for relevant information. This took additional time that was not originally accounted for. Looking forward, areas of expansion identified during the development and implementation of the ALERT project would include changing the user login system to allow more granular permissions for each user. In order to handle a large deployment, there is also a need to enhance the application server to handle a multi-threaded, multi-server deployment. Additionally, there is a need to enhance how notifications are sent to caregivers. Allowing for MMS and automated phone call support would expand the number of potential customers served. What opportunities for training and professional development has the project provided?CreateAbility will work to integrate workflow/training processes with chosen Smart home/sensor installation companies and teams to ensure an effective training and development plan is in place for both the client and care provider. FOR THE OLDER RURAL ADULT: This project provided an opportunity to learn how to introduce technology into the lives of older adults. In the future, each device that is given to designated participant/end user will have a pre-installed video demonstration that will provide education on the purpose of the device/program, what is expected from them and how to care for their device. An ALERT demo will preempt them as to what their avatar-led communications will look like to increase familiarity and ultimately, adoption. FOR THE CARE PROVIDER: This project provided an opportunity to learn how to support the care provider, that could be anywhere in the spectrum of comfort with technology. The CreateAbility Support Center Knowledge Base will soon afford care providers direct access to detailed product knowledge and multimedia support articles for every major CreateAbility application. Organizations will be given access to create, edit, organize, and rate private Knowledge Base articles and categories for their support professionals/resource facilitators to use, in addition to having public access to a robust collection of existing support articles and online demonstrations in the areas of: 1) Quick and scalable implementation of CreateAbility products, 2) Instructions on how to set-up CreateAbility apps on Android and Apple devices, 3) Instructions for configuring apps, and 4) Detailed troubleshooting articles w/visuals The CreateAbility Support Center Knowledge Base is built upon the Echo Knowledge Base Wordpress plugins and contains vital information for using CreateAbility products and an extensive FAQ component regarding all CreateAbility apps and services. How have the results been disseminated to communities of interest?CreateAbility has submitted proposals from Request for Paters from the various associations identified as potential strategic partners. These associations also have opportunities for advertising, sponsorships, content-sharing, and podcast interviews. A sample list is provided here: N4A (National Association of Area Agencies on Aging) ISG (International Society of Gerontology) ASA (American Society on Aging) NCOA (National Council on Aging) CES (Consumer Electronic Show; "Silver Hall" exhibition) NRHA (National Rural Health Association) GSA (Gerontology Society of America) What do you plan to do during the next reporting period to accomplish the goals? Nothing Reported

Impacts
What was accomplished under these goals? Goal 1: Two focus groups were held: The first was with healthcare professionals and caregiver organizations. The second focus group was with older rural adults and a family member, charged with their care. After collecting and analysing the results, this led to the development of a set of features and functions required in the system. These were documented in the design documents that drove the development. Goal 2: The Alert system development. This development was done in parallel and were devided between Larry King performing the App development, and the team of Sutter(PD) and Daiuto developing the cloud -based interface and decision logic. The portion of the ALERT project developed by the team of Sutter (PD) and Daiuto, involved building software components in the cloud, consisting of 5 discrete components: 1. The dashboard is an Internet accessible application that allows caregivers to change settings and view information related to the ALERT project from a web browser. The ALERT dashboard is primarily written in Python using the Django web framework, while data is stored in a SQLite3 database. Several webpages display relevant information related to incoming messages from HomeSense, debug and troubleshooting utilities, traffic monitoring and logging, responses from the older adult, and a process monitor page shows that statuses of the applications running in the cloud. 2. The MQTT broker accepts messages sent from the HomeSense in-home monitoring system. These messages contain relevant information like if an occupant has been in bed too long. We chose Mosquitto (sic) as our broker because it is an open-source application with a robust community for support. Mosquitto(sic) is maintained by the Eclipse project and is written in C. 3. the MQTT shuttle, written in Python and leveraging the Paho-MQTT library, it listens to the MQTT broker for new messages from HomeSense and stores them in a database assessable from the web dashboard. This is required because the MQTT broker does not have persistent storage. 4. the assessment delivery mechanism and email service, which sends assessments at designated times of day for each tablet based on data received from HomeSense. The application receives the assessment responses, and if the person requests assistance, ALERT will send an email or text message (via SMS gateway) to the designated caregiver. 5. the cloud services is the app status page. This application is accessible from a web browser but is separate from the main web dashboard. This was developed to monitor if a particular participant's tablet is on. This component was written in Python using the Flask web framework. Data is stored in a SQLite3 database. The ALERT server application was developed with the Python programming language, as a lightweight and scalable application that communicates between the MQTT messaging service and the ALERT App. The Python Flask module was used to provide REST-style communications between the ALERT App and server via HTTPS protocol. This protocol is also used for communication between the ALERT server and the MQTT messaging service to retrieve tri ggers from sensors in the user's home. A SQLite database is used on the ALERT server to queue messages for each user, and to keep track of when the user consumes a message, in addition to tracking their response. The ALERT server also utilizes Amazon Polly; a service that generates MP3 audio files and JSON text files that contain timestamps and viseme codes that occur at each timestamp. The ALERT server converts this JSON file into a custom format that can be consumed by the ALERT App. The ALERT application was developed with the cross-platform Solar 2D development tools and initially designed for Android tablets. The application employs networking code to allow it to communicate to and from the ALERT server, as well as frame-accurate graphics code to allow real-time playback of the interactive avatar. When not interacting with the user, the application acts as a digital picture frame that cycles through images that are built into the application, or custom images that can be placed into the application's sandbox by a caregiver. The ALERT application contains code to ensure the application runs the device at full power; preventing the tablet from sleeping or darkening the screen. The application running on the tablet, cycles a continuous thread that checks the ALERT server every few seconds for a new message. If one exists, then the tablet App downloads the data that determines if a standardized question should be presented to the user, or a custom question that was designed by the caregiver. If a custom question is presented, then the appropriate binary and text files are downloaded from the ALERT server. Once everything is downloaded the tablet App sends a signal to the server that it has received all data, and the question is then presented to the user. The avatar voices the question to the user and moves their mouth and head in real-time, perfectly synchronized to the rendered MP3 audio file. This is accomplished with several parallel threads that move the avatar's head, mouth, and eye blinks. Specifically, the avatar's mouth motions were coded to mouth each phoneme spoken to appear most natural. Once the user answers the question, that information is sent back to the ALERT server for analysis. Goal 3: The integrated Phase I prototype was evaluated with six older rural adults (more than the 3-5 planned), over a two month period at the end of the grant. The results were that all six study participants (>65, rural) felt that the system was surpiisingly easy and enjoyable to use, and felt a high level of safety and security, knowing that the tablet could summon assistance if they needed it as they answered questions. Three expressed that they were "contributing to society", or "advancing the science in the world of healthcare." On a scales of 1 being not good, to 10 being very good, only two users rated the system a 5 in its current form, with others rating it a 9. All felt the concept was a 9 or 10 if it were improved from its current prototype state. That said, the six study participants had some very useful enahcement requests for the final product (CreateAbility will explore thes in Phase II). To make sure everything is working, it could periodically state that, "All Systems Are Go!" Add the ability to initiate communication by the senior (allowing them to immediately ask for help via the tablet). Link with fall detection pendants or PERS, so that if the pendant detected a fall the avatar would assure them that help was on the way. More rugged case in case it was accidentally knocked off table. Need a sleep mode to dim the brightness or turn off the display during sleeping hours. Need a louder speaker for those with hearing impairments. Also provide the option for multiple alerts for alerting the arrival of new messages. For example: notify every five minutes until the question is answered. Goal 4: This goal was accomplshed in two parts: 1) A financial justification concluded that the incremental cost of the avatar App and tablet, plus the cloud-based interface layer gave a greater than 10X reduction in costs, primarily due to reducing un-necessary trips. 2) Regarding positive impact, while rural adults over 65 years old benefit from ALERT, the primary customers are nationwide Home and Community Based Service (HCBS) companies who wish to expand their market share by tfilling the growing need in the rural market, and have been waiting for a solution that allows them to capture it in a cost-effective manner. These companies will benefit from efficiently deploying their staff to the homes on a as needed basis.

Publications