Prinicpal Investigator: Skubic
Co-Investigators: Abott, He, Ho, Keller, Popescu, RantzAgency: NSF
Project Summary
Americans are living longer and more fulfilling lives. They desire to live as independently as possible, but independent lifestyles come with risks, such as debilitating falls that limit mobility. To address these issues, researchers are developing “smart home” technologies to enhance residents’ safety and monitor health conditions using sensors and other devices. In particular, the continuous assessment of physical function is a key indicator of initial decline in health and functional ability. Identifying and assessing problems while they are still small can provide a window of opportunity for interventions that will alleviate the problem areas before they become catastrophic.
Researchers at the University of Missouri have established an interdisciplinary team dedicated to developing and evaluating technology to keep older adults functioning at high levels and living independently. We leverage ongoing research at a unique local eldercare facility (TigerPlace) to study active sensing and fusion using vision and acoustic sensors for the continuous assessment of a resident’s risk of falling as well as the reliable detection of falls in the home environment. The project investigates the interplay between fall detection and fall risk assessment.
Project Objectives:
- Investigate adaptive, active, anonymized vision sensing for monitoring elders in a home setting
- Investigate adaptive acoustic sensing for monitoring elders in a home setting
- Investigate adaptive sensor fusion and intelligent decision making using heterogeneous sensor data collected at varying time scales, including both quantitative and qualitative data, and incorporating risk factors
- Evaluate the effectiveness of the monitoring system in a realistic physical environment with variable conditions
Project Highlights:
- Fall detection algorithms were developed using an array of microphones, the Microsoft Kinect depth camera, and a two-webcam system. For the webcam system, we extract silhouettes from orthogonal views and construct a 3D voxel model for analysis.
- Passive gait analysis algorithms were developed for the depth camera and the two-webcam system, initially in the lab using vicon marker-based motion capture as ground truth.
- The depth camera and webcam systems were installed in 10 TigerPlace apartments for two years to collect realistic data. Every month, trained stunt actors came into each apartment to fall, and a researcher guided the older adult participants through a set of standardized fall risk assessments. These data were used as ground truth to further refine the fall detection and gait analysis algorithms for in-home use. GPU code was developed to achieve real-time processing for the webcam system.
- At the end of the study, the depth data was found to be more robust and more practical for in-home use. Live fall detection alerts were established and algorithms proposed for gait-change algorithms.
See also:
In-Home Gait: https://www.youtube.com/watch?v=MF6yZyLuuII
In-Home Falls: https://www.youtube.com/watch?v=TFB7YOUmHho
Papers
Banerjee T, Keller JM, Popescu M & Skubic M, “Recognizing Complex Instrumental Activities of Daily Living Using Scene Information and Fuzzy Logic,” Computer Vision and Image Understanding, 2015, 140:68-82.
Stone E & Skubic M, “Fall Detection in Homes of Older Adults Using the Microsoft Kinect,” IEEE Journal of Biomedical and Health Informatics, 2015, 19(1):290-301.
Stone E, Skubic M, Rantz M, Abbott C & Miller S, “Average In-Home Gait Speed: Investigation of a New Metric for Mobility and Fall Risk Assessment of Elders,” Gait and Posture, 2015, 41:57-62.
Banerjee T, Keller J, Skubic M & Stone E, “Day or Night Activity Recognition from Video Using Fuzzy Clustering Techniques,” IEEE Transactions on Fuzzy Systems, 2014, 22(3):483-493.
Banerjee T, Skubic M, Keller JM & Abbott CC, “Sit-To-Stand Measurement For In Home Monitoring Using Voxel Analysis,” IEEE Journal of Biomedical and Health Informatics, 2014, 18(4):1502-1509.
Li Y, Ho KC & Popescu M, “Efficient Source Separation Algorithms for Acoustic Fall Detection Using a Microsoft Kinect,” IEEE Transactions on Biomedical Engineering, 2014, 61(3):745-755.
Li Y, Ho KC, Popescu M & Skubic M, “A Theoretical Study on the Placement of Microphone Arrays for Improving the Localization Accuracy of a Fall,” Proceedings, IEEE International Conference of the Engineering in Medicine and Biology Society, Chicago, IL, August 26-30, 2014, pp 4523-4526.
Stone E & Skubic M, “Testing Real-Time In-Home Fall Alerts with Embedded Depth Video Hyperlink,” International Conference on Smart Homes and Health Telematics (ICOST), Denver, CO, June 25-27, 2014.
Stone E, Skubic M & Back J, “Automated Health Alerts from Kinect-based In-Home Gait Measurements,” Proceedings, IEEE International Conference of the Engineering in Medicine and Biology Society, Chicago, IL, August 26-30, 2014, pp 2961-2964.
Li Y, Banerjee T, Popescu M & Skubic M, “Improvement of Acoustic Fall Detection using Kinect Depth Sensing,” Proceedings, IEEE 2013 International Conference of the Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, July 3-7, 2013.
Stone E & Skubic M, “Mapping Kinect-based In-home Gait Speed to TUG Time: A Methodology to Facilitate Clinical Interpretation,” Proceedings, 7th International Conference on Pervasive Computing Technologies for Healthcare (Pervasive Health), Venice, Italy, May 5-8, 2013.
Stone E & Skubic M, “Unobtrusive, Continuous, In-HomeGait Measurement Using the Microsoft Kinect,” IEEE Transactions on Biomedical Engineering, 2013, 60(10):2925-2932.
Wallace R, Abbott C, Gibson-Horn C & Skubic M, “In-Home Measurement of the Effect of Strategically Weighted Vests on Ambulation,” Proceedings, IEEE 2013 International Conference of the Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, July 3-7, 2013, pp 949-952.
Wang F, Stone E, Skubic M, Keller JM & Abbott C, “Toward a Passive Low-Cost In-Home Gait Assessment System for Older Adults,” IEEE Journal of Biomedical and Health Informatics, 2013, 17(2):346-355.
Banerjee T, Keller JM & Skubic M, “Resident Identification Using Kinect Depth Image Data and Fuzzy Clustering Techniques,” Proceedings, 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego CA, August 28-September 1, 2012, pp 5102-5105.
Li Y, Ho KC & Popescu M, “A Microphone Array System for Automatic Fall Detection,” IEEE Transactions on Biomedical Engineering, 2012, 59(2):1291-1301.
Li Y, Popescu M, & Ho KC, “Improving Automatic Sound-based Fall Detection Using iVAT Clustering and GA-based Feature Selection,” Proceedings, 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, August 28 – September 1, 2012, pp 5867-5870.
Stone E & Skubic M, “Capturing Habitual, In-Home Gait Parameter Trends Using an Inexpensive Depth Camera,” Proceedings, 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego CA, August 28-September 1, 2012, pp 5106-5109.
Stone EE & Skubic M, “Passive, In-Home Gait Measurement Using an Inexpensive Depth Camera: Initial Results,” Proceedings, 6th International Conference on Pervasive Computing Technologies for Healthcare, San Diego CA, May 21-24, 2012, pp 183-186.
Banerjee T, Keller JM, Zhou Z, Skubic M & Stone E, “Activity Segmentation of Infrared Images Using Fuzzy Clustering Techniques,” Proceedings, World Conference on Soft Computing, San Francisco, May 23-26, 2011.
Li Y, Popescu M, & Ho KC, “Improving Acoustic Fall Recognition by Adaptive Signal Windowing,” Proceedings, 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, August 30-September 3, 2011, pp 7589-7592.
Stone E & Skubic M, “Evaluation of an Inexpensive Depth Camera for In-Home Gait Assessment,” Journal of Ambient Intelligence and Smart Environments, 3(4):349-361, 2011.
Stone E & Skubic M, “Passive In-Home Measurement of Stride-to-Stride Gait Variability Comparing Vision and Kinect Sensing,” Proceedings, 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, August 30-September 3, 2011, pp 6491-6494.
Stone EE & Skubic M, “Evaluation of an Inexpensive Depth Camera for Passive In-Home Fall Risk Assessment,” Proceedings, Pervasive Health Conference, Dublin, Ireland, May 21-23, 2011, pp 71-77, Best Paper Award.
Stone EE & Skubic M, “Silhouette Classification Using Pixel and Voxel Features for Improved Elder Monitoring in Dynamic Environments,” Proceedings, 2011 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), Seattle, WA, March 21-25, 2011, pp 655-661.
Wang F, Skubic M, Abbott C & Keller J, “Quantitative Analysis of 180 Degree Turns for Fall Risk Assessment Using Video Sensors,” Proceedings, IEEE Engineering in Medicine and Biology Conference, Boston, MA, August 30-September 3, 2011, pp 7606-7609
Banerjee T, Keller JM, Skubic M & Abbott CC, “Sit-To-Stand Detection Using Fuzzy Clustering Techniques,” Proceedings, IEEE World Congress on Computational Intelligence, Barcelona, Spain, July 18-23, 2010, pp 1-8.
Li Y, Zeng Z, Popescu M & Ho KC, “Acoustic Fall Detection Using a Circular Microphone Array,” Proceedings, 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Buenos Aires, Argentina, August 31-September 4, 2010, pp 2242-2245.
Stone E, Anderson D, Skubic M & Keller J, “Extracting Footfalls from Voxel Data,” Proceedings, 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Buenos Aires, Argentina, August 31-September 4, 2010, pp 1119-1122.
Stone E, Anderson D, Skubic M & Keller J, “Footfall Extraction and Visualization from Voxel Data,” Proceedings, International Society for Gerontechnology 7th World Confreence, Vancouver, Canada, May 27-30, 2010.
Wang F, Skubic M, Abbott C & Keller J, “Body Sway Measurement for Fall Risk Assessment Using Inexpensive Webcams,” Proceedings, 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Buenos Aires, Argentina, August 31-September 4, 2010, pp 2225-2229.