Using the integrated analysis technology platform for physiological and behavioral data developed for LifeMicroScope®, an algorithm is generated that predicts crash risk from truck driver heart beats data and notifies drivers and managers
May 23, 2022
Fig. 1. Technology platform for real-time prediction of accident risk caused by worker fatigue based on physiological data
utilizing the integrated analysis technology platform for physiological and behavioral data developed for LifeMicroScope®
Hitachi, Ltd. has developed a technology to predict the risk of crashes caused by fatigue in real time based on physiological data in collaboration with Hitachi Transport System, Ltd., the Institute of Physical and Chemical Research (RIKEN), and Fatigue and Mental Health Check Center, Inc. (FMCC). This technology utilizes the integrated analysis technology platform for physiological and behavioral data that Hitachi has cultivated through its research and development of LifeMicroScope® to achieve functionality that predicts the risk of crashes due to fatigue using heart beats data of truck drivers on the job and notifies the drivers and managers of this risk. This function is installed in SSCV-Safety*1 (SSCV: Smart & Safety Connected Vehicle), a safety operation management solution of Hitachi Transport System.
In the manufacturing and logistics fields, artificial intelligence (AI) and service robot technologies are being used to save labor and improve efficiency. However, preventing serious accidents from human error resulting from fatigue and stress has become an issue, especially with tasks that require humans to do work, such as in the transportation industry with trucks and aircraft, and in the maintenance and operation of infrastructure systems that support the safety and security of society. Companies that operate trucks and buses have long taken measures to prevent accidents by having managers check the health condition of drivers before work, but it has been difficult to assess in real time drivers’ health condition while driving, including accumulated fatigue and reduced alertness.
Hitachi is conducting research and development with the aim of providing safety and security solutions that predict and notify of, when required, the risk of human error based on the data measured of a worker's physical and mental condition as well as of the work being done. The technologies being utilized are IoT technology and the technology platform for integrating and analyzing data from disparate fields cultivated through the LifeMicroScope®, a system that measures and analyzes human activity using sensor network technology. In order to prevent crashes caused by truck driver fatigue, Hitachi has developed an algorithm that predicts crash risk in real time based on heart beats data and related vehicle behavior data for 1,200 driver-days on the job. By measuring heart beats data with a wearable device, this technology enables drivers and managers to detect the accumulation of fatigue and stress as a predictive sign of a crash, thereby preventing crashes before they occur (Fig. 2).
Hitachi aims to achieve a zero-accident society by applying the newly developed technology to various industries where serious accidents caused by human error are a concern, including transportation outside of logistics, as well as construction, manufacturing, and the maintenance and operation of the infrastructure systems of society. Some of the outcomes of this research were published in the journal PLOS ONE in October 2021*2.
Fig. 2. Achieving safe operation management using technology for predicting crash risks
(1) Technology using physiological data to predict in real time the risk of crashes caused by fatigue
This technology functions to calculate the risk of a crash in real time by determining the autonomic nervous system *3 feature value based on analysis of heart beats data for 1,200 driver-days on the job, and notify drivers and managers when the calculated risk exceeds a threshold (Fig. 3).
First, through joint research between Hitachi and experts in the field of fatigue science, we designed a feature value set (approximately 50 types) of autonomic nervous system related to driver fatigue and stress. Next, using the near-miss*4 classification technique described in the next section, we calculated the probability distribution of the risk level, which indicates the possibility of a near-miss situation occurring for a feature value. The results show that the distribution of near-miss risk expands in the direction of higher near-miss probability depending on the value of the feature (fatigue and stress). Therefore, using approximately 50 types of feature values, we performed a regression analysis to estimate the probability of high-risk situations by feature value, and succeeded in extracting features that narrow the range of risk by up to about 50% toward the low end (Fig. 4). With this novel method of analysis, we have generated an algorithm that can calculate specific autonomic nervous system features from heart beats data alone and then calculate crash risk.
Fig. 3. Algorithm for predicting crash risk using autonomic nervous system feature values
that are highly correlated with the occurrence of near-misses
![Fig. 4. The relationship between the feature values of the autonomic nervous system (parasympathetic nerve activity index) extracted in this study and the probability of occurrence of driving operations (near-miss situations) and level of risk (The range of risk narrows by about 50% toward the low end as the feature values increase [relaxing].)](image/2205_pra-4.png)
Fig. 4. The relationship between the feature values of the autonomic nervous system (parasympathetic nerve activity index) extracted in this study and the probability of occurrence of driving operations (near-miss situations) and level of risk
(The range of risk narrows by about 50% toward the low end as the feature values increase [relaxing].)
(2) Near-miss classification technique through machine learning
We have developed a near-miss classification technique that extracts features of high-risk behavior from vehicle behavior data and outputs the level of risk that is likely to cause the above-mentioned near-miss situations.
In logistics operations, a large amount of data is required to statistically reveal the relationship between physiological data during driving and crash risk. However, the annual number of crashes involving trucks is small (about 1% of the total number of trucks owned), and since cases of crashes involving a particular vehicle or driver are rare, collecting statistically useful data has been difficult. Therefore, we focused on near-miss situations that could lead to crashes.
Although a person can visually identify near-miss situations from in-vehicle camera video, it would take an inordinate amount of time to do so, thus we have developed a technique to automatically classify near-miss situations based on vehicle speed and acceleration data using machine learning (Fig. 5). In this process, experts selected near-miss situations using a portion of data and used them as training data. In order to improve the accuracy of classification, we defined several driving scenes where crashes are likely to occur, such as driving on highways and before and after stopping vehicles, based on actual near-miss cases, and conducted machine learning for each driving scenario. As a result of these learning efforts, it is now possible to automatically classify near-miss situations from vehicle behavior data with an accuracy of 84%. Using this classification technique, we determined the degree of risk, which indicates the probability of occurrence of a near-miss situation, from the speed and acceleration data.
Fig. 5. Technique that uses machine learning to classify near-miss situations
from vehicle behavior data and determine the probability of occurrence and degree of risk
For more information, use the enquiry form below to contact the Research & Development Group, Hitachi, Ltd. Please make sure to include the title of the article.
