Development of Deep Predictive Learning Robot Control Technologies Enabling Real-Time Decision Making and Movement Adapting to Operational and Environmental Changes

*1.

Hiroshi Ito, Kenjiro Yamamoto, Hiroki Mori, Tetsuya Ogata, “Efficient multitask learning with an embodied predictive model for door opening and entry with whole-body control,” Science Robotics, 6 April 2022, Vol 7, Issue 65

*2.

This data is based on internal research by Hitachi, Ltd.

*3.

The free energy principle is a theory of the brain that posits that not only cognition but also behavior is generated by the principle of minimizing a cost function called variational free energy, which is involved in predicting the environment.

Technology details

1. Deep predictive learning technology referencing functions of the brain

Fig. 1. DPL algorithm to minimize prediction errors between real situations and models
referencing the fact that living organisms behave so that error between the real world and the brain is minimized

In typical machine learning, robots mainly use a method to create an optimal predictive model using a large amount of data to apply to a variety of tasks. However, this type of method has had trouble configuring predictive models so that robots can handle all types of situations when it does actually encounter something unexpected. This research set out to propose a deep predictive learning (DPL) algorithm to minimize prediction error between models and the current situation on the premise of a predictive model’s imperfections. Hitachi, Professor Tetsuya Ogata, and his team developed this technology based on a free energy principle that describes how the functions of the brain work to minimize prediction error in the real world. The robot predicts the near future based on visual motion data to guide the next movement with minimal error (gap) from the real world. This innovation allows robots to flexibility take on tasks in untrained situations by continually adapting its operations in real time while tolerating discrepancies between its training and the real world.

2. Motion generation technology using deep predictive learning

Fig. 2. DPL motion generation technology for robots to properly open doors
even with untrained patterns and different door knob positions or shape

Typical robot machine learning had taken a huge amount of trial and error to obtain the desired movement. While this did have the benefits of finding more efficient movements not yet discovered by people, the machine learning required a lot of time and effort. As shown in Figure 2, this technology uses three steps to enable robots to handle tasks with untrained objects or in unknown environments. The method developed to gain the desirable operation without any programing has an operator remotely control the robot to teach the necessary tasks several times (Step 1), and then simply conducts training for several hours in a computer (Step 2). Next, when the robot executes the tasks (Step 3), it recalls its past training and experience, compares it to the current situation, and predicts the necessary movements in real time.

To verify the effectiveness of this technology, this research chose to employ door opening and entry tasks using an actual robot as one example. Even in the simple daily task of opening a door, people must not only recognize the door from its exterior appearance but also the direction the door moves (push/pull), the direction the door opens (right/left), and the position and shape of the door knob. The human brain instantly determines all of this accurately based on past experiences. In the case of a robot opening a door though, all of these decisions have to be built into extensive motion learning and programs to accommodate every situation. This technology can be applied to a wide range of applications under diverse conditions to overcome these challenges. This research has shown that a robot can properly open a door even with untrained door patterns as well as door knob positions and geometries by simply teaching the desired task several times.

3. Mechanism to switch between multiple predictive models in real time

Fig. 3. Mechanism to switch between multiple predictive models in real time
to enable complex tasks not supported by one predictive model

Substantial development costs and a lot of adjustment work is necessary for a robot to handle tasks across multiple processes. This is because engineers need to create programs separately for robots to handle unexpected situations in addition to those to handle the series of expected tasks. The robot must also recognize changes in a situation due to any unexpected disturbance in the work environment and run vast calculations to rework the task, which can cause extended shutdowns and maintenance times.

In this technology, the robot memorizes predictive models for each action, such as opening a door and passing through, and combines them to achieve a series of tasks. Each predictive model generates a predictive image illustrating the circumstances in the near future using sensor data. The robot compares the real camera images (actual images) to its predictive images (Step 1), and then calculates the time difference in real time of indicators (certainty factors) that show the feasibility of a task with what level of accuracy in the current situation (Step 2). The robot then independently selects the predictive model with the highest certainty (Step 3) to execute the right action according to the real-world environment. This real time calculation allows the robot to perform complex tasks which cannot be done with a single predictive model without precisely designing the timing to switch between operations and flow of operations.