Recent room air conditioners are required to provide comfort as well as offer high energy efficiency. Consumers want room air conditioners to cool every corner of a room in summer and warm even the areas under tables in winter, while reducing power charges as much as possible. Such wishes have been realized by Hitachi's room air conditioners.
The products feature "attentiveness" by incorporating what Hitachi dubs as the "Kurashi Camera" (lifestyle imaging camera)—a multi-monitoring system using cameras as sensors to secure comfort.
(Publication: March 15, 2016)
KOMATSUThat's right. Conventionally, research and development efforts for room air conditioners were focused on "energy efficiency" as the valuation standard. More recently, another valuation standard of "ability to provide comfort" has been added. To meet these standards, Hitachi has incorporated its image recognition technology in the form of what we call the "Kurashi Camera," or "lifestyle imaging camera," to our room air conditioners.
KOMATSUWhat triggered us to do so was an inquiry we received from Hitachi Appliances, Inc. about our research activities. As we had an accumulation of image processing-related know-how, we suggested to them that new values might be created by combining image processing technology using cameras with home electric appliances. Air conditioners were among such appliances.
HAMADAWe thought of combining cameras with a variety of home appliances, not only air conditioners but also refrigerators and microwave ovens, for example. In the end, the combination with air conditioners was selected and adopted. I guess it was chosen because cameras had the potential to allow for air conditioners to "provide comfort," which is an ability required of air conditioners in recent years.
KOMATSUInitially, we began the work by incorporating our existing technologies to air conditioners. However, it was not easy to satisfy the needs with that approach. So we looked to customers to directly hear their opinions.
HAMADATo do so, we prepared several proposals on certain functions in advance, and asked customers if they would want such functions if they were available.
KOMATSUWe worked to find out what functions would really make customers happy. After fully understanding their needs, we added new technologies to our existing technology base. This is how the "lifestyle imaging camera" evolved. One of the added technologies was the "room layout detection" function.
HAMADAWell, air conditioners are usually installed in the corners of rooms. Air conditioners have louvers, or slats that control the airflow direction, which swing horizontally. Then, if an air conditioner is installed in the left corner of a room, air conditioning is quite strong on the left side of room, while the airflow doesn't reach the wider space on the right side of the room. This had been a conventional problem with air conditioners.
Moreover, according to survey results of housing conditions, recent condominiums in Japan increasingly have a Japanese-style room located next to the living room, separated from the living room by sliding or movable doors.
KOMATSUBecause of this, customers need air conditioning to perform in accordance with the shape, layout and other arrangements of the room. To meet this requirement, we incorporated a function to detect the room layout using a camera.
Figure 1: Detection of doors
KOMATSUThe room layout detection function conducts two processes. The first is detection of doors. In the process, two images are used to check if there have been any door-like movements. Then, the position of what appears to be a door is determined, and its center coordinates are located. This is how the process is done.
HAMADAWhether it is a door or not is judged by the complexity of the image of the section where there is likely to be a door. Doors are generally flat. On the contrary, when a door is open, there are several things in the room behind the door, and the scene is messy and complicated. So, if the image of the section is complicated, we judge that the door is open.
KOMATSUThe other process is detecting the corners of a room. Linear features, or lines, are detected from the continuity of the shapes of a beam, a column and other objects in an image. Then the intersections of the lines are detected. That's how you can find corners of a room.
In this way, the location of doors and their status of being open or closed, as well as the information on the corners of a room are detected. The louver is controlled based on such information. If the door is shut, the air conditioner sends air to the entire room. If it is open, air is sent straight toward the room behind the door. As such, the air conditioner can control airflow in accordance with the shape, layout and other arrangements of a room.
KOMATSUExactly. The new models have a function called "object detection camera," or "MONO Camera" in Japanese, in the product catalogue. Earlier models incorporated a visible-light camera, which is similar to that installed on digital cameras and smart phones. In addition to it, the new models have a near-infrared camera function that can take photos even at night.
HAMADAWith a visible-light camera, shapes of furniture are hard to see because of the furniture patterns and design and effects of external light. In this regard, a near-infrared camera shows things in a single color and the furniture patterns and designs are not easily visible. As the shapes of furniture stand out, it is very easy to process the images.
KOMATSUUsing the images taken by the near-infrared camera, the furniture is classified into a table, a sofa or other categories. It is also determined whether airflow can go through the respective pieces of the furniture. For example, in the case of a table having long legs, air is sent to the lower part of the table so that the lower parts of the legs of people sitting that are hidden under the table are also warmed.
KOMATSUYou may naturally think so. In fact, we only use a visible-light camera. A filter which lets only near-infrared light pass through entirely covers the visible-light camera, so that it works as a near-infrared camera. At the same time, a near-infrared LED emits light and the camera takes images. But near-infrared light is invisible and cannot be seen.
Simply adding a camera would increase the cost. So we racked our brains on this. In developing the system, we made experiments by holding the filter up over an ordinary camera.
Figure 2: Structure of the object detection camera
HAMADAA single round of development is conducted in a span of roughly a year, and more than half of the period is spent in investigating what functions to be incorporated into the new product. As we do so, we hold extensive discussions to identify what functions should make customers "happy" and encourage them to buy the product.
Naturally, we don't just hit upon such smart ideas as shown in the catalogue from the start. It is hard to say how many plans were discarded. After repeated discussions for half a year or so, we finally decide on the feature functions to be demonstrated on the first page of the catalogue. This is the most laborious process.
HAMADAThat's right. It takes about three months to make the prototypes. We must create the first prototype in a month or so and deliver it to Hitachi Appliances. This is because they must subsequently go through the phase of incorporating the prototype into an air conditioner and confirming if the functions perform without any hitches with the air conditioner. In any way, we must concentrate our efforts into a very short time.
KOMATSUThe launching dates are fixed, and work must be done to meet the deadlines. If our work is delayed, not only the shipment of products but also their mass production are delayed. This would incur more expenses. We work under such pressures.
HAMADASometimes the prototype that is incorporated into an air conditioner may fail to perform at the expected level. In one such case, Mr. KOMATSU visited the plant of Hitachi Appliances and worked until late at night to fix the issue.
KOMATSUYes, I did. As I saw the many people actually working there, I realized how many people from a variety of fields are engaged in the process of creating products. Moreover, each time we made a judgment, it affected a variety of aspects, and I felt the importance of our responsibility as researchers involved in commercializing products. That's why I believe we must conduct our research in a careful and steady manner.
KOMATSUI presented our research to the Information Processing Society of Japan for two straight years, and were commended each time. I was very pleased that our results were highly evaluated from outside the company in terms of technical achievements. Moreover, the air conditioners won the Minister of Economy, Trade and Industry Award of the Energy Conservation Grand Prize.
HAMADAI hear it was the first time in over a decade for Hitachi's air conditioners to win the Award. The company's air conditioners have conventionally demonstrated energy efficiency value that was among the highest in the industry. I believe the products now enjoy even higher evaluations as they provide better comfort due to the "lifestyle imaging camera."
KOMATSUOf course we are pleased that the products have won such awards and increased sales. However, what is most encouraging are the words from those around us saying, "We've bought a Hitachi's air conditioner. It warms every part of the room, and is very nice."
HAMADAExactly. Just the other day, I gave a briefing on the "lifestyle imaging camera 4" at a press conference for the latest room air conditioners. Along with this, we have already started a new round of six months in which we will investigate the next version.
Presently, the "lifestyle imaging camera" is incorporated only in room air conditioners. But there are those asking if it can be applied to business-use air conditioners, which you see installed in the ceilings of office spaces. I believe offices will become quite comfortable if such business-use air conditioners have this function. Also, the camera may possibly be used in the air conditioning in server rooms.
KOMATSUThe room layout detection and object detection functions of the "lifestyle imaging camera" may be applied to calculating the movements of robots, though it is an area far from air conditioners. If layouts are recognized correctly, robots will not bump into walls. If furniture shapes are grasped, robots will be able to go around a table that may be present.
KOMATSUWith the new combination of room air conditioners and cameras, we have succeeded in incorporating a function to provide better comfort. Similarly, we may be able to pioneer new areas by applying cameras in places where no one has thought of incorporating them.
HAMADAMore recently I have focused on the research of in-vehicle cameras. Looking ahead, the era will come when driving a car is automated. Still it is an undeveloped area and there is a huge amount of items to be researched. If such items are addressed one by one and automated driving is realized, a world free from car accidents will no longer be a dream.
KOMATSULooking back, I have been engaged in research subjects that are directly related to commercialization of products since I joined the company. Likewise, a variety of people are involved in supporting the launching of products. This is wonderful, and I am grateful for it.
HAMADAIndeed, many things happen in product development. Launching new products is the most rewarding. It is such a pleasure that hundreds and thousands of models featuring technologies we have developed are offered to the public and that people use them! For this reason, I hope to continue being involved in the commercialization of products.
