Harsha
Autonomous driving and advanced driver-assistance systems (ADAS) are hot topics. Automakers, suppliers and thousands of startups have been working on autonomous driving technologies for decades. The Society of Automotive Engineers ranks the levels of driving automation from level zero (no automation) to level five (full automation). Many expected fully autonomous vehicles to be on the road by now. However, the highest commercial level of automation is still level three, where automation is in effect only in certain, limited conditions.

 
John, from your perspective, what level of automation are customers seeking in the U.S.?

John
We see customer demand for low-cost level-two systems such as steering, braking and acceleration support. These features use either just a camera, or a camera and radar. Higher level-two or lower level-three features are on premium vehicles. The question is, will customers pay for higher-level features on standard vehicles?

Harsha
Dr. Peng, what is the public’s general level of acceptance for these ADAS features?

Huei
Technology for driving assistance systems has matured. Auto emergency braking is widely available now. There's no question that these systems are providing safety benefits. But drivers start to think that they may not need to pay full attention, which is dangerous.

 
People see the benefits of ADAS, but it’s a challenge for people to understand what the system is capable of. For instance, the driver of a level-two car could mistakenly think that the car is fully autonomous. It takes time for customers to understand this new technology, and this is the major barrier in the near term for ADAS. More training time may be needed for dealerships and customers.

Harsha
Subrata, what kind of innovation is being created by Hitachi R&D in this area?

Subrata
At Hitachi R&D, we’ve been working on autonomous driving software and hardware to address complex issues. One of our solutions is a stereo camera that detects obstacles, road surfaces, traffic signs and so on. The system can then control braking, acceleration and steering accordingly to provide better comfort and safer, stress-free driving. In addition, we are working to minimize energy usage, lower emissions, and create seamless traffic networks and other innovations.

 Since there are many different scopes in the autonomous driving area, we work directly with our customers to understand the complex and unknown issues and then develop a solution together that can provide the safety and comfort that autonomous driving requires.

Harsha
Moving to higher levels of autonomous driving will pose challenges for both technology and society. How can society understand and start adopting level-four or higher vehicles in the coming years?

Huei
Challenges experienced by level-four vehicles are similar to those at levels one and two. You must communicate to your customers what your level four is. What are the limits? Is your vehicle speed-limited or weather-limited? Is it geofenced? Is it able to handle a left turn where the view is partially blocked?

 People are sometimes afraid of new technology, and that's natural. Deploying technology in a careful, limited way helps reduce some of the concern. It's up to us to deploy technology carefully and judiciously so that people won’t fear it.

Harsha
John, what are some of the main technical challenges that need to be addressed before reliable systems for autonomous driving are ready for commercial deployment?

John
Level three, conditional automation, is confusing for drivers. You're basically telling them the car will drive itself, that they don't have to pay attention until we need them to, and then they have to quickly pay attention. That's difficult. I can see legal issues and other problems arising in level three. At level four, the vehicle has to still be able to safely get itself off the road, stop, and give the driver plenty of time to decide whether to take over and drive.

 
Since no single sensor can handle every situation, we keep adding more sensors — LiDAR (light detection and ranging), cameras, radar, finite impulse response (FIR) and more. This enormously increases the amount of data that must be processed. We have to collect the data, process it, decide which data to trust depending on the environmental conditions, and then plan the path and actuate the vehicle based on these decisions. This requires server-level processing power. So not only is the sensor suite commercially prohibitive, the amount of computational power necessary makes it unrealistic for individuals to purchase one of these vehicles.

 
We've got to simplify the processing and the software. We've got to decide quickly which sensors we use in which conditions so that we don't have to process all this data. One of the main steps required is reducing the complexity and costs.

Harsha
Right. We've got to think about functional safety and redundancy. With a lane departure warning, you just get the warning and life goes on. But if you're in a level-three or higher vehicle and encounter an issue and the driver's not paying attention, it creates a difficult situation. Any thoughts?

John
Clearly you need redundancy and backup systems, which are expensive. That's why you're seeing the first adoption on small, controlled fleets that are owned by large companies, not by individuals.

Harsha
From a design point of view, are there any techniques or new approaches that R&D can take to simplify not only the design but also the process?

Subrata
Definitely. R&D can focus on finding the optimum combination of technologies that will work in all types of conditions. We can use multiple LiDAR units or cameras, but we need to consider the cost of the vehicle. And, as John mentioned, the enormous amount of data processing required by the electronic control unit for real-time applications results in high power usage. This also leads to heat generation, which ultimately reduces vehicle efficiency — R&D is also focusing on those issues.

 
And we must absolutely ensure that the system works perfectly in all conditions before we deliver it to the customer. It's difficult to ensure this by an actual road test. We need an effective co-simulation platform so that we can evaluate vehicle-level system performance and verify the real-time performance of a combination of sensors in the electronic control unit to help ensure the safety of the autonomous driving systems.

Harsha
Dr. Peng, how do you think connectivity plays a role in enhancing ADAS features?

Huei
Connectivity provides additional useful information for both near-real-time and long-horizon problems. For example, a vehicle that is not connected will need to rely on in-vehicle technology such as sensors, cameras, radar and LiDAR, which won’t help if you have an occluded left turn or intersection.

 At Mcity, we provide a research facility and testing ground for connected and automated vehicles. We work with dozens of partners from the automotive industry. If you are not sure your vehicle can handle unprotected left turns or a roundabout, you can bring it to Mcity to test it. You can control the exact test conditions, and you can repeat that all day long. Although we were created to focus on mobility, we intend to be part of the development of smart cities, and that means connectivity.

 
With connectivity, a roadside camera or LiDAR can see when there are moving objects approaching the vehicle and broadcast that information to the vehicle. That kind of communication will not be widespread quickly, but it could be employed at challenging intersections. This approach uses infrastructure intelligence to help the vehicle’s onboard intelligence. When connectivity is in place, instead of relying on onboard intelligence and sensors, you can use information from a high-definition map or roadside sensors to get information about road curvatures, ice or potholes.

Harsha
Connectivity and data sharing bring regulation. Dr. Peng and Subrata, can you briefly talk about the regulations here in the U.S. and elsewhere?

SubrataRules and regulations for using connected data vary by region. If transportation authorities can use information about the individual's driving history or destination, they can more easily provide traffic data and reduce congestion. The first wide application of connected service will probably be in Middle Eastern countries and China. North American countries will be later because of data privacy restrictions.

HueiNobody wants to be tracked from home to work. If cameras are everywhere, at every intersection, that's a problem. But if you only capture data in selected intersections, that's less of a concern. Still, it's important to be transparent and judicious about placing cameras in intersections.

Harsha
From a business point of view, John, how is connectivity going to benefit us in developing new business models or adding value for the customers? Why should customers pay for this kind of service?

John
Sharing data from a set of vehicles to the cloud and then broadcasting it to every vehicle minimizes the processing power required in each vehicle. You have to collect some data in the autonomous vehicle, but things like road and weather conditions are a little more static and don't need to be collected in real time by every vehicle. That reduces the onboard processing necessary, reducing the complexity.

Harsha
Connected vehicles are often powered by electricity, which is another hot topic. John, from talking to your customers, where do you think we stand now with electric vehicles (EVs)?

John
EVs are greener than internal combustion vehicles because they have lower emissions. However, if it takes a long time to make up for the extra cost of an EV in gas savings, customers will be less likely to purchase one. Some people will purchase EVs and take the financial hit because they're doing something good for the planet. But that's not the majority.

 
Some countries, such as China, are implementing strict regulations to force the move to EVs. That's necessary because the cost of batteries is significant. The volume must go up before prices can come down, and the only way to increase volume is to subsidize it through government programs.

Huei
I agree with John. We cannot afford to keep this a financial decision by individuals. Governments can help. For example, California allows EVs access to HOV carpool lanes, helping EV drivers avoid being stuck in rush-hour traffic. Norway offers free EV parking in some downtown areas.

Subrata
It’s not just the electric vehicle, but the connected autonomous electric vehicle that will change the future. And changes are coming soon for connected EVs. At Hitachi R&D, we’ve demonstrated how to use connected data to improve their efficiency. We’re conducting extensive research on materials and manufacturing process to further reduce the cost of electric motors and inverters.

Harsha
Mcity is one of the premier institutes for connected autonomous vehicles. Dr. Peng, how can Mcity bring together people to solve societal challenges?

Huei
The name “Mcity” reflects our focus on smart cities. We’re building on what we're really good at, which is connected automotive technology, but also data. Mcity was one of the first facilities to use an early prototype of 5G. We have an edge computer with 24 GPU — a lot of computation power. We want to build on top of mobility and move to a better-living laboratory concept. Leaving an impact on people's lives is our objective.

HarshaWhen and how do you see the implementation of autonomous vehicles?

Huei
I think that the current trend — level one, level two — will continue. Level-one automated vehicles will eventually have 99% precision. Level four will start in a shared managed fleet with a commercial application. Today's LiDAR, radar, cameras and GPUs are still expensive, so it makes sense to use them on vehicles that are used 12 hours per day, not two hours per day.

Harsha
John, from a business point of view, where do you think we are headed in the next 10 years?

JohnI think we’ll see a quick acceleration in the low to high level-two applications. The high level-two applications now in premium vehicles will move down to non-luxury vehicles. We can get to level-four capabilities within the 10-year horizon, but that will be limited to shared fleets and geofenced, fixed-route applications.

 
Another growth area is long-haul trucking, where a truck that's now driven eight hours a day can be extended to drive 20 to 24 hours a day. This will reduce shipping times and costs. Also, in urban or densely populated areas, private ownership of vehicles will decrease.

Harsha
Subrata, what are some of the challenges we’ll face in design and technology over this 10-year horizon?

Subrata
I'm optimistic that we will be able to solve the existing problems. At Hitachi R&D, we have been working on high-resolution sensors that use connected data. We’re looking at how to effectively manage connected data for highly reliable autonomous driving, and we’re optimizing the digital network architecture to support real-time applications.

 
Since safety is our first priority, we have to consider using redundant safety controllers until we are sure that the AI-based controller works perfectly in all conditions. The world of the future will be a system of systems, and connected autonomous vehicles will be an integral part of that society. Those vehicles will significantly contribute to safe and comfortable mobility and reduce congestion and emissions. By using Hitachi's social innovation co-creation approach, we can collaborate with our customers to deliver significant contributions to society.

John
Manufacturing and automotive companies know about specifications and regulatory requirements. IT companies know connectivity and software development. Hitachi has both because we're both an IT company and an automotive company. We combine both of these worlds in one company.

Harsha
Working with partners such as Mcity will help us showcase these accomplishments.

Huei
Yes, we look forward to working together with Hitachi and others to continue developing smart city concepts well into the future.