Harman is “bringing cockpits to life”

Harman, a subsidiary of Samsung, has been working with automakers to “bring cockpits to life,” says Huibert Verhoeven, Senior Vice President for Digital Cockpits. In the case of a cockpit, the consolidated operating systems (OS) have different constraints. For example, the digital cluster requires the display to respond quickly, while an infotainment system requires it to execute a large set of applications (web browser, air conditioning, radio and video, Android or iPhone applications). This cluster is safety critical and required to be predictable, while the infotainment needs to be open and secure.

With Harman’s virtualisation process, powered by its type-1 ASIL B certified hypervisor, the key requirement is to provide a safe, flexible, and controlled means of communication. Verhoeven notes that experiences within the car have changed significantly through the years: “Our perspective on cockpits has shifted greatly due to the electrification of vehicles. The user has become much more of a focus.”

Infotainment

For passengers, the entertainment aspect of infotainment has been rapidly advancing to make the experience more enjoyable.

“On the infotainment side, we primarily saw the transition from the screen trying to mimic a smartphone to a set of screens—there are cars right now with up to six screens throughout the vehicle—where the information has to be presented in a way that is easily understandable and optimised for driving,” Verhoeven tells Automotive World. This is a trending industry topic, with many OEMs and suppliers looking to manage multiple screens within the vehicle across different users and displays.

Harman’s digital cockpit works by using scalable hardware and software platforms. The flexible hardware is security operations centre (SOC) agnostic and can support up to seven displays, eight cameras, and 12 microphones, with both controller area network (CAN) and ethernet communication interfaces. Hardware modules combine with modular cluster and IVI software modules to pre-integrate leading features

But away from just entertainment, presenting insightful information, such as road updates and traffic alerts, in a non-distracting manner is an equally important part of the equation. “For the driver, the information you present can inherently be very different from the passenger’s information,” he explains. Can the screen be connected so the user can cast information to any screen, not just visually but also with audio? What if users start creating different audio zones in the vehicle—can passengers pursue individual activities while also retaining communication with everyone else? These are areas that Verhoeven’s cockpit team are researching and developing.

Security for our customers is security in the context of an inherently connected vehicle, which is where curating the apps within the cockpit also becomes important

Announced in February 2022, Harman’s latest cockpit addition, labelled as ‘human-centric’, includes Ready Together for easy in-vehicle entertainment and communication management. The Software Enabled Branded Audio on-demand software platform upgrades the sound quality and feature set of existing unbranded car audio systems. The company also offers augmented reality 3D navigation to enhance the driver and passenger travel experience through high-quality, hyper-realistic graphics.

Safety

“While managing infotainment, it’s also important for automakers and suppliers to consider safety,” says Verhoeven. One focus includes advanced driver assistance systems (ADAS)—generally used to monitor the driver—inside the vehicle. While some automakers have been utilising systems that detect distracted drivers, Harman is looking to advance its solutions to the next level by focusing on what can be done to gain the attention of distracted or drowsy drivers. Potential solutions include increasing the air-conditioning or adjusting the volume on the radio to increase alertness.

Harman calls this solution Ready Care, which uses a camera to capture the driver’s eyelids and determine their engagement and whether they are likely to fall asleep. “Our approach has been fundamentally different. Yes, we detect drowsy drivers, but we fuse it with other factors. It’s not just about the driver falling asleep—what if the driver is on the road looking at the screen? Are they still mentally focused on the task at hand?”

These considerations will become increasingly important as automakers move towards self-driving features. At SAE Levels 3 and 4, there will be situations where the car might be driving, but the driver is still responsible. “Then, the question becomes a little bit more difficult. Most of the time the driver will rely on the vehicle doing things correctly, but they still have to be present and alert to take over when needed.”

Security

In addition to safety, Harman’s digital cockpits are renowned for their audio systems and voice recognition. The cockpit features neural networks that provide artificial intelligence functions like face and natural voice recognition, which work both online and offline. “Security is absolutely an underlying part of voice recognition,” says Verhoeven. One branch of voice recognition includes improving voice identification. Going further, an area of concern includes the example of if a child was to get into a self-driving car and request a journey—the technology for this consideration hasn’t been fully developed yet. “Voice signatures are there, and having the software recognise differences is a mature technology,” he says, as his team continue to develop this further.

Another aspect of security includes the connected vehicle and malware: “Security for our customers is security in the context of an inherently connected vehicle, which is where curating the apps within the cockpit also becomes important.”

Although the future is important, Verhoeven believes that now is a pivotal time in the automotive sector. The next step for the industry is stepping back from focusing wholly on the latest technology innovation and instead moving towards the user experience—“We call it experiences per mile.” Furthermore, Harman could impact the development of cockpits greatly, having formed and chaired an organisation where different players within the ecosystems dig deep into the question ‘What is the user experience?.’