Time of Flight – 3D Gesture Recognition

Use Cases – Hands-free Operation

ToF Gesture Sensor

Automotive-compliant Gesture Sensor

Embedded AI is developing its own automotive-compliant optical sensor based on the principle of Time of Flight (ToF) in the infrared light range. The main task of the sensor is to recognize hand and foot gestures for opening car doors, controlling automotive HMIs, or hands-free control (infection protection) of terminals or other objects. It can also be used for acquiring 3D data as well as measuring distances and geometric structures of objects.

The ToF sensor can be used both inside and outside vehicles, as well as outside the automotive environment.

ToF Measurement Principle

The ToF measurement principle is capable of measuring the travel times of light and calculating distances from them. For this purpose, the scene is illuminated with an infrared light pulse, and the light reflected from objects is received by an IR ToF camera chip specifically designed for this application. By using the light travel time, which is proportional to the distance, distances can be determined for each pixel. Thus, the ToF sensor is capable of generating not only 2D but also 3D camera images.

ToF Chip with Minimal Energy Consumption

The core of the sensor is a ToF chip developed and certified for automotive use, optimized for low-cost applications. The camera chip operates on the ToF principle of pulsed light modulation and has a low resolution of 32×32 pixels, resulting in a similarly low-resolution 3D image. The 32×32 pixel matrix enables low energy consumption and system costs due to the reduced computational power required.

All-in-One Solution – Embedded AI Develops …

  • AI Algorithms for 3D Gesture Recognition
    • Modeling/Simulation in Matlab/Python
    • Real-time Implementation in C++ (cf. Embedded Software)
    • Data Engineering
  • Electronics Development (PCB)
  • Microcontroller (MCU) Selection
  • Embedded Software
    • Hardware Abstraction Layer (HAL)
    • Function Layer
    • Application Layer with AI Algorithms for 3D Gesture Recognition
    • System Software, Diagnostic Software
  • Optical Frontend
    • Evaluation and Selection of Emitting Diode (High-Power VCSEL Diode)
    • Transmitting Optics Integrated into the VCSEL Diode
    • Receiving Optics with Partners
  • Housing Design

Applications in Vehicles

Interior

HMI and Car Infotainment System

The following video demonstrates the gesture-based control of a car infotainment system (CIS) using the ToF gesture sensor from Embedded AI. For the demonstrator, the installation position was chosen to be below the display. Other installation positions for the sensor are also possible.

Exterior

In addition to its use in the vehicle interior, the sensor can also be employed for functions in the exterior area of the vehicle. Mounted at the rear of the vehicle, it serves as a trunk opener, allowing the trunk to be opened with a foot gesture. Positioned on the side (B-pillar), it offers a completely new control panel accessible via hand gestures.

Trunk Opener

In contrast to conventional solutions based on capacitive sensors, the ToF gesture sensor offers the advantage that the operating area is placed behind the vehicle, eliminating the need for users to perform the gesture while standing on one leg. The action area is signaled to the user via a spotlight.
By stepping into the spotlight and then removing the foot from the action area, the gesture is executed.
Thanks to the innovative control concept combined with the precise measuring properties of ToF technology, a very robust and secure gesture control can be ensured.

B-pillar

By installing the sensor in the B-pillar, new control options arise in the exterior area of the vehicle. It is thus not only possible to unlock the vehicle and open doors with a gesture but also various control options emerge. For instance, the process of electric charging can be controlled via the gesture interface on the B-pillar. Further use cases are conceivable.

The following video demonstrates the use of the ToF gesture sensor as a trunk opener and as a control interface in the B-pillar.

Interested?

We hope that we could provide you with a small insight. If you are interested in viewing demos or live applications, please feel free to contact us.

  • E-mail: po[at]embedded.ai
  • Phone: +49 351 6533 3808