Wallboard AI camera

This solution is based on the HMS (Huawei Mobile Services) ML kit face analyzing engine. It is capable of detecting faces on camera stream, analyze faces and return their attributes (such as age, gender, smile, etc.). The application works 2 ways, calculating the tracked face and create a report from the collected data, and change the state of different events depending on the analyzed faces.

The collected data are the following:

Gender: male/female
Age category: (We decided to categorize the ages in order to correct the inaccuracy of the engine.)
- child(0-15)
- junior(16-24)
- young(25-40)
- mature(41-64)
- senior (65+)
Presence Duration: time spent in front of the camera in milliseconds
- Attention duration: time spent in front of the camera and the axis Y was less than +/-15 degrees
Smiling: currently smiling or not (in the report this will be aggregated)
Matrix location X/Y: if the location matrix is available, then the last x/y coordinates

Requirements

You can buy the package (Giada media box with Wallboard firmware and the application on it) from Wallboard only.

Giada DN74 media box
Webcam (HD, FullHD)
Serial cable (communication with client box)
Wallboard Sensor Designer Application (for providing configuration)

UI and states

The application has 4 states which are the following:

Initializing

This state is responsible for initializing the application, checking the permissions, and waiting for the boot completion. The next stage will be one of the main screens depending on if a camera is attached or not. If you remove an attached camera then the application restarts from this state.

Main screen:

Without an attached camera, you will see this screen until there is a webcam attached:

With an attached camera, you will see this screen:

After you attach a webcam to the media box you can see the main screen. On the main screen, you can see the following:

Resolution and the fps of the camera which has been set in the configuration (top-bar)
Number of measurements in 1 minute and in total (top-bar)
Running time of the application (top-bar)
camera stream and the overlay (middle)
Version of the application next to the Wallboard logo
Current summary of the seen faces:
- Number of faces
- Gender ratio (most)
- Average age category
- Smile ratio (most)
- Looking ratio (most)

To start the engine of the application and see the camera source and the analyzer results, you will need to set the configuration. The configuration can be set in the Wallboard Sensor Designer application (in detail later).

To open the recovery menu type ‘ wbd ’ with an attached keyboard.

The recovery menu contains options and some functions for the application. You are able to open the android system settings from the first option, update the application over the cloud or from a flash drive.

Note: Lock/Unlock functions are disabled on Giada devices and Update from a flash drive is currently unavailable.

Configuration

The configuration can be built in the Wallboard Sensor Designer application. There are two ways to set the configuration to the AI camera device:

Overcloud with a connected client application
Connected to the computer and push configuration to the device directly

You can set the parameters of the camera, preview, serial message (simple message/complex JSON object), events:

Id: The ai camera id, which will be attached to the report and event messages.
Camera resolution: the source of the camera resolution (recommended: 1280x720/1920x1080).
Camera fps: the frames per second will be obtained from the webcam. (recommended: 30/60).
Camera facing: the characteristics of the webcam, can be different on each device. (Note: Giada DN74 uses camera facing BACK option.)
Show preview: set the visibility of the camera source.
Highlight faces: set the visibility of the overlay which will show the results of the analyzer (depends on preview).
Simple message: set the structure of the data, sent on serial (simple string / JSON object). Select only if the client is a Samsung device.
Face lost delay: the time that needs to pass in milliseconds after a face disappears to report it.

Other options are the events and parameters of the messages. Note: Smiling range parameters means the percentage of people who need to smile.