The focus of our today’s story is facial recognition and its operation phases. We will share little secrets of storing information and insights into how collected identification data is read, transmitted, and deleted. Kirill Bondarenko, CROC and NNTC expert, and contributor to the development of numerous facial recognition-based solutions (including iFalcon), will help us get down to the bottom of this technology.

facial recognition

Particular tasks and objects typically determine how exactly facial recognition will work. For example, a face recognition system in a mall collects statistical data: gender, age, and other shoppers’ details. This is one thing. Solutions like iFalcon Face Control Mobile are tasked to search for suspects, using a special rogue database. Yet another and completely different case is iFalcon Foreman that can track workers at construction sites. Depending on the place and goals, system components may vary, as well as the set of collected and stored data. To dispel any doubts about this technology, I will describe how facial recognition works in general and how data are collected and transmitted.

how face recognition works

Step #1

Imagine a camera connected to a facial recognition server. Somebody is approaching. The camera sees that person and the video stream is transmitted to the server (the video is not stored on the server; so if a customer wants a video stored, video management systems (VMS) are used). Then the face detection module is activated. The neural network captures human faces from a video stream, segments the face from the image background, and sends it to a facial recognition engine for analysis.

The module is smart. We have set in advance shots of what quality is considered acceptable, including the precise number of fine pixels, lighting, and head tilt. Setting these parameters helps exclude low-quality images, and the system will not waste time on them, but simply wait for a person to come closer. It’s like hunting with a camera for the best shot.

step 2

Step #2

The photo gets to the facial recognition system, where the face extractor module comes into play. This module’s job is to create something like a face mask and convert it into the data format comprehensible by the facial recognition system. There is no need to store photos in the system, since the facial recognition technology only needs that “mask” and not a specific photo.

This is how it works: the system analyzes frames to obtain a so-called face hash, a function that pinpoints and measures face features such as cheekbones, eyes, nose, etc. Legally, face hash is not personally identifiable information, because you need more data than that to identify a person. Moreover, the hash is not necessarily linked to a specific person (or personal record in the system).

step 3

Step #3

Database analysis. Now, having a face hash, the system can check it against a database of face hashes and give a match alert, if any. It should be noted that the system won’t give you a straight answer in a split second and recognize a stranger as “a vicious candy and bicycle thief”. Instead, the system will inform you about the “98.9% match between the face hash of the person X and the face hash of The Notorious Candy Thief“. Then an operator analyzes the notification with the photos and decides whether to detain this citizen or let them bounce their way to the next candy shop.

The system has a confidence percent configuration, which means you can set the rule to raise an alert with a minimum match of 98%. This feature is especially useful for biometric access control systems, like the one NNTC developed for the Oman Arab Bank mobile wallet app.

It is impressive indeed, how modern facial recognition engines do the job in mere seconds. Detecting and segmenting the face from the video stream is the most time-consuming stage due to a very large amount of data to process.

Where does the photo go?

The photo is stored on a dedicated server so that the alerted operator could quickly compare it with the one taken by the facial recognition system. This is optional, so you can always store only the face hash. However, a photo of a wanted criminal is most likely to be stored, because it can help the investigation. Besides, the operator will be able to match photos without any problems. It’s kind of funny to hear yesterday’s carjacker complaining about their face being stored in the police database, isn’t it?

criminal

Where do databases come from?

Not a single respectable facial recognition solution would be sold with any preloaded databases. That’s out of the question. We sell the tool, but never the database. Each customer collects its own database. The database for attendance tracking is enriched by the organization that had the solution installed in its office (with the employees’ consent to use their personal data, of course). If the police are the customer, then the police database shall be connected. If the system registers a random person in the crowd, for whom there is no record in the database, then this person is simply assigned a digit-like ID. It is entirely at the customer’s own discretion what data and how much data to store, and, of course, the facial recognition vendor has no access to these databases.

Read more