FaceNet512
Introduction
FaceNet is a neural network for face re-identification based on the Inception architecture. It regresses a face feature vector (embedding) that can be matched against a database of known face vectors. In this version the embedding has dimension 512. The weights come from the DeepFace implementation.
Note: This is not a face detection model. Faces must be detected and cropped before being passed to FaceNet.
Key Features
- Compact 512-dimensional face embedding suitable for verification and identification tasks.
- Fully quantized int8 model with uint8 input and float32 output.
- Compiled for i.MX 93 Ethos-U65 NPU and converted for i.MX 95 / i.MX 952 Neutron NPU.
Model Description
Modifications
The original Keras model from the DeepFace library is converted directly to TensorFlow Lite with full int8 quantization using 100 calibration images from a commercially-usable ImageNetV2 subset. The input type is uint8 (values 0-255) and the output type is float32.
Model Information
| Information | Value |
|---|---|
| Input shape | RGB face crop (1, 160, 160, 3), uint8 |
| Output shape | Face embedding vector (1, 512), float32 |
| FLOPS | 2.84 GOPS |
| Number of parameters | 23,497,424 |
| Source framework | TensorFlow/Keras (DeepFace 0.0.75) |
| Target platform | MPUs (i.MX 8M Plus, i.MX 93, i.MX 95, i.MX 952) |
Tested Configurations
The quantized int8 model has been tested on i.MX 8M, i.MX 93, i.MX 95 and i.MX 952 using benchmark-model (see i.MX Machine Learning User Guide).
Training and Evaluation
The original model was trained and evaluated on the LFW dataset, achieving 99.65% accuracy according to the DeepFace source.
We re-evaluated the model on LFW deep-funneled images from the Kaggle LFW dataset using the pairsDevTest split (1000 pairs, cosine distance threshold 0.6):
| Model | Accuracy |
|---|---|
| FaceNet512 Keras float32 | 97.5% |
| FaceNet512 TensorFlow Lite int8 | 97.6% |
The evaluation script is evaluate.py.
Conversion/Quantization
The original Keras model is converted directly to TensorFlow Lite with full int8 post-training quantization. 100 random images sampled from a commercially-usable ImageNetV2 subset are used as calibration data, resized to the 160x160 model input and scaled to [0, 1]. Point DATASET_PATH at your local ImageNetV2 subset root (a directory of per-class image subfolders). The conversion is performed using TensorFlow 2.x with:
inference_input_type = tf.uint8inference_output_type = tf.float32OpsSet.TFLITE_BUILTINS_INT8
The i.MX 93 model is compiled from the int8 TFLite model using Vela tag lf-6.18.20_2.0.0 with --accelerator-config ethos-u65-512 --optimise Performance.
The i.MX 95 and i.MX 952 models are converted using eIQ Neutron SDK version 3.1.3.
Download and Run
To regenerate the TFLite model fully quantized in int8 (uint8 input, float32 output), run:
bash recipe.sh
An example showing how to compute the cosine similarity between two face images is in example.py:
python example.py -m original_model/facenet512_uint8_float32.tflite -i face.jpg -i2 face2.jpg
To evaluate the model on the LFW pairsDevTest split, download the required data and run:
# Download LFW deep-funneled images
wget https://vis-www.cs.umass.edu/lfw/lfw-deepfunneled.tgz
tar xzf lfw-deepfunneled.tgz
# Download pairs file
wget https://vis-www.cs.umass.edu/lfw/pairsDevTest.txt
python evaluate.py -m original_model/facenet512_uint8_float32.tflite
Full on-device examples leveraging GStreamer/NNStreamer and this model can be found in nxp-nnstreamer-examples on GitHub
Origin
Model implementation: https://github.com/serengil/deepface/
Original article: Schroff, Florian, Dmitry Kalenichenko, and James Philbin. "FaceNet: A Unified Embedding for Face Recognition and Clustering." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015. https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Schroff_FaceNet_A_Unified_2015_CVPR_paper.html