Template-based facial recognition

November 5, 2021

This post is an edited version of my Twitter thread from 3 November 2021.

On 2 November 2021, Facebook announced that they will delete the data and shut down the facial recognition system on Facebook.

Which data is being deleted? Facebook’s blogpost does not say that they will delete the models that were generated using the data. It also does not say that they will not use people’s image data to train models. It only says “we will delete more than a billion people’s individual facial recognition templates.” Is deleting templates enough?

What are these templates? Templates are not images. Templates are generated using images. To understand how templates fit into a facial recognition system, it can be useful to understand the different steps involved in a template-based facial recognition system 1. Here is a simplified version of the steps involved:

  1. Image collection: A collection of images. People upload images in Facebook. So, collection is easy.
  2. Creating templates: Using a combination of algorithms to process the collected images of a person to smooth out non-facial elements and the background.
  3. Storage: These templates are kept in a database.
  4. Template-matching algorithm (s) 2
  5. Identification system: Use the template matching algorithm to compare and match new images to the templates in the database to identify faces.

Facebook’s announcement only affects steps (3) and (5). As long as they have the critical components (1) access to images, (2) the algorithms to generate templates and (4) the template matching algorithm, generating new templates and turning on the system is a matter of re-running old programs and scripts.

The announcement indicates that they are pausing (not shutting down for good) the use of the system so that they can restart it.

Looking ahead, we still see facial recognition technology as a powerful tool, for example, for people needing to verify their identity, or to prevent fraud and impersonation. We believe facial recognition can help for products like these with privacy, transparency and control in place, so you decide if and how your face is used.

There is also the possibility that Facebook might be using this announcement as part of a tactic to push the argument that without facial recognition, they are unable to make their platform accessible.

This change will also impact Automatic Alt Text (AAT), which creates image descriptions for blind and visually-impaired people. After this change, AAT descriptions will no longer include the names of people recognized in photos but will function normally otherwise.

  1. Template-based facial recognition is only one of the methods. There are other methods for facial recognition systems such as relying on specific features of the object such as the boundary, shape, colour, etc.↩︎

  2. The simplest version of template matching will try to find whether the exact template is present in the new image. More sophisticated algorithms account for different sizes of the face, the angle of the face, the brightness and quality of the image, etc.↩︎