Teaching Linux to Recognize My Face

If you bought a laptop in the last five years and immediately put Linux on it, there is a good chance you paid for hardware you have never once used. Right next to your normal webcam sits a dedicated infrared camera and a little IR emitter. On Windows, that hardware runs Windows Hello. On Linux, its primary job is gathering dust while you mistype your sudo password three times in a row because your fingers are cold.

Yes, projects like Howdy have existed for a while, and they deserve credit for proving this works. Setups that run Python directly inside the PAM authentication flow have always made me a little nervous, though. Loading an entire Python interpreter and heavy machine learning libraries every single time you run sudo apt update adds noticeable lag, and a routine system upgrade that breaks a Python package can suddenly make logging into your own computer a very exciting puzzle.

So I built Linux-Hello. The goal was simple: make facial authentication on Linux feel instant, native, and boringly reliable, the same way fprintd handles fingerprint readers.

Keeping PAM on a Strict Diet

Anyone who has ever edited files in /etc/pam.d/ knows the mild adrenaline rush of wondering if you just locked yourself out of your own machine. (Pro tip: always keep a second terminal open with an active root shell before touching PAM. Ask me how I know.)

When a program asks PAM to authenticate you, that code runs inside the calling process. Stuffing OpenCV, neural network weights, and camera drivers directly into a shared library loaded by sudo or GDM is asking for trouble. Instead, Linux-Hello splits the work in two.

The PAM module itself is a tiny Rust shared library, barely 500 KB, with almost no dependencies. When you run sudo, pam_linux_hello.so just opens a Unix socket at /run/linux-hello.sock, asks the background daemon if you are sitting in front of the screen, and waits for a yes or no. If the daemon is stopped, or the socket is missing, or your laptop lid is closed in a dock, the module immediately steps aside and hands you back the normal password prompt. No hanging for ten seconds, no cryptic errors.

Meanwhile, the heavy lifting lives in linux-hellod, a systemd service that keeps the ONNX models loaded and warm in memory. Because the models are already initialized, actual face verification takes well under 100 milliseconds. By the time your pinky finger leaves the Enter key after typing sudo whoami, the IR light has blinked once and your command is already running.

Wrestling with Infrared Hardware

Talking to cameras on Linux means talking to V4L2. Rather than pulling in a huge multimedia framework, Linux-Hello talks directly to /dev/video* using ioctl calls and memory-mapped (mmap) buffers.

First, there is a strict rule baked into camera discovery: it will only talk to an actual infrared camera (checking for grayscale formats like GREY or IR device metadata). It will never fall back to your normal RGB webcam on /dev/video0, because unlocking a machine with an ordinary color webcam is how you end up getting pwned by someone holding up your LinkedIn profile picture.

Once you actually open an IR stream, hardware quirks start showing up fast. Infrared cameras work by strobing an IR LED, which means half the frames coming off the sensor are pitch black ambient frames and the other half are lit frames. Linux-Hello filters the stream on the fly so the face detector only sees frames where the emitter actually fired.

Even weirder, a fair number of laptop UVC cameras boot up with their hardware auto-exposure Region of Interest set to a broken 1x1 pixel box at the top-left corner of the frame. Left alone, the camera cranks the exposure to the moon and turns your face into a glowing white orb that no neural network on earth can recognize. Linux-Hello resets the hardware exposure region to the full frame on startup, and once it spots your face, tells the camera firmware to meter exposure specifically on your face coordinates.

No, My Hoodie Is Not the System Administrator

Under the hood, a C++17 bridge passes frames into OpenCV's DNN engine running two compact ONNX models from the OpenCV Zoo: YuNet for finding faces and five facial landmarks (both eyes, nose tip, and mouth corners), and SFace for extracting a 128-dimensional embedding vector that gets compared via cosine similarity.

During early testing, I learned a fun truth about fast face detectors: if you stare at enough grayscale noise, crumpled hoodies, curtain folds, and couch pillows start looking a lot like human faces.

To keep inanimate household objects from gaining root privileges, Linux-Hello runs an anatomical sanity check on every detection before it even bothers with recognition. The bounding box has to have human proportions. The eyes have to be a reasonable distance apart. The mouth has to be below the eyes, and the nose has to sit between them. It also checks pixel contrast and dynamic range around the eye landmarks to make sure both eyes are actually open and visible, which stops it from matching if your eyes are covered by a hand or if you are asleep at your desk.

Enrollment works much like Windows Hello. Rather than taking a single flat snapshot and hoping for the best, the CLI and GUI walk you through a five-pose registration: center, left, right, up, and down. It estimates head yaw and pitch from your facial landmarks, makes you hold each angle steady for three frames, and stores those five templates locally. That way, it still recognizes you when you are slouching in your chair at 11 PM.

A GUI That Doesn't Care Which Desktop You Use

Because Linux users can never agree on a desktop environment, I built linux-hello-gui as a standalone app using eframe and egui. Whether you run GNOME, KDE Plasma, XFCE, or a tiling window manager you spent all weekend configuring in Lua, it just opens and works at 60 FPS.

You get a live IR camera preview with landmark overlays, the five-pose enrollment ring, a live recognition debugger that shows your cosine similarity score in real time, and one-click buttons (via Polkit pkexec) to toggle PAM between system-wide authentication and a safer sudo-only mode. And if you live in the terminal, the linux-hello CLI lets you test images offline, inspect cameras, and stream live debug telemetry without ever touching a mouse.

It is open source under the Apache 2.0 license, packages cleanly into a .deb for Debian and Ubuntu (using pam-auth-update properly instead of hacking config files with sed), and builds from source in a couple of commands on Fedora and Arch. If you have an IR camera sitting idle at the top of your screen, give it a spin.