py-faceblur/DEV_GUIDE.md

PyFaceBlur Developer & AI Guide

This document provides architectural context, data flow maps, and strict guidelines for human developers and AI coding assistants working on the py-faceblur codebase.

Note to AI Assistants: Read this entire document before proposing architectural changes or adding dependencies to this project.

Architecture & Data Flow

The project is structured as a linear, sequential CLI pipeline. It extracts frames, detects faces, clusters them by identity, asks the user for input, and re-encodes the video.

1. Module Responsibilities

• src/faceblur/app.py: The main entry point. Orchestrates the sequential CLI flow using rich for UI/progress and questionary for user prompts.
• src/faceblur/video.py: Uses ffmpeg (via subprocess) to extract frames from the input video at a specified interval.
• src/faceblur/detect.py: Wraps the UniFace library. Uses RetinaFace for bounding boxes and landmarks, and ArcFace for generating 512-dimensional facial embeddings.
• src/faceblur/cluster.py: Takes the 512-dim embeddings and runs DBSCAN (from scikit-learn) using the cosine distance metric to group identical faces into clusters.
• src/faceblur/blur.py: Contains the image manipulation logic. Responsible for drawing blurs (Gaussian, pixelate, etc.) and handling the linear interpolation of bounding boxes between keyframes.
• src/faceblur/encode.py: Probes the video, auto-detects hardware encoders, and pipes raw BGR bytes from OpenCV to an ffmpeg subprocess to re-encode the video with blurs applied.
• src/faceblur/cli.py: The legacy/debug CLI entry point. It runs the pipeline but dumps raw frames and face crops to disk instead of re-encoding a video.

2. The Data Pipeline

1. Extraction: Video -> Every Nth Frame saved to /tmp/.../frames/
2. Detection: Frame -> FaceData dataclass (contains id, bbox (x1, y1, x2, y2), embedding, landmarks).
3. Clustering: List of FaceData -> List of Cluster dataclasses.
4. Interpolation (Encoding):
   • A lookup table of keyframes is built.
   • For frames between keyframes, bounding boxes are linearly interpolated.
   • If a face enters or leaves, the bounding box is held static to the nearest known location to prevent split-second privacy exposures.
5. Muxing: OpenCV reads frames -> Blur applied -> Piped as rawvideo to FFmpeg -> FFmpeg copies original audio and encodes video.

Core Dependencies

• UniFace (uniface): The core AI engine for detection and recognition. Do not swap this out for MediaPipe, dlib, or plain OpenCV Haar cascades. It uses ONNX runtime under the hood.
• FFmpeg: Required on the host system. Used for both initial frame extraction and final video re-encoding.
• OpenCV (opencv-python): Used exclusively for image reading/writing and drawing the actual blur arrays. Not used for video demuxing/muxing (FFmpeg handles that).
• scikit-learn: Used purely for the DBSCAN clustering algorithm.
• Rich & Questionary: Used for the terminal user interface.

Guidelines & Strict Rules for AI Agents

When modifying this codebase, adhere to the following rules:

1. Hardware Encoding Fallback is Sacred: In encode.py, the find_best_encoder() function meticulously probes the system for av1_vaapi, hevc_vaapi, h264_vaapi, h264_nvenc, etc., using a 1280x720 test video. Do not simplify or break this logic. If you modify it, ensure hardware encoders are prioritized over CPU (libopenh264).

2. Bounding Box Format: Bounding boxes are strictly formatted as (x1, y1, x2, y2). Do not attempt to use (x, y, w, h) anywhere in the pipeline.

3. Dependency Discipline: Do not add heavy dependencies (like PyTorch or TensorFlow) unless explicitly requested by the user. UniFace running on ONNX is specifically chosen to keep the footprint light (~50MB).

4. UI Paradigm: The CLI uses a synchronous, sequential prompt flow (questionary). Do not introduce asynchronous UI frameworks (like Textual) or background worker threads. The application blocks while waiting for user input, and blocks while processing with a rich progress bar.

5. Temp File Cleanup: The pipeline generates gigabytes of raw frames in a temp directory during execution. Ensure any new exit paths or error states properly trigger the shutil.rmtree(temp_dir) cleanup block in app.py.