diff --git a/DEV_GUIDE.md b/DEV_GUIDE.md new file mode 100644 index 0000000..92890a1 --- /dev/null +++ b/DEV_GUIDE.md @@ -0,0 +1,57 @@ +# PyFaceBlur Developer & AI Guide + +This document provides architectural context, data flow maps, and strict guidelines for human developers and AI coding assistants working on the `py-faceblur` codebase. + +> **Note to AI Assistants:** Read this entire document before proposing architectural changes or adding dependencies to this project. + +## Architecture & Data Flow + +The project is structured as a linear, sequential CLI pipeline. It extracts frames, detects faces, clusters them by identity, asks the user for input, and re-encodes the video. + +### 1. Module Responsibilities + +- **`src/faceblur/app.py`**: The main entry point. Orchestrates the sequential CLI flow using `rich` for UI/progress and `questionary` for user prompts. +- **`src/faceblur/video.py`**: Uses `ffmpeg` (via subprocess) to extract frames from the input video at a specified interval. +- **`src/faceblur/detect.py`**: Wraps the **UniFace** library. Uses RetinaFace for bounding boxes and landmarks, and ArcFace for generating 512-dimensional facial embeddings. +- **`src/faceblur/cluster.py`**: Takes the 512-dim embeddings and runs **DBSCAN** (from `scikit-learn`) using the **cosine distance** metric to group identical faces into clusters. +- **`src/faceblur/blur.py`**: Contains the image manipulation logic. Responsible for drawing blurs (Gaussian, pixelate, etc.) and handling the linear interpolation of bounding boxes between keyframes. +- **`src/faceblur/encode.py`**: Probes the video, auto-detects hardware encoders, and pipes raw BGR bytes from OpenCV to an `ffmpeg` subprocess to re-encode the video with blurs applied. +- **`src/faceblur/cli.py`**: The legacy/debug CLI entry point. It runs the pipeline but dumps raw frames and face crops to disk instead of re-encoding a video. + +### 2. The Data Pipeline + +1. **Extraction**: Video -> Every Nth Frame saved to `/tmp/.../frames/` +2. **Detection**: Frame -> `FaceData` dataclass (contains `id`, `bbox (x1, y1, x2, y2)`, `embedding`, `landmarks`). +3. **Clustering**: List of `FaceData` -> List of `Cluster` dataclasses. +4. **Interpolation (Encoding)**: + - A lookup table of keyframes is built. + - For frames between keyframes, bounding boxes are **linearly interpolated**. + - If a face enters or leaves, the bounding box is held **static** to the nearest known location to prevent split-second privacy exposures. +5. **Muxing**: OpenCV reads frames -> Blur applied -> Piped as `rawvideo` to FFmpeg -> FFmpeg copies original audio and encodes video. + +## Core Dependencies + +- **UniFace (`uniface`)**: The core AI engine for detection and recognition. Do not swap this out for MediaPipe, dlib, or plain OpenCV Haar cascades. It uses ONNX runtime under the hood. +- **FFmpeg**: Required on the host system. Used for both initial frame extraction and final video re-encoding. +- **OpenCV (`opencv-python`)**: Used exclusively for image reading/writing and drawing the actual blur arrays. *Not* used for video demuxing/muxing (FFmpeg handles that). +- **scikit-learn**: Used purely for the `DBSCAN` clustering algorithm. +- **Rich & Questionary**: Used for the terminal user interface. + +## Guidelines & Strict Rules for AI Agents + +When modifying this codebase, adhere to the following rules: + +1. **Hardware Encoding Fallback is Sacred**: + In `encode.py`, the `find_best_encoder()` function meticulously probes the system for `av1_vaapi`, `hevc_vaapi`, `h264_vaapi`, `h264_nvenc`, etc., using a `1280x720` test video. Do not simplify or break this logic. If you modify it, ensure hardware encoders are prioritized over CPU (`libopenh264`). + +2. **Bounding Box Format**: + Bounding boxes are strictly formatted as `(x1, y1, x2, y2)`. Do not attempt to use `(x, y, w, h)` anywhere in the pipeline. + +3. **Dependency Discipline**: + Do not add heavy dependencies (like PyTorch or TensorFlow) unless explicitly requested by the user. UniFace running on ONNX is specifically chosen to keep the footprint light (~50MB). + +4. **UI Paradigm**: + The CLI uses a synchronous, sequential prompt flow (`questionary`). Do not introduce asynchronous UI frameworks (like Textual) or background worker threads. The application blocks while waiting for user input, and blocks while processing with a `rich` progress bar. + +5. **Temp File Cleanup**: + The pipeline generates gigabytes of raw frames in a temp directory during execution. Ensure any new exit paths or error states properly trigger the `shutil.rmtree(temp_dir)` cleanup block in `app.py`. \ No newline at end of file