PersonaLive: Tạo ảnh động chân dung thời gian thực cho live stream với CVPR 2026

PersonaLive la mot framework diffusion mo nguon ma cho phep tao anh dong chan dung thoi gian thuc tu mot anh tham chieu va video dan dong, giai quyet van de tao noi dung hoat hinh khuon mai cho live streaming ma khong can quay phim. Duoc chap nhan tai CVPR 2026, tool nay cua nhom GVCLab la lua chon danh cho developer va researcher muon xay dung ung dung tao video chan dung vo tan tren GPU pho thong (12GB VRAM). Ban co the xem chi tiet tai repository goc tren GitHub.

Nội dung

Thank you for reading this post, don't forget to subscribe!

Nhung diem chinh

Tao anh dong vo tan: Sinh khung hinh lien tuc khong gioi han nho co che streaming strategy, chi can 12GB VRAM.
Ho tro thoi gian thuc: Tich hop WebUI va ComfyUI de chay truc tiep trong live stream, co the thay anh tham chieu dong.
Tang toc TensorRT: Chuyen doi model sang TensorRT giup tang toc do inferen len gap 2 lan (can build engine rieng cho moi device).
Huan luyen 3 giai doan: Tu warm-up anh don den tinh chinh temporal module, yeu cau 8x H100 nhung huan luyen tu do.
Tuong thich RTX 50-Series: Can tat xformers de tranh crash, co huong dan rieng cho Windows va Blackwell GPU.

PersonaLive la gi va giai quyet van de gi?

PersonaLive la mot khuon kho diffusion co kha nang phat truc tuyen (streamable) duoc thiet ke dac biet cho live streaming. No nhan mot anh chan dung tinh (reference image) va mot video dan dong (driving video) de tao ra mot chuoi khung hinh vo tan voi nhan dang va bieu cam duoc giu nguyen. Khac voi cac phuong phap truyen thong can xu ly toan bo doan phim, PersonaLive phat hien bo nho dem va stream tung phan de giam tai nguyen.

Cac van de chinh ma no giai quyet bao gom: chi phi xu ly cao trong live streaming (GPU re tien van co the chay), thieu tinh nhat quan nhan dang qua cac khung hinh, va kho khan khi can mo rong do dai video. Voi co che streaming strategy, PersonaLive co the tao video dai bat ky ma khong can bo nho GPU khoi luong thap. Day la cong cu ly tuong cho cac nha phat trien ung dung nhai hoat hinh hay tro choi.

Ban co the kham pha them tai PersonaLive GitHub.

Nhung tinh nang noi bat cua PersonaLive

Tao anh dong vo tan voi streaming strategy

Tinh nang quyet dinh cua PersonaLive la kha nang sinh khung hinh lien tuc (streaming gen). Khi bat --stream_gen=True, no chi load mot phan nho cua video hien tai vao bo nho, xu ly tung doan roi giai phong, giong nhu LiveStreaming. Dieu nay cho phep chay tren GPU chi 12GB VRAM ma van tao ra video dai hang nghin khung.

Tang toc TensorRT va tinh tuong thich

Nguoi dung co the chuyen doi mo hinh sang TensorRT de tang toc do inference len gap doi. Tuy nhien can chu y rang build engine mat khoang 20 phut va co the gay sai khac nhe ve chat luong. Mo hinh TensorRT di kem tu H100, nhung tac gia khuyen nen build lai tren may cua ban de dam bao tuong thich toi uu.

WebUI va ComfyUI

PersonaLive di kem WebUI tich hop Node.js de chay truc tiep trinh duyet. Nguoi dung co the tai anh tham chieu, kich hoat Fuse Reference, roi nhan nut Start Animation de xem ket qua. Ngoai ra con co ban ComfyUI-PersonaLive do cong dong pha trien, giup tich hop vao workflow ComfyUI.

Ho tro nhieu dong GPU

Tool ho tro xformers de tiet kiem bo nho (mac dinh bat). Tuy nhien, nguoi dung RTX 50-Series (Blackwell) can tat xformers de tranh crash: python inference_offline.py --use_xformers False. Cung co huong dan rieng cho Windows RTX 50-Series tu cong dong.

Huong dan cai dat va su dung co ban

De bat dau voi PersonaLive, ban can clone repo, tao moi truong conda, cai dat dependencies, tai pretrained weights, va chay infer. Duoi day la cac buoc toi thieu.

Clone repo va tao moi truong: git clone https://github.com/GVCLab/PersonaLive va conda create -n personalive python=3.10, sau do pip install -r requirements_base.txt.
Tai pretrained weights: Chay python tools/download_weights.py de tu dong tai cac thanh phan can thiet (sd-image-variations-diffusers, sd-vae-ft-mse, personalive weights). Ban cung co the tai thu cong ve folder ./pretrained_weights.
Chay infer offline: python inference_offline.py. Tham so tuy chinh: --reference_image, --driving_video, --stream_gen, --use_xformers. Vi du: python inference_offline.py --reference_image './path/photo.jpg' --driving_video './path/video.mp4'.
Chay WebUI (online): Cai dat Node.js 18+, chay source web_start.sh, sau do mo trinh duyet tai http://0.0.0.0:7860. Upload anh, nhan Fuse Reference va Start Animation.
Tang toc TensorRT (optional): Cai dat pip install -r requirements_trt.txt, chay python torch2trt.py, sau do chay inference online voi --acceleration tensorrt. Neu gap loi PyCUDA, cai bang conda: conda install -c conda-forge pycuda 'numpy<2.0'.

Luu y: Ban co the dieu chinh Driving FPS trong WebUI de giam lao dong tinh toan. Neu GPU ban thap, giam FPS xuong hoac tang multiplier len.

Ai nen dung - ai khong nen dung PersonaLive?

Nen dung: Nhung nguoi phat trien ung dung can hoat hinh chan dung thoi gian thuc nhu live streaming, game, hoac chatbot co khuon mat. Cac researcher trong linh vuc diffusion va electron muon nghien cuu phuong phap streaming cung se thay gia tri. Neu ban co GPU 12GB+ va muon thu nghiem, tool nay rat de dung nho WebUI.

Khong nen dung: Neu ban khong co GPU hoac chi co card tich hop, PersonaLive khong chay. Neu ban can ket qua hoan hao ve chat luong (nhin mat cuc ky chinh xac), thi hieu nang co the bi giam khi dung TensorRT do sai khac nho. Doi voi nguoi dung Windows va RTX 50-Series, can bo qua mot so buoc thu cong de tranh crash.

Dong thoi, day la du lieu nghien cuu - khong khuyen khich tao noi dung trai phep theo Disclaimer cua du an. Neu ban la content creator khong co ky nang lap trinh, co the gap kho khi cai dat moi truong Python va conda.

Cau hoi thuong gap

PersonaLive co the tao video bao lau?

PersonaLive co the tao video vo han nho streaming strategy. Mac dinh infer offline gioi han 100 khung (-L 100), nhung ban co the set bat ky so luong nao. Khi dung streaming, bo nho chi chua mot phan video, vi vay khong co gioi han do VRAM.

Co can GPU nang de chay infer offline khong?

Khong nhat thiet. PersonaLive co the chay tren GPU 12GB VRAM nho xformers va streaming gen. Vi du voi GTX 1080 Ti (11GB) hoac RTX 3060 (12GB) deu du. Neu ban bat TensorRT, can GPU ho tro.

Tool co ho tro tieng Viet khong?

Giao dien WebUI hien tai bang tieng Anh mac dinh. Ban co the custom lai frontend. Phan chuong trinh xu ly khong phu thuoc ngon ngu, nen anh dong co the ap dung cho bat ky khuon mat nao.

Co the thay anh tham chieu khi dang chay live stream khong?

Co. Trong WebUI, sau khi nhan Fuse Reference va bat dau animation, ban van co the tai len anh tham chieu khac va nhan lai Fuse Reference de cap nhat. Tinh nang nay duoc them vao ban cap nhat cuoi nam 2025.

Huong dan huan luyen co kho khong?

Huan luyen tu do (train from scratch) thuong yeu cau 8x H100 va mat ~48h cho ca ba giai doan. Tuy nhien, nguoi dung chi can infer co the tai weights co san. Neu muon fine-tune, can chuan bi dataset voi videos va boxes, chay preprocessing tools.

Ket luan

PersonaLive la mot buoc tien thuc te trong linh vuc tao hoat hinh chan dung thoi gian thuc, dac biet phu hop voi live streaming. No ket hop duoc ca chat luong cao cua diffusion, su linh hoat cua streaming strategy, va kha nang tang toc TensorRT. Neu ban co GPU va muon xay dung ung dung ban chat co khuon mat dong, PersonaLive la mot lua chon dang de xem xet. Hay clone repo, chay thu va dung quen danh sao (star) tren GitHub de ung ho tac gia nhe!