RCLI

Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG

cactus

Kernels & AI inference engine for phones

Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

distributed-llama

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

PowerInfer

High-speed Large Language Model Serving for Local Deployment

MNN

MNN is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba. Full multimodal LLM Android App:[MNN-LLM-Android](./apps/Android/MnnLlmChat/README.md). MNN TaoAvatar Android - Local 3D Avatar Intelligence: apps/Android/Mnn3dAvatar/README.md