colibri

A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.

Loading community details…
Loading project owner…
Loading launchers…
Loading check-ins…

Community

Create post
Ask or share about colibri…
Loading...
colibri on Pinokio