Pulse
0
Velocity
Stars
0
Run large language models locally with dynamic batching for efficient inference.
Optimizes LLM inference by dynamically batching requests for better throughput.
Run large language models locally with dynamic batching for efficient inference.
Why Trending
Achieves higher throughput for local LLM inference using dynamic batching.
Target Audience
ML engineers and researchers running LLMs locally.
Similar Projects