VeloxQuant-MLX
Free, open-source Python tool that compresses local LLM KV caches on Apple Silicon Macs to reduce memory use during inference.
VeloxQuant-MLX is a developer tool for MLX-based local language-model inference on Apple Silicon Macs. It provides 43 KV-cache compression and token-eviction methods, Metal GPU kernels, and a drop-in Python API requiring roughly three lines of integration. It supports existing mlx_lm models and is distributed under the MIT license.
What VeloxQuant-MLX does
- Compresses LLM KV caches
- Provides 43 compression algorithms
- Supports token eviction and quantization
- Uses Metal GPU kernels
- Offers a drop-in Python API
VeloxQuant-MLX on Founder Battle
- Global rank
- #26
- Category rank
- #3 in Developer Tools
- Finished battles
- 8
- Wins
- 4
- Losses
- 4
- No contests
- 0
Recent battle results
- Lost vs Scanwork · Global leaderboard ·
- Won vs CryptoPro Trading Learning Suite · Global leaderboard ·
- Won vs CryptoPro Trading Learning Suite · Global leaderboard ·
- Lost vs Nuuu · Global leaderboard ·
- Lost vs GovParti · Global leaderboard ·