ExLlamaV2

Fast inference library for quantized Llama models on consumer GPUs with EXL2 format. — extracted from the official website or Wikipedia.

Learn ExLlamaV2

Recommended resources to get started

Interested in this technology?

Feel free to reach out if you would like to discuss this technology or explore how it can be applied to your projects.