SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

An innovative library for efficient LLM inference via low-bit quantization

Reimplement RetinaFace use C++ and TensorRT

tensorrt int8 量化yolov5 onnx模型

TensorRT int8 量化部署 yolov5s 模型，实测3.3ms一帧！

Pure-Rust, CPU-only OCR engine for Baidu Unlimited-OCR (a DeepSeek-OCR-derived 3B MoE VLM). One fixed model, custom int4/int8 kernels, no ML framework, no GPU. Pre-Phase-0 scaffold.

a simple pipline of int8 quantization based on tensorrt.

RepVGG TensorRT int8 量化，实测推理不到1ms一帧！

Nodes to run Hunyuan Image 3 locally with BF16 and NF4 quantized options in Comfyui

👀 Apply YOLOv8 exported with ONNX or TensorRT(FP16, INT8) to the Real-time camera

Trustworthy onboard satellite AI in PyTorch→ONNX→INT8 with calibration, telemetry, and a PhiSat-2 EO tile-filter demo.

World's First NMS-Free YOLOv26n on ESP32-P4. Features end-to-end Int8 QAT and custom C++ optimizations achieving 30% faster inference than the official ESP-DL YOLOv11n (1.7s vs 2.4s).

nanodet int8 量化，实测推理2ms一帧！

NCNN+Int8+YOLOv4 quantitative modeling and real-time inference

INT8 calibrator for ONNX model with dynamic batch_size at the input and NMS module at the output. C++ Implementation.

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.

TensorRT Int8 Python version sample. TensorRT Int8 Python 实现例子。TensorRT Int8 Pythonの例です

An stress and benchmark utility for NVIDIA GPUs. Measures performance across various precisions (FP64, FP32, TF32, FP16, INT8) and monitors real-time vitals like power, temperature, and clock speeds.

RISCV Vector Kernel C/LLVM-IR generator

MT-Yolov6 TensorRT Inference with Python.