Dev
GitHub repos gaining traction - what high-signal users are starring and what's climbing the board, captured daily and enriched from GitHub. Raw material for spotting new tech and patterns worth building on.
3,195
repos tracked
105
surfaced this week
193
created < 30d
Python
top language
73 repos
-
An open-source inference engine for AI functions in SQL
-
LLM inference in C/C++
-
🧱 Experimental library for building inference engines
-
A high-throughput and memory-efficient inference and serving engine for LLMs
-
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
-
Implement a reasoning LLM in PyTorch from scratch, step by step
-
SGLang is a high-performance serving framework for large language models and multimodal models.
-
Exact probabilistic inference by compilation into tree decision diagrams
-
Machine Learning Engineering Open Book
-
Code for 'Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling'
-
A high-throughput and memory-efficient inference and serving engine for LLMs
-
Port of OpenAI's Whisper model in C/C++
-
Phonon TTS inference using xn
-
DeepInfer: High-performance, large-scale inference framework for DeepSeek models
-
Rails as a specification; the deployment target is a build flag
-
Use Hugging Face with JavaScript
-
Mathematical reconstruction and verification of dive tables and decompression algorithms, minimal-compartment models, robust inference from incomplete dive histories, and empirical decompression-risk analysis.
-
A high-throughput and memory-efficient inference and serving engine for LLMs
-
A high-throughput and memory-efficient inference and serving engine for LLMs
-
Programmable chat templates for LLM training and inference.
-
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
-
The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
-
Compile natural-language function descriptions into reusable local PAW programs.
-
A Long-Horizon Benchmark for Self-Improving Inference Systems
-
Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild
-
Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
-
LLM inference in C/C++
-
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
-
LLM inference in C/C++
-
TPU inference for vLLM, with unified JAX and PyTorch support.
-
Evidence-bound, deadline-aware inference coordination for continuous video and physical agents
-
A measurement-first lab for low-latency local camera inference with Glance.
-
LLM inference in C/C++
-
LLM inference in C/C++
-
★ 0 jxmorris12/vllmA high-throughput and memory-efficient inference and serving engine for LLMs
-
Batched single-token choice inference for open language models, compatible with TypeSafe
-
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
-
A high-throughput and memory-efficient inference and serving engine for LLMs
-
Accurate, large-scale, and extensible simulator for LLM inference Systems
-
FlashInfer: Kernel Library for LLM Serving
-
RTX 6000 Pro Wiki — Running Large LLMs (Qwen3.5-397B, Kimi-K2.5, GLM-5) on PCIe GPUs without NVLink
-
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
-
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
-
Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
-
Infrastructure for continually self‑improving agents
-
Embarrassingly parallel fasttext batch inference in Rust.
-
Sub-real-time MiniMax H3 on Hopper: 13.506 s for a 14.375 s 768p video with audio on 8xH100. Four runtime patches removing 9.72 s of non-model overhead from FastVideo.
-
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
-
dInfer: An Efficient Inference Framework for Diffusion Language Models
-
A Simple Transport Layer For Teleoperation And Inference