Dev
GitHub repos gaining traction - what high-signal users are starring and what's climbing the board, captured daily and enriched from GitHub. Raw material for spotting new tech and patterns worth building on.
3,195
repos tracked
105
surfaced this week
193
created < 30d
Python
top language
114 repos
-
MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
-
Our library for RL environments + evals
-
A framework for few-shot evaluation of language models.
-
Learn harness engineering by building Claude Code from scratch. Free, open-source course: 8 articles, 6 videos, 1 codebase.
-
A benchmark for evaluating AI agents on realistic business workflows
-
Anserini is a Lucene toolkit for reproducible information retrieval research
-
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.
-
A Python library encapsulating best practices for rubric-based evaluation of LLM/VLM outputs using AI judges such as LLMs and decision models.
-
Local-first lab for testing AI agent workflows end to end: emulated Slack (signed events, buttons), local S3/SQS, deterministic authorization, and independent evidence instead of self-grading. Optional Google AX (Agent Executor) + Agent Substrate lane on GKE/gVisor, reported per assertion. Independent; not affiliated with Google.
-
A framework for building decision-model harnesses for any kind of task. Your program offers the model a short list of next steps, and the model picks one at a time until it can answer.
-
RankLLM is a Python toolkit for reproducible information retrieval research using rerankers, with a focus on listwise reranking.
-
Portable SOL visual policy evaluation, empirical latency profiles, and research evidence
-
Tree Decision Diagrams for Rust, Python and C: canonical circuits for Boolean functions, with model counting and weighted evaluation
-
the LLM vulnerability scanner
-
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
-
A toy eval suite for tracing generalization dynamics of LM pre-training
-
Semantic search in one SQLite file. No model, no server at query time.
-
Evaluation harness for OpenHands V1.
-
Executable RL environment for diagnosing and repairing distributed PyTorch training failures, with sandboxed execution and correctness-verified recovery rewards.
-
Ridiculously fast symbolic expressions
-
RewardBench: the first evaluation tool for reward models.
-
An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark and a playthrough app.
-
The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
-
Evaluation tools shared across anserini, pyserini, and pygaggle
-
cotrust.ai
-
Framework for evaluating and improving agents
-
Feather agent harness, training integrations, and evaluation tools for knowledge work
-
★ 0 neubig/harborHarbor is a framework for running agent evaluations and creating and using RL environments.
-
Track your Data Science. Skore's open-source Python library accelerates ML model development with automated evaluation reports, smart methodological guidance, and comprehensive cross-validation analysis.
-
ICMI Working Paper #16 — A Test of Faith: Christian Correctives to Evaluation Awareness. Experimental code, reasoning traces, and reproducible analysis for a BrowseComp-style honeypot study on Claude Opus 4.6.
-
VirtueBench V2: Multi-dimensional virtue evaluation benchmark for LLMs with tripartite and Ignatian temptation models
-
The platform for continuously improving AI agents.
-
📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG
-
Find code by asking what it does. A CLI for coding agents that uses Jev to discover relevant files and source context.
-
Automatic evals for LLMs
-
Official repository for "RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies".
-
Homework assignments for Evaluating and Improving AI Agents
-
Loom TUI and Character Lab for developing AI characters from base models: branch text, curate anthologies, simulate conversations, and evaluate training data.
-
Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains
-
Evaluation Resources
-
Generative Representational Instruction Tuning
-
★ 1.7k ai-evals-course/evals-skillsSkills that guide AI coding agents to help you build product-specific AI evals.
-
Hosted, link-only submission site for the Sutro MNIST-medium time challenge on Modal, with the evaluator harness and a deployment runbook
-
★ 0 xeophon/harborHarbor is a framework for running agent evaluations and creating and using RL environments.
-
JevBench by Benchmark Heaven: benchmark and leaderboard for AI decision models, measuring typed decision accuracy, calibration, latency and cost.
-
Local typed decisions, contrastive data curation, and model evaluation.
-
Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.