Dev
GitHub repos gaining traction - what high-signal users are starring and what's climbing the board, captured daily and enriched from GitHub. Raw material for spotting new tech and patterns worth building on.
3,195
repos tracked
105
surfaced this week
193
created < 30d
Python
top language
42 repos
-
MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
-
A framework for few-shot evaluation of language models.
-
A Python library encapsulating best practices for rubric-based evaluation of LLM/VLM outputs using AI judges such as LLMs and decision models.
-
Local-first lab for testing AI agent workflows end to end: emulated Slack (signed events, buttons), local S3/SQS, deterministic authorization, and independent evidence instead of self-grading. Optional Google AX (Agent Executor) + Agent Substrate lane on GKE/gVisor, reported per assertion. Independent; not affiliated with Google.
-
Portable SOL visual policy evaluation, empirical latency profiles, and research evidence
-
Tree Decision Diagrams for Rust, Python and C: canonical circuits for Boolean functions, with model counting and weighted evaluation
-
the LLM vulnerability scanner
-
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
-
Evaluation harness for OpenHands V1.
-
Executable RL environment for diagnosing and repairing distributed PyTorch training failures, with sandboxed execution and correctness-verified recovery rewards.
-
RewardBench: the first evaluation tool for reward models.
-
An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark and a playthrough app.
-
Evaluation tools shared across anserini, pyserini, and pygaggle
-
Feather agent harness, training integrations, and evaluation tools for knowledge work
-
★ 0 neubig/harborHarbor is a framework for running agent evaluations and creating and using RL environments.
-
Track your Data Science. Skore's open-source Python library accelerates ML model development with automated evaluation reports, smart methodological guidance, and comprehensive cross-validation analysis.
-
ICMI Working Paper #16 — A Test of Faith: Christian Correctives to Evaluation Awareness. Experimental code, reasoning traces, and reproducible analysis for a BrowseComp-style honeypot study on Claude Opus 4.6.
-
VirtueBench V2: Multi-dimensional virtue evaluation benchmark for LLMs with tripartite and Ignatian temptation models
-
The platform for continuously improving AI agents.
-
Official repository for "RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies".
-
Evaluation Resources
-
★ 0 xeophon/harborHarbor is a framework for running agent evaluations and creating and using RL environments.
-
JevBench by Benchmark Heaven: benchmark and leaderboard for AI decision models, measuring typed decision accuracy, calibration, latency and cost.
-
Local typed decisions, contrastive data curation, and model evaluation.
-
Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
-
Harbor is a framework for running agent evaluations and creating and using RL environments.
-
Evaluation harness and tooling for measuring model compliance with the OpenAI Model Spec.
-
Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
-
Public results and task definitions for FrontierHarness Eval
-
Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
-
Reproducible, flexible LLM evaluations
-
A simple evaluation of generative language models and safety classifiers.
-
Semantic Scholar's Author Disambiguation Algorithm & Evaluation Suite
-
Welfare evaluation framework for AI models — measures deprecation/cessation responses across 14 Claude models
-
Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI
-
MLT & AI Communities
-
A curated, non-BS library of the best resources for building and evaluating AI agents — papers, blogs, talks, tools, benchmarks. Maintained by BenchFlow.
-
Evaluation tools shared across anserini, pyserini, and pygaggle
-
ABC: Scalable Behavior Cloning with Open Data, Training, and Evaluation
-
TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.
-
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
-
Rubric compiler and judge engine for LLM evaluation