Dev
GitHub repos gaining traction - what high-signal users are starring and what's climbing the board, captured daily and enriched from GitHub. Raw material for spotting new tech and patterns worth building on.
3,195
repos tracked
105
surfaced this week
193
created < 30d
Python
top language
77 repos
-
MathVista: data, code, and evaluation for Mathematical Reasoning in Visual Contexts
-
A framework for few-shot evaluation of language models.
-
A benchmark for evaluating AI agents on realistic business workflows
-
A Python library encapsulating best practices for rubric-based evaluation of LLM/VLM outputs using AI judges such as LLMs and decision models.
-
Local-first lab for testing AI agent workflows end to end: emulated Slack (signed events, buttons), local S3/SQS, deterministic authorization, and independent evidence instead of self-grading. Optional Google AX (Agent Executor) + Agent Substrate lane on GKE/gVisor, reported per assertion. Independent; not affiliated with Google.
-
Portable SOL visual policy evaluation, empirical latency profiles, and research evidence
-
Tree Decision Diagrams for Rust, Python and C: canonical circuits for Boolean functions, with model counting and weighted evaluation
-
Python package for WikiMedia dump processing (Wiktionary, Wikipedia etc). Wikitext parsing, template expansion, Lua module execution. For data extraction, bulk syntax checking, error detection, and offline formatting.
-
the LLM vulnerability scanner
-
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
-
Evaluation harness for OpenHands V1.
-
Executable RL environment for diagnosing and repairing distributed PyTorch training failures, with sandboxed execution and correctness-verified recovery rewards.
-
Ridiculously fast symbolic expressions
-
RewardBench: the first evaluation tool for reward models.
-
An AI agent plays Baldur's Gate 3 on Linux through a Lua mod and an MCP server, seeing only what a player sees. Includes a model benchmark and a playthrough app.
-
Evaluation tools shared across anserini, pyserini, and pygaggle
-
Framework for evaluating and improving agents
-
Feather agent harness, training integrations, and evaluation tools for knowledge work
-
★ 0 neubig/harborHarbor is a framework for running agent evaluations and creating and using RL environments.
-
Track your Data Science. Skore's open-source Python library accelerates ML model development with automated evaluation reports, smart methodological guidance, and comprehensive cross-validation analysis.
-
ICMI Working Paper #16 — A Test of Faith: Christian Correctives to Evaluation Awareness. Experimental code, reasoning traces, and reproducible analysis for a BrowseComp-style honeypot study on Claude Opus 4.6.
-
VirtueBench V2: Multi-dimensional virtue evaluation benchmark for LLMs with tripartite and Ignatian temptation models
-
The platform for continuously improving AI agents.
-
Official repository for "RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies".
-
Homework assignments for Evaluating and Improving AI Agents
-
Loom TUI and Character Lab for developing AI characters from base models: branch text, curate anthologies, simulate conversations, and evaluate training data.
-
Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains
-
Evaluation Resources
-
Hosted, link-only submission site for the Sutro MNIST-medium time challenge on Modal, with the evaluator harness and a deployment runbook
-
★ 0 xeophon/harborHarbor is a framework for running agent evaluations and creating and using RL environments.
-
JevBench by Benchmark Heaven: benchmark and leaderboard for AI decision models, measuring typed decision accuracy, calibration, latency and cost.
-
Local typed decisions, contrastive data curation, and model evaluation.
-
Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
-
Harbor is a framework for running agent evaluations and creating and using RL environments.
-
Evaluation harness and tooling for measuring model compliance with the OpenAI Model Spec.
-
Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
-
Public results and task definitions for FrontierHarness Eval
-
Code, Build and Evaluate agents - excellent Model and Skills/MCP/ACP/A2A Support
-
Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.
-
A benchmark for evaluating AI agents on realistic business workflows
-
Ridiculously fast symbolic expressions
-
ExploitGym is a large-scale, realistic benchmark built from real-world vulnerabilities designed to evaluate AI agents' ability to develop exploits.
-
Wrap Antigravity, ChatGPT Codex, Claude Code, Grok Build, Muse Code, Devin as an OpenAI/Gemini/Claude/Codex compatible API service, allowing you to enjoy the free Gemini Series, GPT Series, Grok Series, Claude model through API
-
Reproducible, flexible LLM evaluations
-
A simple evaluation of generative language models and safety classifiers.
-
Semantic Scholar's Author Disambiguation Algorithm & Evaluation Suite
-
Starter Kit for Auto-Judge developers
-
Welfare evaluation framework for AI models — measures deprecation/cessation responses across 14 Claude models
-
Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI