Introducing Systems-Bench: Evaluating the Performance of AI Agents on End-to-End SWE Workflows
A benchmark for multi-tool, long-horizon software engineering tasks in production grade systems.
Towards Greater Reliability and Autonomy in Software Engineering Agents
AI coding agents have become remarkably capable within the IDE - starting from autocomplete, to single-file edits, to making changes across the entire repository. However, true software engineering requires operating beyond the editor.