Engineering Blog

SWE-agent, Open-Source AI: Stronger Alternative to Devin

Software development is a complex field, and even experienced engineers can encounter bugs and issues from time to time. But what if there was a way to automate some of the troubleshooting process?

This is where SWE-agent comes in. SWE-agent is a new system that leverages the power of large language models (LLMs) like GPT-4 to tackle bugs and resolve issues found in real GitHub repositories.

State-of-the-Art Performance

On a standard test set known as SWE-bench, SWE-agent achieves an impressive 12.29% success rate in resolving issues. This puts it at the forefront of AI-powered software engineering tools.

The Secret Sauce: Agent-Computer Interface (ACI)

SWE-agent’s success hinges on a concept called the Agent-Computer Interface (ACI). ACI essentially provides a set of instructions and feedback mechanisms that allow the LLM to effectively navigate a software repository. These instructions include:

  • Browsing the repository
  • Viewing and editing code files
  • Executing code

Just like how humans need clear instructions to perform tasks, LLMs benefit from a well-designed ACI. SWE-agent’s creators have incorporated features specifically helpful for LLMs working with code, such as:

  • Lint checks: To ensure code edits are syntactically correct before submission.
  • Customizable file viewer: Limited to displaying 100 lines at a time for better focus.
  • Search functionality: To allow the LLM to find specific code sections.
  • Informative messages: Feedback on whether a command produced any output.

Getting Started with SWE-agent

SWE-agent is open-source and available for anyone to experiment with. However, it does require some setup:

  1. Install Docker, then start Docker locally.
  2. Install Miniconda, then create the swe-agent environment with conda env create -f environment.yml
  3. Activate using conda activate swe-agent.
  4. Run ./setup.sh to create the swe-agent docker image.
  5. Create a keys.cfg file at the root of this repository and fill in the following:

GITHUB_TOKEN: ‘GitHub Token Here (required)’
OPENAI_API_KEY: ‘OpenAI API Key Here if using OpenAI Model (optional)’
ANTHROPIC_API_KEY: ‘Anthropic API Key Here if using Anthropic Model (optional)’
TOGETHER_API_KEY: ‘Together API Key Here if using Together Model (optional)’

See the following links for tutorials on obtaining AnthropicOpenAI, and Github tokens.

The included instructions detail two key functionalities:

  • Inference: This is where SWE-agent analyzes a GitHub issue and attempts to fix it by generating a pull request.
  • Evaluation: (Currently limited to SWE-bench issues) This step assesses whether the generated pull request actually resolves the reported issue.

Join the Community

The SWE-agent team is actively seeking contributions and welcomes:

  • If you’d like to ask questions, learn about upcoming features, and participate in future development, join our Discord community!
  • If you’d like to contribute to the codebase, we welcome issues and pull requests!
  • If you’d like to see a post or tutorial about some topic, please let us know via an issue.

With its promising results and open-source nature, SWE-agent represents a significant step towards AI-powered software development. So, if you’re looking to streamline your coding workflow, SWE-agent is definitely worth exploring.

Reference to the Article

Previous Post