GitLab Duo vs. GitHub Copilot:

The Ultimate DevOps AI Showdown

Over the past two years, we have seen the rise in AI tools enabling businesses and organizations to “do more with less” or, as I would like to put it, “doing more and paying more to do more.” In my time focusing on DevSecOps, I have seen several tools released—AI capabilities for Atlassian (Atlassian Intelligence and Rovo), ServiceNow AI, and Salesforce Einstein. Each tool promises an extensive use of refined large language models (LLM), unique functionality, pricing, and service offerings that meet or exceed expectations.

Two AI products that we are going to compare today are direct competitors with one another, with evangelists on either side: GitLab Duo and GitHub Copilot.

Figure 1: Opening GitLab Duo Quick Chat

To help you and your team’s assessments, we judged each product on the criteria listed below:

User: Type of user the product is designed for (e.g., developers, leads, testers, project managers, etc.).
Team Size: Preferred team size.
LLM Used: Which LLM(s) are used for each AI capability? This distinction forms the backbone of the capabilities and functionality found in each AI tool.
Scalability: How scalable is the product? Can it be used only by small teams, or can it be used by large teams in enterprise (or ultimate) deployments?
Administration: Comparison of administration tools for Duo and Copilot, including analytics and dashboard capabilities.
Deployment Type: Self-hosted, cloud, SaaS, or other.
Functionality: List of functionality and capabilities of each. (If you’d like to learn more about this topic, review our comprehensive buyer guide, which goes into more detail on GitLab Duo and GitHub Copilot)
Feedback: Developer feedback thus far: positive, negative, or neutral?
Price: List out the pricing for both.

Criteria

GitLab Duo

GitHub Copilot

User	Focused on the project team, including developers, leads, testers, security, PMs, and more	Focused on the individual and smaller development teams, most developers, and some testers
Team Size	Medium to large	Small to medium
LLM(s) Used	Google Cloud’s Vertex AI Codey and Anthropic’s Claude Other LLMs as needed	OpenAI Codex (GPT-3) Ability to openly customize the LLM as needed but not point it to another LLM
Scalability	Large teams to enterprise	Large teams to enterprise
Admin	AI Impact Dashboard: Displays key metrics and trends for a project or group—e.g., percentage of users that engage with code suggestions every month, code suggestions acceptance rate, and percentage of users who engage with Duo Chat	Policy management, access management, usage data, audit logs, and exclude files for administrator configurations and updates
Deployment Type	Open core, available on multiple deployment models, including self-hosted, SaaS, and cloud options	Cloud only, tied to Azure, locking in customers to use Azure as their cloud platform Cannot be self-hosted, must be cloud-based
Functionality	All-in-one tool capabilities Issue tracking and management, wiki, VCS, CI/CD, security scanning and deployment	Focused on VCS capabilities Limited capabilities around wiki and issue tracking
Feedback	Positive: Developers, leads, and project managers really like the transparency GitLab Duo provides across all team members.	Positive: Developers and leads like the code suggestion and other capabilities accessible from the GitHub Copilot chat.
Price	$19–$39 per user per month Available for premium and ultimate	$10–$39 per user per month Available for single users, business, and enterprise users

When attempting code suggestions and other functionalities centered on software development, we have found both Duo and Copilot to be very functional and accurate, generating secure code and upholding any training data licensing rules. Of course, you should always verify what is being generated (especially if a feature is in Beta or Public Preview).

Figure 2: GitHub Copilot – Writing a Unit Test

So, where do we at GitSimple shake out? While we are impressed with the accuracy and capabilities of both tools, we highly recommend GitLab Duo—specifically, the Enterprise release—for several reasons:

Teams and Scalability: GitLab Duo focuses on the team; Duo delivers useful capabilities for developers, testers, security, PMs, and other users alike, resulting in efficiencies across the entirety of the DevSecOps lifecycle. In contrast, GitHub Copilot is more focused on smaller teams or the lone developer.
Deployment Capabilities: While GitHub Copilot is only available for cloud, GitLab Duo allows for deployment of self-hosted, SaaS, and cloud offerings.
LLM Diversity: Depending on the task, GitLab Duo uses one or more LLMs, which increases the accuracy of the output from Duo for your team. GitLab also provides additional customization (self-hosted models in beta) to extend your LLM capabilities, allowing teams to work outside the GitLab AI Gateway.

Interested in implementing GitLab Duo but don’t know where to start? Check out our comprehensive buyer guide, or contact us directly.