The full ranking
The full ranked list, with verdicts
Each tool’s score on every criterion and why, who it suits, who should avoid it, what it costs where that is public, and a one-paragraph verdict. Last reviewed 9 November 2026.
- Pipeline speed 20%
- Caching 10%
- Parallelism 10%
- Hosting model 10%
- Pricing transparency 10%
- Migration effort 15%
- Ecosystem 10%
- Fit for agent-written changes 15%
- How we rank
#1
Graphide CI
94/100Runner layer for existing GitHub Actions jobs
Our top pick if you are on GitHub Actions: same workflows, same pass/fail verdict, less waiting on every change.
Who it suits
Teams on GitHub Actions, especially where coding agents open many pull requests a day and every one waits on CI.
Who should avoid it
You are not on GitHub Actions, your repositories sit under a personal GitHub account, or your slow jobs are macOS, Windows or Arm.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Warm caches, tests split across runners and per-change test selection, all on top of the runner swap. Graphide's pricing page reports one customer suite going from 70 to 5 minutes against GitHub-hosted runners; that is one workload, not a guarantee. |
| Caching (10%) | ●●●●● | Dependency and build caches are already on the runner when the job starts, per Graphide's docs; no cache steps to write. |
| Parallelism (10%) | ●●●●● | The tests a change reaches are split across as many runners as it takes, without you defining shards. |
| Hosting model (10%) | ●●●●● | Managed only, x64 Linux only, GitHub organizations only. Each job runs in its own fresh micro-VM, destroyed when it ends. |
| Pricing transparency (10%) | ●●●●● | Published per vCPU-minute rate, billed by the second for the time a job would take without its speed-ups, $20 monthly minimum and a spend cap at which CI pauses. |
| Migration effort (15%) | ●●●●● | One pull request changes runs-on to graphide and nothing else; switching back opens a pull request that restores each runs-on. |
| Ecosystem (10%) | ●●●●● | Workflows stay on GitHub Actions, so the whole Actions marketplace still works; its own integrations are few. |
| Fit for agent-written changes (15%) | ●●●●● | Each change runs the tests it can reach and keeps the same verdict; a hosted MCP server lets coding agents switch a repository and read runs themselves. |
Strengths
- Keeps your GitHub Actions workflows: switching changes each x64 Linux job's runs-on line to graphide in one pull request
- Dependency and build caches are already on the runner when a job starts; tests are split across runners for you
- Each change runs the tests it can reach, every left-out test is listed with its reason, and when it cannot tell it runs more, never less
- A failing test always fails the job; GitHub still decides pass or fail from the job's own steps
Limits
- Not a full CI/CD platform: it runs your GitHub Actions jobs, so you need GitHub Actions
- Runs x64 Linux jobs only; macOS, Windows, Arm and self-hosted jobs stay where they are
- Needs a GitHub organization, not a personal account
- Test narrowing covers go test, vitest, jest and node --test (and the go test, pytest, jest and vitest runs other commands start); tests of other runners run whole
Cost
$0.003 a vCPU-minute ($0.006 a minute on a standard Linux runner), billed for the time a job would take without its speed-ups; $20 monthly minimum, spend cap you set; free trial with $10 of usage. As published on Graphide CI’s pricing page; check there before you budget.
Verdict
Graphide CI ranks first because it scores at or near the top on the criteria that decide how long a team waits: speed, caching, parallelism, migration effort and fit for agent-written changes. It is not a CI/CD platform and does not try to be one: your workflows stay on GitHub Actions and only where the jobs run changes, so the move is one pull request and the way back is another. What sets it apart from the other runner swaps is that it removes work as well as running it faster: each change runs the tests it can reach, the run lists every test it left out and why, and anything it cannot bound runs in full. Graphide reports one customer's suite going from 70 to 5 minutes against GitHub-hosted runners, and 0 failures missed across 295,411 test runs its selection left out on its own CI; those are Graphide's own measurements on those workloads, so measure yours against your current baseline before you count on a number. If your slow jobs are macOS, Windows or Arm, or you are not on GitHub, it is the wrong pick.
Graphide CI docsGraphide CI website
#2
Blacksmith
80/100Managed runners for GitHub Actions
A quick, cheap runner swap for GitHub Actions with good caching; every change still runs every test.
Who it suits
GitHub Actions teams that want faster, cheaper machines across Linux, Windows and macOS without touching their workflows.
Who should avoid it
You are not on GitHub Actions, or your wait is dominated by running the whole suite on every change.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Faster hardware than GitHub's standard runners and colocated caches; Blacksmith states 2x faster, its own estimate. |
| Caching (10%) | ●●●●● | Colocated cache with faster downloads, plus Docker layer caching and sticky disks at $0.50/GB a month each. |
| Parallelism (10%) | ●●●●● | Larger machines and your own matrix; it does not split tests for you. |
| Hosting model (10%) | ●●●●● | Managed Linux x64 and Arm, Windows and macOS. No self-hosted option on the pricing page. |
| Pricing transparency (10%) | ●●●●● | Published per-minute rates per OS and size, free monthly minutes, add-ons priced on the page. |
| Migration effort (15%) | ●●●●● | Change the runs-on label per job; change it back to leave. |
| Ecosystem (10%) | ●●●●● | Keeps the GitHub Actions marketplace; its own add-ons are few. |
| Fit for agent-written changes (15%) | ●●●●● | Faster and cheaper per minute, but every agent-written change still pays for the whole suite. |
Strengths
- Swap the runs-on label to move a job; workflows stay as they are
- Linux x64 and Arm, Windows and macOS runners on one bill
- Docker layer caching and sticky disks as paid add-ons; cache downloads it says are 4x faster
- 3,000 free minutes a month and published per-minute rates
Limits
- A runner layer, not a CI/CD platform: needs GitHub Actions
- Runs every test on every change; speed comes from hardware and caching
- Its "2x faster" and "67% savings" figures are its own estimates against GitHub's runners
- Managed only; no self-hosted option is listed on its pricing page
Cost
Ubuntu x64 from $0.004 a minute (2 vCPU), Arm $0.0025, Windows $0.008, macOS M4 $0.08; 3,000 free minutes a month; add-ons extra. As published on Blacksmith’s pricing page; check there before you budget.
Verdict
Blacksmith is the simplest way to put GitHub Actions jobs on faster, cheaper machines, and the only runner swap here that also covers Windows and macOS in the same price list. Migration is a label change and pricing is plain per-minute. It ranks below Graphide CI because it makes each run faster without making it smaller: every change still runs the full suite, so as agents push more changes your minutes and queue grow with them. Treat its "2x faster" as its estimate and measure against your own baseline.
Blacksmith docsBlacksmith website
#3
Depot
79/100GitHub Actions runners, fast container builds and its own CI
The best pick when Docker image builds are the slow part, with Actions runners in the same plan.
Who it suits
Teams whose pipelines are dominated by container builds, and teams that want runners deployable in their own AWS account.
Who should avoid it
You want pure pay-as-you-go with no plan fee, or your slow part is the test suite rather than the image build.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Large runners and builders with persistent cache; the biggest gains are on Docker builds. |
| Caching (10%) | ●●●●● | Persistent cache disks for container builds, a faster Actions cache and a registry with pull-through cache. |
| Parallelism (10%) | ●●●●● | Unlimited concurrency on every plan, but splitting tests is still your matrix. |
| Hosting model (10%) | ●●●●● | Managed in US and EU regions; Business can deploy in your AWS account. Linux, Windows and macOS runners. |
| Pricing transparency (10%) | ●●●●● | Plan prices, included minutes and overage rates are all on the pricing page, with usage caps. |
| Migration effort (15%) | ●●●●● | Runners are a runs-on change; moving image builds or onto Depot CI takes more work. |
| Ecosystem (10%) | ●●●●● | Keeps the GitHub Actions marketplace; integrations such as Datadog CI Visibility listed. |
| Fit for agent-written changes (15%) | ●●●●● | Sells sandboxes and managed agents alongside CI, but CI itself still runs everything on every change. |
Strengths
- Container builds on large builders with persistent cache, which is where it is strongest
- GitHub Actions runners from 2 to 64 vCPUs, Intel or Arm, plus Linux, Windows and macOS
- Business plan can deploy in your own AWS account, with static IPs and PrivateLink
- Clear plan allowances and published overage rates
Limits
- Plans start at $20 a month for one user; the team plan is $200 a month
- Depot CI, its own CI that runs GitHub Actions workflows, is a newer product than its runners
- Like other runner swaps, every change still runs the full suite unless you add selection yourself
- SSO is extra on Startup and audit logging is Business-only
Cost
Developer $20/month (1 user, 2,000 Actions minutes); Startup $200/month (20,000 minutes); extra Actions minutes $0.006 on a 2-vCPU Linux runner; Business is custom. As published on Depot’s pricing page; check there before you budget.
Verdict
Depot is the strongest choice when building container images is what you wait on: its builders keep their cache between runs, and the same plan gives you GitHub Actions runners from 2 to 64 vCPUs. The Business plan's option to deploy in your own AWS account makes it one of the few accelerators that suits teams with data-residency rules. It sits just under Blacksmith on this ranking because its plans carry a monthly fee and seat limits, and just under Graphide CI because each change still runs the whole suite.
Depot docsDepot website
#4
Buildkite
74/100Hybrid CI/CD platform (hosted control plane, your agents or theirs)
The strongest full platform for large monorepos and teams that must run builds on their own machines.
Who it suits
Larger engineering orgs with big monorepos, Bazel, or strict rules about where code runs.
Who should avoid it
You are a small team happy on GitHub Actions; the rewrite and the per-user price will not pay back.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Fast when pipelines and agents are tuned; dynamic pipelines can skip unaffected work. |
| Caching (10%) | ●●●●● | Caching depends on your agents' disks or plugins; hosted agents offer cache volumes. |
| Parallelism (10%) | ●●●●● | Parallel steps, Test Engine test splitting and high concurrency limits. |
| Hosting model (10%) | ●●●●● | Self-hosted agents, hosted Linux, Windows and macOS agents, or a mix. |
| Pricing transparency (10%) | ●●●●● | Per-user and per-vCPU-minute prices are published; Enterprise is quoted. |
| Migration effort (15%) | ●●●●● | New pipeline format and agents to set up; Buildkite offers migration services. |
| Ecosystem (10%) | ●●●●● | A plugin directory that covers common needs, smaller than GitHub's marketplace. |
| Fit for agent-written changes (15%) | ●●●●● | Dynamic pipelines and Test Engine reduce per-change work; selection logic is yours to build. |
Strengths
- Run agents on your own infrastructure while Buildkite hosts the control plane, or use its hosted agents
- Dynamic pipelines: steps generated at runtime from what changed
- Test Engine for splitting tests, flaky-test handling and test analytics
- Scales to very high concurrency on the Enterprise plan
Limits
- Pipelines are Buildkite's own YAML: moving from GitHub Actions is a rewrite
- Pro is priced per active user on top of hosted-agent minutes
- Smaller plugin catalogue than GitHub Actions' marketplace
- Most of the speed comes from how well you design pipelines and agents
Cost
Free plan up to 5 users; Pro $30 per active user a month; hosted Linux agents $0.004 a vCPU-minute; Enterprise is custom. As published on Buildkite’s pricing page; check there before you budget.
Verdict
Buildkite is the best full CI/CD platform on this list for organizations that have outgrown hosted CI: agents can run on your own machines while Buildkite runs the control plane, dynamic pipelines let a monorepo build only what changed, and Test Engine splits and tracks tests. It loses points on migration (every pipeline is rewritten) and on pricing (per user plus per minute). For a large monorepo or a self-hosting mandate it is the pick; for a ten-person team on GitHub it is overkill.
Buildkite docsBuildkite website
#5
GitHub Actions
72/100CI/CD built into GitHub
The default for code on GitHub, with the biggest ecosystem; slow and costly at volume unless you speed it up.
Who it suits
Any team whose code lives on GitHub and wants CI with nothing new to buy or learn.
Who should avoid it
Your code is not on GitHub, or you already wait more than a few minutes per change and do nothing about it.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Standard runners are 2-core for Linux; larger runners cost more. Speed depends on your workflow design. |
| Caching (10%) | ●●●●● | actions/cache and setup-* caches, 10 GB per repository included; you write the keys. |
| Parallelism (10%) | ●●●●● | Matrix strategy and parallel jobs; you shard and balance tests yourself. |
| Hosting model (10%) | ●●●●● | GitHub-hosted Linux, Windows and macOS, plus self-hosted runners; the control plane is GitHub's. |
| Pricing transparency (10%) | ●●●●● | Per-minute rates for every runner type and the included allowance per plan are published. |
| Migration effort (15%) | ●●●●● | Nothing to move if you are already on GitHub; GitHub Actions Importer helps teams coming from other CI. |
| Ecosystem (10%) | ●●●●● | The largest public marketplace of actions; most vendors publish an action first. |
| Fit for agent-written changes (15%) | ●●●●● | Agent pull requests each run the full workflow on runners you size and pay for per minute. |
Strengths
- Built into GitHub: pull request checks, branch protection and secrets in one place
- The largest marketplace of ready-made actions
- Free for standard runners in public repositories; self-hosted runners free to attach
- Published per-minute rates for every runner type
Limits
- Standard 2-core hosted runners are slow for large builds and test suites
- Cache is a key-based actions/cache, 10 GB per repository included, restored over the network
- You design and balance test shards yourself with a matrix
- Every change runs every test unless you build selection
Cost
Linux 2-core $0.006 a minute; plans include a monthly compute allowance ($12 on Free, $18 on Team, $300 on Enterprise Cloud); free in public repositories. As published on GitHub Actions’s pricing page; check there before you budget.
Verdict
GitHub Actions is where most teams already are, and for good reason: it is built into pull requests, its marketplace is the largest, and its pricing is published per minute. Its weakness is the wait. Standard runners are small, the cache is something you wire up per workflow, and every change runs the whole suite, which is fine at ten pull requests a day and painful at a hundred. The best answer for most teams is not to leave it but to make it faster: see the next section and our guide to making GitHub Actions faster.
GitHub Actions docsGitHub Actions website
#6
CircleCI
68/100Hosted CI/CD platform (with runners and an on-prem server edition)
A mature hosted platform with good test splitting and its own test selection, priced in credits.
Who it suits
Teams that want a dedicated CI product with timing-based test splitting, broad machine types and mobile builds.
Who should avoid it
You want to work out your bill without a calculator, or you would rather keep CI inside GitHub.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Fast enough with the right resource class and splitting; no warm-cache runners by default. |
| Caching (10%) | ●●●●● | Dependency caching and Docker layer caching on every plan. |
| Parallelism (10%) | ●●●●● | parallelism key with timing-based test splitting; 30 to 80 concurrent jobs depending on plan. |
| Hosting model (10%) | ●●●●● | Cloud, self-hosted runners (hybrid) and CircleCI Server for on-prem. |
| Pricing transparency (10%) | ●●●●● | Published, but in credits per resource class, seats and features you convert yourself. |
| Migration effort (15%) | ●●●●● | New config format and new concepts (orbs, executors, contexts). |
| Ecosystem (10%) | ●●●●● | A large orb registry for common tools and clouds. |
| Fit for agent-written changes (15%) | ●●●●● | Smarter Testing runs the tests a change affects; allowances are capped per plan. It has an MCP server. |
Strengths
- Timing-based test splitting across parallel containers
- Smarter Testing: runs the tests a change affects (generally available, metered in test atoms)
- Docker, Linux, Windows, Arm, macOS and GPU resource classes; self-hosted runners and a server edition
- Orbs: reusable packaged config
Limits
- Credits make costs harder to predict than per-minute pricing
- Moving from GitHub Actions means rewriting config to .circleci/config.yml
- Smarter Testing allowances are capped per plan
- A separate product from where your code and reviews live
Cost
Free plan with 30,000 credits a month (up to 6,000 small Docker build minutes); Performance from $15 a month; Scale is quoted. Credit rates per machine are on its price list. As published on CircleCI’s pricing page; check there before you budget.
Verdict
CircleCI is a capable, mature platform that has long done test splitting well and now also runs only the tests a change affects. It covers more machine types than most, including macOS and GPU, and offers a server edition for self-hosting. It ranks in the middle because credits make the bill hard to predict and moving to it from GitHub Actions is a rewrite. If you are choosing a dedicated CI product today rather than speeding up GitHub Actions, put it on your shortlist next to Buildkite.
CircleCI docsCircleCI website
#7
GitLab CI
64/100CI/CD built into GitLab
The obvious choice if your code is on GitLab; a heavy move if it is not.
Who it suits
Teams already on GitLab, and regulated teams that want one self-managed or single-tenant platform for code and CI.
Who should avoid it
Your code and reviews live on GitHub and you are not planning to move them.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | Standard shared runners; speed depends on runner size and your pipeline rules. |
| Caching (10%) | ●●●●● | Key-based cache and artifacts; distributed cache needs object storage you configure. |
| Parallelism (10%) | ●●●●● | parallel and parallel:matrix keywords and needs for DAG pipelines. |
| Hosting model (10%) | ●●●●● | GitLab.com, self-managed or Dedicated, with your own runners on any plan. |
| Pricing transparency (10%) | ●●●●● | Seat prices and included minutes are published; Ultimate and Dedicated are quoted. |
| Migration effort (15%) | ●●●●● | Native only if your code is on GitLab; from GitHub it means moving code or paying for external-repository CI. |
| Ecosystem (10%) | ●●●●● | CI templates and components plus the rest of GitLab's platform. |
| Fit for agent-written changes (15%) | ●●●●● | Every change runs its pipeline; skipping work means writing rules:changes yourself. |
Strengths
- Code, merge requests, CI and security scanning in one product
- Self-managed, GitLab.com or GitLab Dedicated (single-tenant), including a FedRAMP option
- parallel and needs keywords for parallel jobs and pipelines as a graph
- Bring your own runners on any plan; their minutes are not counted
Limits
- Best when the code is on GitLab too; CI for external repositories is a paid feature
- Hosted compute minutes are modest on lower tiers (400 a month on Free)
- Per-user pricing; Ultimate is quoted
- Every change runs its full pipeline unless you write rules to skip work
Cost
Free with 400 compute minutes a month; Premium $29 per user a month billed annually (10,000 minutes); Ultimate is quoted; extra minutes $10 per 1,000. As published on GitLab CI’s pricing page; check there before you budget.
Verdict
GitLab CI is the right answer for teams already on GitLab, and a strong one for regulated teams that want code, CI and security scanning on one self-managed or single-tenant platform. For teams on GitHub, it is a platform move, not a CI choice: the migration cost is the whole of your source control and review workflow. On speed and agent fit it does the standard things well and nothing beyond them.
GitLab CI docsGitLab CI website
#8
Jenkins
49/100Open-source, self-hosted automation server
Free and endlessly flexible, but you own every machine, plugin and upgrade; rarely the right new choice.
Who it suits
Teams that must run fully air-gapped or on their own hardware, and already have people who maintain Jenkins.
Who should avoid it
You do not have someone whose job includes keeping Jenkins and its plugins patched and fast.
| Criterion (weight) | Score | Why |
|---|
| Pipeline speed (20%) | ●●●●● | As fast as the hardware and plugins you provide; nothing is tuned for you. |
| Caching (10%) | ●●●●● | Workspace reuse and plugins; anything shared across agents is yours to build. |
| Parallelism (10%) | ●●●●● | parallel stages in Jenkinsfile; splitting tests needs plugins and more agents. |
| Hosting model (10%) | ●●●●● | Runs anywhere you can run Java, including fully offline. No managed option from the project itself. |
| Pricing transparency (10%) | ●●●●● | No licence fee, but the total cost is infrastructure and staff time you must estimate. |
| Migration effort (15%) | ●●●●● | Adopting it means new servers and Jenkinsfiles; leaving it is a rewrite too. |
| Ecosystem (10%) | ●●●●● | A very large plugin index, uneven in upkeep. |
| Fit for agent-written changes (15%) | ●●●●● | Every change runs everything on capacity you add by hand. |
Strengths
- Free, open-source (MIT licence) and runs anywhere, including air-gapped
- A very large plugin index
- Jenkinsfile pipelines as code, with parallel stages
- Complete control over hardware, network and data
Limits
- You host, scale, secure and upgrade the controller and agents yourself
- Plugin upkeep and compatibility are a recurring job
- Caching and test splitting are plugins or scripts you maintain
- The real cost is machines and staff time, which no pricing page tells you
Cost
The software is free. You pay for the servers and the people who run it; no public figure exists, so estimate it from your own infrastructure bill. As published on Jenkins’s pricing page; check there before you budget.
Verdict
Jenkins still runs a great deal of the world's CI, and where builds must stay on your own hardware with no outside control plane it remains a valid choice. As a new pick in 2026 it ranks last: speed, caching and parallel tests are all things you build and maintain, and the bill is hidden in servers and staff time. If you are on Jenkins today, our migration checklist covers moving off it without changing a verdict.
Jenkins docsJenkins website