AI adoption

Here’s what most teams get wrong about rolling out Copilot

They give everyone a license on Monday and check adoption numbers on Friday.

I’ve seen this pattern across multiple organizations, and the result is usually the same: a spike in usage for the first two weeks, a gradual decline, and a leadership meeting where someone asks whether the $19/month per seat was worth it.

Rolling out GitHub Copilot to a team isn’t a software installation. It’s a change management problem with a keyboard attached.

I’ve been managing engineers through AI adoption for over a year now. The first team-wide Copilot rollout taught me things I wish I’d known before day one. Here’s the playbook I’d use if I had to do it again — structured around the first 90 days.

Days 1–30: Set Up the Pilot, Not the Rollout

Don’t start with “everyone gets a seat.” Start with one team that has:

– A manager or team lead who will actually use Copilot and talk about it in standups
– One or two experienced developers who can test advanced workflows and help peers
– Work that’s well-suited to agentic tools — investigating a codebase, making coordinated changes across files, writing tests, or preparing pull requests
– Stable delivery metrics you can use as a baseline

GitHub’s own playbook recommends this exact approach: start with one team or a small group of established teams. The reasoning is straightforward — participants learn from each other, and you can separate team practices from individual preferences.

Critically, don’t pick only the enthusiasts. Include a representative mix of experience levels. You need to surface the common setup, trust, and workflow barriers before you scale.

Record your baseline before the pilot starts. Lead time to merge, pull request cycle time, defect rate — whatever your team already tracks. Without a baseline, the 90-day mark is just a date, not a measurement.

GitHub’s Engineering System Success Playbook emphasizes this: define your goals before you pick your metrics. Usage numbers tell you whether people are using the tool. Engineering outcomes tell you whether the tool is improving anything.

Days 31–60: The Two-Week Onboarding Sprint

Around day 30, run a focused two-week sprint. This isn’t a training course — it’s a real-work test.

GitHub’s recommended approach: give the team two or three approved workflows that match their current work, such as:

– Using Copilot CLI to investigate a failure, plan a multi-file change, run tests, or review local changes
– Using the Copilot app to work from an issue, make changes in an agent session, and open a pull request
– Moving between tools when useful (you can open the Copilot app from Copilot CLI with copilot app)

Launch with the whole team. Explain the outcomes you want to improve, the activation target, and the guardrails. Have the manager or team lead complete one of the selected workflows in the team’s actual codebase — live, in front of everyone. Show where human review and approval remain required.

Then complete a first task together. Pair experienced developers with newer team members so everyone starts a session and completes a small, low-risk task on the team’s real codebase.

GitHub’s white paper on training and onboarding developers on GitHub Copilot includes a sample 90-day plan that starts training 45 days before rollout — champions, reminders, and continuous training rather than a single point-in-time event.

What most teams skip: After the sprint, debrief as a group. What surprised you? What worked? What didn’t? Use that debrief to adjust the workflows, remove what didn’t help, and share practices from the sessions that did.

Days 61–90: Measure What Actually Matters

This is where most rollouts go off the rails. They look at adoption numbers — are people using it? — and call it a success.

Usage is not impact. This is the single most important thing I’ve learned from watching teams adopt Copilot.

GitHub’s own research showed compelling individual-level results: developers using Copilot completed coding tasks 55% faster, with 88% saying they felt more productive and 73% saying they were more in flow. An Accenture study of their own developers found that 81.4% installed Copilot on day one, 67% used it at least five days per week, and pull requests increased by 8.69% with a 15% increase in merge rates.

But here’s what the individual numbers don’t tell you: a 2025 CACM study found no statistically significant improvement in pull request count or lines of code after Copilot adoption at the organizational level. Individual efficiency gains did not translate to measurable throughput.

Why? Because engineering productivity is determined by the entire delivery system, not just coding. As Plandek’s framework puts it, you need to measure across four pillars: Focus, Speed, Predictability, and Quality. Faster coding means nothing if review, testing, deployment, or planning bottlenecks absorb the gains.

At the 90-day mark, your leadership meeting should answer these questions:

– Did lead time to merge improve, or did it stay flat?
– Did defect rates change? (Faster output with more bugs is worse, not better.)
– Where did developers actually use Copilot, and where did they discard its suggestions?
– What did the team tell you in retros about whether Copilot helped or hurt their workflow?

GitHub’s impact dashboard compares users in different adoption cohorts — not the same users before and after. The documentation is honest about this limitation: “Differences between cohorts can reflect their work or experience as well as their use of Copilot.” Record any other changes (staffing, project complexity, release schedules) that might explain results independently of Copilot.

What I’d Do Differently

If I were rolling out Copilot to a team today, here’s what I’d change from my first attempt:

1. I’d spend more time on the pilot selection. My first rollout picked the most enthusiastic developers. The CACM study showed that 7% of assigned users reported zero usage in the three months after deployment. You need to understand why — concerns about accuracy, lack of time, or lack of knowledge — before you scale. The study found the primary reasons for non-use were accuracy concerns and lack of time and knowledge to use Copilot.

2. I’d track quality alongside speed from day one. GitHub’s research noted that Accenture saw an 84% increase in successful builds, suggesting AI-assisted pull requests were often of higher quality when reviewed. But that’s not guaranteed. If your team ships faster with more defects, you’ve made things worse.

3. I’d identify Copilot champions early. GitHub’s “Taking it to the streets” guide defines champions as the developers people look up to, rather than the leadership team. They’re the ones using the tool and sharing real experiences with peers. This drives adoption far more effectively than any announcement from above.

4. I’d stop treating training as a one-time event. The white paper recommends continuous, multi-form training: workshops, office hours, and informal channels. Shift from formal training to specific, on-demand support as adoption grows.

5. I’d be honest about the efficiency-throughput gap. The CACM research found that despite improvements in motivation and perceived skills, there was no immediate increase in key engineering metrics. This isn’t a failure of Copilot — it’s a reminder that tools don’t fix broken processes. As the research concluded: “realizing the potential of AI-powered tools requires continued innovation and optimization of software engineering operations.”

What I’d Tell Another Manager Starting This Tomorrow

Pick one team. Record your baseline. Run a two-week sprint on real work. Measure impact, not adoption. And be prepared for the possibility that Copilot makes individual developers faster without making the team ship more — because that’s exactly what the data says can happen.

The teams that get the most out of Copilot aren’t the ones with the most seats. They’re the ones that treat the rollout as an engineering problem, not a software installation.

Question for anyone reading this who’s already rolled out Copilot: What metric surprised you the most — or didn’t move at all? I’m genuinely curious what the real-world data looks like outside the case studies.

Sources: GitHub Docs — Rolling out GitHub Copilot at scale, GitHub — Achieving your company’s engineering goals with Copilot, GitHub — Taking it to the streets: Rolling out GitHub Copilot, GitHub — Training and onboarding developers on GitHub Copilot (white paper), Axify — GitHub Copilot Best Practices for Engineering Teams (Oct 2026), Plandek — How to measure Copilot’s impact on productivity, GitHub Research — Quantifying Copilot’s impact on developer productivity, GitHub/Accenture — Quantifying Copilot’s impact in the enterprise, CACM — The Efficiency-Throughput Gap with GitHub Copilot (2025).