Engineering Competency Matrix: The One We Actually Run (and What AI Changed)

QUICK ANSWER

An engineering competency matrix is a grid that maps the skills an engineer needs against the levels they move through, from junior to principal, with a clear description of what each level looks like in practice. It turns vague titles into observable behavior, so hiring, promotion, and pay decisions stop running on gut feel. It is also the standard an engineer should measure themselves against when they write a software engineer self review. The best ones are short, opinionated, and built around what your team actually values, not copied off the internet. It plays out role by role, including a senior Rails developer job description.

We run an engineering competency matrix at Full Scale across more than 350 engineers and the 200-plus companies we have built teams for. It is how we decide who is a senior engineer, who is ready to be one, and what we are actually promising a client when we say we are sending them a senior. Before you can score anyone against a matrix, you have to define the software engineer levels your team actually uses.

Most articles about this topic are written by people who have never had to use one in a real promotion meeting.

This one isn’t. I have been hiring and leveling engineers for twenty years, first at VinSolutions, then at Stackify, and now across 350-plus of them at Full Scale. I have sat in the room where two managers each think their person deserves the same raise, and the only thing keeping the conversation honest is the matrix on the screen. So this is the version with the scar tissue in it, including the part nobody else is talking about: what AI just did to the whole idea of “junior.”

What an engineering competency matrix actually is

Strip away the consultant language and it is a table.

Down one side you list the things you care about in an engineer. Across the top you list the levels, usually something like junior, mid, senior, staff, and principal. In each cell you write what that competency looks like at that level, in plain behavior a manager could actually observe.

That last part is the whole game. “Strong communicator” is useless. “Writes design docs that a teammate can implement without a meeting” is something you can look at a person and check.

A good matrix answers three questions that every engineering org argues about:

• What does “senior” mean here, specifically, so two managers grade the same person the same way?
• What does this engineer need to do differently to reach the next level?
• When we hire, what level are we actually hiring for, and how do we test for it?

Get those three right and the matrix quietly fixes a dozen downstream problems: messy job descriptions, promotion debates that turn political, the muddy line between junior, mid, and senior developers, and the awkward review where nobody can say why the rating is what it is.

The matrix we use (steal this one)

Here is a working version, close to what we run internally. Six competencies, five levels, behavior in every cell. You can copy it straight into a sheet and start editing.

Competency	Junior (L1)	Mid (L2-3)	Senior (L4)	Staff (L5)	Principal (L6)
Technical execution	Finishes well-scoped tasks with guidance	Delivers a whole feature on their own	Owns a service and its tradeoffs	Sets technical direction across teams	Steers architecture for the company
Code quality and review	Writes working code, learns from review	Adds tests, gives useful review comments	Enforces quality through review, not nagging	Defines the patterns others follow	Sets the bar the whole org reviews against
Systems and architecture	Understands one service	Traces a request across a few services	Designs for failure, not just the happy path	Designs platform-level capabilities	Defines reference architecture and kills bad ideas early
Problem framing and judgment	Builds what the ticket says	Spots when the ticket is wrong	Reframes a vague ask into the real problem	Finds problems leadership hasn’t named yet	Picks which problems are worth the company’s time
Ownership and autonomy	Needs a check-in most days	Runs a feature without hand-holding	Runs for months without a manager in the loop	Creates work from business goals	Operates like an owner of the business
Collaboration and mentorship	Asks good questions	Helps onboard the next hire	Mentors two or three engineers	Grows five to ten engineers’ careers	Shapes hiring and culture at scale

A few things to notice, because they are deliberate.

Coding is one row, not the whole table. As engineers get more senior, they spend less time writing code and more time deciding what code should exist. In my book Product Driven, the through-line is that the hard part of software was never the typing, it was understanding the problem well enough to know what to build. A matrix that only measures code rewards the wrong thing at the top.

The other thing: judgment and ownership climb faster than technical skill as you go right. That is the real shape of an engineering career, and most matrices miss it because they are easier to write for the technical rows.

We pair this with a downloadable template (an editable Google Sheet and an Excel version) so a team can fork it instead of starting from a blank page. I would rather you start from something opinionated and argue with it than build a perfect-looking grid that nobody believes.

Pick a rating scale that isn’t theater

A matrix needs a way to say how well someone meets a level, not just which row they are in. A simple five-point scale works:

1. Below the level
2. Developing toward it
3. Solidly at the level
4. Exceeding it
5. Sets the standard others learn from

The trap is treating the number as the answer. The number should be where a conversation about evidence ends up, after you have looked at the actual work. Two engineers can both “deliver features independently.” One shipped three clean features with tests and zero production bugs. The other shipped four and caused two incidents that ate a sprint to clean up. Same sentence, different rating, and the only way you know is by looking at the actual work.

So tie every score to something you can point at, like a merged pull request, a design doc, an incident write-up, or a teammate they unblocked. Tell your engineers to keep a running list of what they did, because nobody remembers in March what they shipped in July.

What AI just changed about leveling

Here is the part the other guides haven’t caught up to.

When 84% of developers are now using or planning to use AI coding tools, up from 76% a year earlier, “writes working code” stops being a useful test for a junior. The machine writes working code. (Stack Overflow’s 2025 Developer Survey puts daily AI use among professionals at 51%, and notably, trust in the output is falling, not rising.) That is exactly why a modern .NET developer job description has to screen for judgment, not syntax recall.

So a matrix written in 2022 is now grading people on the easy part.

The skills that used to show up at senior, reviewing code you didn’t write, catching the subtle bug, knowing when the generated answer is confidently wrong, framing the problem before any code exists, now matter at the junior and mid levels too. AI raised the floor on output and raised the bar on judgment at the same time.

If you are refreshing your matrix this year, that is the edit to make. Push “evaluates and corrects code, including AI-generated code” down into your mid-level rows. Weight problem framing and review earlier than you used to. The competency that AI cannot fake is knowing whether the thing you built is the right thing, and that has always been the expensive part.

The job didn’t get easier. The expensive skill just moved earlier in the career.

How to build yours, step by step

You do not need a committee and a quarter. You need a weekend and the willingness to be opinionated. Here is the order I would go in.

1. Give it one owner. A matrix designed by committee turns into mush, every stakeholder adding their pet competency until it is forty rows long. Put a director or a strong EM in charge and let them take input without taking dictation.
2. Write the levels first, then the rows. Decide how many levels you actually have. A 20-person team does not need six. Name them, then list the six to eight competencies that matter for your work.
3. Fill the cells with behavior, not adjectives. Every cell should describe something a manager could watch someone do. If you can’t observe it, you can’t grade it, and you definitely can’t coach it.
4. Calibrate before you ship it. Take three real engineers everyone knows well and grade them against the draft. If the matrix says your obvious senior is a mid, the matrix is wrong, not the engineer. Adjust until it matches reality.
5. Put it where people work. Link it from the job descriptions, the review template, and the promotion form. If it lives in a forgotten wiki page, people treat it like one more doc nobody reads. Put it in the path they already walk and it gets used.
6. Revisit it once a year. Roles change, tools change, and as the AI point above shows, the definition of “junior” can shift under you in twelve months.

When you don’t need a matrix yet

Here is the contrarian part: plenty of teams build one too early.

If you are ten engineers and one manager who has read everyone’s code, you do not have a leveling problem. The manager already knows who is senior and who is close. A formal matrix at that size is busywork that makes you feel organized.

The matrix starts earning its keep when you can no longer hold the whole team in your head, or when two managers start grading the same kind of work differently. That is usually somewhere past twenty or thirty engineers, or the day you add a second manager. Build it the quarter before you need it, not three years early.

Keep it simple and boring until you can’t.

Using the matrix where it counts: hiring and promotion

The matrix earns its keep in two rooms.

The first is the interview. Once a level has observable behavior attached, your interview questions write themselves. For the judgment row, you ask, “Tell me about a time the ticket was wrong. How did you figure that out, and what did you do?” For systems thinking, you ask someone to walk through how a service they owned could fail and how they would know. For ownership, you ask about a project they pushed through when nobody told them to. For mentorship, you ask who they have helped level up and how they did it. You are testing for the behavior the matrix already named, instead of vibing your way to a yes. For an Apple-platform example, see our interview questions for iOS engineers.

The second room is the promotion or calibration meeting, and this is where a matrix saves you from yourself.

Without one, the loudest manager wins and the quiet engineer who shipped the hard thing gets passed over. With one, every manager has to bring evidence against the same anchors, and “I just feel like they’re ready” doesn’t survive contact with “show me where they did staff-level work.” It won’t kill bias completely, but it does kill the lazy version of it.

The matrix also tells you who is ready before the title catches up. The engineers worth promoting are already doing the next level’s work without being asked: taking the vague project nobody scoped, unblocking other people, quietly raising the bar in code review. The ones who are not ready show the opposite pattern, strong output but siloed, slipping deadlines, or unwilling to mentor anyone. Watch the behavior across two review cycles before you call it, because one great quarter is not a new baseline.

This is also the antidote to a hiring mistake I see constantly, which is chasing the cheapest developer instead of the right level. I have a name for that, cheapshoring, and it burns more companies than bad code ever has. A matrix forces the honest question: what level do we actually need, and are we paying for it or pretending the rate is the same thing as the skill?

One matrix, even when the team is spread across the world

Most matrices quietly assume everyone sits in one building. Almost no real engineering team does anymore.

This is the part I have spent the last several years living. Full Scale runs a staff augmentation model, which means our engineers in the Philippines join a client’s existing team and work directly inside it. For that to work, a client’s “senior” and our “senior” have to mean the same thing. The matrix is what makes that true.

There are smart people all over the world. The thing that breaks distributed teams is not skill, it is a fuzzy bar that lets one office grade differently than another. A shared matrix is the fix. Everyone is measured against the same behavior, wherever they log in from.

One opinion I will not soften: you always need technical leadership in-house. The matrix tells you what good looks like. It does not replace someone senior on your side who owns the standard and holds the line on it. Software development is about communication more than anything else, and a grid is not a substitute for a person who can tell whether the work is actually good.

For a deeper look at how that integrated model works, we wrote up the dedicated team model separately.

The don’ts (where these things go wrong)

I have watched competency matrices fail more often than they succeed, and it is nearly always the same handful of reasons.

• Overbuilding it. Forty competencies and nine levels feels thorough and is unusable. If an engineer can’t hold the shape of it in their head, they will ignore it. Fewer rows, sharper definitions.
• Adjectives instead of behavior. “Excellent collaborator” grades itself differently for every manager. Write down the specific action you would watch someone do.
• Measuring only the technical rows. They are the easiest to write, so lazy matrices stop there and accidentally tell your engineers that mentoring and judgment don’t count. They count more as you go up, not less.
• Freezing it. A matrix you never revisit becomes a lie within a year or two, especially right now. Build it to be edited.
• Shipping it in silence. Drop a grading grid on a team with no explanation and they hear “we are about to rank and cut you.” Roll it out as a development tool, in person, or it dies as bureaucracy.

Real matrices worth stealing from

You do not have to invent this from scratch. Some of the best engineering orgs publish theirs.

• CircleCI open-sourced their engineering competency matrix and the thinking behind it. It is the canonical reference, and refreshingly opinionated, leadership shows up at every level and technical skills are only about a fifth of the whole thing.
• progression.fyi collects real, published career frameworks from companies like Monzo and others, side by side, so you can compare structures.
• The Open Competency Matrix on GitHub is a community-built, editable starting point if you want a data structure rather than prose.
• Bradford Fults’ software engineer leveling matrix is a clean, lightweight public example released under Creative Commons.

Read two or three, then throw out everything that doesn’t fit how your team actually works. A matrix you borrowed wholesale describes someone else’s company.

Interviewing for a specific stack? See our guides for Java, React, Ruby on Rails, Python, and .NET developers.

See also our Vue and Flutter developer interview guides.

Hire engineers who already fit your matrix

A competency matrix tells you exactly what level you need. The next problem is filling it.

That is the part we handle. Full Scale places senior engineers, 7-plus years of experience on average, into your team in about two weeks, and we keep them: our developer retention runs 93%. They are measured against the same kind of matrix laid out above, so a senior we send is a senior by your definition, not a title we made up.

If you are tired of guessing whether a hire is really at the level you need, book a discovery call and we will talk through what your team is actually missing.