Performance review calibration: what it is and how to run a calibration meeting

Performance review calibration is a meeting where managers compare their draft performance ratings against one shared standard before the ratings are final. Each manager explains the ratings they propose. The group questions ratings that look out of line and adjusts them where the evidence supports it. The result is that “exceeds expectations” means the same thing in every team.

Calibration exists because ratings drift by manager. One manager rates generously. Another rates harshly. A third puts the whole team in the middle. Without a shared check, a rating says as much about the manager as about the work.

TL;DR

• Calibration is a group review of draft ratings, held before those ratings feed pay or promotion decisions.
• Its job is consistency across managers. In one large multinational, calibration committees changed about 25% of ratings, mostly downward (Demeré, Sedatole and Woods, 2019).
• It can add bias as well as remove it. The same study found committees pulled ratings toward the middle.
• A good meeting runs on evidence prepared in advance, not on whoever argues hardest.
• AI helps most before the meeting, by gathering evidence and flagging rating patterns. The meeting time then goes to the hard cases.

What is performance calibration?

Performance calibration is the step in a performance cycle where ratings are checked for consistency across managers. It usually takes the form of a calibration meeting, also called a calibration session or calibration committee. Managers bring their proposed ratings and the evidence behind them. HR or a senior leader runs the discussion.

Calibration does not replace the manager’s judgment. The manager still knows the work best. Calibration tests that judgment against how other managers apply the same standard.

Why is calibration necessary in performance reviews?

Three distortions show up in most rating rounds:

• Leniency. Some managers rate almost everyone high, so a top rating stops meaning anything.
• Centrality. Some managers rate almost everyone in the middle. That hides both the strongest performers and the real problems.
• Recency and halo. The rating reflects the last few weeks, or one strong trait, instead of the whole period.

Vague evidence makes all three worse. An analysis of over 200 performance reviews at one large technology company found vague praise in more reviews for women than for men (57% versus 43%). Developmental feedback was tied to business outcomes more often for men than for women (60% versus 40%) (Correll and Simard, 2016). A rating built on vague evidence is hard to defend and hard to calibrate.

Calibration also gives leaders a view across teams. With every team’s ratings side by side, it becomes clear where one department rates high and where the same skill gap repeats across teams.

What does the research say about calibration committees?

A 2019 study in Management Science looked at what calibration committees do with real ratings. Demeré, Sedatole and Woods used rating data from a large multinational organization. They found that:

• Committees adjusted ratings sparingly, in about 25% of cases.
• Downward adjustments were more frequent, and larger, than upward ones.
• Committees lowered ratings from managers who rated above average, and raised ratings from managers who rated below average.
• The net effect was more consistency across managers and less leniency, but more ratings in the middle.
• Committees were less likely to change the rating of an employee further from them in the hierarchy. They deferred to the manager closest to the work.

The practical reading is simple. Calibration works for consistency. Watch the middle of the scale. If every hard case ends at “meets expectations”, the meeting has traded one bias for another.

How does a calibration meeting work?

A calibration meeting usually follows five steps.

1. Managers submit draft ratings with evidence. Each rating comes with examples of work, goals met or missed, and feedback from the period. Employees do not see the ratings yet.
2. The facilitator prepares the view. Ratings are laid out by team and by manager, so the group sees each manager’s spread before the discussion starts.
3. The group starts with the outliers. That means a team rated far above the others, a rating that does not match the evidence, or a borderline promotion case.
4. Managers explain, peers question. The manager presents the evidence. The others ask how the same standard would apply to their own people. A rating changes only when the evidence supports the change.
5. Decisions and reasons are written down. Each changed rating gets a short reason. Managers then share the final ratings and the feedback with their employees.

Most of the meeting’s quality is decided in step one. When the evidence is thin, the discussion turns into opinion.

Who sits on a calibration committee?

A calibration committee is usually the managers whose ratings are under review, a facilitator from HR or people operations, and the leader one level above those managers. The facilitator keeps time and holds the group to the rubric. The senior leader settles disagreements and checks consistency across teams.

Keep the group small enough that every manager speaks to their own ratings. A large organization runs several committees, one per department or level, and then compares results across them.

What are the biggest challenges with calibration?

Unclear expectations

If roles have no written expectations, there is no standard to calibrate against. The meeting becomes a debate about what good looks like, instead of a check on whether someone met it. Setting expectations for each role and level comes first.

Manager readiness

New managers often find it hard to apply a standard consistently. Few feel confident questioning a more senior peer’s ratings in the room.

Process overload

A calibration round can take hours of meetings across a department. Without evidence prepared in advance, much of that time goes to reconstructing what happened during the period.

Reinforcing bias

The loudest voice can move a rating more than the evidence does. And as the research shows, calibration itself can push ratings toward the middle.

How does calibration improve fairness and insights?

Done well, calibration delivers three things:

1. A level standard. Employees are measured against the same bar, whatever their team or manager.
2. Better talent decisions. Promotion, pay and development decisions draw on ratings that can be compared.
3. Performance insights. Patterns across teams show where skills, effectiveness or management need attention.

How can AI help with performance review calibration?

AI helps most before the meeting starts. It can gather performance information from the period, summarize the evidence behind each draft rating, and flag patterns such as one manager rating far above the rest. That moves the meeting time from reconstructing the past to deciding the hard cases.

AI should not set the final rating. The point of calibration is that people stand behind a judgment about a person. Use AI to prepare the evidence and spot inconsistencies. Keep the decision with the managers.

How does Taito.ai help with performance calibration?

Taito.ai is a people operations system for teams of 30 to 300. It runs performance cycles with AI analysis, so the evidence is ready before the calibration meeting:

• Evidence gathered for you. AI performance analysis gathers and analyzes performance information from any data source, so managers write less by hand.
• One standard per role. A generated job catalog sets expectations for every role and seniority level, and analyzes how well each employee meets them.
• Results you can act on. Performance insights help people leaders act on the results of a performance cycle.
• Policy built in. Cycles stay in line with your organization’s performance management policies, for example for ISO 27001 or SOC 2 certification.

Sources

• Harvard Business Review (2016) Research: Vague Feedback Is Holding Women Back (Correll and Simard; the 57% versus 43% and 60% versus 40% figures).
• Management Science (2019) The Role of Calibration Committees in Subjective Performance Evaluation Systems, vol. 65, no. 4, pp. 1562–1585 (Demeré, Sedatole and Woods; the 25% adjustment rate and the direction of adjustments).