The Review & Collaborate features provide valuable insights into organizational behavior with respect to how individuals are collaborating with their peers during the code review process. 

Review Workflow shows all the Pull Requests that were open at any time during the selected period. PRs opened before the selected period are included if they were open during the selected period. Use this report as your starting place for a birds-eye view of all Pull Request activity.

Review Collaboration shows collaboration stats between the submitters and the reviewers. 

Submitter Metrics quantify how submitters are responding to comments, engaging in discussion, and incorporating suggestions. Submitter metrics are: 

  • Responsiveness is the average time it takes to respond to a comment with either another comment or a code revision;
  • Comments addressed is the percentage of Reviewer comments that were responded to with a comment or a code revision;
  • Receptiveness is the ratio of follow-on commits to comments. It’s important to remember that Receptiveness is a ‘goldilocks’ metric—you’d never expect this metric to go up to 100%, and if you did it’d be indicative of a fairly unhealthy dynamic where every single comment lead to a change;
  • Unreviewed PRs is the percentage of PRs submitted that had no comments.

Reviewer Metrics provide a gauge for whether reviewers are providing thoughtful, timely feedback. Reviewer metrics are:

  • Reaction time is the average time it took to respond to a comment;
  • Involvement is the percentage of PRs a reviewer participated in. It’s important to note that this metric is a highly context-based metric. At an individual or team level, “higher” is not necessarily better as it can point to a behavior where people are overly-involved in the review process, but there are certain situations where you’d expect to see Involvement very high, sometimes from a particular person on the team and other times from a group that’s working on a specific project;
  • Influence is the ratio of follow-on commits to comments made in PRs;
  • Review coverage represents the percentage of PRs reviewed.
  • Robust comments have a length over 200 characters;
  • Regular comments span between 100-200 characters;
  • Trivial comments are under 100 characters long.

The sharing Index offers a daily overview of how broadly information is being shared amongst a team by looking at who's reviewing whose PRs. If you hover over a column in the chart, it will show you the number of PRs, the number of Active Reviewers and Submitters, as well as the Sharing Index.

Sharing Index metrics are:

  • PRs is the total number of PRs that were reviewed;
  • Sharing Index measures how broadly information is being shared amongst a team by looking at who is reviewing whose PRs;
  • Active Reviewers is the count of active users who actually reviewed a PR in the selected time period;
  • Submitters is the total number of users who submitted a PR in the selected time period.

If you hover over a profile, it will show you the connections between engineers.

PR Resolution shows a time to close work trend graph for the selected period as well as pull requests metrics.

There are six metrics that comprise the PR Resolution report:

  • Time to Resolve is the average time it takes to close a Pull Request;
  • Time to First Comment is the average time between when a Pull Request is opened and the time the first Reviewer comments;
  • Follow-on Commits is the average number of code revisions once a Pull Request is opened for review;
  • Reviewers is the average number of reviewers per Pull Request;
  • Reviewer Comments is the average number of reviewer comments per Pull Request;
  • Avg. Comments per Reviewer is the average number of comments per Reviewer.

The Fundamentals feature gives you the ability to see your team's performance across the fundamental productivity metrics for both submitters and reviewers.

Submitter Fundamentals provide a bigger picture regarding the submitters metrics. You can select which team's performance to view, which repositories' contribution to analyze and what period to inspect. Each metric has its own average value which you can see in the tab view or on the left side of the chart.
This tab analyzes the Responsiveness (the average time it takes to respond to a comment with either another comment or a code revision). Click on the metrics tabs (Responsiveness, Comments Addressed, Receptiveness and Unreviewed PRs) to scroll through them. Each metric has a corresponding chart.

If you hover over a column in the chart, you'll see the metric's value for that particular period.

This tab analyzes the Comments Addressed metric (the percentage of Reviewer comments that were responded to with a comment or a code revision). 

This tab evaluates the Receptiveness metric (the ratio of follow-on commits to comments).

This tab investigates the Unreviewed PRs metric (the percentage of PRs submitted that had no comments).

Reviewer Fundamentals offers an overview regarding the reviewers metrics. You can select which team's performance to view, which repositories' contribution to analyze and what period to inspect. Each metric has its own average value which you can see in the tab view or on the left side of the chart.
This tab analyzes the Reaction Time (the average time it took to respond to a comment) metric.

This tab analyzes the Involvement (the percentage of PRs a reviewer participated in).

This tab analyzes the Influence (the ratio of follow-on commits to comments made in PRs).

This tab analyzes the Review Coverage (the percentage of PRs reviewed).

Did this answer your question?