Pull Requests Impact

Understanding Impact

Impact is a way to measure the magnitude of code changes that are happening, in a way that goes beyond simplistic measurements like lines of code.

Impact attempts to answer the question: “Roughly how much cognitive load did the contributor/team carry when implementing these changes?”

Impact is a measure of work size that takes the following into account:

The amount of code in the change
What percentage of the work is edits to old code
The surface area of the change (think ‘number of edit locations’)
The number of files affected
The severity of changes when old code is modified
How this change compares to others from the project history

One contributor makes an addition of 100 new lines of code to a single file. Compare that to another contributor effort which touches three files at multiple insertion points where they add a total of 16 lines while removing 24 lines.

The significance of each contribution can’t be boiled down to just the amount of code being checked in. Even without knowing specifics, it’s likely that the second set of changes were more difficult to implement, given that they involved several spot-edits to old code.

On one hand, we have someone adding 100 lines of new code to a single file, while on the other hand, only 16 lines of new code are written, but there’s a bit more going on here:

this change required modifying previous work
the edits happened in 4 different locations
3 different files were affected

Even without knowing the severity of changes or comparing to historical changes, it’s probably safe to assume that the second contribution was more ‘expensive,’ and therefore carries a higher impact score.

Although the first set of changes is technically more code, the added complexity of the work from the second change could arguably make that change set at least as much work, and possibly more — even in the simplistic representation above, it’s clear that there’s more than just lines of code at play.

How to Use Pull Requests Impact

Quality Standard Optimization: Use insights from the impact of Pull Requests to refine or establish code quality standards and best practices, ensuring that contributions consistently enhance project quality.
Tailored Training Programs: Leverage data from the Pull Requests Impact metric to customize training sessions, addressing specific areas that frequently influence pull requests negatively, and enhancing contributor' skills in these key areas.
Performance Analysis: Incorporate the Pull Requests Impact metric into performance evaluations, providing a factual basis to assess individual contributions and their effectiveness towards achieving project goals.

Strategic Use of Pull Requests Impact

Strategic Project Planning: Analyze longitudinal trends in the impact of Pull Requests to make informed decisions regarding project direction or necessary adjustments in project scope, aligning development efforts with strategic objectives.
Code Quality Enhancement: Regularly review the impact of Pull Requests to set or adjust code quality standards, ensuring continuous improvement in project outputs and contributor performance.
Focused Contributor Training: Use insights from this metric to tailor training and development initiatives, concentrating on prevalent issues that negatively affect Pull Requests, thus boosting overall project quality.

Considerations for Implementation

Continual Feedback and Adjustment: Use the Pull Requests Impact metric as a tool for ongoing feedback and adjustment, enabling continuous improvement in coding practices and project alignment.
Balanced Evaluation Approach: Ensure that the metric is used as part of a broader set of evaluation tools to provide a well-rounded view of contributors' contributions and impacts.