Nudge: Accelerating Overdue Pull Requests Towards Completion

by Maddila, Chandra and Upadrasta, Sai Surya and Bansal, Chetan and Nagappan, Nachiappan and Gousios, Georgios and van Deursen, Arie

You can get a pre-print version from here.

Abstract

Pull requests are a key part of the collaborative software development and code review process today. However, pull requests can also slow down the software development process when the reviewer(s) or the author do not actively engage with the pull request. In this work, we design an end-to-end service, Nudge, for accelerating overdue pull requests towards completion by reminding the author or the reviewer(s) to engage with their overdue pull requests. First, we use models based on effort estimation and machine learning to predict the completion time for a given pull request. Second, we use activity detection to reduce false positives. Lastly, we use dependency determination to understand the blocker of the pull request and nudge the appropriate actor(author or reviewer(s)). We also do a correlation analysis to understand the statistical relationship between the pull request completion times and various pull request and developer related attributes. Nudge has been deployed on 147 repositories at Microsoft since 2019. We do a large scale evaluation based on the implicit and explicit feedback we received from sending the Nudge notifications on 8,500 pull requests. We observe significant reduction in completion time, by over 60%, for pull requests which were nudged thus increasing the efficiency of the code review process and accelerating the pull request progression.

Bibtex record

@article{MSBNGD20,
  title = {Nudge: Accelerating Overdue Pull Requests Towards Completion},
  author = {Maddila, Chandra and Upadrasta, Sai Surya and Bansal, Chetan and Nagappan, Nachiappan and Gousios, Georgios and van Deursen, Arie},
  year = {2020},
  eprint = {2011.12468},
  archiveprefix = {arXiv},
  primaryclass = {cs.SE},
  url = {https://arxiv.org/pdf/2011.12468.pdf}
}

The paper