Can Issues Reported at Stack Overflow Questions be Reproduced? An Exploratory Study
Software developers often look for solutions to their code level problems at Stack Overflow. Hence, they frequently submit their questions with sample code segments and issue descriptions. Unfortunately, it is not always possible to reproduce their reported issues from such code segments. This phenomenon might prevent their questions from getting prompt and appropriate solutions. In this paper, we report an exploratory study on the reproducibility of the issues discussed in 400 questions of Stack Overflow. In particular, we parse, compile, execute and even carefully examine the code segments from these questions, spent a total of 200 man hours, and then attempt to reproduce their programming issues. The outcomes of our study are two-fold. First, we find that 68% of the code segments require minor and major modifications in order to reproduce the issues reported by the developers. On the contrary, 22% code segments completely fail to reproduce the issues. We also carefully investigate why these issues could not be reproduced and then provide evidence-based guidelines for writing effective code examples for Stack Overflow questions. Second, we investigate the correlation between issue reproducibility status (of questions) and corresponding answer meta-data such as the presence of an accepted answer. According to our analysis, a question with reproducible issues has at least three times higher chance of receiving an accepted answer than the question with irreproducible issues
Mon 27 MayDisplayed time zone: Eastern Time (US & Canada) change
11:55 - 12:30 | Session VII: Collaboration & Communication (Part 2)MSR 2019 Technical Papers at Place du Canada Chair(s): Kelly Blincoe University of Auckland | ||
11:55 15mFull-paper | Can Issues Reported at Stack Overflow Questions be Reproduced? An Exploratory Study MSR 2019 Technical Papers Saikat Mondal University of Saskatchewan, Masud Rahman University of Saskatchewan , Chanchal K. Roy University of Saskatchewan Pre-print | ||
12:10 15mFull-paper | Exploratory Study of Slack Q&A Chats as a Mining Source for Software Engineering Tools MSR 2019 Technical Papers Preetha Chatterjee University of Delaware, USA, Kostadin Damevski Virginia Commonwealth University, Lori Pollock University of Delaware, USA, Vinay Augustine , Nicholas A. Kraft ABB Corporate Research Pre-print | ||
12:25 6mShort-paper | Impacts of Daylight Saving Time on Software Development MSR 2019 Technical Papers Junichi Hayashi Osaka University, Yoshiki Higo Osaka University, Shinsuke Matsumoto Osaka University, Shinji Kusumoto Osaka University Pre-print |