TY - GEN
T1 - Processing Large Datasets of Fined Grained Source Code Changes
AU - Levin, Stanislav
AU - Yehudai, Amiram
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/9
Y1 - 2019/9
N2 - In the era of Big Code, when researchers seek to study an increasingly large number of repositories to support their findings, the data processing stage may require manipulating millions and more of records. In this work we focus on studies involving fine-grained AST level source code changes. We present how we extended the CodeDistillery source code mining framework with data manipulation capabilities, aimed to alleviate the processing of large datasets of fine grained source code changes. The capabilities we have introduced allow researchers to highly automate their repository mining process and streamline the data acquisition and processing phases. These capabilities have been successfully used to conduct a number of studies, in the course of which dozens of millions of fine-grained source code changes have been processed.
AB - In the era of Big Code, when researchers seek to study an increasingly large number of repositories to support their findings, the data processing stage may require manipulating millions and more of records. In this work we focus on studies involving fine-grained AST level source code changes. We present how we extended the CodeDistillery source code mining framework with data manipulation capabilities, aimed to alleviate the processing of large datasets of fine grained source code changes. The capabilities we have introduced allow researchers to highly automate their repository mining process and streamline the data acquisition and processing phases. These capabilities have been successfully used to conduct a number of studies, in the course of which dozens of millions of fine-grained source code changes have been processed.
KW - Software Evolution, Empirical Software Engineering, Mining Software Repositories, AST
UR - http://www.scopus.com/inward/record.url?scp=85077181623&partnerID=8YFLogxK
U2 - 10.1109/ICSME.2019.00064
DO - 10.1109/ICSME.2019.00064
M3 - ???researchoutput.researchoutputtypes.contributiontobookanthology.conference???
AN - SCOPUS:85077181623
T3 - Proceedings - 2019 IEEE International Conference on Software Maintenance and Evolution, ICSME 2019
SP - 382
EP - 385
BT - Proceedings - 2019 IEEE International Conference on Software Maintenance and Evolution, ICSME 2019
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2019 IEEE International Conference on Software Maintenance and Evolution, ICSME 2019
Y2 - 30 September 2019 through 4 October 2019
ER -