IN4334 - Machine Learning for Software Engineering

Course information

General description

Software repositories archive valuable software engineering data, such as source code, execution traces, historical code changes, mailing lists, and bug reports. This data contains a wealth of information about a project’s status and history. Doing data science on software repositories, researchers can gain empirically based understanding of software development practices, and practitioners can better manage, maintain, and evolve complex software projects.

In the recent years, the advances in Machine Learning and AI technologies, as demonstrated by the successful application of Deep Neural Networks in various domains did not go unoticed in the field of Software Engineering. Researchers have applied DNNs to tackle issues such as automated program repair, code summarization, code structure representation, etc.

IN4334 is a seminar course that aims to give students a deep understanding of and hands-on approach on how deep neural networks and NLP techniques are used by today’s industry leaders to represent knoweledge and solve existing problems in novel ways.

Learning Objectives

This course will enable students to:

• Understand and analyze related work in the area of software analytics
• Apply appropriate research methods to extract data from software repositories
• Evaluate the applicability of results in the software analytics literature on practical problems
• Analyze data from software repositories and extract new insights

Course Organization

• 5 ECTS: This means that you need to devote at least 140 hours of study for this course, per person. Given that the course runs in a period of 7 weeks, the workload is around 20 hours a week.

• Lectures: The course consists of 14 2-hour lectures. You are not required, but you are strongly encouraged, to attend. We will be discussing 2-3 papers (presentations given either by the lecturer or by teams) in terms of techniques, insights and impact.

• Homework: Before each lecture, you must read and prepare questions about the papers that will be discussed during the lecture. You can find the list of the papers to read on the beginning of each week’s lecture.

• Lecturers: The course is supervised by Georgios Gousios and Maurício Aniche, who are responsible for the content, assignments and exams. Several people will provide extra lectures in topics of their expertise.

• Course work: To finish the course you will need to:

  • Give a 10-min presentation of 1-2 papers
  • Apply ML to a software engineering problem (either replicating a recent paper or proposing a new approach)
  • Review and grade each other’s work
  • Give a pitch and present a poster

• Groups: You will work in groups of 3-4 persons. You are free to choose your group partners.

• Labs: Unsupervised, optional. 4 hours per week, designed to give you a place and time to work together.

The course projects

During the course, you will choose a software engineering problem and will propose a ML-based solution for that problem.

See the list of suggested projects (and existing papers that try to tackle it). Your job will be to either:

1. Replicate an existing paper: Replication is a topic much touted but seldom practiced in the software engineering community. It is, however, a core aspect of science, especially empirical. You can download readily available data sets published together with the paper, requesting the data from the original authors or by applying the same techniques on different data.

2. Propose a completely new approach to the problem (highly appreciated!). Did you find a way to improve the existing work? Did you see the problem from a perspective that current research hasn’t explored yet? Your task will be to collect data and test your hypothesis.

You will implement different ML/DL models. You are required to use Python and more specifically, Pytorch. Check our curated list of tutorials that might help you in getting started with different NLP, DL, and ML topics.

Required reading for week 1:

• Quantitative methods in Software Engineering
• Discussion presentation template
• This paper, by Pradel and Sen [1]

Contents

Date	Week	Lecture	Topic	Lecturer
3/9	1	1	Course Introduction, How to read a paper in a group	GG
4/9	1	2	Enough data science to become dangerous, notebook	GG
11/9	2	1	Enough neural networks to become dangerous, Hand-written number recognition with PyTorch, Sing like Freddie via LSTMs notebook	MFA
12/9	2	2	Enough NLP to become dangerous, Building your ML pipeline	MFA / GG
17/9	3	1	Representing code	Group 3
18/9	3	2	Code embeddings	Group 8
24/9	4	1	Source code analysis	Group 7 / VM
25/9	4	2	NLP-based program analysis	Group 11 / VM
1/10	5	1	Finding bugs	Group 4 / MA
2/10	5	2	Code summarization	Group 5 / MA
8/10	6	1	Feedback session	MFA / GG
9/10	6	2	Code Completion	Group 1
15/10	7	1	Transfer Learning	Group 2 / VK
16/10	7	2	Repairing bugs	Group 6 / VK

Lecturers

• GG: Georgios Gousios
• MFA: Maurício Aniche

Guest lecturers

• MA: Miltos Allamanis (Microsoft Research)
• VM: Vadim Markovtsev (source{d})
• VK: Vladimir Kovalenko (TU Delft / JetBrains)

Assessment

The final course grade will be calculated as:

• 80% - Results of the research
• 20% - Presentation

All deliverables will be peer-reviewed by 2 other teams. The peer-review grade is 50% of the final grade per grade item.

Bibliography

[1]

M. Pradel and K. Sen, “DeepBugs: A learning approach to name-based bug detection,” Proc. ACM Program. Lang., vol. 2, no. OOPSLA, pp. 147:1–147:25, Oct. 2018.

Copyright

The course contents are copyrighted (c) 2018 - onwards by TU Delft and their respective authors and licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.