Course Introduction

In this course, we will learn

What software analytics is.
What you can do with it.
How to do it.

We will also learn

How to read papers (this lecture!)
How to do systematic literature reviews
How to write about papers

Course Teacher

Georgios Gousios

Georgios Gousios, http://gousios.org/

Schedule

Date	Week	Lecture	Topic	Lecturer
3/9	1	1	Course introduction, Quantitative methods in Software Engineering	MB
5/9	1	2	Discussion groups	GG
10/9	2	1	Process Analytics	AR / GG
12/9	2	2	Testing Analytics, Testing Paper	students, MB
17/9	3	1	Build Analytics, Building Paper	students, MB
19/9	3	2	Bug Prediction	students, MB
24/9	4	1	Software Ecosystem Analytics	JH

Schedule (2)

Date	Week	Lecture	Topic	Lecturer
26/9	4	2	Release Engineering Analytics	students, AR
1/10	5	1	Results: Survey on software analytics	students
3/10	5	2	Code review	students, GG
8/10	6	1	Runtime and Performance Analytics, Analytics at work: Adyen	Mauricio Aniche, MK
10/10	6	2	App store analytics	students, MK
15/10	7	1	Analytics at work: ING	Hennie Huijgens
17/10	7	2	Results: Replication project results	students

How will we study

The course is a seminar. Therefore:

We will study papers, lots of them!
We will write nice summaries
We will present the results of our study in the classroom
We will replicate papers
We will work in groups (3-4 people)

Course deliverables: The survey

We will work together on a survey of modern software analytics. The topics we will cover are:

Level	Topics
Product	source code, repository, builds, static analysis, ecosystems
Process	PR/Code reviews, release engineering, communities, agile, testing etc
Runtime	logs, crashes, security, performance
Community	app stores, social media, user acceptance

All topics to work on correspond to the contents of the project’s TOC.

All topics have an instructor attached to them. You can find the responsible instructor on the course’s home page

Working on the survey (1)

Each topic page lists 5-10 publications that can serve as a starting point to get you aquainted with the research area.
We will follow Kitchenham’s survey protocol [2].
Each group will select a topic. You can express interest in a topic. All topics need to be covered. Some groups may get topics they don’t like.
Software Analytics is very applied; outside of the scientific literature we will therefore also write a section covering
- non-scientific experience reports (e.g., blog posts)
- practical tools (Open-Source or commercial)
- a look outside the academic ivory tower (is it used at all?)

Working on the survey (2)

The survey is a Bookdown document (template), available on GitHub. Fork it and file PRs. Every group works on one chapter (one file).

D By group vote: American English (60%)
Linebreaks after 99 chars (i.e., no lines with 100 chars)
Only use pure RMarkdown

How to read a paper

Keshav, Srinivasan. “How to read a paper.” ACM SIGCOMM Computer Communication Review 37.3 (2007): 83-84.

D Read the above paper. After 10 minutes, we will discuss it.

How to get a paper

At Uni!
Via VPN (to Uni). TU Delft has contracts with most major Tech publishers to get you free access to papers.

Q How much does one article cost?

There are alternatives …

We use Google Scholar to retrieve metrics. There is other tools such as Scopus. Even the IEEE Xplore Digital Library gives metrics.

D Use the different services to derive number of reads, number of citations, and the BibTeX entry for: Beller, Moritz, Georgios Gousios, and Andy Zaidman. “How (much) do developers test?.” Software Engineering (ICSE), 2015 IEEE/ACM 37th IEEE International Conference on. Vol. 2. IEEE, 2015. [beller2015much]

Course deliverables: Paper Replication

Replication: Redoing a published study following exactly the same research protocol, using the same (exact) or different subjects.

Replication is often touted, seldom practiced. To change this, we will replicate existing studies, as follows:

You will select a recent (max age: 4 years) paper from the papers you surveyed
You propose it to the instructor responsible for your research area, along with justification on why you need to replicate it.
You do the work and write a paper with your findings

How to work

Each week: i) you produce a reviewable deliverable, ii) review the deliverables of two other groups, and iii) prepare for the classes.
Survey task duration: 5 weeks (starting today!)
Replication task duration: 5 weeks (starting on week 3)

A deliverable is either a document and/or source code.

How to review:

Documents: print, read and annotate with comments
Source code: use GitHub

Preparation for classes comprises reading a set of papers for the lecture topic (usually 2-3).

Grades

The final course grade will be calculated as:

50% - Quality of survey
40% - Replication of existing work
20% - Presentation of results

All deliverables will be peer-reviewed and cross-graded by 2 other teams. The peer-review grade is 50% of the final grade per grade item. The results add up to 110%.

TODO for today

Form a group
Study the course materials and select a topic
Register your group and prefered topics. Use this form: https://goo.gl/forms/LrbxTbxIkcm0Wd3p2

Slide symbols

Q A question with a known answer; this will be revealed, but we should work together towards it!
D An open discussion item; we need to think and discuss.

Lecture notes

Freely available on the web, on Georgios’s homepage (http://gousios.org/teaching.html)
You can print/download them before the lecture and bring them along to make additional notes.
We are looking forward to improve them! If you have suggestions, find mistakes etc, let me know!

Bibliography

[1]

B. Kitchenham, “Procedures for performing systematic reviews,” Keele, UK, Keele University, vol. 33, no. 2004, pp. 1–26, 2004.

[2]

B. Kitchenham, O. P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, “Systematic literature reviews in software engineering–a systematic literature review,” Information and software technology, vol. 51, no. 1, pp. 7–15, 2009.