ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference

by Mir, Amir M. and Latoškinas, Evaldas and Gousios, Georgios

You can get a pre-print version from here.
You can view the publisher's page here.
See the paper's associated code repository: saltudelft/many-types-4-py-dataset

Bibtex record

@inproceedings{MLG21,
  author = {Mir, Amir M. and Latoškinas, Evaldas and Gousios, Georgios},
  booktitle = {2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)},
  title = {ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference},
  month = may,
  year = {2021},
  volume = {1},
  pages = {585-589},
  doi = {10.1109/MSR52588.2021.00079},
  publisher = {IEEE Computer Society},
  address = {Los Alamitos, CA, USA},
  abstact = {
                 In this paper, we present ManyTypes4Py, a large Python dataset for machine
                 learning (ML)-based type inference. The dataset contains a total of 5,382
                 Python projects with more than 869K type annotations. Duplicate source code
                 files were removed to eliminate the negative effect of the duplication bias.
                 To facilitate training and evaluation of ML models, the dataset was split
                 into training, validation and test sets by files. To extract type
                 information from abstract syntax trees (ASTs), a light- weight static
                 analyzer pipeline is developed and accompanied with the dataset. Using this
                 pipeline, the collected Python projects were analyzed and the results of the
                 AST analysis were stored in JSON-formatted files. The ManyTypes4Py dataset
                 is shared on zenodo and its tools are publicly available on GitHub.
                 
                 },
  url = {https://arxiv.org/pdf/2104.04706.pdf},
  github = {saltudelft/many-types-4-py-dataset}
}