Skip to Main content Skip to Navigation
New interface
Conference papers

A Large-scale Dataset of (Open Source) License Text Variants

Abstract : We introduce a large-scale dataset of the complete texts of free/open source software (FOSS) license variants. To assemble it we have collected from the Software Heritage archive-the largest publicly available archive of FOSS source code with accompanying development history-all versions of files whose names are commonly used to convey licensing terms to software users and developers. The dataset consists of 6.5 million unique license files that can be used to conduct empirical studies on open source licensing, training of automated license classifiers, natural language processing (NLP) analyses of legal texts, as well as historical and phylogenetic studies on FOSS licensing. Additional metadata about shipped license files are also provided, making the dataset ready to use in various contexts; they include: file length measures, detected MIME type, detected SPDX license (using ScanCode), example origin (e.g., GitHub repository), oldest public commit in which the license appeared. The dataset is released as open data as an archive file containing all deduplicated license files, plus several portable CSV files for metadata, referencing files via cryptographic checksums.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03624198
Contributor : Stefano Zacchiroli Connect in order to contact the contributor
Submitted on : Thursday, March 31, 2022 - 2:49:10 PM
Last modification on : Tuesday, April 5, 2022 - 3:33:39 AM

Files

main.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Stefano Zacchiroli. A Large-scale Dataset of (Open Source) License Text Variants. The 2022 Mining Software Repositories Conference, May 2022, Pittsburgh, Pennsylvania, United States. ⟨10.1145/3524842.3528491⟩. ⟨hal-03624198⟩

Share

Metrics

Record views

461

Files downloads

67