Multilingual LibriSpeech (MLS) dataset is a large multilingual corpus suitable for speech research. The dataset is derived from read audiobooks from LibriVox and consists of 8 languages - English, German, Dutch, Spanish, French, Italian, Portuguese, Polish.
Consists of train, dev and test sets for each language. Also, includes small training set for limited supervision (10hr, 1hr or 10 minutes of labelled speech).
| Language | Download Link (Original: flac) |
Download Link (Compressed: opus) |
|---|---|---|
| English * | mls_english.tar.gz (2.4T) | mls_english_opus.tar.gz (651G) |
| German | mls_german.tar.gz (115G) | mls_german_opus.tar.gz (29G) |
| Dutch | mls_dutch.tar.gz (86G) | mls_dutch_opus.tar.gz (23G) |
| French | mls_french.tar.gz (61G) | mls_french_opus.tar.gz (16G) |
| Spanish | mls_spanish.tar.gz (50G) | mls_spanish_opus.tar.gz (14G) |
| Italian | mls_italian.tar.gz (15G) | mls_italian_opus.tar.gz (3.8G) |
| Portuguese | mls_portuguese.tar.gz (9.3G) | mls_portuguese_opus.tar.gz (2.5G) |
| Polish | mls_polish.tar.gz (6.2G) | mls_polish_opus.tar.gz (1.6G) |
Consists of language modelling corpus and pre-trained 3-gram and 5-gram LMs.
| Language | Download Link |
|---|---|
| English | mls_lm_english.tar.gz (44G) |
| German | mls_lm_german.tar.gz (2.7G) |
| Dutch | mls_lm_dutch.tar.gz (1.4G) |
| French | mls_lm_french.tar.gz (4.8G) |
| Spanish | mls_lm_spanish.tar.gz (1.2G) |
| Italian | mls_lm_italian.tar.gz (1.7G) |
| Portuguese | mls_lm_portuguese.tar.gz (558M) |
| Polish | mls_lm_polish.tar.gz (30M) |
| About | Download Link |
|---|---|
| Downloaded text from LibriVox books | lv_text.tar.gz (2.0G) |
| Unrated dev/test transcripts (before human rating) | unrated_transcripts.tar.gz (2.8M) |
All the above links are hosted AWS S3 bucket and can be downloaded using AWS CLI tools as well. For downloading using AWS CLI tools, create an AWS account, put the credentials in the CLI tools and all the resources can be downloaded for free. To get the S3 bucket path of an URL, replace https://dl.fbaipublicfiles.com/ with s3://dl.fbaipublicfiles.com/. For example, the S3 bucket path of URL https://dl.fbaipublicfiles.com/mls/mls_polish.tar.gz is s3://dl.fbaipublicfiles.com/mls/mls_polish.tar.gz.
More details on these files and their directory structure can be found in README files included in the .tar.gz files.
You can cite the data using the following BibTeX entry:
@article{Pratap2020MLSAL,
title={MLS: A Large-Scale Multilingual Dataset for Speech Research},
author={Vineel Pratap and Qiantong Xu and Anuroop Sriram and Gabriel Synnaeve and Ronan Collobert},
journal={ArXiv},
year={2020},
volume={abs/2012.03411}
}
NOTE: We have made few updates to the MLS dataset after our INTERSPEECH paper was submitted to include more #hours and also improve the quality of transcripts. To avoid confusion (by having multiple versions), we are making ONLY one release with all the improvements included. For accurate dataset statistics and baselines, please refer to the arXiv paper - https://arxiv.org/abs/2012.03411.
* This resource can also be downloaded in parts from the 100GB splits - mls_english_parts_list.txt and mls_english_opus_parts_list.txt for the original and compressed versions respectively. After downloading all the splits, run `cat` on the files to create a single tar file and verify the md5 checksum.