Baidu's ERNIE 3.0 paper discloses a 4TB pre-training corpus drawn from Baidu Search content, web text and encyclopedias
5 de jul. de 2021 · Coletado automaticamente de fontes públicas. Não revisado por uma pessoa.
Este item só está disponível no idioma original (inglês).
What happened
Baidu researchers published the ERNIE 3.0 technical paper, describing a model with 10 billion parameters trained on a 4TB corpus of plain texts plus a large-scale knowledge graph.
What the corpus contains, per Baidu's own paper
The 4TB corpus spans 11 categories and is built from ERNIE 2.0 data (including Baike encyclopedia entries and Wikipedia), Baidu Search content (including Baijiahao posts, Zhidao Q&A, Tieba forum posts and Experience articles), web text, long and short Q&A, poetry and couplets, domain-specific medical, legal and financial data, and a Baidu knowledge graph with more than 50 million facts.
What this establishes — and what it does not
This establishes, from Baidu's own technical disclosure, that material published by users and creators on Baidu's own platforms (Baijiahao, Zhidao, Tieba) and on the web was used as pre-training data for the ERNIE line that led to Ernie Bot. It does not establish the licence status of any individual work in that corpus.