KR-BERT
A Small-Scale Korean-Specific Language Model
Git Hub
snunlp/KR-BERT
KoRean based BERT pre-trained models (KR-BERT) for Tensorflow and PyTorch - snunlp/KR-BERT
github.com
| Mulitlingual BERT(Google) | KorBERT(ETRI) | KoBERT(SKT) | KR-BERT character | KR-BERT sub-character | |
| vocab size | 119,547 | 30,797 | 8,002 | 16,424 | 12,367 |
| parameter size | 167,356,416 | 109,973,391 | 92,186,880 | 99,265,066 | 96,145,233 |
| data size | - (The Wikipedia data for 104 languages) |
23GB 4.7B morphemes |
- (25M sentences, 233M words) |
2.47GB 20M sentences, 233M words |
2.47GB 20M sentences, 233M words |
반응형
'Generative AI > Language Model' 카테고리의 다른 글
| [Language Model] Attention Model (0) | 2022.05.04 |
|---|---|
| [Language Model] BERTopic (0) | 2021.06.14 |
| [NLP] 통계적 언어 모델(Statistical Language Model, SLM) (0) | 2021.04.12 |
| [Language Model] T5(Text-to-Text Transfer Transformer) (0) | 2021.04.08 |
| [Language Model] Multilingual BERT (0) | 2021.04.08 |