Sooftware NLP - 한국어 사전학습 모델 (Korean Pre-trained Language Model)
한국어 사전학습 모델 (Korean Pre-trained Language Model)
공개된 한국어 사전학습 모델을 기록합니다. 크게 아래 3개의 모델 계열로 구분했으며, 모델 사이즈는 정확하지 않을 수 있습니다.
-
Encoder Model (BERT 계열)
-
Decoder Model (GPT 계열)
-
Encoder-Decoder Model (Seq2seq 계열)
Encoder Model (BERT 계열)
| Model Name | Size | Link |
|---|---|---|
| LASSL BERT | Small (10M) | link |
| LASSL RoBERTa | Small (10M) | link |
| TUNiB ELECTRA (Ko) | Small (10M) | link |
| TUNiB ELECTRA (Ko-En) | Small (10M) | link |
| KoELECTRA | Small (10M) | link |
| KcELECTRA | Small (10M) | link |
| Soongsil-BERT | Small (10M) | link |
| Ko-CHAR-ELECTRA | Small (10M) | link |
| Distill-KoBERT | 30M | link |
| KoBERT | Base (110M) | link |
| KorBERT | Base (110M) | link |
| HanBERT | Base (110M) | link |
| KcBERT | Base (110M) | link |
| Soongsil-BERT | Base (110M) | link |
| KLUE-RoBERTa | Base (110M) | link |
| KoELECTRA | Base (110M) | link |
| KcELECTRA | Base (110M) | link |
| TUNiB-ELECTRA (Ko) | Base (110M) | link |
| TUNiB-ELECTRA (Ko-En) | Base (110M) | link |
| LMKor Albert | Base (110M) | link |
| LMKor Bert | Base (110M) | link |
| LMKor Funnel | Base (110M) | link |
| LMKor ELECTRA | Base (110M) | link |
| KalBERT | Base (110M) | link |
| LASSL BERT | Base (110M) | link |
| KoBigBird | Base (110M) | link |
| Ko-CHAR-BERT | Base (110M) | link |
| Ko-CHAR-ELECTRA | Base (110M) | link |
| BrainSBERT | Base (110M) | link |
| KPFBERT | Base (110M) | link |
| KcBERT | Large (340M) | link |
| KLUE-RoBERTa | Large (340M) | link |
Decoder Model (GPT 계열)
| Model Name | Size | Link |
|---|---|---|
| LASSL GPT2 | Small (20M) | link |
| LMKor KoGPT2 | Base (110M) | link |
| SKT KoGPT2 | Base (110M) | link |
| SKT KoGPT Trinity | 1.2B | link |
| KakaoBrain KoGPT | 6B | link |
Encoder-Decoder Model (Seq2seq 계열)
| Model Name | Size | Link |
|---|---|---|
| Cosmoquester BART | Mini (13M) | link |
| KE-T5 (Ko-En) | Small (60M) | link |
| KE-T5 (Ko) | Small (60M) | link |
| LMKor T5 | Small (60M) | link |
| Cosmoquester BART | Small (40M) | link |
| LMKor BERT Shared | Base (130M) | link |
| SKT KoBART | Base (130M) | link |
| Cosmoquester BART | Base (130M) | link |
| KcT5 | Base (220M) | link |
| KE-T5 (Ko-En) | Base (220M) | link |
| KE-T5 (Ko) | Base (220M) | link |
| Asian BART (Ko) | Base (360M) | link |
| pko-t5 (Ko) | Base (250M) | link |
| Wisenut KoT5 (Ko) | Base (220M) | link |
| mBART | Large (610M) | link |
| KE-T5 (Ko-En) | Large (770M) | link |
| KE-T5 (Ko) | Large (770M) | link |
| pko-t5 (Ko) | Large (800M) | link |
Subscribe to SOOFTWARE
Get the latest posts delivered right to your inbox