nlp, paper, LLaMA2LLaMA2 Meta(전 Facebook)가 이번 7월 18일에 LLaMA2를 공개했습니다. 🎉 LLaMA2 관련 내용이 담긴 논문 과 함께 7B, 13B, 70B 모델을 공개했습니다. 이전 LLaMA와 다르게 LLaMA…
nlp, paper, Sooftware NLP - Generation with RetrievalGeneration with Retrieval 이번에 딥마인드에서 RETRO(Retrieval-Enhanced Transformer) 라는 모델을 내놓았습니다. 문서 retrieval + GPT 기반 모델인데, 7B 모델임에도 불구하고 2…
nlp, paper, Sooftware NLP - Fine-grained Post-training for Improving Retrieval-based Dialogue Systems Paper ReviewFine-grained Post-training for Improving Retrieval-based Dialogue Systems Paper Review Paper: https://aclanthology.org/2021.naacl-main.12…
speech, nlp, paper, Sooftware NLP - Textless NLPTextless NLP: Generating expressive speech from raw audio paper / code / pre-train model / blog Name: Generative Spoken Language Model (GSLM…
nlp, paper, Sooftware NLP - Efficient Attention Paper ReviewEfficient Attention: Attention with Linear Complexities Shen Zhuoran et al. Abstract Dot-product attention은 들어오는 인풋 길이에 따라 memory…
nlp, paper, Sooftware NLP - Luna: Linear Unified Nested AttentionLuna: Linear Unified Nested Attention USC + CMU + Facebook AI 2021.06 code Abstract 트랜스포머의 Multi Headed Self Attention…
nlp, paper, Sooftware NLP - P-Tuning Paper ReviewGPT Understands, Too Xiao Liu et al. Tsinghua University etc. arXiv pre-print Abstract GPT를 파인튜닝하는 방법은 Narural Language Understanding (NLU…
speech, paper, Sooftware Speech - Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Paper ReviewPushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang et al., 2020 Google Research, Brain Team Reference…
nlp, paper, Sooftware NLP - Longformer Paper ReviewLongformer: The Long-Document Transformer Paper Code Iz Beltagy et al. Introduction 트랜스포머는 긴 시퀀스는 처리하지 못한다는 한계를 가지고 있음 이유는 시퀀스 길이에 O(n^…
speech, paper, Sooftware Speech - EMNLP Paper Review: SpeechEMNLP Paper Review: Speech Adaptive Feature Selection for End-to-End Speech Translation (Biao Zhang et al) Incremental Text-to-Speech…
nlp, parallelism, paper, Sooftware NLP - Megatron LM Paper ReviewMegatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi et al. 2019. NVIDIA Corp. Summary…
speech, tts, paper, Sooftware Speech - One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Paper ReviewOne Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Tomáš Nekvinda, Ondřej Dušek Charles University INTERSPEECH, 202…
nlp, paper, Sooftware NLP - RoBERTa Paper ReviewRoBERTa paper / code Abstract BERT를 제대로 학습시키는 법을 제안 BERT는 엄청난 모델이지만, Original BERT 논문에서 하이퍼파라미터에 대한 실험이 제대로 진행되지 않음 BERT…
nlp, paper, Sooftware NLP - Electra Paper ReviewBelow is just about everything you’ll need to style in the theme. Check the source code to see the many embedded elements within paragraphs…
speech, paper, Sooftware Speech - Wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representationswav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael…
speech, paper, Sooftware Speech - Conformer Paper ReviewConformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati et al. Google Inc. INTERSPEECH, 2020 Reference Conformer…
speech, paper, Sooftware Speech - ClovaCall Paper ReviewClovaCall: Korean Goal-Oriented Dialog Speech Corpus for Automatic Speech Recognition of Contact Centers image 논문링크 2020-04-2…
speech, paper, Sooftware Speech - STATE-OF-THE-ART SPEECH RECOGNITION WITH SEQUENCE-TO-SEQUENCE MODEL Paper Review「STATE-OF-THE-ART SPEECH RECOGNITION WITH SEQUENCE-TO-SEQUENCE MODEL」 Review title https://arxiv.org/abs/1712.0176…
speech, paper, Sooftware Speech - Attention-Based Models for Speech Recognition Paper ReviewAttention-Based Models for Speech Recognition Paper Review title http://papers.nips.cc/paper/5847-attention-based-models-for-speech…
speech, paper, Sooftware Speech - SpecAugment Paper ReviewSpecAugment: 「A Simple Data Augmentation Method for Automatic Speech Recognition」 Review title https://arxiv.org/abs/1904.08779 Abstract…
speech, paper, Sooftware Speech - DeepSpeech Paper ReviewDeep Speech: Scaling up end-to-end speech recognition title https://arxiv.org/pdf/1412.5567.pdf (Awni Hannun et al. 2014) Abstract…
speech, paper, Sooftware Speech - Listen, Attend and Spell Paper Review「Listen, Attend and Spell」 Review title https://arxiv.org/abs/1508.01211 (William Chan et al. 2015) Introduction 어텐션 기반 Seq2seq…
