Subscribe to SOOFTWARE

Stay up to date! Get all the latest & greatest posts delivered straight to your inbox

paper

A collection of 22 posts

LLaMA2 cover image
nlp, paper, 

LLaMA2

LLaMA2 Meta(전 Facebook)가 이번 7월 18일에 LLaMA2를 공개했습니다. 🎉 LLaMA2 관련 내용이 담긴 논문 과 함께 7B, 13B, 70B 모델을 공개했습니다. 이전 LLaMA와 다르게 LLaMA…

Sooftware NLP - Generation with Retrieval cover image
nlp, paper, 

Sooftware NLP - Generation with Retrieval

Generation with Retrieval 이번에 딥마인드에서 RETRO(Retrieval-Enhanced Transformer) 라는 모델을 내놓았습니다. 문서 retrieval + GPT 기반 모델인데, 7B 모델임에도 불구하고 2…

Sooftware NLP - Fine-grained Post-training for Improving Retrieval-based Dialogue Systems Paper Review cover image
nlp, paper, 

Sooftware NLP - Fine-grained Post-training for Improving Retrieval-based Dialogue Systems Paper Review

Fine-grained Post-training for Improving Retrieval-based Dialogue Systems Paper Review Paper: https://aclanthology.org/2021.naacl-main.12…

Sooftware NLP - Textless NLP cover image
speech, nlp, paper, 

Sooftware NLP - Textless NLP

Textless NLP: Generating expressive speech from raw audio paper / code / pre-train model / blog Name: Generative Spoken Language Model (GSLM…

Sooftware NLP - Efficient Attention Paper Review cover image
nlp, paper, 

Sooftware NLP - Efficient Attention Paper Review

Efficient Attention: Attention with Linear Complexities Shen Zhuoran et al. Abstract Dot-product attention은 들어오는 인풋 길이에 따라 memory…

Sooftware NLP - Luna: Linear Unified Nested Attention cover image
nlp, paper, 

Sooftware NLP - Luna: Linear Unified Nested Attention

Luna: Linear Unified Nested Attention USC + CMU + Facebook AI 2021.06 code Abstract 트랜스포머의 Multi Headed Self Attention…

Sooftware NLP - P-Tuning Paper Review cover image
nlp, paper, 

Sooftware NLP - P-Tuning Paper Review

GPT Understands, Too Xiao Liu et al. Tsinghua University etc. arXiv pre-print Abstract GPT를 파인튜닝하는 방법은 Narural Language Understanding (NLU…

Sooftware Speech - Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Paper Review cover image
speech, paper, 

Sooftware Speech - Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Paper Review

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang et al., 2020 Google Research, Brain Team Reference…

Sooftware NLP - Longformer Paper Review cover image
nlp, paper, 

Sooftware NLP - Longformer Paper Review

Longformer: The Long-Document Transformer Paper Code Iz Beltagy et al. Introduction 트랜스포머는 긴 시퀀스는 처리하지 못한다는 한계를 가지고 있음 이유는 시퀀스 길이에 O(n^…

Sooftware Speech - EMNLP Paper Review: Speech cover image
speech, paper, 

Sooftware Speech - EMNLP Paper Review: Speech

EMNLP Paper Review: Speech Adaptive Feature Selection for End-to-End Speech Translation (Biao Zhang et al) Incremental Text-to-Speech…

Sooftware NLP -  Megatron LM Paper Review cover image
nlp, parallelism, paper, 

Sooftware NLP - Megatron LM Paper Review

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism ​ Mohammad Shoeybi et al. 2019. NVIDIA Corp. ​ Summary…

Sooftware Speech - One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Paper Review cover image
speech, tts, paper, 

Sooftware Speech - One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Paper Review

One Model, Many Languages: Meta-learning for Multilingual Text-to-Speech Tomáš Nekvinda, Ondřej Dušek Charles University INTERSPEECH, 202…

Sooftware NLP - RoBERTa Paper Review cover image
nlp, paper, 

Sooftware NLP - RoBERTa Paper Review

RoBERTa paper / code Abstract BERT를 제대로 학습시키는 법을 제안 BERT는 엄청난 모델이지만, Original BERT 논문에서 하이퍼파라미터에 대한 실험이 제대로 진행되지 않음 BERT…

Sooftware NLP - Electra Paper Review cover image
nlp, paper, 

Sooftware NLP - Electra Paper Review

Below is just about everything you’ll need to style in the theme. Check the source code to see the many embedded elements within paragraphs…

Sooftware Speech - Wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations cover image
speech, paper, 

Sooftware Speech - Wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations

wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, Michael…

Sooftware Speech - Conformer Paper Review cover image
speech, paper, 

Sooftware Speech - Conformer Paper Review

Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati et al. Google Inc. INTERSPEECH, 2020 Reference Conformer…

Sooftware Speech - ClovaCall Paper Review cover image
speech, paper, 

Sooftware Speech - ClovaCall Paper Review

ClovaCall: Korean Goal-Oriented Dialog Speech Corpus for Automatic Speech Recognition of Contact Centers image 논문링크 2020-04-2…

Sooftware Speech - STATE-OF-THE-ART SPEECH RECOGNITION WITH SEQUENCE-TO-SEQUENCE MODEL Paper Review cover image
speech, paper, 

Sooftware Speech - STATE-OF-THE-ART SPEECH RECOGNITION WITH SEQUENCE-TO-SEQUENCE MODEL Paper Review

「STATE-OF-THE-ART SPEECH RECOGNITION WITH SEQUENCE-TO-SEQUENCE MODEL」 Review title https://arxiv.org/abs/1712.0176…

Sooftware Speech - Attention-Based Models for Speech Recognition Paper Review cover image
speech, paper, 

Sooftware Speech - Attention-Based Models for Speech Recognition Paper Review

Attention-Based Models for Speech Recognition Paper Review title http://papers.nips.cc/paper/5847-attention-based-models-for-speech…

Sooftware Speech - SpecAugment Paper Review cover image
speech, paper, 

Sooftware Speech - SpecAugment Paper Review

SpecAugment: 「A Simple Data Augmentation Method for Automatic Speech Recognition」 Review title https://arxiv.org/abs/1904.08779 Abstract…

Sooftware Speech - DeepSpeech Paper Review cover image
speech, paper, 

Sooftware Speech - DeepSpeech Paper Review

Deep Speech: Scaling up end-to-end speech recognition title https://arxiv.org/pdf/1412.5567.pdf (Awni Hannun et al. 2014) Abstract…

Sooftware Speech - Listen, Attend and Spell Paper Review cover image
speech, paper, 

Sooftware Speech - Listen, Attend and Spell Paper Review

「Listen, Attend and Spell」 Review title https://arxiv.org/abs/1508.01211 (William Chan et al. 2015) Introduction 어텐션 기반 Seq2seq…