luke-japanese
luke-japanese is the Japanese version of LUKE (Language Understanding with Knowledge-based Embeddings), a pre-trained knowledge-enhanced contextualized representation of words and entities. LUKE treats words and entities in a given text as independent tokens, and outputs contextualized representations of them. Please refer to our GitHub repository for more details and updates.
This model contains Wikipedia entity embeddings which are not used in general NLP tasks. Please use the lite version for tasks that do not use Wikipedia entities as inputs.
luke-japaneseは、単語とエンティティの知識拡張型訓練済みTransformerモデルLUKEの日本語版です。LUKEは単語とエンティティを独立したトークンとして扱い、これらの文脈を考慮した表現を出力します。詳細については、GitHub リポジトリを参照してください。
このモデルは、通常のNLPタスクでは使われないWikipediaエンティティのエンベディングを含んでいます。単語の入力のみを使うタスクには、lite versionを使用してください。
Experimental results on JGLUE
The experimental results evaluated on the dev set of JGLUE are shown as follows:
| Model | MARC-ja | JSTS | JNLI | JCommonsenseQA |
|---|---|---|---|---|
| acc | Pearson/Spearman | acc | acc | |
| LUKE Japanese base | 0.965 | 0.916/0.877 | 0.912 | 0.842 |
| Baselines: | ||||
| Tohoku BERT base | 0.958 | 0.909/0.868 | 0.899 | 0.808 |
| NICT BERT base | 0.958 | 0.910/0.871 | 0.902 | 0.823 |
| Waseda RoBERTa base | 0.962 | 0.913/0.873 | 0.895 | 0.840 |
| XLM RoBERTa base | 0.961 | 0.877/0.831 | 0.893 | 0.687 |
The baseline scores are obtained from here.
Citation
@inproceedings{yamada2020luke,
title={LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention},
author={Ikuya Yamada and Akari Asai and Hiroyuki Shindo and Hideaki Takeda and Yuji Matsumoto},
booktitle={EMNLP},
year={2020}
}