Demo
Please try this ➤➤➤ Colab Notebook Demo (click me!)
| Context | Response | human_vs_machine score |
|---|---|---|
| I love NLP! | I'm not sure if it's a good idea. | 0.000 |
| I love NLP! | Me too! | 0.605 |
The human_vs_machine score predicts how likely the response is from a human rather than a machine.
DialogRPT-human-vs-machine
Dialog Ranking Pretrained Transformers
How likely a dialog response is upvoted 👍 and/or gets replied 💬?
This is what DialogRPT is learned to predict. It is a set of dialog response ranking models proposed by Microsoft Research NLP Group trained on 100 + millions of human feedback data. It can be used to improve existing dialog generation model (e.g., DialoGPT) by re-ranking the generated response candidates.
Quick Links:
We considered the following tasks and provided corresponding pretrained models.
| Task | Description | Pretrained model |
|---|---|---|
| Human feedback | given a context and its two human responses, predict... | |
updown | ... which gets more upvotes? | model card |
width | ... which gets more direct replies? | model card |
depth | ... which gets longer follow-up thread? | model card |
| Human-like (human vs fake) | given a context and one human response, distinguish it with... | |
human_vs_rand | ... a random human response | model card |
human_vs_machine | ... a machine generated response | this model |
Contact:
Please create an issue on our repo
Citation:
@inproceedings{gao2020dialogrpt,
title={Dialogue Response RankingTraining with Large-Scale Human Feedback Data},
author={Xiang Gao and Yizhe Zhang and Michel Galley and Chris Brockett and Bill Dolan},
year={2020},
booktitle={EMNLP}
}