Kirk Tung
Kirk Tung is a tiny language model trained on the Corpus Kirkensis, meant to produce output which is entertainingly silly as an experiment in using the Llama architecture.
Tung has a Libre and Open dataset, meaning that the content it was trained on is nearly 100% under a free license, with few exceptions.
Technical details
Kirk Tung was trained on 1.1B tokens over the course of 4 hours on a single A100 GPU.
| Name | Value |
|---|---|
| Architecture | Llama |
| Context Length | 1024 |
| Vocab size | 32606 |
| RoPE theta | 10000 |
num_attention_heads = 8
num_key_value_heads = 2
num_hidden_layers = 10
hidden_size = 512
intermediate_size = 1728
tie_word_embeddings = True
Training hyperparameters
| Name | Value |
|---|---|
| Per-device batch size | 12 |
| Learning rate | 5e-4 |
| Learning scheduler | Cosine |
| Weight decay | 0.1 |
| Epochs | 1 |
| Gradient accumulation steps | 8 |
Example completions
Prompt: "Once upon a time,"
Once upon a time, the two brothers would be able to use their powers to defeat the first."
Rosa was a very wise man. He had a love for a woman and a friend. He was so strong, he felt very happy that he had to be.
"You never thought you could be like you. I am a woman, a man who never wanted to go to the house, and never wanted to go to the house that night."
The two brothers sat in silence. The three brothers had a good laugh. "Birds, I can see you. Look for me."
The brothers tried to raise their hands. "I won't," they say. "You don't have to be a mother. I will be a mother. You don't have to be a mother," she says.
The brothers moved into their respective houses. The brothers were very happy. They were always happy. They always thought of it.
The brothers were very happy. The brothers knew that they would never have to go back home. They did not want to be friends. They never wanted to be alone again. The brothers were always happy. But they were always happy. They had to be happy. They had no friendship. They would
Prompt: "Artificial intelligence is"
Artificial Intelligence is the source of these data.
Witness the following:
"1. The United Nations has been stricter in internal affairs, and the agenda has been growing; the people have become more and more bankrupt, the world a power under international law. The foreign powers have become many times more bankrupt, and the foreign powers have become much more constraining of all their financial resources. It has allowed for still more operations than any other nation in the world. The U.S. Government has found its way into international agreements, financial concessions and a continuous flow of railways locomotives into the borders. This solution has been seen as possible two long colonization attempts."https://www.theguardian.com/world/2014/sep/10/go-topics-for-the-existing-response-event-at-the-united-from-the-loyal-response-president-on/
- The First International Commission for the Investigation of National Socialist Crimes
In October 1945, before the Great Purge, Germany invaded Poland from the East (not to be confused with the Red Army), the Central Powers, and the Baltic states. (Silesia was not a neutral
How to use
The easiest way to use Kirk Tung programatically is with a pipeline in HF transformers.
$ pip install transformers torch
import torch
from transformers import pipeline
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
pipe = pipeline(
"text-generation", model="rtc2022/kirk-tung", device=device
)
Support the creator
Support the creation of more models like this one by directly supporting the main creator: