CosmicFish-300M

A 300M parameter language model with modern architecture improvements developed by Mistyoz AI.

Quick Start

The easiest way to chat with CosmicFish is using our chat.py script:

# Download the chat script from this repository
wget https://huggingface.co/MistyozAI/CosmicFish-300M/resolve/main/chat.py

# Install dependencies
pip install transformers huggingface-hub termcolor safetensors

# Run the chat interface (automatically downloads model)
python chat.py

The chat.py script handles all model loading, generation, and provides the best chat experience with live streaming, repetition penalty, and conversation commands.

Model Details

• Parameters: 369M
• Architecture: CosmicFish (RoPE, GQA, SwiGLU, RMSNorm)
• Context Length: 2048 tokens
• Vocabulary: 50,257 tokens
• Training Data: CosmicSet 2.0
• Developer: Mistyoz AI
• Repository: MistyozAI/CosmicFish-300M
• Format: Safetensors

Usage

Installation

pip install transformers huggingface-hub termcolor safetensors torch

Quick Chat Interface

from transformers import GPT2Tokenizer
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
import torch
import json
import os

# Download model from Hugging Face Hub
cache_dir = snapshot_download(repo_id="MistyozAI/CosmicFish-300M")

# Load tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

# Load config
with open(os.path.join(cache_dir, "config.json")) as f:
    config_dict = json.load(f)

# Load model weights from safetensors
state_dict = load_file(os.path.join(cache_dir, "model.safetensors"))

# Note: Full model class available in the repository
print("Model downloaded and ready for use!")

Advanced Generation with Repetition Penalty

def generate_with_repetition_penalty(model, tokenizer, prompt, max_tokens=100, temperature=0.5, penalty=1.2):
    input_ids = torch.tensor(tokenizer.encode(prompt)).unsqueeze(0)
    generated = input_ids.clone()
    
    for _ in range(max_tokens):
        with torch.no_grad():
            logits, _ = model(generated)
        
        next_token_logits = logits[:, -1, :] / temperature
        
        # Apply repetition penalty
        if penalty > 1.0:
            for token_id in set(generated[0].tolist()):
                if next_token_logits[0, token_id] > 0:
                    next_token_logits[0, token_id] /= penalty
                else:
                    next_token_logits[0, token_id] *= penalty
        
        probs = torch.nn.functional.softmax(next_token_logits, dim=-1)
        next_token = torch.multinomial(probs, num_samples=1)
        
        if next_token.item() == tokenizer.eos_token_id:
            break
            
        generated = torch.cat([generated, next_token], dim=1)
    
    return tokenizer.decode(generated[0], skip_special_tokens=True)

Loading Model with Safetensors

from safetensors.torch import load_file
from modeling_cosmicfish import CosmicFish, CosmicConfig
import json

def load_cosmicfish_model(model_path):
    # Load config
    with open(os.path.join(model_path, "config.json")) as f:
        config_dict = json.load(f)
    
    # Create model config
    config = CosmicConfig(
        vocab_size=config_dict["vocab_size"],
        block_size=config_dict["block_size"], 
        n_layer=config_dict["n_layer"],
        n_head=config_dict["n_head"],
        n_embd=config_dict["n_embd"],
        bias=config_dict["bias"],
        dropout=0.0,
        use_rotary=config_dict["use_rotary"],
        use_swiglu=config_dict["use_swiglu"],
        use_gqa=config_dict["use_gqa"],
        n_query_groups=config_dict["n_query_groups"]
    )
    
    # Create model
    model = CosmicFish(config)
    
    # Load weights from safetensors (secure format)
    state_dict = load_file(os.path.join(model_path, "model.safetensors"))
    
    # Handle weight sharing (lm_head.weight shares with transformer.wte.weight)
    if 'lm_head.weight' not in state_dict and 'transformer.wte.weight' in state_dict:
        state_dict['lm_head.weight'] = state_dict['transformer.wte.weight']
    
    model.load_state_dict(state_dict)
    model.eval()
    
    return model

Chat Interface

def chat_with_model():
    conversation = []
    
    while True:
        user_input = input("You: ")
        if user_input.lower() in ['quit', 'exit']:
            break
        
        context = "Below is a conversation between a human and an AI assistant.\n\n"
        for human, ai in conversation:
            context += f"Human: {human}\nAssistant: {ai}\n\n"
        context += f"Human: {user_input}\nAssistant:"
        
        # Generate response with repetition penalty
        response = generate_with_repetition_penalty(
            model, tokenizer, context, 
            max_tokens=150, temperature=0.7, penalty=1.2
        )
        
        # Extract just the assistant's response
        response = response.split("Assistant:")[-1].split('\n')[0].strip()
        print(f"CosmicFish: {response}")
        
        conversation.append((user_input, response))

chat_with_model()

Architecture

CosmicFish uses several modern improvements over standard transformers:

• RoPE (Rotary Position Embeddings): Better position encoding than absolute positions
• GQA (Grouped-Query Attention): Reduces memory usage with 4 query groups
• SwiGLU: More effective activation function than ReLU/GELU
• RMSNorm: Simpler, more stable normalization than LayerNorm

Training

• Dataset: CosmicSet 2.0
• Sequence Length: 2048 tokens
• Training Steps: ~130K iterations
• Hardware: Nvidia RTX Pro 6000 x1

Performance

• Speed: Varies by hardware (not benchmarked)
• Memory: ~1GB RAM
• File Size: 738.6MB
• Loading: Fast and secure with safetensors

Model Format

This model uses safetensors format for:

• Security: Safe loading without arbitrary code execution
• Performance: Faster loading compared to pickle-based formats
• Memory efficiency: Zero-copy loading when possible
• Cross-platform compatibility: Works consistently across different environments

Limitations

• May produce less accurate responses
• 2048 token context limit
• English only
• Training data cutoff applies
• May generate incorrect information
• Cannot browse internet or access real-time data

License

Apache 2.0 - see LICENSE file.

Credit

If you use CosmicFish-300M, please credit Mistyoz AI.