LunarAI - Fine-tuned DeepSeek Coder V2 Lite for Spigot Plugin Development

Model Description

LunarAI is a custom language model fine-tuned from the deepseek-ai/DeepSeek-Coder-V2-Lite-Base model. It has been specialized to act as an AI programming assistant, with a particular focus on Spigot/Minecraft plugin development.

This model is designed to provide accurate code examples, explanations, and guidance related to the Spigot API and general Java programming concepts relevant to creating Minecraft server plugins.

Training Details

• Base Model: deepseek-ai/DeepSeek-Coder-V2-Lite-Base
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Dataset: A custom dataset (spigot_dataset.jsonl) focused on Spigot/Minecraft plugin development, including common tasks, event handling, and API usage.
• Adapter Size: Approximately 1.1 GB (LoRA adapter before merge)
• Training Framework: Axolotl

Model Files

This repository contains two main versions of the fine-tuned model:

1. Full Merged Model (Safetensors): The complete model with the LoRA adapter merged into the base model's weights. This is the standard Hugging Face format, ideal for further development or use with transformers.

   • Files: model-00001-of-00007.safetensors through model-00007-of-00007.safetensors (totaling ~31.4 GB)
   • Configuration files: config.json, tokenizer.json, special_tokens_map.json, etc.

2. Quantized GGUF Model (for Ollama): A highly optimized, quantized version of the merged model in GGUF format, specifically designed for efficient local inference with tools like Ollama.

   • File: model.gguf (~16.7 GB, q8_0 quantization)

How to Use LunarAI with Ollama (Recommended for Local Inference)

To run LunarAI locally using Ollama, follow these steps:

1. Ensure Ollama is Installed: If you don't have Ollama, install it from ollama.com.

2. Download model.gguf: You can download the model.gguf file directly from this repository's "Files" tab, or use ollama pull ThePegasusGroup/LunarAI if Ollama supports direct pulling of GGUF files from the Hub (this might require a Modelfile first).

3. Create a Modelfile: In the same directory as your downloaded model.gguf, create a file named Modelfile with the following content:

   # Tell Ollama which GGUF file to use
   FROM ./model.gguf
   
   # Set the chat template for DeepSeek Coder
   TEMPLATE """{% for message in messages %}{% if message['role'] == 'user' %}{{ 'You are an AI programming assistant, utilizing the Deepseek Coder model, developed by Deepseek Company. Follow the user\'s instructions carefully. Respond using markdown.' }}\n### Instruction:\n{{ message['content'] }}\n### Response:\n{% elif message['role'] == 'assistant' %}{{ message['content'] }}{% if not loop.last %}\n{% endif %}{% endif %}{% endfor %}"""
   
   # Set a default parameter
   PARAMETER temperature 0.7
   

4. Create the Model in Ollama:

   ollama create LunarAI -f ./Modelfile
   

5. Run LunarAI:

   ollama run LunarAI
   

   You can then start asking it questions related to Spigot plugin development!

How to Load the Merged Model with Hugging Face Transformers

If you wish to load the full, unquantized merged model for further development or advanced usage with the transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Replace 'ThePegasusGroup/LunarAI' with the actual repo ID if you renamed it
model_id = "ThePegasusGroup/LunarAI"

# Load the model
# Ensure you have sufficient VRAM (GPU memory) or RAM for this large model
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16, # Or torch.float16, or torch.float32 depending on your hardware
    device_map="auto",
    trust_remote_code=True # Required for DeepSeek-Coder-V2-Lite-Base architecture
)

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

print("LunarAI model loaded successfully!")