L3.1-Pneuma-Allades-8B

L3.1-Pneuma-Allades-8B is a unique 8B parameter model, carefully merged using the della_linear method to integrate the best qualities of two exceptional models: L3.1-Pneuma-8B-v1 and Llama3.1-Allades-8B. This model is finely tuned to handle advanced tasks in creative writing, reasoning, roleplay, and instruction-following.

This model is designed for anyone seeking to enhance AI-assisted storytelling, detailed reasoning, or dynamic interactive dialogues. The combination of the Pneuma and Allades models ensures a well-balanced performance in both technical and creative realms. 🧠✨

🌐 Family Tree

This model is the result of a fusion between the following:

• mergekit-community/L3.1-Pneuma-8B-v1: A comprehensive 8B model designed to excel in logical reasoning and long-form content generation. It was merged with a focus on instruction-following and roleplaying.
• nbeerbower/Llama3.1-Allades-8B: A finetuned Llama 3.1 model, focused on creative writing, reasoning, and roleplay. Trained with a mix of public and private datasets to push the boundaries of creativity and interaction in AI language models.

🧬 Model Lineage

A: mergekit-community/L3.1-Pneuma-8B-v1

• Designed for advanced reasoning tasks, particularly in creative problem solving and interactive roleplay scenarios.
• This model leverages the strength of instruction-following tasks with a highly optimized configuration, making it a robust choice for interactive experiences.

B: nbeerbower/Llama3.1-Allades-8B

• Tuned with creative writing, storytelling, and dynamic roleplay in mind.
• It integrates five datasets that focus on reasoning, storytelling, and narrative structure, providing a well-rounded experience for creative and logical task performance.

⚙️ Merge Details

The L3.1-Pneuma-Allades-8B model was merged using the della_linear method, ensuring a balanced integration of the two base models' distinct strengths. Here’s the exact configuration used for the merge:

out_dtype: bfloat16
dtype: float32
tokenizer_source: base
merge_method: della_linear
parameters:
  int8_mask: true
  density: 0.5
  epsilon: 0.04
  lambda: 1.05
base_model: nbeerbower/Llama3.1-Allades-8B
models:
  - model: nbeerbower/Llama3.1-Allades-8B
    parameters:
      weight:
        - filter: v_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: o_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: up_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: gate_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: down_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - value: 1
  - model: mergekit-community/L3.1-Pneuma-8B-v1
    parameters:
      weight:
        - filter: v_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: o_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: up_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: gate_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: down_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - value: 0

🔮 Key Features & Capabilities

1. Instruction Following:

• Pneuma's structure allows this model to excel in precise instruction-following tasks, ensuring clear and coherent responses.

2. Creative Writing & Storytelling:

• With Allades' fine-tuned datasets, this model is capable of generating detailed and creative narratives, perfect for roleplaying, interactive storytelling, and fantasy creation.

3. Dynamic Roleplay:

• Combining the reasoning power of Pneuma with Allades' focus on roleplay and creativity, this model can generate interactive, logical, and emotionally-driven dialogues in any roleplay scenario.

🚀 Use Cases

• Immersive Storytelling: Whether you’re writing a novel, developing interactive character dialogue, or creating a magical world, this model offers both creativity and structure.
• Instructional Content: From educational guides to complex instructional scenarios, this model adapts well to delivering step-by-step processes in a clear and structured format.
• AI-Assisted Roleplay: Perfect for game development, NPC dialogues, and other roleplaying applications where both logic and creativity are required.

📜 License

This model is open-sourced under the Apache-2.0 License, allowing free use, modification, and distribution with proper attribution.