QwQ-LCoT-3B-Instruct

QwQ-LCoT-3B-Instruct Model Card

The QwQ-LCoT-3B-Instruct model is a lightweight, instruction-tuned language model designed for complex reasoning and explanation tasks. It is fine-tuned on the Qwen2.5-3B-Instruct base model using the QwQ-LongCoT-130K dataset, focusing on long-chain-of-thought (LCoT) reasoning for enhanced logical comprehension and detailed output generation.

Sample Long CoT:

Key Features:

1. Long Chain-of-Thought Reasoning:

   • Specifically designed to generate comprehensive, step-by-step explanations for complex queries.

2. Lightweight and Efficient:

   • With only 3 billion parameters, it is optimized for systems with limited computational resources without compromising reasoning capabilities.

3. Instruction Optimization:

   • Fine-tuned to follow prompts and provide concise, actionable, and structured responses.

Training Details:

• Base Model: Qwen2.5-3B-Instruct
• Dataset: amphora/QwQ-LongCoT-130K
  • Comprising 133,000 annotated samples focusing on logical tasks and structured thinking.

Capabilities:

1. Text Generation:

   • Provides detailed, structured, and logical text outputs tailored to user prompts.

2. Reasoning Tasks:

   • Solves step-by-step problems in math, logic, and science.

3. Educational Assistance:

   • Generates coherent explanations for academic and research purposes.

4. Dialogue and Summarization:

   • Handles conversational queries and summarizes long documents effectively.

Usage Instructions:

1. Setup: Download all model files and ensure compatibility with the Hugging Face Transformers library.

2. Loading the Model:

   from transformers import AutoModelForCausalLM, AutoTokenizer
   
   model_name = "prithivMLmods/QwQ-LCoT-3B-Instruct"
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   model = AutoModelForCausalLM.from_pretrained(model_name)
   

3. Generate Long-Chain Reasoning Outputs:

   input_text = "Explain the process of photosynthesis step-by-step."
   inputs = tokenizer(input_text, return_tensors="pt")
   outputs = model.generate(**inputs, max_length=300, temperature=0.5)
   print(tokenizer.decode(outputs[0], skip_special_tokens=True))
   

4. Customize Output Generation:
   Modify the generation_config.json file for different scenarios:

   • temperature: Controls randomness (lower = deterministic, higher = creative).
   • max_length: Sets response length.
   • top_p: Adjusts sampling for diversity in outputs.