DNA-R1
We introduce DNA-R1, a specialized reasoning model optimized for Korean language based on Microsoft's Phi-4. By applying large-scale reinforcement learning (RL) using the same methodology as DeepSeek-R1, we have significantly enhanced the model's Korean reasoning capabilities. This model demonstrates deep understanding of Korean text and exhibits exceptional reasoning abilities across mathematics, coding, and general reasoning tasks.
Training Methodology
Our comprehensive training pipeline consists of three strategic stages:
- Stage 1: Initial SFT with a large Korean non-reasoning dataset (760k examples) reused from our DNA 1.0 8B Instruct training pipeline
- Stage 2: Strategic integration of Korean reasoning patterns from DeepSeek R1 using a specialized Korean reasoning dataset (300k examples)
- Stage 3: Advanced reinforcement learning with GRPO using a combined Korean/English reasoning dataset, with format, accuracy, and language consistency as rewards
DNA-R1 has learned reasoning patterns specifically tailored for Korean language, and demonstrates capabilities such as self-verification, reflection, and generation of long chains-of-thought (CoT). This represents a significant milestone for the AI research community in the Korean language environment.
Model Specifications
- Developed by: Dnotitia Inc.
- Supported Languages: Korean, English
- Model Release Date: Mar 6, 2025
- Number of Parameters: 14B
- License: CC BY-NC 4.0
Technical Details
Multi-Stage Training Pipeline
We implemented a sophisticated training approach to enhance Phi-4's Korean reasoning capabilities:
- Initial Foundation (Stage 1): Supervised Fine-Tuning using our extensive Korean non-reasoning dataset from the established DNA 1.0 8B Instruct training pipeline
- Reasoning Integration (Stage 2): Specialized adaptation of DeepSeek R1's reasoning patterns with Korean-specific optimization through a meticulously curated dataset
- Advanced Refinement (Stage 3): Reinforcement learning optimization using GRPO to perfect reasoning in both Korean and English, with comprehensive reward signals for format structure, factual accuracy, and language consistency
This methodical approach enables DNA-R1 to develop sophisticated chain-of-thought (CoT) reasoning for complex problem solving, resulting in a model finely calibrated for Korean language reasoning while maintaining robust general capabilities.
Performance Highlights
Our Korean-specific multi-stage training pipeline significantly enhances the Phi-4 base model's understanding of Korean context, reasoning depth, and response capabilities. The model excels at:
- Generating nuanced Korean chains-of-thought (CoT)
- Performing rigorous self-verification
- Solving multi-step complex problems
- Maintaining cultural and linguistic context in reasoning
- Distinguishing between deep thinking and concise answers using the
<think>and<answer>tags
Evaluation Results
Below, we present our evaluation results for the DNA-R1 model across math, coding, science, Korean, and general-performance benchmarks. Despite being only 14B in size, the DNA-R1 model demonstrates superior performance compared to many larger models across various benchmarks.
- The highest scores are in bold form, and the second-highest scores are underlined.
- All benchmarks are evaluated with lm-eval and skythought-eval.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
tokenizer = AutoTokenizer.from_pretrained('dnotitia/DNA-R1')
model = AutoModelForCausalLM.from_pretrained('dnotitia/DNA-R1', device_map='auto')
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
conversation = [
{"role": "user", "content": """
어려서부터 우리 집은 가난했었고
남들 다하는 외식 몇 번 한 적이 없었고
일터에 나가신 어머니 집에 없으면
언제나 혼자서 끓여 먹었던 라면
그러다 라면이 너무 지겨워서
맛있는 것 좀 먹자고 대들었었어
그러자 어머님이 마지못해 꺼내신
숨겨두신 비상금으로 시켜주신
짜장면 하나에 너무나 행복했었어
하지만 어머님은 왠지 드시질 않았어
어머님은 짜장면이 싫다고 하셨어
어머님은 짜장면이 싫다고 하셨어
야이야~야 그렇게 살아가고
그렇게 후회하고 눈물도 흘리고
야이야~야 그렇게 살아가고
너무나 아프고 하지만 다시 웃고
---
친구가 쓴 시인데, 여기서 친구의 어머니가 짜장면이 싫다고 하신 이유는?사랑or희생?"""},
]
inputs = tokenizer.apply_chat_template(conversation,
add_generation_prompt=True,
return_dict=True,
return_tensors="pt").to(model.device)
_ = model.generate(**inputs, streamer=streamer)
License
This model is released under CC BY-NC 4.0 license. If you have any questions or commercial usage inquiries, please Contact us.
Citation
If you use or discuss this model in your academic research, please cite the project to help spread awareness:
@misc{dnar12025,
title={DNA R1},
author={Jungyup Lee and Jemin Kim and Sang Park and SeungJae Lee},
year={2025},
publisher={HuggingFace},
url={https://huggingface.co/dnotitia/DNA-R1}
}