Llama 3.2 Physics Fine-tuned Model

This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on kejian/arxiv-physics-debug-v0. Mostly for concept proofing, don't trust it for real physics (I mean, even Claude 3.5 can be wrong on graduate physics plenty of times, let alone a 1B model)!

new: Training script: benhaotang/LoRa-Finetune-Navi

Model description

• Base model: meta-llama/Llama-3.2-1B-Instruct
• Training data: kejian/arxiv-physics-debug-v0
• Fine-tuning type: LoRA
• Use case: Physics domain questions
• Training Arguments:
  • Learning Rate: 2e-5
  • Epochs: 3
  • Gradient Accumulation Steps: 8
• Training setup: Ubuntu 24.04 with RX7800XT

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("benhaotang/llama3.2-1B-physics-finetuned")
tokenizer = AutoTokenizer.from_pretrained("benhaotang/llama3.2-1B-physics-finetuned")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
# Example usage
text = "Give me a short intodcution to renormalization group(RG) flow in physcis?\n"
inputs = tokenizer(text, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_length=2048)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example output:

I'll start by explaining the concept of renormalization group flow, then we can discuss the different types of renormalization group flow and their applications.

Step 1: Introduction to Renormalization Group (RG) Flow

The Renormalization Group (RG) is a theoretical framework used in physics to study the behavior of physical systems at very small distances or high energies. It is a powerful tool for understanding the dynamics of systems that exhibit scale invariance, meaning that their properties remain the same under certain transformations of their parameters.

Step 2: RG Flow in Physics

In physics, the RG flow is a mathematical transformation that describes how physical parameters, such as energy or length, evolve over time or space in a system. It is typically represented as a map between a set of initial parameters and a set of final parameters, where the map is defined by a set of equations that describe how the parameters change as the system evolves. The RG flow is often used to study the behavior of systems at the critical point, where the system's properties change dramatically as the parameters change.

Step 3: Types of RG Flow

There are several types of RG flow, including:

• Continuum RG Flow: This is the most common type of RG flow, where the system is treated as a continuum, and the equations of motion are derived from a set of partial differential equations.
• Discrete RG Flow: This type of RG flow is used in systems with discrete variables, such as lattice gauge theory.
• Asymptotic RG Flow: This type of RG flow is used in systems with a power-law dependence of the parameters on the energy scale.
• Non-Asymptotic RG Flow: This type of RG flow is used in systems with a logarithmic dependence of the parameters on the energy scale.

Step 4: Applications of RG Flow

The RG flow is a powerful tool for understanding the behavior of physical systems at very small distances or high energies. Some of the applications of RG flow include:

• Quantum Field Theory: RG flow is used to study the behavior of quantum field theories, such as the Standard Model of particle physics.
• Condensed Matter Physics: RG flow is used to study the behavior of materials at the nanoscale, such as superconductors and superfluids.
• High-Energy Physics: RG flow is used to study the behavior of high-energy particles, such as quarks and leptons.