qwen38-27b-terraform

Writes Terraform (HCL) from a plain-language description of the infrastructure you want. Merged fine-tune for engineers drafting .tf files across AWS, GCP, Azure, Kubernetes and eleven other provider families.

This is a merged fine-tune of Qwen/Qwen3.8-27B: the LoRA adapter has been fused into the base weights with merge_and_unload(), so it loads as an ordinary transformers checkpoint with no PEFT dependency. It was trained with 8-bit-quantized-base LoRA via TRL SFT.

Model details

The config here carries no quantization_config — 8-bit was a training-time choice, not a property of these weights.

Intended use

Direct use. Given a one-paragraph description of infrastructure, emit the Terraform configuration that implements it: resource, data, variable, output, provider and terraform blocks as needed, raw HCL with no prose and no markdown fences.

The prompt format is part of the contract. The model was trained on one exact rendering; see How to get started. Prompting it as a general chat model will not reproduce its behaviour.

How to get started

Prompt format

The model uses the base model's ChatML template with thinking disabled. Training rendered each sample as (newlines shown literally):

<|im_start|>system
{SYSTEM}<|im_end|>
<|im_start|>user
{INSTRUCTION}

```
{PLAIN_LANGUAGE_DESCRIPTION}
```<|im_end|>
<|im_start|>assistant
<think>

</think>

{HCL}<|im_end|>

Three things follow from this, and all three bite in practice:

1. The user turn is instruction, then a blank line, then the description inside a fenced code block. The instruction names the provider family ("Write a Terraform configuration for AWS that satisfies the requirement below. Emit only HCL — no explanation, no markdown fences."); the fenced block holds the intent.
2. enable_thinking=False is required. Every training sample carried the empty <think>\n\n</think> block, so the trained generation prefix ends after </think>. The template's default (enable_thinking=True) stops at <think> instead and invites the model to reason — off-contract, and not what the loss was computed against.
3. Use apply_chat_template; do not hand-build the string.

System prompt

This is the exact system prompt the adapter was trained under, taken from the training configuration. Changing it moves the model off its trained contract.

You are a Terraform expert. Given a plain-language description of infrastructure, output the complete, valid Terraform HCL configuration (including any required terraform, provider, resource, data, variable, and output blocks) that implements it. Output raw HCL only—no markdown, no explanations, no commentary.

Loading it

from_pretrained requests files by name, so it fetches the config, tokenizer and safetensors shards only — the GGUF in this repo costs transformers users nothing. If you instead download the repo by hand, filter it, or you will pull all 72 GB:

hf download SASVAAI/qwen38-27b-terraform --exclude "*.gguf" --local-dir ./qwen38-27b-terraform

Requires a transformers build that knows model_type: qwen3_5. Note that AutoModelForCausalLM cannot load this architecture — use AutoModelForImageTextToText, which is what the multimodal config maps to.

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

MODEL = "SASVAAI/qwen38-27b-terraform"

tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL, dtype=torch.bfloat16, device_map="auto",
    attn_implementation="flash_attention_2",   # "sdpa" if flash-attn is unavailable
)
model.eval()

SYSTEM = (
    "You are a Terraform expert. Given a plain-language description of "
    "infrastructure, output the complete, valid Terraform HCL configuration "
    "(including any required terraform, provider, resource, data, variable, "
    "and output blocks) that implements it. Output raw HCL only—no markdown, "
    "no explanations, no commentary."
)

instruction = (
    "Write a Terraform configuration for AWS that satisfies the requirement "
    "below. Emit only HCL — no explanation, no markdown fences."
)
description = (
    "Create a VPC named \"Ntier\" in the dev environment with a CIDR block "
    "defined by the `vpc_range` variable."
)

messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": f"{instruction}\n\n```\n{description}\n```"},
]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
enc = tokenizer(text, return_tensors="pt").to(model.device)

out = model.generate(**enc, max_new_tokens=2048, do_sample=False,
                     temperature=None, top_p=None, top_k=None,
                     pad_token_id=tokenizer.pad_token_id)
print(tokenizer.decode(out[0][enc["input_ids"].shape[-1]:], skip_special_tokens=True))

Run locally with Ollama

The Q4_K_M GGUF ships in this repo, so there is nothing to build. Either of these works:

# A. One command — Ollama pulls the GGUF straight from this repo
ollama run hf.co/SASVAAI/qwen38-27b-terraform

# B. Or fetch the GGUF plus the Modelfile and build the tag locally
hf download SASVAAI/qwen38-27b-terraform \
    --include "*Q4_K_M.gguf" "Modelfile" --local-dir .
ollama create qwen38-terraform -f Modelfile
ollama run qwen38-terraform

Either way you download 16.8 GB, not the 55.6 GB of bf16 safetensors — Ollama only fetches the GGUF.

The two paths are not quite equivalent. ollama run hf.co/... never reads a Modelfile. The Hub serves a GGUF repo to Ollama through its own OCI-compatible registry: it picks the Q4_K_M GGUF as the model layer and attaches this repo's params and system files as the parameter and system-prompt layers. Those two files are kept byte-equivalent to the Modelfile's PARAMETER lines and SYSTEM block, so both routes land on the same greedy, system-prompted configuration. Path B is the one to use if you want to edit any of it.

Requirements

Ollama v0.34.0 or newer, and a llama.cpp build at commit 7cf1c54 or later. The GGUF declares architecture qwen35; older runtimes will refuse to load it.

Verified transcript

Built and run end-to-end on this host from the GGUF in this repo (Ollama v0.34.0, 1× H200, 48 CPU threads):

ollama create qwen38-terraform -f Modelfile     # real 1m1.888s
ollama show qwen38-terraform

  Model
    architecture        qwen35
    parameters          27.3B
    context length      262144
    embedding length    5120
    quantization        Q4_K_M

  Capabilities
    tools
    thinking
    completion

  Parameters
    num_ctx        8192
    num_predict    2048
    stop           "<|im_end|>"
    stop           "<|endoftext|>"
    temperature    0
    top_k          1

Then a held-out validation prompt (an AWS VPC with counted subnets), through /api/generate with "think": false and "options": {"num_thread": 48}:

Four things worth noting, all confirmed on this build:

• prompt_eval_count is 195 — the same count the transformers tokenizer produces for the same system+user messages. The Modelfile SYSTEM block plus the GGUF's embedded chat template reconstruct the training-time prompt exactly.
• eval_count is 137 — the same number of tokens bf16 emitted for this prompt under greedy decoding.
• num_ctx is load-bearing. Ollama sizes the default context from available VRAM and on this host chose 262,144. The Modelfile pins 8,192 to match MAX_SEQ_LEN at training time.
• Send "think": false. ollama show lists a thinking capability, which comes from the base model's template, not from this fine-tune. Training used an empty <think></think> block on every sample, so thinking-on is off contract. The tools capability is likewise inherited and untrained here.

Building your own GGUF (optional)

Needed only if you want a quantization other than the shipped Q4_K_M.

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
pip install -r requirements/requirements-convert_hf_to_gguf.txt

python convert_hf_to_gguf.py /path/to/qwen38-27b-terraform \
    --outfile terraform-qwen38-27b-f16.gguf --outtype f16

cmake -B build && cmake --build build --config Release -j
./build/bin/llama-quantize terraform-qwen38-27b-f16.gguf \
    qwen38-27b-terraform-Q4_K_M.gguf Q4_K_M

Measured on this repo's weights:

llama-quantize reported model size = 52115.19 MiB → quant size = 16021.46 MiB, in 137.4 s on 16 threads. Other K-quants land between these two; the two rows above are the only ones actually built and run here, so they are the only ones quoted.

The GGUF carries 866 tensors against 1,199 in the safetensors — the converter emits the text model only. The vision tower is a separate --mmproj run and is not part of this repo's GGUF, which is consistent with the vision tower never having been trained.

The MTP layer is not optional

This is the one step that fails silently if you build the merge yourself. config.json declares mtp_num_hidden_layers: 1, and llama.cpp's converter reads it: it sets qwen35.block_count = 65 (64 transformer layers + 1) and remaps the 15 mtp.* tensors onto blk.64.*. If your merged checkpoint is missing them, conversion still succeeds — and then Ollama refuses to load:

check_tensor_dims: tensor 'blk.64.attn_norm.weight' not found

PEFT's merge_and_unload() followed by save_pretrained() drops the mtp.* tensors, because they are not part of the wrapped LM graph. They are restored verbatim from the base checkpoint in this repo's third shard — an exact operation, since the adapter contains no MTP weights to merge. Reloading the spliced repo and re-running that prompt reproduced the pre-splice generation token-for-token, so the splice does not perturb the language-model path. ollama show reporting 27.3B rather than 26.9B is the quick check that the layer is present.

Stop tokens

The Modelfile's explicit stop "<|im_end|>" / stop "<|endoftext|>" lines are belt-and-braces. The completion above ended on done_reason: "stop" with the model emitting <|im_end|> itself.

Quantization fidelity

That same prompt was run twice under greedy decoding (temperature 0 / top_k 1): once against the bf16 safetensors in this repo via transformers, once against the Q4_K_M GGUF via Ollama.

• Both emitted 137 tokens and both produced the same two resources with the same arguments.
• They are not byte-identical: 396 characters from bf16 against 398 from Q4_K_M. The entire difference is two inserted blank lines, one before each tags = { block.
• Both pass terraform validate (terraform 1.16.2, aws provider 6.64.0) with the three referenced variables declared.

One example is one example. Q4_K_M on this model stays on-task and on-format here and changes whitespace. If generated HCL is going to be applied, run terraform validate and terraform plan against it.

Training details

Data

Each pair is one .tf file containing at least one resource block, paired with a plain-language instruction. Provider mix:

Method

8-bit QLoRA (bitsandbytes load_in_8bit) with a LoRA adapter over the language model only. Supervised fine-tuning on completions and prompts together — standard causal LM loss over the full rendered sequence. 8 × H200, torchrun --nproc_per_node 8.

The adapter targets the ten projection names this architecture actually uses — in_proj_qkv, in_proj_z, out_proj (Gated DeltaNet linear-attention layers), q_proj, k_proj, v_proj, o_proj (the full-attention layers, every 4th), and gate_proj, up_proj, down_proj (MLP). That resolves to 400 modules, all in the language model, none in the vision tower. The vision tower and the MTP layer were not trained.

Final hyperparameters

DoRA, rsLoRA, LoRA+, NEFTune, Liger kernels and PiSSA/LoftQ init were all available and all left off.

Observed training metrics

Evaluation

eval_loss = 0.352946, measured on these merged bf16 weights over the full 4,004-row held-out validation split.

This is a teacher-forced loss: the model is scored on how well it predicts the reference HCL token by token, with the reference supplied as context.

Environmental impact

Reported rather than estimated, since the hardware is shared and per-job power was not instrumented:

Carbon emitted was not measured; the grid region and PUE of the host are not known to the authors.

Framework versions

qwen3_5 is not in any stable transformers release as of 2026-09-23. A git-main install is required to load this model.

Citation

Cite this model, the base model, the dataset and the training stack:

@misc{qwen38_27b_terraform_2026,
  title  = {qwen38-27b-terraform},
  author = {{SASVA AI Model Cognition Labs (MCL) Team}},
  year   = {2026},
  url    = {https://huggingface.co/SASVAAI/qwen38-27b-terraform}
}

@misc{qwen38-27b,
  title  = {Qwen3.8-27B},
  author = {Qwen Team},
  year   = {2026},
  url    = {https://huggingface.co/Qwen/Qwen3.8-27B}
}

@dataset{terrads,
  title     = {TerraDS: A Dataset of Terraform Infrastructure-as-Code},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.20339474}
}

@misc{trl,
  title  = {TRL: Transformer Reinforcement Learning},
  author = {von Werra, Leandro and others},
  year   = {2020},
  url    = {https://github.com/huggingface/trl}
}