PorTAL refit for Gemma 4 E2B
This is a native PorTAL artifact
refitted onto google/gemma-4-E2B. Its 14-task latent table and canonical
LoRA-generating core were learned jointly from Qwen3-1.7B and Qwen3-4B and
frozen during refitting. Only a fresh Gemma 4 alignment was trained.
Gemma 4 alternates sliding- and full-attention projection widths and reuses KV states in later layers. The artifact records the exact sparse q/v projection targets and generates rank-8 LoRA factors only for projections that exist in the text decoder.
Evaluation
One seed was evaluated on the pinned 14-task validation suite using continuation
log-probability divided by character length (acc_norm). Gold continuation
token-mean NLL was tracked separately for checkpoint selection.
| Model | Macro acc_norm |
|---|---|
| Frozen Gemma 4 E2B | 0.5729 |
| PorTAL-adapted | 0.7363 |
| Absolute lift | +0.1634 |
The selected checkpoint was epoch 4. Across the same tasks, independently trained rank-16 full-text-decoder LoRA baselines averaged 0.7084. Relative to the frozen base, PorTAL retained 120.6% of their aggregate lift.
These are research benchmark results for this exact artifact and evaluation
recipe, not a general performance guarantee. Full selected-epoch results are in
metrics.json.
Refit recipe
- Base:
google/gemma-4-E2Batd29ff6b45f081a49ee2733a859c9c9c2d95d1a6f - Frozen source carrier:
RampPublic/portal-qwen3-4b@v0.2.0 - Dataset:
RampPublic/portallib-tasksatffc3c0e44f529bf64a5ae62ed5db090952db97ea - Refit data: deterministic nested sample of up to 1,000 examples per task
- Optimization: 5 epochs, batch size 4, alignment LR
1e-3, linear decay with 10% warmup, seed 0 - Trainable parameters: target-base alignment only; task latents and canonical core remained frozen
- Checkpoint selection: maximum macro validation
acc_norm, with lower gold NLL as the tie-breaker - Architecture: q/v targets, rank 8, alpha 16, task latent 256, layer embedding 32, hidden 512, canonical width 1024
Usage
Install portallib>=0.2.0, then load the immutable artifact revision:
from portallib import PortalModel
portal = PortalModel.from_pretrained(
"RampPublic/portal-gemma-4-e2b",
revision="v0.2.0",
)
portal.export_peft("rte", "./portal-rte-gemma-4-e2b")
See the release recipe for the full task list, evaluation definition, target topology, and refitting procedure. The artifact is Apache-2.0; the benchmark dataset contains components under multiple upstream licenses documented on its dataset card.