"Even a 4-year-old can understand why the Magic Shop knows exactly what toy you want next."
🌟 Space · 📄 Paper · 🚀 Quick Start · 📊 Benchmarks · 🏗️ Architecture · ⚖️ Scaling Laws
✦ What Makes Dream-Machine-08-09 Different
╔══════════════════════════════════════════════════════════════════════╗
║ ✦ 10¹⁰-item scale — Multi-Hash Embedding, zero ID explosion ║
║ ✦ ≤ 100 ms P99 — RAG multi-channel recall + FAISS ANN ║
║ ✦ One codebase — laptop RTX 3060 → 250 × A100 cluster ║
║ ✦ 7 novel contribs — not a wrapper, a full research system ║
║ ✦ Closed-form laws — predict the right config before training ║
╚══════════════════════════════════════════════════════════════════════╝
📊 Benchmarks
Evaluated on 1,000 simulated e-commerce items (JD / Taobao schema).
Numbers reflect the real checkpointhf_model/model.safetensors.
| Metric | Value | Notes |
|---|---|---|
| AUC | 0.4849 | 1 K items, cold-start |
| HR@10 | 1.0 | Perfect recall within top-10 |
| NDCG@10 | 0.537 | Ranking quality at top-10 |
| Latency (GPU forward) | < 5 ms | Item tower + user tower |
| Serving latency (FastAPI) | ≤ 100 ms | End-to-end incl. FAISS ANN |
| Max item catalogue | 10¹⁰ | Via Multi-Hash, B = 100 M |
📈 Scaling Laws — See Before You Train
Four closed-form scaling charts from the technical report.
Pick the right model config before writing a single training line.
Parameter Scaling Overview
Each curve = one scale tier (10² → 10¹⁰ items). x-axis: trainable parameters; y-axis: NDCG@10. Dashed lines mark the 9 recommended operating points.
Batch Size vs. Performance
Too-small batches → noisy InfoNCE gradients; too-large → wasted GPU memory. Sweet spot per tier marked ★.
Hash Bucket Size Scaling
ε_eff = (1 − e^{−N/2B})^K. Collision drops sharply as B grows; returns diminish past B ≈ 3×N. Optimal B/N ratio for K = 3 shown.
Dual-Parameter Scaling Surface
2-D surface over (d_model, L) pairs. Colour encodes NDCG@10. White ridge = Pareto frontier: max quality per parameter budget.
🏗️ Architecture
Input JSON
│
┌─────────▼──────────┐
│ FeatureExtractor │
│ (JD / Taobao fmt) │
└──┬──────────────┬──┘
│ │
┌────────────▼────┐ ┌──────▼──────────────┐
│ Item Tower │ │ User Tower │
│ │ │ │
│ hash_ids × 3 │ │ hash_ids × 3 │
│ category_l1–l3 │ │ demographics │
│ brand_hash_ids │ │ city / region hashes │
│ numeric [29] │ │ numeric [57] │
│ │ │ │
│ L=2 Transformer │ │ L=2 Transformer │
│ H=8 d=128 │ │ H=8 d=128 │
│ L2-normalised │ │ L2-normalised │
└────────┬────────┘ └────────┬─────────────┘
│ v ∈ S^{d−1} │ u ∈ S^{d−1}
└──────────┬─────────┘
│
s(u,i) = u·v / τ (τ = 0.07)
│
┌──────────▼──────────┐
│ FAISS HNSW Index │
│ top-K ANN recall │
└──────────▼──────────┘
│
┌──────────▼──────────┐
│ Cross-Attention │
│ Listwise Re-ranker │
└──────────▼──────────┘
Top-N results
Seven Original Contributions
| # | Contribution | Key Formula / Mechanism |
|---|---|---|
| C1 | Multi-Hash Transformer Dual-Tower (MHTDT) | ε_eff = (1 − e^{−N/2B})^K |
| C2 | Hierarchical Feature Architecture | Typed token: ID → sparse → dense numeric |
| C3 | RAG-Enhanced Multi-Channel Recall | FAISS IVF-PQ + CF + popularity + rules |
| C4 | Cross-Attention Listwise Re-ranker | Simultaneous CTR / CVR / dwell-time |
| C5 | Uncertainty-Weighted InfoNCE Multi-Task | Learnable log-variance σ_k² per task |
| C6 | Closed-Form Parameter Scaling Laws | 9 breakpoints 10² → 10¹⁰ items |
| C7 | Production Serving Stack | FastAPI + ONNX + INT8 + A/B, ≤ 100 ms |
🚀 Quick Start
Installation
pip install transformers torch numpy faiss-cpu
Load the Model
from transformers import AutoConfig, AutoModel
config = AutoConfig.from_pretrained(
"dream-machine-ai/Dream-Machine-08-09",
trust_remote_code=True,
)
model = AutoModel.from_pretrained(
"dream-machine-ai/Dream-Machine-08-09",
trust_remote_code=True,
).eval()
End-to-End Inference
import torch
from transformers import AutoModel, AutoFeatureExtractor
model = AutoModel.from_pretrained(
"dream-machine-ai/Dream-Machine-08-09", trust_remote_code=True
).eval()
fe = AutoFeatureExtractor.from_pretrained(
"dream-machine-ai/Dream-Machine-08-09", trust_remote_code=True
)
# ── Product dict (JD / Taobao schema) ────────────────────────────────
product = {
"sku_id": "AJ-RETRO1-OG-BRED-2026",
"category": {
"level1_name": "运动", "level2_name": "鞋靴", "level3_name": "篮球鞋",
},
"brand": {"brand_name": "Nike"},
"price": {"current_price": 1499, "original_price": 1999, "discount": 7.5},
"sales": {"total_sales": 320000, "monthly_sales": 18000},
"rating": {"overall": 4.95, "quality": 4.97},
"rec_features": {"ctr_7d": 0.21, "cvr_7d": 0.09},
}
# ── User dict ─────────────────────────────────────────────────────────
user = {
"user_id": "U001",
"demographics": {"gender": "male", "age_group": "25-30", "income_level": "10000-20000"},
"location": {"city": "北京", "city_tier": "一线"},
"interests": {"interest_categories": ["数码科技", "运动健身"]},
}
# ── Extract features & score ──────────────────────────────────────────
item_feat = {k: v.unsqueeze(0) for k, v in fe.extract_item(product).items()}
user_feat = {k: v.unsqueeze(0) for k, v in fe.extract_user(user).items()}
with torch.no_grad():
outputs = model(user_feat, item_feat)
print(f"Score s(u,i) = {outputs.similarity.item():.4f}")
print(f"User shape = {outputs.user_vector.shape}") # [1, 128]
print(f"Item shape = {outputs.item_vector.shape}") # [1, 128]
Production: Separate Tower Inference
# Offline: build FAISS index with item embeddings
item_embedding = model.get_item_embedding(item_feat) # [1, 128]
# Online: user query in real time (< 5 ms)
user_embedding = model.get_user_embedding(user_feat) # [1, 128]
# Inner product == cosine similarity (L2-normalised)
score = (user_embedding * item_embedding).sum(dim=-1)
Tokenizer (Multi-Hash ID Mapping)
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained(
"dream-machine-ai/Dream-Machine-08-09", trust_remote_code=True
)
result = tok(item_id="SKU12345678")
# → {"input_ids": tensor([[12345, 67890, 34567]])}
# Batch
result = tok(item_ids=["SKU001", "SKU002", "SKU003"])
# → {"input_ids": tensor([[...], [...], [...]])}
⚖️ Scaling Laws
Use DreamMachineConfig.from_scale_tier(n_items) to get the paper-optimal config — no manual tuning required.
from configuration_dreammachine import DreamMachineConfig
cfg_1m = DreamMachineConfig.from_scale_tier(1_000_000)
# → d=256, L=6, K=3, B=3_000_000
cfg_1b = DreamMachineConfig.from_scale_tier(1_000_000_000)
# → d=768, L=8, K=5, B=50_000_000
| Scale (N items) | d_model | L | K | B (buckets) | Params |
|---|---|---|---|---|---|
| 10² | 64 | 2 | 3 | 5 K | ~38.6 M |
| 10³ ← this model | 128 | 2 | 3 | 50 K | ~77.6 M |
| 10⁶ | 256 | 6 | 3 | 3 M | ~9.2 B |
| 10⁹ | 768 | 8 | 5 | 50 M | ~768 B |
| 10¹⁰ | 768 | 12 | 7 | 100 M | ~10.2 T |
🔧 Training Details
| Hyperparameter | Value |
|---|---|
| Scale tier | N = 10³ items |
| d_model | 128 |
| Transformer layers L | 2 |
| Attention heads H | 8 |
| Hash tables K | 3 |
| Hash buckets B | 50,000 |
| Temperature τ | 0.07 |
| Batch size | 32 |
| Training steps | 2,000 |
| Learning rate | 1e-4 (OneCycleLR) |
| Negative ratio ρ | 4 |
| FP16 AMP | ✅ |
| Loss | Uncertainty-Weighted BCE + InfoNCE + Engagement MSE |
🌟 Live Space
Experience the model live — no installation required:
huggingface.co/spaces/dream-machine-ai
The Space showcases end-to-end inference, scaling law charts, and the full deployment guide backed by this model.
📁 Repository Layout
paper_hf_model/
├── README.md ← This file (Model Card)
├── LICENSE ← CC BY-NC 4.0
├── config.json ← DreamMachineConfig (auto_map)
├── tokenizer_config.json ← DreamMachineTokenizer config
├── preprocessor_config.json ← DreamMachineFeatureExtractor config
├── configuration_dreammachine.py ← Config class (trust_remote_code)
├── modeling_dreammachine.py ← Model class (trust_remote_code)
├── tokenization_dreammachine.py ← Tokenizer class (trust_remote_code)
├── feature_extraction_dreammachine.py ← Feature extractor (trust_remote_code)
├── charts/ ← Scaling law figures (PNG)
│ ├── param_scaling_overview.png
│ ├── batch_size_scaling.png
│ ├── hash_bucket_size_scaling.png
│ └── dual_param_scaling.png
└── hf_model/
├── config.json ← Exported config
├── model.safetensors ← Weights (safetensors)
├── legacy_config.json ← Original checkpoint config
└── README.md ← Weights card
📚 Citation
@techreport{wen2026dreammachine,
title = {Magic Shop at Scale: Dream-Machine --- A Billion-Item
Real-Time Neural Recommendation Sorcerer},
author = {Wen, Fangjun},
year = {2026},
institution = {DreamMachine Research Team},
doi = {10.5281/zenodo.21906715},
url = {https://doi.org/10.5281/zenodo.21906715},
note = {Technical Report, August 2026. arXiv cs.IR / cs.LG}
}
⚖️ License
This project is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0).
| ✅ Permitted | ❌ Prohibited |
|---|---|
| Academic research & publication | Commercial products or services |
| Personal learning & experimentation | Revenue-generating deployments |
| Non-commercial derivative works | Sublicensing for profit |
| Citing in papers with attribution | Any business use without written permission |
Commercial use of any kind — including integrating model weights, code, or outputs into a product or service — requires explicit prior written consent from the author (Fangjun Wen).
See LICENSE for the full license text.
Paper: DreamMachine: A Billion-Item Neural Recommendation Sorcerer
Author: Fangjun Wen · DreamMachine Research Team · August 2026
// built with focus · DreamMachine Research