ESPnet3 tse model

Packed model bundle generated from egs3/librimix/tse.

Model

• Repository: wyz/librimix_tse_clean_2mix_training_td_speakerbeam_16k
• Recipe: egs3/librimix/tse
• Task: tse
• System: tse
• Creator: wyz
• Created: 2026-08-28T11:24:04
• Git: 55776eaf5 (dirty)

Usage

from espnet3.publication import InferenceModel

model = InferenceModel.from_pretrained("wyz/librimix_tse_clean_2mix_training_td_speakerbeam_16k", trust_user_code=True)
result = model(sample)

Packaging

• Bundle: model_pack
• Exp dir: ./exp/training_td_speakerbeam_16k
• Strategy: copy experiment outputs; include extra recipe assets; register named artifact files; apply exclude filters

Results

Training config

num_device: 1
num_nodes: 1
task: espnet3.systems.tse.task.TSETask
recipe_dir: .
data_dir: ./data
exp_tag: training_td_speakerbeam_16k
exp_dir: ./exp/training_td_speakerbeam_16k
stats_dir: ./exp/stats
dataset_dir: /path/to/your/dataset
create_dataset:
  recipe_dir: .
  dataset_dir: /path/to/your/dataset
dataset:
  _target_: espnet3.components.data.data_organizer.DataOrganizer
  recipe_dir: .
  train:
  - data_src_args:
      split: 2mix_16k_max_train_mix-clean
      ignore_key_prefix:
      - text_spk
      - utt_id
      - num_spk
      use_espnet_preprocessor: true
  valid:
  - data_src_args:
      split: 2mix_16k_max_dev_mix-clean
      ignore_key_prefix:
      - text_spk
      - utt_id
      - num_spk
      use_espnet_preprocessor: true
  - data_src_args:
      split: 2mix_16k_max_test_mix-clean
      ignore_key_prefix:
      - text_spk
      - utt_id
      - num_spk
      use_espnet_preprocessor: true
  test: null
  preprocessor:
    _target_: espnet2.train.preprocessor.TSEPreprocessor
    train: true
    train_spk2enroll: ./data/2mix_16k_max_train_mix-both/spk2enroll.json
    enroll_segment: 48000
    load_spk_embedding: false
    load_all_speakers: true
    num_spk: 2
    _convert_: all
  _convert_: all
model:
  encoder: conv
  encoder_conf:
    channel: 256
    kernel_size: 32
    stride: 16
  decoder: conv
  decoder_conf:
    channel: 256
    kernel_size: 32
    stride: 16
  extractor: td_speakerbeam
  extractor_conf:
    layer: 8
    stack: 4
    bottleneck_dim: 256
    hidden_dim: 512
    skip_dim: 256
    kernel: 3
    causal: false
    norm_type: gLN
    pre_nonlinear: prelu
    nonlinear: linear
    i_adapt_layer: 7
    adapt_layer_type: mul
    adapt_enroll_dim: 256
    use_spk_emb: false
  model_conf:
    num_spk: 2
    share_encoder: true
  criterions:
  - name: snr
    conf:
      eps: 1.0e-07
    wrapper: fixed_order
    wrapper_conf:
      weight: 1.0
optimizer:
  _target_: torch.optim.Adam
  lr: 0.001
  eps: 1.0e-08
  weight_decay: 0
  _convert_: all
scheduler:
  _target_: torch.optim.lr_scheduler.StepLR
  step_size: 200000
  gamma: 0.98
  _convert_: all
scheduler_interval: step
scheduler_monitor: null
best_model_criterion:
- - valid/loss
  - 3
  - min
seed: null
init: null
parallel:
  env: local
  n_workers: 1
dataloader:
  collate_fn:
    _target_: espnet2.train.collate_fn.CommonCollateFn
    int_pad_value: -1
    _convert_: all
  train:
    total_shards: 1
    dist_world_size: 1
    iter_factory:
      _target_: espnet2.iterators.chunk_iter_factory.ChunkIterFactory
      shuffle: true
      collate_fn:
        _target_: espnet2.train.collate_fn.CommonCollateFn
        int_pad_value: -1
        _convert_: all
      batches:
        type: unsorted
        shape_files:
        - ./exp/stats/train/feats_shape
        batch_size: 1
        batch_bins: -1
      batch_size: 4
      chunk_length: 48000
      chunk_shift_ratio: 0.5
      excluded_key_prefixes:
      - enroll_ref
      num_workers: 4
      _convert_: all
  valid:
    total_shards: 1
    dist_world_size: 1
    iter_factory:
      _target_: espnet2.iterators.sequence_iter_factory.SequenceIterFactory
      shuffle: false
      collate_fn:
        _target_: espnet2.train.collate_fn.CommonCollateFn
        int_pad_value: -1
        _convert_: all
      batches:
        type: unsorted
        shape_files:
        - ./exp/stats/valid/feats_shape
        batch_size: 1
        batch_bins: -1
      _convert_: all
trainer:
  accelerator: auto
  devices: 1
  num_nodes: 1
  accumulate_grad_batches: 1
  check_val_every_n_epoch: 1
  gradient_clip_val: 5.0
  log_every_n_steps: 100
  max_epochs: 100
  logger:
  - _target_: lightning.pytorch.loggers.TensorBoardLogger
    save_dir: ./exp/training_td_speakerbeam_16k/tensorboard
    name: tb_logger
    _convert_: all
  strategy: ddp_find_unused_parameters_true
  precision: 32
  limit_train_batches: 1.0
fit: {}

Citing ESPnet

@inproceedings{watanabe2018espnet,
  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and
    Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner
    and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
  title={{ESPnet}: End-to-End Speech Processing Toolkit},
  year={2018},
  booktitle={Proceedings of Interspeech},
  pages={2207--2211},
  doi={10.21437/Interspeech.2018-1456}
}

@article{ESPnet3-Someki2026,
  title={{ESPnet3}: Infrastructure for Scalable Speech and Audio Research in the Foundation Model Era},
  author={Someki, Masao and Polok, Alexander and Carvalho, Carlos and Lin, Chyi-Jiunn and Yang, Da-Hee and Shi, Jiatong and Tian, Jinchuan and Soplin, Nelson Enrique Yalta and Cornell, Samuele and Arora, Siddhant and Teixeira, Francisco and Wang, Wei and Chen, William and Abad, Alberto and Li, Chenda and Watanabe, Shinji and Zhang, Wangyou},
  journal={arXiv preprint arXiv:2606.21854},
  year={2026},
}