GP-VAC Tagger (en_gp_vac_tagger)

GP-VAC Tagger is a spaCy pipeline that automatically identifies verb-argument constructions (VACs) in English text. Its label scheme covers 85 verb patterns from Grammar Patterns 1: Verbs (Francis, Hunston, & Manning, 1996), and it was trained on both native-speaker and L2 learner English so that it can be used for learner corpus research.

Each VAC label is attached to the verb token that heads the pattern, as a single-token entity (doc.ents / token.ent_type_). Because the pipeline also keeps spaCy's standard components (from en_core_web_trf), the same Doc gives you lemmas, POS tags, and dependency parses as well.

Intended use

• Automatic annotation of VACs in written English corpora, including learner corpora (e.g., frequency, diversity, and verb–construction association analyses in usage-based SLA and corpus linguistics).
• Research and education. Commercial use is not permitted under the license.

Out of scope: spoken or transcribed speech (not evaluated), languages other than English, and high-stakes decisions about individual learners (e.g., automated scoring or placement) without human checking.

How to use

Installation

pip install https://huggingface.co/fuku0827/en_gp_vac_tagger/resolve/main/en_gp_vac_tagger-1.0.0-py3-none-any.whl

This installs the pipeline as a Python package together with its dependencies (spacy, spacy-transformers, spacy-curated-transformers). A GPU is recommended; to use one, install a CUDA-enabled PyTorch build first.

Alternatively, download the repository files and load the pipeline from the directory:

from huggingface_hub import snapshot_download
import spacy

path = snapshot_download("fuku0827/en_gp_vac_tagger", ignore_patterns=["*.whl"])
nlp = spacy.load(path)

Tagging VACs

import spacy

spacy.prefer_gpu()  # uses a GPU if one is available
nlp = spacy.load("en_gp_vac_tagger")

doc = nlp("She wants her students to read more books.")
for token in doc:
    if token.ent_type_:
        print(token.text, token.lemma_, token.ent_type_)

Example output (produced with this release):

wants      want       VAC_V_N_TO_INF
read       read       VAC_V_N_OBJ

If you only need the VAC labels, you can skip the frozen components to save memory and time. ner_transformer and ner must both be kept:

nlp = spacy.load(
    "en_gp_vac_tagger",
    exclude=["transformer", "tagger", "parser", "attribute_ruler", "lemmatizer"],
)

Label scheme

spaCy entity labels cannot contain hyphens or lowercase pattern notation, so each Grammar Patterns (GP) pattern is mapped to an upper-case label. n = noun group, pl-n = plural noun group, adj = adjective group, -ing / to-inf / inf / -ed = non-finite clause types, that / wh = finite clause types; words in italics are fixed lexical items.

Training

Data

The model was trained on 8,082 manually annotated sentences (153,259 tokens) from five sources. The training, validation, and test data are not distributed with this model, because the source corpora have their own terms of use.

The sentences were split at random into a training set (6,465 sentences, 80%) and a validation set (1,617 sentences, 20%). The validation set was used for checkpoint selection. This release is the version trained without sentence-level resampling of the training data; a variant trained with nsCRD resampling (Wang & Wang, 2022) did not improve accuracy.

Procedure

• The transformer, tagger, parser, attribute_ruler, and lemmatizer components are taken from en_core_web_trf 3.8.0 and were frozen during training.
• Only ner_transformer (initialised from roberta-base) and ner (spaCy TransitionBasedParser.v2) were trained.
• Main settings: Adam (initial learning rate 5e-5, 250 warm-up steps, linear decay), dropout 0.1, gradient accumulation 3, batch size 2,000 padded tokens, up to 20,000 steps, evaluation every 200 steps, patience 1,600 steps. The checkpoint with the best validation score was kept (model-best).
• Hardware: one NVIDIA A100 GPU (Google Colaboratory).

The full training configuration is included as config.cfg.

Evaluation

Scores are entity-level precision, recall, and F1 from spaCy's NER scorer. Because every VAC entity is a single verb token, a prediction counts as correct only when both the token and the label match. Micro averages pool all labels; Macro averages are unweighted means over labels; Macro 3+ is restricted to labels that occur at least three times in the test set.

All test sets were annotated separately from the training data. Per-label validation scores are stored in meta.json (performance.ents_per_type).

Limitations

• The most frequent error type involves prepositional phrases: whether a PP is a pattern element (e.g., V n for n) or an adjunct is often ambiguous. Patterns such as V n with n, V n in n, and V n for n have lower F1 than simple patterns, which can add noise to frequency or association estimates for prepositional VACs.
• Learner grammatical errors are the second most frequent error source, although many sentences with verb-form errors or word-class confusions are labelled correctly.
• In be + -ed sequences where -ed could be an adjective or a passive participle, the model may assign both VAC_V_ADJ (to be) and VAC_V_N_OBJ_PASS (to the participle). There V n is sometimes over-applied. Rule-based post-processing can reduce both.
• Tokenization and spelling errors in the input can cause errors; normalising the text first may help.
• Labels are assigned to single verb tokens, so the pipeline does not mark multi-word units such as phrasal verbs and is not suited to analyses that treat phrasal verbs as units.
• All training and test annotations were made by a single annotator; inter-annotator agreement has not been assessed.
• Labels that are rare in the test data have unstable per-label scores; interpret them with care.
• The model was trained and evaluated on written texts only.

License

The model is released under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). The components taken from en_core_web_trf (MIT License, © ExplosionAI GmbH) and the roberta-base weights used to initialise ner_transformer (MIT License) remain under their original licenses; see LICENSE and LICENSES_SOURCES. Users are responsible for complying with the terms of any corpora they process with this model.

Citation

A citation for the accompanying publication will be added here. Until then, please cite this repository:

@misc{gp_vac_tagger,
  author       = {Kohei Fukuda},
  title        = {{GP-VAC Tagger}: A Grammar Pattern-based Verb-Argument Construction Tagger for {English}},
  year         = {2026},
  version      = {1.0.0},
  howpublished = {\url{https://huggingface.co/fuku0827/en_gp_vac_tagger}}
}

References

• Berzak, Y., Kenney, J., Spadine, C., Wang, J. X., Lam, L., Mori, K. S., Garza, S., & Katz, B. (2016). Universal Dependencies for learner English. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long papers) (pp. 737–746). https://doi.org/10.18653/v1/P16-1070
• Davies, M. (2008–). The Corpus of Contemporary American English (COCA). https://www.english-corpora.org/coca/
• Francis, G., Hunston, S., & Manning, E. (1996). Collins COBUILD grammar patterns 1: Verbs. HarperCollins Publishers.
• Geertzen, J., Alexopoulou, T., & Korhonen, A. (2014). Automatic linguistic annotation of large scale L2 databases: The EF-Cambridge Open Language Database (EFCamDat). In Selected proceedings of the 2012 Second Language Research Forum (pp. 240–254). Cascadilla Proceedings Project.
• Ishikawa, S. (2023). The ICNALE guide: An introduction to a learner corpus study on Asian learners' L2 English. Routledge. https://doi.org/10.4324/9781003252528
• Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692.
• Shatz, I. (2020). Refining and modifying the EFCAMDAT: Lessons from creating a new corpus from an existing large-scale English learner language database. International Journal of Learner Corpus Research, 6(2), 220–236. https://doi.org/10.1075/ijlcr.20009.sha
• Silveira, N., Dozat, T., de Marneffe, M.-C., Bowman, S. R., Connor, M., Bauer, J., & Manning, C. (2014). A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14) (pp. 2897–2904). European Language Resources Association.
• Wang, X., & Wang, Y. (2022). Sentence-level resampling for named entity recognition. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 2151–2165). https://doi.org/10.18653/v1/2022.naacl-main.156