create_config module

Create config.json file

usage: create_config.py [-h] [--path PATH] [--type {gpt2,mt5,led}] [--vocab_size VOCAB_SIZE]

Named Arguments

--path

Path where the config.json file will be created

Default: “models/gpt2_event_tokenizer”

--type

Possible choices: gpt2, mt5, led

Type of pre-trained model

Default: “gpt2”

--vocab_size

Vocabulary Size of the tokenizer

Default: 30000