NL-FM-Toolkit
Getting Started:
Introduction
Installation
Training a Tokenizer from Scratch
Creating Model Configuration File
Training a Masked Language Model from Scratch
Training a Causal Language Model from Scratch
Training a Sequence Labeler
Training a Sequence Classifier
Scripts
Modules
train_tokenizer module
create_config module
tokenize_corpus module
run_clm module
run_mlm module
run_seq_to_seq_pretrain module
run_tc module
run_seq module
data_collator_for_seq_to_seq module
NL-FM-Toolkit
Modules
data_collator_for_seq_to_seq module
View page source
data_collator_for_seq_to_seq module