Welcome to NL-FM-Toolkit’s documentation!

Getting Started:

• Introduction
• Installation
• Training a Tokenizer from Scratch
• Creating Model Configuration File
• Training a Masked Language Model from Scratch
• Training a Causal Language Model from Scratch
• Training a Sequence Labeler
  • Convert CoNLL file to JSON format
  • Training a Token classifier
  • Hyper-Parameter Tuning
  • Fine-Tuning using best Hyper-Parameter
• Training a Sequence Classifier
  • Hyper-Parameter Tuning
  • Fine-Tuning using best Hyper-Parameter

Scripts

• Modules
  • train_tokenizer module
  • create_config module
  • tokenize_corpus module
  • run_clm module
  • run_mlm module
  • run_seq_to_seq_pretrain module
  • run_tc module
  • run_seq module
  • data_collator_for_seq_to_seq module

Indices and tables

• Index

• Module Index

• Search Page