Contents Menu Expand Light mode Dark mode Auto light/dark mode
IBM Generative AI Python SDK (Tech Preview)
IBM Generative AI Python SDK (Tech Preview)
  • Getting Started
  • V3 Migration Guide
  • V2 Migration Guide
  • Examples
    • Text
      • Stream answer from a model
      • Text Moderations
      • Chat with a model
      • Get embedding vectors for text data
      • Simple text generation
      • Compare a set of hyperparameters
      • Complex text generation example with a progress bar
      • Tokenize text data
      • Experimental
        • Sentence Similarity
        • Text Classification
        • Document reranking
    • Models
      • Show information about supported models
    • Tunes
      • Tune a custom model (Prompt Tuning)
    • Deployment
      • Get all deployed models
      • Fine tune and deploy custom model
    • Prompts
      • Create a custom prompt with variables
    • System Prompts
      • Working with system prompts
    • Files
      • Working with files
    • Users
      • Show information about current user
    • Requests
      • Working with your requests
      • Provide feedback to the request
    • Tags
      • Working with tags
    • Folders
      • Working with folders
    • Tasks
      • Working with tasks
    • Extra
      • Overriding built-in services
      • Enable/Disable logging for SDK
      • Error Handling
      • Shutdown Handling
      • Text generation with custom concurrency limit and multiple processes
      • Retrieve metadata for given service method
      • Customize underlying API (httpx) Client
      • Vector Databases
        • Chroma DB
          • Create ChromaDB Embedding Function
    • Extensions
      • Transformers (HuggingFace)
        • Run Transformers Agents
      • lm_eval
        • lm-evaluation-harness CLI usage
        • lm-evaluation-harness advanced usage
      • LangChain
        • Streaming response from LangChain
        • Chat with a model using LangChain
        • Use LangChain generation with a custom template.
        • LangChain SQL Agent
        • Text generation using LangChain
        • LangChain agent
        • LangChain Embeddings
        • Serialize LangChain model to a file
        • QA using native LangChain features
      • LLamaIndex
        • Use a model through LLamaIndex
        • LlamaIndex Embeddings
      • LocalServer
        • Customize behavior of local client
        • Use a local server with a custom model
  • FAQ
  • Changelog

Versions

  • main (unreleased)
  • v3.0.0
  • v2.3.0
  • v2.2.0
  • v2.1.1
  • v2.1.0
  • v2.0.0
Back to top

lm_eval#

Before you start

To use the following extension, first install it by running pip install 'ibm-generative-ai[lm_eval]'.

  • lm-evaluation-harness CLI usage
  • lm-evaluation-harness advanced usage
Next
lm-evaluation-harness CLI usage
Previous
Run Transformers Agents
Copyright © 2024, IBM Research
Made with Sphinx and @pradyunsg's Furo