Pre-work¶

The labs in the workshop are Jupyter notebooks. The notebooks can be run on your computer or remotely on the Google Colab service. Check out Running the Granite Notebooks section on how to setup the way you want to run the notebooks.

Running the Granite Notebooks¶

The notebooks can be run:

Locally on your computer OR
Remotely on the Google Colab service

Follow the instructions in one of the sections that follow on how you would like to run the notebooks.

Running the Granite Notebooks Locally¶

It is recommended if you want to run the lab notebooks locally on your computer that you have:

A computer or laptop
Knowledge of Git and Python

If not, then it recommended to go to the Running the Granite Notebooks Remotely (Colab) section instead.

Running the lab notebooks locally on your computer requires the following steps:

Local Prerequisites
Clone the Granite Workshop Repository
Serving the Granite AI Models
Install Jupyter

Local Prerequisites¶

Git
Python 3.10 or 3.11

Clone the Granite Workshop Repository¶

Clone the workshop repo and cd into the repo directory.

git clone https://github.com/IBM/granite-workshop.git
cd granite-workshop

Serving the Granite AI Models¶

Lab 1: Document Summarization with Granite, Lab 2: Retrieval Augmented Generation (RAG) with Langchain and Lab 4: Generating Bash Code with Granite Code require Granite models to be served by an AI model runtime so that the models can be invoked or called. There are 2 options to serve the models as follows:

Replicate AI Cloud Platform
Running Ollama Locally OR

Replicate AI Cloud Platform¶

Replicate is a cloud platform that will host and serve AI models for you.

Create a Replicate account. You will need a GitHub account to do this.
Add credit to your Replicate Account (optional). To remove a barrier to entry to try the Granite models on the Replicate platform, use this link to add a small amount of credit to your Replicate account.
Create a Replicate API Token.
Set your Replicate API Token as an environment variable in your terminal where you will run the notebook:
```
export REPLICATE_API_TOKEN=<your_replicate_api_token>
```

Running Ollama Locally¶

If you want to run the AI models locally on your computer, you can use Ollama. You will need to have a computer with:

GPU processor
At least 32GB RAM

Tested system

This was tested on a Macbook with an M1 processor and 32GB RAM. It maybe possible to serve models with a CPU and less memory.

If you computer is unable to serve the models, then it is recommended to go to the Replicate AI Cloud Platform section instead.

Running Ollama locally on your computer requires the following steps:

Download and install Ollama, if you haven't already. Ollama v0.3.14+ is required, so please upgrade if on an earlier version.

On macOS, you can use Homebrew to install with
```
brew install ollama
```
Start the Ollama server. You will leave this running during the workshop.
```
ollama serve
```
In another terminal window, pull down the Granite models you will want to use in the workshop. Larger models take more memory to run but can give better results.
```
ollama pull granite3-dense:2b
ollama pull granite3-dense:8b
```
For Lab 4: Generating Bash Code with Granite Code, you will also need at least one of the following Granite Code models.
```
ollama pull granite-code:3b
ollama pull granite-code:8b
```

Install Jupyter¶

Use a virtual environment

Before installing dependencies and to avoid conflicts in your environment, it is advisable to use a virtual environment (venv).

Create virtual environment:
```
python3 -m venv --upgrade-deps venv
```
Activate the virtual environment by running:
```
source venv/bin/activate
```
Install Jupyter notebook in the virtual environment:
```
python3 -m pip install --require-virtualenv notebook ipywidgets
```
For more information, see the Jupyter installation instructions
To open a notebook in Jupyter (in the active virtual environment), run:
```
jupyter notebook <notebook-file-path>
```

Running the Granite Notebooks Remotely (Colab)¶

Running the lab notebooks remotely using Google Colab requires the following steps:

Colab Prerequisites
Serving the Granite AI Models for Colab

Colab Prerequisites¶

Google Colab requires a Google account that you're logged into

Serving the Granite AI Models for Colab¶

Lab 1: Document Summarization with Granite, Lab 2: Retrieval Augmented Generation (RAG) with Langchain and Lab 4: Generating Bash Code with Granite Code require Granite models to be served by an AI model runtime so that the models can be invoked or called.

Replicate AI Cloud Platform for Colab¶

Replicate is a cloud platform that will host and serve AI models for you.

Create a Replicate account. You will need a GitHub account to do this.
Add credit to your Replicate Account (optional). To remove a barrier to entry to try the Granite Code models on the Replicate platform, use this link to add a small amount of credit to your Replicate account.
Create a Replicate API Token.
Add your Replicate API Token to the Colab Secrets manager to securely store it. Open Google Colab and click on the 🔑 Secrets tab in the left panel. Click "New Secret" and enter REPLICATE_API_TOKEN as the key, and paste your token into the value field. Toggle the button on the left to allow notebook access to the secret.