README.md

Running temporary Large Language Models (LLMs) with Ollama

This directory outlines two distinct scenarios and approaches, differing in the method of running the base Ollama server and the LLM:

1. An approach utilizing the official Ollama container image, which encompasses the entire software stack and necessary binaries to operate Ollama.
2. An approach involving manual setup of Ollama within your user directories, requiring you to download binaries and modify paths accordingly.

Furthermore, this directory includes two examples:

• Using a standard REST API request to prompt the LLM
• Engaging with the LLM via the ollama-python library.

Please find more information to Ollama in the following links:

• https://github.com/ollama/ollama
• https://github.com/ollama/ollama-python

1. Running Ollama with the official container

... follows soon ...

2. Downloading and running Ollama manually

Before beeing able to execute Ollama and run the exaples, you need to download Ollama and make it available to the upcoming workflow steps. Additionally, we use a Python virtual environment, to demonstrate how Ollama can be used via the ollama-python library.

Execute the following instructions ONCE to download Ollama and create the virtual environment:

# Specify the Ollama root directory, where binaries should be placed and where venv should be created, such as:
export OLLAMA_ROOT_DIR=${HOME}/ollama

# initialize environment variables that refer to installation and virtual environment
source set_paths.sh

# Download the Ollama binaries and create the venv
zsh download_and_create_venv.sh

Now you can execute the examples, either in the current shell or by submitting a batch job that runs the examples on a backend node:

# run in current active shell
zsh submit_job_venv.sh

# submit batch job
sbatch submit_job_venv.sh