added description for container-based workflow

machine-and-deep-learning/ollama/README.md

@@ -12,15 +12,40 @@ Please find more information to Ollama in the following links:
 - https://github.com/ollama/ollama
 - https://github.com/ollama/ollama-python
 
-## 1. Running Ollama with the official container
+## 0. Prerequisites
 
-... follows soon ...
+To demonstrate how to use Ollama with the `ollama-python` library, you first need to create a Python virtual environment. Run the following command **ONCE**:
+```bash
+# Specify the Ollama root directory, where binaries should be placed and where venv should be created, such as:
+export OLLAMA_ROOT_DIR=${HOME}/ollama
+
+# initialize environment variables that refer to installation and virtual environment
+source set_paths.sh
+
+# create the venv
+zsh create_venv.sh
+```
+
+## 1. Running Ollama
+
+ℹ️ **Note:** Examples here run `ollama serve` and `ollama run` in the background to enable concise demonstrations from a single script or shell. However, official examples also show that these commands can be run in separate shells on the same node instead.
+
+## 1.1. Running Ollama with the official container
+
+Since an Ollama container will be centrally provided on our HPC system **very soon**, you can start using the examples right away, either in your current shell or by submitting a batch job to run them on a backend node:
+```bash
+# run in current active shell
+zsh submit_job_container.sh
+
+# submit batch job
+sbatch submit_job_container.sh
+```
 
-## 2. Downloading and running Ollama manually
+## 1.2. Downloading and running Ollama manually
 
-Before beeing able to execute Ollama and run the exaples, you need to download Ollama and make it available to the upcoming workflow steps. Additionally, we use a Python virtual environment, to demonstrate how Ollama can be used via the `ollama-python` library.
+Before beeing able to execute Ollama and run the examples, you need to download Ollama and make it available to the upcoming workflow steps.
 
-Execute the following instructions **ONCE** to download Ollama and create the virtual environment:
+Execute the following instructions **ONCE** to download Ollama:
 ```bash
 # Specify the Ollama root directory, where binaries should be placed and where venv should be created, such as:
 export OLLAMA_ROOT_DIR=${HOME}/ollama
@@ -28,8 +53,8 @@ export OLLAMA_ROOT_DIR=${HOME}/ollama
 # initialize environment variables that refer to installation and virtual environment
 source set_paths.sh
 
-# Download the Ollama binaries and create the venv
-zsh download_and_create_venv.sh
+# download and extract the binariesthe venv
+zsh download_and_extract.sh
 ```
 
 Now you can execute the examples, either in the current shell or by submitting a batch job that runs the examples on a backend node:

machine-and-deep-learning/ollama/download_and_create_venv.sh → machine-and-deep-learning/ollama/create_venv.sh

@@ -2,12 +2,6 @@
 
 # create required directory
 mkdir -p ${OLLAMA_ROOT_DIR}
-mkdir -p ${OLLAMA_INSTALL_DIR}
-
-# download Ollama binaries
-cd ${OLLAMA_INSTALL_DIR}
-curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
-tar -xzf ollama-linux-amd64.tgz
 
 # create Python virtual
 module load Python

machine-and-deep-learning/ollama/download_and_extract.sh

+#!/usr/bin/zsh
+
+# create required directory and download Ollama binaries
+mkdir -p ${OLLAMA_INSTALL_DIR} && cd ${OLLAMA_INSTALL_DIR}
+curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
+tar -xzf ollama-linux-amd64.tgz
\ No newline at end of file

machine-and-deep-learning/ollama/set_paths.sh

@@ -2,7 +2,12 @@
 
 # path where Ollama binaries will be placed after download and extraction
 export OLLAMA_INSTALL_DIR=${OLLAMA_ROOT_DIR}/install
+
 # path to Python virtual environment
 export OLLAMA_VENV_DIR=${OLLAMA_ROOT_DIR}/venv_ollama
+
+# path to Ollama container image
+export OLLAMA_COINTAINER_IMAGE=${HOME}/ollama/ollama.sif
+
 # extend path to make it executable in the shell
 export PATH="${OLLAMA_INSTALL_DIR}/bin:${PATH}"
\ No newline at end of file

machine-and-deep-learning/ollama/submit_job_container.sh

+#!/usr/bin/zsh
+############################################################
+### Slurm flags
+############################################################
+
+#SBATCH --time=00:15:00
+#SBATCH --partition=c23g
+#SBATCH --nodes=1
+#SBATCH --ntasks-per-node=1
+#SBATCH --cpus-per-task=24
+#SBATCH --gres=gpu:1
+
+############################################################
+### Load modules or software
+############################################################
+
+# specify your Ollama root directory
+export OLLAMA_ROOT_DIR=${HOME}/ollama
+
+# set dependent paths
+source set_paths.sh
+
+# load Python and activate venv
+module load Python
+source ${OLLAMA_VENV_DIR}/bin/activate
+
+############################################################
+### Parameters and Settings
+############################################################
+
+# print some information about current system
+echo "Job nodes: ${SLURM_JOB_NODELIST}"
+echo "Current machine: $(hostname)"
+nvidia-smi
+
+############################################################
+### Execution (Model Training)
+############################################################
+
+# Note: setsid is utilized to gracefully stop processes and subprocesses
+
+# start the container that serves and runs the model
+setsid apptainer exec -e --nv ${OLLAMA_COINTAINER_IMAGE} \
+    bash -c "OLLAMA_HOST=0.0.0.0 ollama serve & sleep 5 && ollama run llama3.2 && tail -f /dev/null" &> log_container.log &
+
+# remember ID of process that has just been started in background
+export proc_id_container=$!
+
+# wait until model is up
+sleep 5
+
+# Example: prompt against LLM via REST API (Note: streaming is typically only useful when using a Chat frontends)
+echo "========== Example REST API =========="
+curl http://localhost:11434/api/generate -d '{"model": "llama3.2", "prompt":"Why is the sky blue?", "stream": false}'
+echo "\n"
+
+# Example: prompt against LLM through ollama-python
+echo "========== Example Python via ollama-python =========="
+python3 ollama-example.py
+
+# cleanup: stop container and model: also kills serve and run processes
+kill -9 -- -${proc_id_container}

machine-and-deep-learning/ollama/submit_job_venv.sh

@@ -37,8 +37,10 @@ nvidia-smi
 ### Execution (Model Training)
 ############################################################
 
+# Note: setsid is utilized to gracefully stop processes and subprocesses
+
 # run server in background and redirect output
-ollama serve &> log_ollama_serve.log &
+setsid ollama serve &> log_ollama_serve.log &
 # remember ID of process that has just been started in background
 export proc_id_serve=$!
 
@@ -61,7 +63,4 @@ echo "========== Example Python via ollama-python =========="
 python3 ollama-example.py
 
 # cleanup: stop model and kill serve and run processes
-ollama stop llama3.2
-kill -9 ${proc_id_serve}
-# kill remaining ollama procs if not already done
-ps aux | grep '[o]llama' | awk '{print $2}' | xargs -r kill -9
+kill -9 -- -${proc_id_serve}