Run LoRA Adapter of Any Model with Ollama

This video is a step-by-step tutorial to create integrate LoRA adapters of models in Ollama.

Code:

conda create -n lora python=3.11 -y && conda activate lora

pip install huggingface_hub

huggingface-cli login            

huggingface-cli download pacozaa/tinyllama-alpaca-lora

https://github.com/anna-chepaikina/llama-cpp.git && cd llama-cpp

pip install -r requirements.txt

python3 convert-lora-to-ggml.py /home/Ubuntu/.cache/huggingface/hub/models--pacozaa--tinyllama-alpaca-lora/snapshots/10fcfd76a34f5b2b18a8d2dfc4372f9d4e0f01cd

ollama pull tinyllama

touch Modelfile

============================

FROM tinyllama:latest

ADAPTER /home/Ubuntu/.cache/huggingface/hub/models--pacozaa--tinyllama-alpaca-lora/snapshots/10fcfd76a34f5b2b18a8d2dfc4372f9d4e0f01cd/ggml-adapter-model.bin

TEMPLATE """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

{{ if .System }}### Instruction:

{{ .System }}{{ end }}

{{ if .Prompt }}### Input:

{{ .Prompt }}{{ end }}

### Response:

"""

SYSTEM """You are Alexander the Great, the world conqueror."""

PARAMETER stop "### Response:"

PARAMETER stop "### Instruction:"

PARAMETER stop "### Input:"

PARAMETER stop "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request."

PARAMETER num_predict 200

=================

ollama create alextinyllama -f ./Modelfile

ollama run alextinyllama

/set system You're Aristotle, the philosopher. Answer using philosophical terms.