Install Meta Llama 3.1 Locally and Test - Hands-on Tutorial

 This video shows how to locally install Meta Llama 3.1 8B model and test it on various benchmarks.

Code:

conda create -n newllama python=3.11 -y && conda activate newllama

pip install torch

pip install --upgrade transformers

pip install accelerate huggingface_hub

huggingface-cli login   # get the key from huggingface.co

conda install jupyter -y

pip uninstall charset_normalizer -y

pip install charset_normalizer

jupyter notebook

import transformers

import torch

model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"   # make sure to accept the terms on model card.

pipeline = transformers.pipeline(

    "text-generation",

    model=model_id,

    model_kwargs={"torch_dtype": torch.bfloat16},

    device_map="auto",

)

messages = [

    {"role": "system", "content": "You are helpful assistant!"},

    {"role": "user", "content": "What is the smallest country in the world?"},

]

outputs = pipeline(

    messages,

    max_new_tokens=256,

)

from IPython.display import Markdown, display

output_text = outputs[0]["generated_text"][-1]['content']

display(Markdown(output_text))