How to Install AutoCoder on Google Colab in GGUF Format

 This video shows a step-by-step process to install AutoCoder in GGUF format on Google Colab for free.

Code:

!wget https://huggingface.co/ukung/AutoCoder_S_6.7B-GGUF/resolve/main/AutoCoder_S_6.7B-q5_k_m.gguf?download=true

import torch

!pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama(

      model_path="/content/AutoCoder_S_6.7B-q5_k_m.gguf",

      n_gpu_layers=32,

      seed=1337,

      n_ctx=2048,

)

output = llm(

      "Q: write a script to say hello? A: ",

      max_tokens=1500,

      stop=["Q:", "\n"],

      echo=True

)

print(output['choices'][0])