How to download multiple files with from_pretrained?

[lidj22]

I have no problem downloading single-file models, but for larger ones like llama3 70B Q6_K, they are split to multiple files.

When I run

import llama_cpp
llm = llama_cpp.Llama.from_pretrained("bartowski/Smaug-Llama-3-70B-Instruct-32K-GGUF", "*Q6_K.gguf")

I get the error

huggingface_hub.utils._errors.EntryNotFoundError: 404 Client Error. (Request ID: Root=1-6690e5d5-4978736529503d587c114cbc;e2cb1a51-9148-40e0-8856-28c1397c7cb9)

Entry Not Found for url: https://huggingface.co/bartowski/Smaug-Llama-3-70B-Instruct-32K-GGUF/resolve/main/Smaug-Llama-3-70B-Instruct-32K-Q6_K.gguf.

I think this is because "Q6_K.gguf" corresponds to the folder containing the models. The folder structure is

Smaug-Llama-3-70B-Instruct-32K-Q6_K.gguf/
    Smaug-Llama-3-70B-Instruct-32K-Q6_K-00001-of-00002.gguf
    Smaug-Llama-3-70B-Instruct-32K-Q6_K-00002-of-00002.gguf

So I ran

import llama_cpp
llama_cpp.Llama.from_pretrained("bartowski/Smaug-Llama-3-70B-Instruct-32K-GGUF", "Smaug-Llama-3-70B-Instruct-32K-GGUF/Smaug-Llama-3-70B-Instruct-32K-Q6_K.gguf/*.gguf")

but this time I get a different error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/miniconda3/lib/python3.12/site-packages/llama_cpp/llama.py", line 2052, in from_pretrained
    raise ValueError(
ValueError: No file found in bartowski/Smaug-Llama-3-70B-Instruct-32K-GGUF that match Smaug-Llama-3-70B-Instruct-32K-GGUF/Smaug-Llama-3-70B-Instruct-32K-Q6_K.gguf/*.gguf

Available Files:
["Smaug-Llama-3-70B-Instruct-32K-Q5_K_L.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q6_K.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q8_0.gguf", ".gitattributes", "README.md", "Smaug-Llama-3-70B-Instruct-32K-IQ1_M.gguf", "Smaug-Llama-3-70B-Instruct-32K-IQ2_M.gguf", "Smaug-Llama-3-70B-Instruct-32K-IQ2_XS.gguf", "Smaug-Llama-3-70B-Instruct-32K-IQ2_XXS.gguf", "Smaug-Llama-3-70B-Instruct-32K-IQ3_M.gguf", "Smaug-Llama-3-70B-Instruct-32K-IQ3_XXS.gguf", "Smaug-Llama-3-70B-Instruct-32K-IQ4_XS.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q2_K.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q2_K_L.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q3_K_M.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q3_K_S.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q3_K_XL.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q4_K_L.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q4_K_M.gguf", "Smaug-Llama-3-70B-Instruct-32K-Q5_K_M.gguf", "Smaug-Llama-3-70B-Instruct-32K.imatrix"]

I know I can wget the files and cat them together, but I wanted to check if the project has this feature to begin with?

[tylerebowers]

Normally you can specifiy additional_files= as a parameter, though there seems to be an issue with this currently.