What is the issue i below code, am getting the below error even after the authentication, what is incorrect in this

[Pavan-Kolachena]

Body

Error :
TypeError: HuggingFaceLLM.init() got an unexpected keyword argument 'HUGGINGFACEHUB_API_TOKEN'

if am removing the token details from below code then am getting authentication error

!pip install llama-index-llms-huggingface
import torch
from llama_index.llms.huggingface import HuggingFaceLLM
import os

Retrieve your token from the environment variable

huggingface_token = os.environ.get("HUGGINGFACEHUB_API_TOKEN")

llm = HuggingFaceLLM(
context_window=4096,
max_new_tokens=256,
generate_kwargs={"temperature": 0.7, "do_sample": False},
system_prompt=system_prompt,
query_wrapper_prompt=query_wrapper_prompt,
tokenizer_name="mistralai/Mistral-7B-Instruct-v0.1",
model_name="mistralai/Mistral-7B-Instruct-v0.1",
device_map="auto",
stopping_ids=[50278, 50279, 50277, 1, 0],
tokenizer_kwargs={"max_length": 4096},
# uncomment this if using CUDA to reduce memory usage
model_kwargs={"torch_dtype": torch.float16},
# Pass the token to the constructor
HUGGINGFACEHUB_API_TOKEN = "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"
)

Guidelines

• I have read and understood this category's guidelines before making this post.

[deep-sengupta]

The HuggingFaceLLM class doesn't accept the HUGGINGFACEHUB_API_TOKEN as a keyword argument directly. Instead, you should authenticate with Hugging Face by setting the token as an environment variable.

Just modify the code----
Set the Hugging Face token as an environment variable in your script (before using HuggingFaceLLM)

import os
import torch
from llama_index.llms.huggingface import HuggingFaceLLM

os.environ["HUGGINGFACEHUB_API_TOKEN"] = "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"

llm = HuggingFaceLLM(
context_window=4096,
max_new_tokens=256,
generate_kwargs={"temperature": 0.7, "do_sample": False},
system_prompt=system_prompt,
query_wrapper_prompt=query_wrapper_prompt,
tokenizer_name="mistralai/Mistral-7B-Instruct-v0.1",
model_name="mistralai/Mistral-7B-Instruct-v0.1",
device_map="auto",
stopping_ids=[50278, 50279, 50277, 1, 0],
tokenizer_kwargs={"max_length": 4096},
# uncomment this if using CUDA to reduce memory usage
model_kwargs={"torch_dtype": torch.float16},
)

Ensure the token is correctly set as an environment variable.
Verify it by printing it before initializing the model:

print(os.environ.get("HUGGINGFACEHUB_API_TOKEN"))

This approach uses the environment variable for authentication, which is the correct method for libraries that rely on external API tokens.