o S"g>P@sNddlZddlmZddlZddlmZmZmZmZm Z m Z m Z ddl m Z ddl mZmZmZddlmZddlmZmZmZddlmZmZdd lmZdd lmZdd lmZdd l m!Z!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*ddl+m,Z,ddl-m.Z.m/Z/m0Z0dZ1e2e3Z4GdddeZ5e dddGdddeZ6e6Z7dS)N)BytesIO)AnyCallableDictListOptionalSequenceUnion) deprecated)AsyncInferenceClientInferenceClient model_info) ModelInfo)DEFAULT_EMBED_BATCH_SIZE BaseEmbedding Embedding)Field PrivateAttr)CallbackManager)Pooling)MultiModalEmbedding) get_cache_dirinfer_torch_device)#DEFAULT_HUGGINGFACE_EMBEDDING_MODEL format_query format_text!get_query_instruct_for_model_name get_text_instruct_for_model_name) ImageType)SentenceTransformer)retrystop_after_attemptwait_exponentialic#seZdZUdZeedddZeed<edddZ e ed <ed d d Z e e ed <edd d Ze e ed<edd d Ze e ed<eZeed<eZe ed<eZe ed<eZe ee ed<eddd d d ddded dd d dd fde de e de de ed e e de e d e de ede edede e de de e d e ed!e d"e ee f fd#d$ Zed%e fd&d'Zeed(e d)d*d+d,dd- dKd.ee!e e"fd/e e d%eee#fd0d1Z$ dKd.ee!e e"fd/e e d%eee#fd2d3Z%d4e d%ee#fd5d6Z&d4e d%ee#fd7d8Z'd9e d%ee#fd:d;Z(d9e d%ee#fdee d%eee#fd?d@Z*dAe+d%ee#fdBdCZ,dAe+d%ee#fdDdEZ-dFee+d%eee#fdGdHZ.dFee+d%eee#fdIdJZ/Z0S)LHuggingFaceEmbeddinga HuggingFace class for text and image embeddings. Args: model_name (str, optional): If it is a filepath on disc, it loads the model from that path. If it is not a path, it first tries to download a pre-trained SentenceTransformer model. If that fails, tries to construct a model from the Hugging Face Hub with that name. Defaults to DEFAULT_HUGGINGFACE_EMBEDDING_MODEL. max_length (Optional[int], optional): Max sequence length to set in Model's config. If None, it will use the Model's default max_seq_length. Defaults to None. query_instruction (Optional[str], optional): Instruction to prepend to query text. Defaults to None. text_instruction (Optional[str], optional): Instruction to prepend to text. Defaults to None. normalize (bool, optional): Whether to normalize returned vectors. Defaults to True. embed_batch_size (int, optional): The batch size used for the computation. Defaults to DEFAULT_EMBED_BATCH_SIZE. cache_folder (Optional[str], optional): Path to store models. Defaults to None. trust_remote_code (bool, optional): Whether or not to allow for custom models defined on the Hub in their own modeling files. This option should only be set to True for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine. Defaults to False. device (Optional[str], optional): Device (like "cuda", "cpu", "mps", "npu", ...) that should be used for computation. If None, checks if a GPU can be used. Defaults to None. callback_manager (Optional[CallbackManager], optional): Callback Manager. Defaults to None. parallel_process (bool, optional): If True it will start a multi-process pool to process the encoding with several independent processes. Great for vast amount of texts. Defaults to False. target_devices (Optional[List[str]], optional): PyTorch target devices, e.g. ["cuda:0", "cuda:1", ...], ["npu:0", "npu:1", ...], or ["cpu", "cpu", "cpu", "cpu"]. If target_devices is None and CUDA/NPU is available, then all available CUDA/NPU devices will be used. If target_devices is None and CUDA/NPU is not available, then 4 CPU devices will be used. This parameter will only be used if `parallel_process = True`. Defaults to None. num_workers (int, optional): The number of workers to use for async embedding calls. Defaults to None. **model_kwargs: Other model kwargs to use tokenizer_name (Optional[str], optional): "Deprecated" pooling (str, optional): "Deprecated" model (Optional[Any], optional): "Deprecated" tokenizer (Optional[Any], optional): "Deprecated" Examples: `pip install llama-index-embeddings-huggingface` ```python from llama_index.core import Settings from llama_index.embeddings.huggingface import HuggingFaceEmbedding # Set up the HuggingFaceEmbedding class with the required model to use with llamaindex core. embed_model = HuggingFaceEmbedding(model_name = "BAAI/bge-small-en") Settings.embed_model = embed_model # Or if you want to Embed some text separately embeddings = embed_model.get_text_embedding("I want to Embed this text!") ``` zMaximum length of input.r)default descriptiongt max_lengthTzNormalize embeddings or not.r$r% normalizez%Instruction to prepend to query text.N)r%r$query_instructionzInstruction to prepend to text.text_instructionz$Cache folder for Hugging Face files. cache_folder_model_device_parallel_process_target_devicesr F model_nametokenizer_namepoolingmodel tokenizerembed_batch_sizetrust_remote_codedevicecallback_managerparallel_processtarget_devicesc s| pt} | p t} d|fd| fd|fd|ffD]\}}|dkr't|dq|dur0tdt|f| | | |p;t||p@t|dd |}|rN||_n|j}tj| ||||||d | |_ ||_ ||_ ||_ dS) Nr4r5r3r2r z4 is deprecated. Please remove it from the arguments.z+The `model_name` argument must be provided.)querytext)r8r,r7prompts)r6r9r1r'r)r*r+) rr ValueErrorrrrmax_seq_lengthsuper__init__r.r-r/r0)selfr1r2r3r'r*r+r)r4r5r6r,r7r8r9r:r; model_kwargsvariablevalue __class__i/mnt/skqttb/ctump_chatbot/chatbot/lib/python3.10/site-packages/llama_index/embeddings/huggingface/base.pyrBwsX       zHuggingFaceEmbedding.__init__returncCdS)Nr#rIclsrIrIrJ class_namezHuggingFaceEmbedding.class_name ) multiplierminmax)stopwaitreraiseinputs prompt_namec Csz1|jr!|jj|jd}|jj|||j||jd}|jj|dn |jj||j||jd}| WSt yF}z t d|d}~ww)a/ Generates embeddings with retry mechanism. Args: inputs: List of texts or images to embed prompt_name: Optional prompt type Returns: List of embedding vectors Raises: Exception: If embedding fails after retries )r;)pool batch_sizer\normalize_embeddings)r])r^r\r_zEmbedding attempt failed: N) r/r-start_multi_process_poolr0encode_multi_processr6r)stop_multi_process_poolencodetolist Exceptionloggerwarning)rCr[r\r]emberIrIrJ_embed_with_retrys2 z&HuggingFaceEmbedding._embed_with_retrycCs |||S)a Generates Embeddings with input validation and retry mechanism. Args: sentences: Texts or Sentences to embed prompt_name: The name of the prompt to use for encoding Returns: List of embedding vectors Raises: ValueError: If any input text is invalid Exception: If embedding fails after retries )rj)rCr[r\rIrIrJ_embeds zHuggingFaceEmbedding._embedr<cC|j|gdddS)z Generates Embeddings for Query. Args: query (str): Query text/sentence Returns: List[float]: numpy array of embeddings r<r\rrkrCr<rIrIrJ_get_query_embedding z)HuggingFaceEmbedding._get_query_embeddingct|j|IdHS)z Generates Embeddings for Query Asynchronously. Args: query (str): Query text/sentence Returns: List[float]: numpy array of embeddings N)asyncio to_threadrprorIrIrJ_aget_query_embedding z*HuggingFaceEmbedding._aget_query_embeddingr=crr)z Generates Embeddings for text Asynchronously. Args: text (str): Text/Sentence Returns: List[float]: numpy array of embeddings N)rsrt_get_text_embeddingrCr=rIrIrJ_aget_text_embeddingrvz)HuggingFaceEmbedding._aget_text_embeddingcCrl)z Generates Embeddings for text. Args: text (str): Text/sentences Returns: List[float]: numpy array of embeddings r=rmrrnrxrIrIrJrw(rqz(HuggingFaceEmbedding._get_text_embeddingtextscCs|j|ddS)z Generates Embeddings for text. Args: texts (List[str]): Texts / Sentences Returns: List[List[float]]: numpy array of embeddings r=rmrnrCrzrIrIrJ_get_text_embeddings4s z)HuggingFaceEmbedding._get_text_embeddings img_file_pathcCs||gdS)z Generate embedding for an image.rrnrCr}rIrIrJ_get_image_embedding@sz)HuggingFaceEmbedding._get_image_embeddingc ||S)z/Generate embedding for an image asynchronously.)rr~rIrIrJ_aget_image_embeddingDs z*HuggingFaceEmbedding._aget_image_embeddingimg_file_pathscCs ||S)z(Generate embeddings for multiple images.rnrCrrIrIrJ_get_image_embeddingsHs z*HuggingFaceEmbedding._get_image_embeddingscr)z7Generate embeddings for multiple images asynchronously.)rrrIrIrJ_aget_image_embeddingsNs z+HuggingFaceEmbedding._aget_image_embeddings)N)1__name__ __module__ __qualname____doc__rDEFAULT_HUGGINGFACE_LENGTHr'int__annotations__r)boolr*rstrr+r,rr-rr.r/r0rrrrrrB classmethodrOr r!r"r rfloatrjrkrpruryrwr|rrrrr __classcell__rIrIrGrJr#&s =      D   .          r#zDeprecated in favor of `HuggingFaceInferenceAPIEmbedding` from `llama-index-embeddings-huggingface-api` which should be used instead.always)actioncs2eZdZUdZeejddZeee d<edddZ ee e d<edddZ ee e d <edd dZ ee e d <edd dZee edfe d <edddZeee d<edddZee e fe d<edddZee e fe d<edddZee e d<eZde d<eZde d<eZde d<dee effddZdeddffd d! Zde ddfd"d#Zdedd$fd%d&Ze de fd'd(Z!d)e de"fd*d+Z#d,e$e de%e"fd-d.Z&d/e de"fd0d1Z'd)e de"fd2d3Z(d,e%e de%e"fd4d5Z)d/e de"fd6d7Z*d)e de"fd8d9Z+d,e%e de%e"fd:d;Z,Z-S)< HuggingFaceInferenceAPIEmbeddingz Wrapper on the Hugging Face's Inference API for embeddings. Overview of the design: - Uses the feature extraction task: https://huggingface.co/tasks/feature-extraction z?Pooling strategy. If None, the model's default pooling is used.r(r3Nz.Instruction to prepend during query embedding.r*z-Instruction to prepend during text embedding.r+z8Hugging Face model name. If None, the task will be used.r1uHugging Face token. Will default to the locally saved token. Pass token=False if you don’t want to send your token to the server.tokenzThe maximum number of seconds to wait for a response from the server. Loading a new model in Inference API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.timeoutzAdditional headers to send to the server. By default only the authorization and user-agent headers are sent. Values in this dictionary will override the default values.headersz)Additional cookies to send to the server.cookieszhOptional task to pick Hugging Face's recommended model, used when model_name is left as default of None.taskr _sync_clientr _async_clientzCallable[..., ModelInfo]_get_model_inforKcCs|j|j|j|j|jdS)zAExtract the Hugging Face InferenceClient construction parameters.)r4rrrr)r1rrrrrCrIrIrJ_get_inference_client_kwargss z=HuggingFaceInferenceAPIEmbedding._get_inference_client_kwargskwargsc s|ddur+|dd}tj|d|d<td|dd|dt|dd d tjd i|td i||_ t d i||_ t |_ dS) z\ Initialize. Args: kwargs: See the class-level Fields. r1Nr)rz'Using Hugging Face's recommended model z given task .T)flushrI)getr get_recommended_modelrfdebugprintrArBrrr rr r)rCrrrGrIrJrBs   z)HuggingFaceInferenceAPIEmbedding.__init__c Csn|jjdd}z|j||vrtd|jdWdSty6}ztd|dt|d|d}~ww) z Confirm the contained model_name is deployed on the Inference API service. Args: task: Hugging Face task to check within. A list of all tasks can be found here: https://huggingface.co/tasks all) frameworksz1The Inference API service doesn't have the model z deployed.z Input task z not in possible tasks rN)rlist_deployed_modelsr1r?KeyErrorlistkeys)rCr all_modelsexcrIrIrJvalidate_supporteds"z3HuggingFaceInferenceAPIEmbedding.validate_supportedrcKs|j|jfi|S)z4Get metadata on the current model from Hugging Face.)rr1)rCrrIrIrJget_model_infosz/HuggingFaceInferenceAPIEmbedding.get_model_infocCrL)NrrIrMrIrIrJrOrPz+HuggingFaceInferenceAPIEmbedding.class_namer=c s|j|IdH}t|jdkr|S|jdd}t|jdkr&|Sz||WStyD}z td|j d|d}~ww)NrRr)axiszPooling is required for zG because it returned a > 1-D value, please specify pooling as not None.) rfeature_extractionlenshaperdsqueezer3 TypeErrorr?r1)rCr= embeddingrrIrIrJ_async_embed_singles"  z4HuggingFaceInferenceAPIEmbedding._async_embed_singlerzcs$fdd|D}tj|IdHS)z Embed a sequence of text, in parallel and asynchronously. NOTE: this uses an externally created asyncio event loop. csg|]}|qSrI)r.0r=rrIrJ szFHuggingFaceInferenceAPIEmbedding._async_embed_bulk..N)rsgatherrCrztasksrIrrJ_async_embed_bulksz2HuggingFaceInferenceAPIEmbedding._async_embed_bulkr<cCt||S)z~ Embed the input query synchronously. NOTE: a new asyncio event loop is created internally for this. )rsrunrurorIrIrJrpz5HuggingFaceInferenceAPIEmbedding._get_query_embeddingcCr)z} Embed the text query synchronously. NOTE: a new asyncio event loop is created internally for this. )rsrryrxrIrIrJrwrz4HuggingFaceInferenceAPIEmbedding._get_text_embeddingcsRtzfdd|D}t|Wnwdd|DS)z Embed the input sequence of text synchronously and in parallel. NOTE: a new asyncio event loop is created internally for this. csg|] }|qSrI) create_taskryrlooprCrIrJrszIHuggingFaceInferenceAPIEmbedding._get_text_embeddings..cSsg|]}|qSrI)result)rrrIrIrJrs)rsnew_event_looprun_until_completerYcloserrIrrJr|s z5HuggingFaceInferenceAPIEmbedding._get_text_embeddingsc |jt||j|jdIdHSN)r=)rrr1r*rorIrIrJru  z6HuggingFaceInferenceAPIEmbedding._aget_query_embeddingcrr)rrr1r+rxrIrIrJryrz5HuggingFaceInferenceAPIEmbedding._aget_text_embeddingcs"jfdd|DdIdHS)Ncsg|] }t|jjqSrI)rr1r+rrrIrJrszJHuggingFaceInferenceAPIEmbedding._aget_text_embeddings..)rz)rr{rIrrJ_aget_text_embeddingss   z6HuggingFaceInferenceAPIEmbedding._aget_text_embeddings).rrrrrrCLSr3rrr*rr+r1rr rrrrrrrrrrrrrrBrrrrOrrrrrrprwr|ruryrrrIrIrGrJrUsh   "r)8rsiorloggingtypingrrrrrrr r huggingface_hubr r r huggingface_hub.hf_apir%llama_index.core.base.embeddings.baserrr llama_index.core.bridge.pydanticrrllama_index.core.callbacksr*llama_index.embeddings.huggingface.poolingr,llama_index.core.embeddings.multi_modal_baserllama_index.core.utilsrr(llama_index.embeddings.huggingface.utilsrrrrrllama_index.core.schemarsentence_transformersrtenacityr r!r"r getLoggerrrfr#r!HuggingFaceInferenceAPIEmbeddingsrIrIrIrJs: $        1E