Normally, when using huggingface to load pretrained model and load data from huggingface, it will check if these models and dataset are available in its cached location. By default, the location is
#for mac & ubuntu, the root location of huggingface data saved
~/.cache/huggingfaceMore details for huggingface environment variables refer to Huggingface environment variables. Sometimes you want to save these models in specific locations. For example, when you develop models & services using huggingface pretrained moldels and then deploy them into clound server API service. Normally for security issue, the server will disable online download models from huggingface. In the case, you need to tell huggingface to load models from specified location and offline download these models and transfer to the location. You need set these environment variables in the bash configure file. The bash file in Mac & Ubuntu as:
#In Mac, bash file is
~/.bash_profile
open and modify the bash file and add the following script
#set huggingface env
export HF_HOME=/Users/abc/hf_home
export HF_HUB_CACHE=$HF_HOME/hub
export HF_ASSETS_CACHE=$HF_HONE/assets
export HF_HUB_OFFLINE=0
#similarly in Ubuntu, bash file is
~/.bashrc
Add the above export as in Mac
For exmaple, when using the stable diffusion to generate image, the stable diffusion 2.1 is loaded like
import torch
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler
model_id = "stabilityai/stable-diffusion-2-1"
# Use the DPMSolverMultistepScheduler (DPM-Solver++) scheduler here instead
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float32)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)If HF_HOME is specified as above example, then you check /Users/abc/hf_home/hub, then you can find the model saved in your specified model as
Then you can copy all HF_HOME to any machine, you only need to setup huggingface environment varibales in new machine. Then when running above stable diffusion model loading, huggingface does not need to download models from remote.
