1.准备模型权重：
https://huggingface.co/tloen/alpaca-lora-7b
2.基座模型
decapoda-research/llama-7b-hf
3.核心代码
BASE_MODEL = os.environ.get("BASE_MODEL",None)
LORA_MODEL = os.environ.get("LORA_MODEL", None)
HF_CHECKPOINT = os.environ.get("HF_CHECKPOINT", None)
tokenizer = LlamaTokenizer.from_pretrained(BASE_MODEL)
base_model = LlamaForCausalLM.from_pretrained(
BASE_MODEL,
load_in_8bit=False,
torch_dtype=torch.float16,
device_map={"": "cpu"},
)
lora_model = PeftModel.from_pretrained(
base_model,
LORA_MODEL,
device_map={"": "cpu"},
torch_dtype=torch.float16,
)
lora_model = lora_model.merge_and_unload()
lora_model.train(False)
# did we do anything?
assert not torch.allclose(first_weight_old, first_weight)
lora_model_sd = lora_model.state_dict()
deloreanized_sd = {
k.replace("base_model.model.", ""): v
for k, v in lora_model_sd.items()
if "lora" not in k
}
LlamaForCausalLM.save_pretrained(
base_model, HF_CHECKPOINT, state_dict=deloreanized_sd, max_shard_size="400MB"
)
4.运行代码结果
先指定环境变量：
export BASE_MODEL=decapoda-research/llama-7b-hf
export LORA_MODEL=tloen/alpaca-lora-7b
export HF_CHECKPOINT=./hf_ckpt_merge
已经合并进去并且导出huggingface的格式：