加载报错

[itbithubman]

System Info / 系統信息

conda python=3.11

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

INFO:sat:[RANK 0] > initializing model parallel with size 2
[Language processor version]: chat
[rank1]: Traceback (most recent call last):
[rank1]: File "/home/tt/CogVLM/basic_demo/cli_demo_sat.py", line 161, in
[rank1]: main()
[rank1]: File "/home/tt/CogVLM/basic_demo/cli_demo_sat.py", line 56, in main
[rank1]: tokenizer = llama2_tokenizer(args.local_tokenizer, signal_type=language_processor_version)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/tt/CogVLM/utils/utils/language.py", line 37, in llama2_tokenizer
[rank1]: tokenizer = LlamaTokenizer.from_pretrained(tokenizer_path)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/tt/anaconda3/envs/coga/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2163, in from_pretrained
[rank1]: return cls._from_pretrained(
[rank1]: ^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/tt/anaconda3/envs/coga/lib/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2397, in _from_pretrained
[rank1]: tokenizer = cls(*init_inputs, **init_kwargs)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/tt/anaconda3/envs/coga/lib/python3.11/site-packages/transformers/models/llama/tokenization_llama.py", line 171, in init
[rank1]: self.sp_model = self.get_spm_processor(kwargs.pop("from_slow", False))
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: File "/home/tt/anaconda3/envs/coga/lib/python3.11/site-packages/transformers/models/llama/tokenization_llama.py", line 204, in get_spm_processor
[rank1]: model = model_pb2.ModelProto.FromString(sp_model)
[rank1]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank1]: google.protobuf.message.DecodeError: Error parsing message

Expected behavior / 期待表现

我是在modelscope上下载的vicuna-7b-v1.5