Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

修复某些提问导致rerank张量扩展大小不匹配的问题 #4331

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

xldistance
Copy link
Contributor

可以测试模型提问rerank内容,原版rerank会出错:

对代码进行简洁和高效的改写,并优化逻辑。写出改进的完整代码后详细解释修改后的代码与原始代码之间的不同之处,包括任何逻辑优化和结构改进。在此过程中使用简体中文进行沟通,并确保回答不会有重复或不相关的内容。def get_local_datetime(timestamp:Union[str, float,int],hours:int = 8):
    """
    1.把timestamp或者str的datetime转换时区
    2.hours = 8:转化datetime 0时区到东八区,hours = 0:东八区datetime转化
    """
    if isinstance(timestamp,str):
        # 去除时区字符串
        if "+" in timestamp:
            timestamp = timestamp.split("+")[0]
        if "Z" in timestamp:
            if "." in timestamp:
                local_time = pd.to_datetime(timestamp,format="%Y-%m-%dT%H:%M:%S.%fZ")
            else:
                local_time = pd.to_datetime(timestamp,format="%Y-%m-%dT%H:%M:%SZ")
        elif "T" in timestamp:
            if "." in timestamp:
                local_time = pd.to_datetime(timestamp,format="%Y-%m-%dT%H:%M:%S.%f")
            else:
                local_time = pd.to_datetime(timestamp,format="%Y-%m-%dT%H:%M:%S")
        else:
            if "." in timestamp:
                local_time = pd.to_datetime(timestamp,format="%Y-%m-%d %H:%M:%S.%f")
            else:
                local_time = pd.to_datetime(timestamp,format="%Y-%m-%d %H:%M:%S")
    elif isinstance(timestamp,float):
        local_time = pd.to_datetime(timestamp,unit = "s")
    elif isinstance(timestamp,int):
        #秒时间戳
        if len(str(timestamp)) == 10:
            local_time = pd.to_datetime(timestamp,unit = "s")
        #毫秒时间戳
        elif len(str(timestamp)) == 13:
            local_time = pd.to_datetime(timestamp,unit = "ms")
        #微妙时间戳
        elif len(str(timestamp)) == 16:
            local_time = pd.to_datetime(timestamp,unit = "us")
        #纳秒时间戳
        elif len(str(timestamp)) == 19:
            local_time = pd.to_datetime(timestamp,unit = "ns")
    return (local_time+timedelta(hours=hours)).tz_localize(TZ_INFO)  #pandas timestamp添加时区

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Jun 26, 2024
@imClumsyPanda
Copy link
Collaborator

您好,首先感谢您对项目发起 PR,稍微晚点开发组会对本 PR 进行 review。

另外,0.3.x 版本中项目在逐渐向模型都通过 api 形式接入的方式进行转变,因此当前 reranker 方式可能会发生调整。另外您当前代码中涉及引入 torch 和 llama-index 依赖库,可能需要考虑减少对上述两个库的依赖,或调整 poetry 中依赖要求。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants