Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ability to specify text size splitter and overlap block size for loaders #27

Closed
converseKarl opened this issue Apr 23, 2024 · 2 comments
Closed

Ability to specify text size splitter and overlap block size for loaders #27

converseKarl opened this issue Apr 23, 2024 · 2 comments
Assignees
@adhityan
Labels
enhancement New feature or request

Comments

@converseKarl
Copy link

Be able to set the text split size for the text splitter server
@converseKarl converseKarl changed the title embedding text size blocks and overlap as part of settings Ability to specify text size splitter and overlap block size for loaders Apr 23, 2024
@adhityan
Copy link
Collaborator

Hey @converseKarl, thanks for pointing out this could be useful. I am in the middle of adding support for Docx files and a few other topics. Would you be interested in sending in a PR for this? Otherwise I will get to it after the current set of tasks.

@adhityan adhityan added the enhancement New feature or request label Apr 24, 2024
@adhityan
Copy link
Collaborator

Added support for this in version 0.0.74. Two optional parameters (chunkSize and chunkOverlap) exist in all loaders to configure this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adhityan adhityan

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@adhityan @converseKarl