Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repeatedly sends numbers nonstop #32

Open
psycho-baller opened this issue Dec 18, 2023 · 1 comment
Open

Repeatedly sends numbers nonstop #32

psycho-baller opened this issue Dec 18, 2023 · 1 comment

Comments

@psycho-baller
Copy link

psycho-baller commented Dec 18, 2023

Hey guys, first of all thank you guys for putting the time to build this, it's been really helpful for working with LLMs.

I faced this issue while using it for working with llama_index:

> How can I store a pdf into a vector store with llama_index

  To store a PDF into a vector store with LlamaIndex, you would first need to convert the PDF into a format that LlamaIndex can understand. LlamaIndex
  works with text data, so you would need to extract the text from the PDF. Python has several libraries that can do this, such as PyPDF2 or PDFMiner.

  Once you have the text data, you can create a Document object in LlamaIndex. A Document object represents a piece of text that can be indexed and
  queried. You can create a Document object using the Document class in LlamaIndex. The Document class takes two arguments: text and metadata. The text
  argument is the text to be indexed, and the metadata argument is a dictionary of additional information about the text.

  After creating the Document object, you can add it to a VectorStoreIndex. The VectorStoreIndex class in LlamaIndex represents an index of text that
  can be queried. You can add documents to the index using the from_documents class method. This method takes a list of Document objects and an optional
  show_progress argument. If show_progress is set to True, a progress bar will be displayed during index construction.

  Here is an example of how you might do this:


   from llama_index import VectorStoreIndex, Document
   ...
  Sources:
   ...
   15 https://docs.llamaindex.ai/en/stable/examples/low_level/retrieval.html#load-into-vector-store●


Run code? (y/n): n
> What if the pdf is from the internet, I don't want to download it locally, is there a way to make Llama_index to read that pdf from the internet and create a vector database from it?

  Based on the provided contexts, LlamaIndex doesn't directly support loading PDFs from the internet. The examples given in the documentation involve
  downloading the PDF to a local directory first, then loading it into
  LlamaIndex[1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]...

It kept on sending numbers, it reached [514] then I decided to stop it.

This might not be an issue with your code, but just wanted to let y'all know.

Also one Question I have is, does Context have context to the previous Qs asked my me. I'm guessing not yet, right?

@adrwz
Copy link
Collaborator

adrwz commented Dec 18, 2023

Thanks for raising this issue! Looking into it.

Also one Question I have is, does Context have context to the previous Qs asked my me. I'm guessing not yet, right?

It should! One note: we've seen that retrievals from the library embeddings override past messages especially with gpt-4 (with 8k tokens). Consider using gpt-4-1106-preview (128k) for better results here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants