Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Extension silently overwrites models with the same filename. #288

Open
NonaSuomi opened this issue May 26, 2024 · 3 comments
Open
Labels
bug Something isn't working WIP Being worked on

Comments

@NonaSuomi
Copy link

Describe the bug.

When attempting to download, if a file with the same name already exists in the designated folder, the extension will simply overwrite the pre-existing file. Given that it is particularly common for creators to just use "artistname.ext" with style LoRA's, for example, this can cause significant problems.

Steps to reproduce the problem.

  1. Have a file in the folder that Browser+ will save to
  2. Download a file with the same filename

Expected behavior

Prompt the user for how to handle the file conflict and/or rename the new file being downloaded. Ideally, it could prepend something unique such as the the the model's hash, or the ID portion of the model's AIR onto the downloaded filename.

System info

  • Extension version: commit 5ab99d5
  • OS: Win10 22H2
  • SD-WebUI version: 1.9.3
  • Python: 3.10.6

Console logs

N/A

Additional information

No response

@NonaSuomi NonaSuomi added the bug Something isn't working label May 26, 2024
@BlafKing
Copy link
Owner

Thanks for reporting this issue!

I'm not exactly sure how this could occur, since the extension checks if each model from the search results is installed by doing the following:

  1. It first looks for a saved hash, which gets saved if you download a model with this extension so that it can reliably detect it as being installed later.
  2. If a hash has not been found, it then tries to find a matching filename compared to the filename of the selected model.

So, when you attempt to download a model with the same filename as another model, it should show up as already being installed and because of that it should not be downloadable.

Could you please clarify in more detail which model(s) you had saved and which you tried to download? as I'm not completely sure how this issue could occur.

Regardless, I'll add an extra check to ensure that a model does not get overwritten. I'll append something unique to the filename, as you suggested. 👍🏻

@BlafKing BlafKing added the WIP Being worked on label May 28, 2024
@NonaSuomi
Copy link
Author

Okay, so I've done a little more poking around and have found that there's two presumably-related issues that I'm running into here:

To begin with, some of the checkpoints, LoRAs, etc. I have in my models directory are from sites other than CivitAI. When I installed the extension I first went to the settings and input my API key, then I went to the extension tab and the "update models" subtab, checked the "All" box and hit "Load models" to let it scan my files to match what it could, then ran "update info & tags" followed by "update model preview" operations. From there I went to the Browser subtab.

To begin with, I checked the console output for a model that it couldn't find a match for, and tried searching for a CivitAI model of the same artist- in this case Afrobull. The model I had already can be found on PixAI as model 1619930580734245232. It is 72MB and has a CRC32 of 3DFADF71. The version I downloaded with the extension was CivitAI model 6285, which is 72MB and has a CRC32 of 56D140A2. Something that might be worth noting here is that for some reason the hashes that CivitAI displays do not match what I get when checking the downloaded files in either 7zip or in hashtab. Regardless, from here things proceeded as my original post- I told the extension to download model 6285, and when I headed back to txt2img, I noticed that my old LoRA was missing, and found that the file had been overwritten.


The next issue, which I assume is probably related, is that if a model was downloaded directly from within the extension itself, it will show any other model with a matching filename as installed as well. You can see this using the same model above (6285). It uses the filename "Afrobull.safetensors" which is the exact same filename used by model (442032). If you download one through the extension, the other one will then also marked as [Installed] in the version field, and you won't have the option to download the second model without hitting "Delete model" first.

Another odd thing, and this appears to get to the heart of it, but this filename-matching appears to be case-sensitive, which can cause the extension to overwrite its own downloads. Again, this can be seen using the Afrobull models as an example- model (348816) has two versions, and v1.0 uses the filename "afrobull.safetensors" note the lower-case 'a'. If either model (6285) or (442032) has already been installed, model (348816) will not be detected as installed, but when downloaded it will overwrite the preexisting model file.


Given this info, it seems that the extension only looked for a case-sensitive filename match, and when the file I already had was a case-INsensitive match, it was overwritten. I'm not sure if this is perhaps due to some misconfiguration on my part, but on my install at least it doesn't appear to be doing any kind of hashing at the point of model download/installation.

@onyxlee
Copy link

onyxlee commented Jun 20, 2024

We need a different identifier for each model. The original file name, the lora name can all be duplicated. Now, when the file with the same name exists, ALL the other loras with the same name all appear to be installed.
This issue has been there for a very very long time.
Hope you can fix this. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working WIP Being worked on
Projects
None yet
Development

No branches or pull requests

3 participants