Added UnicodeDecodeError workaround for modules/llamacpp_model.py

[nclok1405]

Sometimes text generation randomly fails on Notebook and Default tab for me with UnicodeDecodeError at llamacpp_model.py.
This commit handles invalid characters gracefully. It's a workaround (probably not a real fix) but I hasn't noticed any ill side effects and this makes text-generation-webui actually usable for me.
I'm using RTX3060 (driver 535.171.04, CUDA Version: 12.2).

Checklist:

• I have read the Contributing guidelines.

[oobabooga]

Maybe this workaround would be better?

ae86292#diff-fde6e7e6b479c25c1048235a48ebd54438ec1ffd360883651832f2c052b05f71R91

[nclok1405]

Maybe this workaround would be better?

ae86292#diff-fde6e7e6b479c25c1048235a48ebd54438ec1ffd360883651832f2c052b05f71R91

No it doesn't seem fix for me. The same error happens with latest dev branch.

Traceback (most recent call last):
File "/home/myuser/apps/text-generation-webui/modules/callbacks.py", line 61, in gentask
ret = self.mfunc(callback=_callback, *args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/myuser/apps/text-generation-webui/modules/llamacpp_model.py", line 141, in generate
prompt = self.decode(prompt)
^^^^^^^^^^^^^^^^^^^
File "/home/myuser/apps/text-generation-webui/modules/llamacpp_model.py", line 117, in decode
return self.model.detokenize(ids).decode('utf-8')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbc in position 0: invalid start byte

[oobabooga]

I mean, is it better to try to decode as latin in case of exception in the line that you changed, instead of adding the replace argument?