Support filename cleanup as per RFC-6266

[asfaltboy]

I found myself writing custom logic to sanitize a filename that would be downloadable on major browsers and able to open it on any operating system. This value is then provided to the content_disposition_filename attribute of the PrivateStorageDetailView class.

Instead, I think PrivateStorageDetailView._encode_filename_header should take care of it instead. We can do this by stripping permitted tokens as per RFC6266. I found the rfc6266 package on PYPI does exactly that, so one option would be to use it and then call in on build_header with the optional filename_compat agument whenever we encounter an older browser (UA == MSIE enough ?)

While this may seem like a purely selfish act, it would be lovely for us not to think about content disposition, and browser support, and header specifications at all when an awesome storage package such as this can do it for us 😄 !

[Chrescht]

I am also having issues with letters like "é" when uploading a file.
(Apache will throw a 500 error which quite brutal I think.)

[Chrescht]

@vdboor can you please shine some light on this? I'm surprised this is not getting more attention. Or maybe everybody else only uses ascii for their filenames.

[asfaltboy]

Note that _encode_filename_header already provides solutions for some user agents with simple encoding to UTF-8.

We ought to build a list of clients where this doesn't work to support our use case. Sadly I have note noted (and can't recall) my exact use case. @Chrescht by Apache did you mean Apache Benchmark, or Java Apache HttpClient, or something else?

By the way, this is a relatively small project that seems to accept contributions. We can start by writing some (failing) e2e tests with e.g microsoft/playwright (this one allows computing suggested filename on the fly too).

[Chrescht]

@asfaltboy I'm talking about the apache web server. The original error comes from Django not being able to handle the provided filname.

I'm not asking about any user agents. My issue is with upload files named e.g. "carte d'identité", the letter é will cause trouble here because the default encoding won't know how to handle it.
How to deal with this without having to write some custom filename cleaning filter?

[asfaltboy]

Sorry I still don't get it, do you mean you run a Django app behind Apache httpd server? And when a user requests a file download, the response's content-disposition header breaks the http server, or what? Can you describe your use case step-by step please 🙏 ?

[Chrescht]

@asfaltboy of course, here you go:
Yes, I'm running Django behind an apache server but this is about django-private-storage and uploading a file with non-ascii characters:

1. PrivateFleField in my model: id_file = PrivateFileField(upload_to='id/', max_file_size=size_3mb, null=True, blank=True) and add it to a form.
2. upload a PDF file with filename containing the character é as an example: "carte d'identité.pdf"
3. Django throws an UnicodeEncodeError

UnicodeEncodeError at /url_removed/

'ascii' codec can't encode character '\xe9' in position 56: ordinal not in range(128)

Request Method: 	POST
Request URL: 	url_removed
Django Version: 	3.2
Exception Type: 	UnicodeEncodeError
Exception Value: 	

'ascii' codec can't encode character '\xe9' in position 56: ordinal not in range(128)

Python 3.8.5 and Django 3.2

[asfaltboy]

I think we're talking about two different issues, from the above it looks to be that you're having an error during the POST (as you mention while uploading), while my issues is about PrivateStorageDetailView which is the view that serves the download of the private file after it's already been uploaded (it doesn't even implement the post method).

You may want to open a separate issue, and probably ought to include the error traceback to aid debugging.

[Chrescht]

You're right, I opened a new ticket #69.