Support for PQC algs

[iyanmv]

Is your feature request related to a problem? Please describe.
I would to try the new PQC algs to generate PDF signatures

Describe the solution you'd like
I would like to work on this, but probably I would need to get some hints in what direction this could be done based on your experience with PDF signatures and other Python libraries.

Describe alternatives you've considered
liboqs is a C library developed by the Open Quantum Safe project that provides a common API to try the new PQC algorithms for KEM and DS. There is also an official Python wrapper for the library and a provider for OpenSSL 3.x. This provider can be used to issue certificates with the new algs with the exactly same commands as traditional algs. For example, this command can be used to generated a self-signed certificate using the hybrid ML-DSA 87 (Dilithium5) + Curve448 mode:

openssl req -provider default -provider oqsprovider \
    -x509 -newkey mldsa87_ed448 -days 365 \
    -keyout root_ca.key -out root_ca_cert.pem

And OpenSSL can also be used to generate CMS signatures. For example, with:

openssl cms -provider default -provider oqsprovider \
    -in message.txt -sign -signerroot_ca.pem -inkey root_ca.key \
    -md sha512 -text -out message.sig

I wonder how difficult it would be to integrate pyHanko with liboqs.

Additional context
Add any other context or screenshots about the feature request here.

When I try to use the CLI to sign using this hybrid certificate I get the following error:

2024-04-16 14:41:16,670 - pyhanko.sign.signers.pdf_cms - ERROR - Could not load cryptographic material
2024-04-16 14:41:16,670 - cli - ERROR - Generic processing error.

So after checking that module I guess the relevant packages used that implement the signature algorithms are asn1crypto and pyca/cryptography.

So I guess the questions is: would these packages need to support PQC algs, or is it possible to integrate pyHanko directly with liboqs. Perhaps this could be behind a --experimental flag?

[MatthiasValvekens]

You know, I've played with that idea myself a few years ago, but I ended up not doing anything with it for lack of time.

I think this is worth pursuing, but there are a few things you have to know:

• asn1crypto is only used for constructing CMS objects & parsing certificates in pyHanko, it doesn't actually do any cryptographic operations. Having said that, it is used to identify algorithms as well, so it needs to know about the OIDs used to identify PQC signing algorithms. When I last checked (several years ago) there were no "official" OIDs published by any mainstream standards body yet, but perhaps that has changed in the meantime.
• I have no idea about pyca/cryptography's roadmap for post-quantum cryptography. As long as there's no PQC support there, integrating with something like liboqs is probably the way to go. Fortunately, it seems that there are Python bindings already available.
• At the time of writing, pyhanko-certvalidator still contains its own "algorithm dispatch" logic. This is purely for historical reasons, and we should probably harmonise the two sooner rather than later, but as long as that's the case, you will need to integrate liboqs with pyhanko-certvalidator as well if you want to be able to validate certificates signed with PQC algorithms.
• No mainstream PDF software (especially not of the user-facing variety) currently supports PQC. AFAIK there are also no mainstream standards yet for hardware-managed PQC keys, or even how to signal that they are being used (see my comment about OIDs above). These things aren't blockers for experimentation, but I'm a bit hesitant to maintain a feature like this in-tree, given that it's almost certainly going to require breaking changes to comply with future standards.
• Signing with PQC using liboqs will probably be straightforward. Validation will require some additional knowledge since the validation engine wasn't really designed to support plugging in new crypto (although I'm open to changing that).

So, in summary: I wholeheartedly support this experiment and I think it's quite doable, but given that interoperability with other implementations is just not achievable at this point, I don't think I can merge a contribution of that nature (yet). It'd make for a nice showcase, dependent lib, and/or advanced usage example.

What do you think?

[iyanmv]

Hey @MatthiasValvekens, thanks so much for the detailed answer!

• asn1crypto is only used for constructing CMS objects & parsing certificates in pyHanko, it doesn't actually do any cryptographic operations. Having said that, it is used to identify algorithms as well, so it needs to know about the OIDs used to identify PQC signing algorithms. When I last checked (several years ago) there were no "official" OIDs published by any mainstream standards body yet, but perhaps that has changed in the meantime.

A list of OIDs can be found here, but they are not final and probably will change in the future. Other projects, e.g. wolfSSL use the same OIDs.

• I have no idea about pyca/cryptography's roadmap for post-quantum cryptography. As long as there's no PQC support there, integrating with something like liboqs is probably the way to go. Fortunately, it seems that there are Python bindings already available.

I was talking before in #pyca and a developer told me the following:

«I am quite interested in having PQC, but any support landing in main will probably be predicated on the finalization of FIPS 203-205 and OpenSSL actually landing it in core. Until then experiments with the oqs provider are a good idea though!»

• At the time of writing, pyhanko-certvalidator still contains its own "algorithm dispatch" logic. This is purely for historical reasons, and we should probably harmonise the two sooner rather than later, but as long as that's the case, you will need to integrate liboqs with pyhanko-certvalidator as well if you want to be able to validate certificates signed with PQC algorithms.

Alright, thanks for pointing in the right direction ;)

• No mainstream PDF software (especially not of the user-facing variety) currently supports PQC. AFAIK there are also no mainstream standards yet for hardware-managed PQC keys, or even how to signal that they are being used (see my comment about OIDs above). These things aren't blockers for experimentation, but I'm a bit hesitant to maintain a feature like this in-tree, given that it's almost certainly going to require breaking changes to comply with future standards.

Of course, of course. I agree it doesn't make sense to implement and support this already as a "main" feature. I just wanted to do a quick experiment/demo. For some PDF readers (maybe not mainstream) to support PQC signatures, first other libraries have to support them. For example, KDE Okular and Gnome Document Viewer (Evince) both use poppler, which relies on NSS and gpgme to support cryptographic signatures. I guess these libraries will start to support PQC sooner than later.

Regarding the HSM support for PQC keys, you can check this table from the PKI Consortium. I think most of the big players will suport PQC keys by the end of this year. Then probably by 2026 we will start to see the first CAs supporting PQC algs.

• Signing with PQC using liboqs will probably be straightforward. Validation will require some additional knowledge since the validation engine wasn't really designed to support plugging in new crypto (although I'm open to changing that).

Yes, I think signing is simple, I was more worried about the CMS object and PDF peculiarities, that I'm not familiar at all. So I guess I could start trying that, just focusing on the signing process.

So, in summary: I wholeheartedly support this experiment and I think it's quite doable, but given that interoperability with other implementations is just not achievable at this point, I don't think I can merge a contribution of that nature (yet). It'd make for a nice showcase, dependent lib, and/or advanced usage example.

What do you think?

Sounds good! I think if I manage to make this to work it could be added to oqs-demos.

[MatthiasValvekens]

Hi @iyanmv,

I also took liboqs for a spin last weekend. With a bit of hacking I was able to get it to produce & validate Dilithium signatures in PDFs... Now that I have a bit of time to write down my findings, I figured I'd post them here :).

The good news is that the areas of the codebase that would have to change to accommodate this are actually fairly reasonable to generalise in a way that makes it possible to "plug in" something liboqs-capable without having to ship PQC support in-tree.

OID registration in asn1crypto (as discussed above)

The only thorny thing here is that the signature_algo helpers in asn1crypto are tricky to monkeypatch. But I have a fairly generic workaround for that: replacing all usages of algo.signature_algo with

        try:
            algo_name = algo.signature_algo
        except ValueError:
            algo_name = algo['algorithm'].native

(also in pyhanko-certvalidator)

Making validation pluggable

We could make the validator use a generic interface that just takes an ASN.1 SignedDigestAlgorithm spec, a message and a signature. The default implementation would support the current algorithm choices, but we could allow overriding that. For my experiment, I just changed the implementation in pyHanko on a sandbox branch.

Signing

Once I had the OID plumbing set up correctly, it was trivial to create a Signer subclass that worked with liboqs, so that's good.

Open questions

• liboqs's installation/distribution process isn't really standard (which is OK, I guess, because it's meant for research use, but it is a showstopper for even including it as an optional dependency)
• Public key encoding requirements are unclear. I just generated a bogus X.509 cert with the subject key OID set to the same OID as I used for marking the signature algorithm, and dumped the output from liboqs directly in there. This works for testing purposes, but for interop there should probably be a published standard. From a cursory reading of https://www.ietf.org/archive/id/draft-ietf-lamps-dilithium-certificates-03.html and the liboqs source, it seems that liboqs's encoding is compatible with this draft, but I also came across another (older) draft that specified something slightly different, so apparently the jury's still out on this one.

Test data

oqs-out.pdf
keys-and-certs.zip

[iyanmv]

Wow! So cool! I also did some experiments but I didn't arrive to touch anything in pyHanko, I was playing around with pypa/cryptography to use the oqsprovider. Could you push the branch where you did all those changes so I can have a look?

Regarding your first open question: Yes, liboqs is unfortunately not available in many distros, but I actually think it would be easy to package (perhaps I open some issues in Debian and Fedora to try to convince them). For Arch Linux I maintain an AUR PKGBUILD that provides a shared library, together with the documentation, and CMake config file. This allows other packages to easily use the library with cmake. For example, the provider for OpenSSL that I mention before that I also maintain uses this shared library. WolfSSL also can use this shared library when available (if compiled with the experimental features enabled).

Regarding the second open questions, I think we have to wait for the final versions of FIPS 203, 204 and 205. NIST will define the OIDs there.