Failed compiling object src/openssl.o

[luaVolk]

OS: Fedora 31
luarocks: 3.0.3
openssl: 1.1.1

Installing https://luarocks.org/luaossl-20190731-0.src.rock

Applying patch config.h.diff...
Hunk 1 found at offset 2...
gcc -O2 -fPIC -I/usr/include -c src/openssl.c -o src/openssl.o -D_REENTRANT -D_THREAD_SAFE -DCOMPAT53_PREFIX=luaossl -D_GNU_SOURCE -I/usr/include -I/usr/include
src/openssl.c:2292:3: error: conflicting types for ‘EVP_KDF_CTX’
 2292 | } EVP_KDF_CTX;
      |   ^~~~~~~~~~~
In file included from /usr/include/openssl/err.h:20,
                 from src/openssl.c:68:
/usr/include/openssl/ossl_typ.h:100:31: note: previous declaration of ‘EVP_KDF_CTX’ was here
  100 | typedef struct evp_kdf_ctx_st EVP_KDF_CTX;
      |                               ^~~~~~~~~~~
src/openssl.c:2294:13: error: conflicting types for ‘EVP_KDF_CTX_free’
 2294 | static void EVP_KDF_CTX_free(EVP_KDF_CTX *kctx) {
      |             ^~~~~~~~~~~~~~~~
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:26:6: note: previous declaration of ‘EVP_KDF_CTX_free’ was here
   26 | void EVP_KDF_CTX_free(EVP_KDF_CTX *ctx);
      |      ^~~~~~~~~~~~~~~~
src/openssl.c:2322:21: error: conflicting types for ‘EVP_KDF_CTX_new_id’
 2322 | static EVP_KDF_CTX *EVP_KDF_CTX_new_id(int id) {
      |                     ^~~~~~~~~~~~~~~~~~
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:25:14: note: previous declaration of ‘EVP_KDF_CTX_new_id’ was here
   25 | EVP_KDF_CTX *EVP_KDF_CTX_new_id(int id);
      |              ^~~~~~~~~~~~~~~~~~
src/openssl.c:2401: warning: "EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND" redefined
 2401 | #define EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND EVP_PKEY_HKDEF_MODE_EXTRACT_AND_EXPAND
      | 
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:55: note: this is the location of the previous definition
   55 | # define EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND  0
      | 
src/openssl.c:2407: warning: "EVP_KDF_HKDF_MODE_EXTRACT_ONLY" redefined
 2407 | #define EVP_KDF_HKDF_MODE_EXTRACT_ONLY EVP_PKEY_HKDEF_MODE_EXTRACT_ONLY
      | 
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:56: note: this is the location of the previous definition
   56 | # define EVP_KDF_HKDF_MODE_EXTRACT_ONLY        1
      | 
src/openssl.c:2411: warning: "EVP_KDF_HKDF_MODE_EXPAND_ONLY" redefined
 2411 | #define EVP_KDF_HKDF_MODE_EXPAND_ONLY EVP_PKEY_HKDEF_MODE_EXPAND_ONLY
      | 
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:57: note: this is the location of the previous definition
   57 | # define EVP_KDF_HKDF_MODE_EXPAND_ONLY         2
      | 
src/openssl.c:2414:12: error: conflicting types for ‘EVP_KDF_vctrl’
 2414 | static int EVP_KDF_vctrl(EVP_KDF_CTX *kctx, int cmd, va_list args) {
      |            ^~~~~~~~~~~~~
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:30:5: note: previous declaration of ‘EVP_KDF_vctrl’ was here
   30 | int EVP_KDF_vctrl(EVP_KDF_CTX *ctx, int cmd, va_list args);
      |     ^~~~~~~~~~~~~
src/openssl.c:2579:12: error: conflicting types for ‘EVP_KDF_ctrl’
 2579 | static int EVP_KDF_ctrl(EVP_KDF_CTX *kctx, int cmd, ...) {
      |            ^~~~~~~~~~~~
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:29:5: note: previous declaration of ‘EVP_KDF_ctrl’ was here
   29 | int EVP_KDF_ctrl(EVP_KDF_CTX *ctx, int cmd, ...);
      |     ^~~~~~~~~~~~
src/openssl.c:2590:15: error: conflicting types for ‘EVP_KDF_size’
 2590 | static size_t EVP_KDF_size(EVP_KDF_CTX *kctx) {
      |               ^~~~~~~~~~~~
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:32:8: note: previous declaration of ‘EVP_KDF_size’ was here
   32 | size_t EVP_KDF_size(EVP_KDF_CTX *ctx);
      |        ^~~~~~~~~~~~
src/openssl.c:2608:12: error: conflicting types for ‘EVP_KDF_derive’
 2608 | static int EVP_KDF_derive(EVP_KDF_CTX *kctx, unsigned char *out, size_t *outlen) {
      |            ^~~~~~~~~~~~~~
In file included from src/openssl.c:615:
/usr/include/openssl/kdf.h:33:5: note: previous declaration of ‘EVP_KDF_derive’ was here
   33 | int EVP_KDF_derive(EVP_KDF_CTX *ctx, unsigned char *key, size_t keylen);
      |     ^~~~~~~~~~~~~~
src/openssl.c: In function ‘kdf_derive’:
src/openssl.c:2401:46: error: ‘EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND’ undeclared (first use in this function)
 2401 | #define EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND EVP_PKEY_HKDEF_MODE_EXTRACT_AND_EXPAND
      |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:12135:5: note: in expansion of macro ‘EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND’
12135 |     EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:2401:46: note: each undeclared identifier is reported only once for each function it appears in
 2401 | #define EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND EVP_PKEY_HKDEF_MODE_EXTRACT_AND_EXPAND
      |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:12135:5: note: in expansion of macro ‘EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND’
12135 |     EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:2407:40: error: ‘EVP_KDF_HKDF_MODE_EXTRACT_ONLY’ undeclared (first use in this function)
 2407 | #define EVP_KDF_HKDF_MODE_EXTRACT_ONLY EVP_PKEY_HKDEF_MODE_EXTRACT_ONLY
      |                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:12138:5: note: in expansion of macro ‘EVP_KDF_HKDF_MODE_EXTRACT_ONLY’
12138 |     EVP_KDF_HKDF_MODE_EXTRACT_ONLY,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:2411:39: error: ‘EVP_KDF_HKDF_MODE_EXPAND_ONLY’ undeclared (first use in this function)
 2411 | #define EVP_KDF_HKDF_MODE_EXPAND_ONLY EVP_PKEY_HKDEF_MODE_EXPAND_ONLY
      |                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/openssl.c:12141:5: note: in expansion of macro ‘EVP_KDF_HKDF_MODE_EXPAND_ONLY’
12141 |     EVP_KDF_HKDF_MODE_EXPAND_ONLY,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/openssl.c:12414:
/usr/include/sys/sysctl.h: At top level:
/usr/include/sys/sysctl.h:21:2: warning: #warning "The <sys/sysctl.h> header is deprecated and will be removed." [-Wcpp]
   21 | #warning "The <sys/sysctl.h> header is deprecated and will be removed."
      |  ^~~~~~~
src/openssl.c: In function ‘randL_stir’:
src/openssl.c:12481:3: warning: ‘sysctl’ is deprecated [-Wdeprecated-declarations]
12481 |   if (0 != sysctl(mib, countof(mib), data, &n, (void *)0, 0))
      |   ^~
In file included from src/openssl.c:12414:
/usr/include/sys/sysctl.h:70:12: note: declared here
   70 | extern int sysctl (int *__name, int __nlen, void *__oldval,
      |            ^~~~~~

Error: Failed installing dependency: https://luarocks.org/luaossl-20190731-0.src.rock - Build error: Failed compiling object src/openssl.o

[daurnimator]

OpenSSL 1.1.1 shouldn't have the kdf type: are you running some weird openssl variant with kdf backported from 1.2.0?

[luaVolk]

Im just using the one from Fedoras repos

➜ openssl version   
OpenSSL 1.1.1d FIPS  10 Sep 2019

[daurnimator]

Im just using the one from Fedoras repos

Looks like they backported kdf and broke API: https://src.fedoraproject.org/rpms/openssl/blob/HEAD/f/openssl.spec#_63

You might be able to work around it by passing HAVE_EVP_KDF_CTX=1 when compiling luaossl

[luaVolk]

That worked! Thank you.

[daurnimator]

@t8m

[t8m]

We needed to backport the EVP_KDF support for use in OpenSSH and libssh to 1.1.1. I do not see how "we broke the API".

[daurnimator]

We needed to backport the EVP_KDF support for use in OpenSSH and libssh to 1.1.1. I do not see how "we broke the API".

The issue is that we also backported it in luaossl, with the backport active if the openssl version number is < 1.2.0. Is there a different check you'd recommend?

[t8m]

You could check by #ifdef EVP_KDF_HKDF_MODE_EXTRACT_AND_EXPAND for example. If this macro is defined, then it means the EVP_KDF_CTX type is backported.

[webtoucher]

Is it already fixed? When will it be released? Or please tell me how can I work around this? I have the same problem on centos 7, lua 5.1.4, OpenSSL 1.1.1c FIPS. I'm trying this:

luarocks install --tree ... luaossl 20190731-0 CRYPTO_INCDIR=/usr/include/openssl11/ OPENSSL_INCDIR=/usr/include/openssl11/ CFLAGS="-DHAVE_EVP_KDF_CTX=1"

...
gcc -DHAVE_EVP_KDF_CTX=1 -I/usr/include -c src/openssl.c -o src/openssl.o -D_REENTRANT -D_THREAD_SAFE -DCOMPAT53_PREFIX=luaossl -D_GNU_SOURCE -I/usr/include/openssl11/ -I/usr/include/openssl11/
src/openssl.c: In function ‘kdf_derive’:
src/openssl.c:12199:2: warning: passing argument 3 of ‘EVP_KDF_derive’ makes integer from pointer without a cast [enabled by default]
  if (EVP_KDF_derive(kctx, out, &outlen) <= 0)
  ^
In file included from src/openssl.c:615:0:
/usr/include/openssl11/openssl/kdf.h:33:5: note: expected ‘size_t’ but argument is of type ‘size_t *’
 int EVP_KDF_derive(EVP_KDF_CTX *ctx, unsigned char *key, size_t keylen);
     ^
gcc -DHAVE_EVP_KDF_CTX=1 -I/usr/include -c vendor/compat53/c-api/compat-5.3.c -o vendor/compat53/c-api/compat-5.3.o -D_REENTRANT -D_THREAD_SAFE -DCOMPAT53_PREFIX=luaossl -D_GNU_SOURCE -I/usr/include/openssl11/ -I/usr/include/openssl11/
gcc -shared -o _openssl.so -L/usr/lib64/lua/5.1 src/openssl.o vendor/compat53/c-api/compat-5.3.o -L/usr/lib64 -L/usr/lib64 -Wl,-rpath,/usr/lib64: -Wl,-rpath,/usr/lib64: -lssl -lcrypto -lpthread -lm -ldl
/usr/bin/ld: src/openssl.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: vendor/compat53/c-api/compat-5.3.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

Error: Build error: Failed compiling module _openssl.so

What am I doing wrong?

[webtoucher]

@daurnimator, please answer

[daurnimator]

@daurnimator, please answer

I consider fedora's backport to be the bug here.

Or please tell me how can I work around this?

set HAVE_EVP_KDF_CTX=1

luarocks install --tree ... luaossl 20190731-0 CRYPTO_INCDIR=/usr/include/openssl11/ OPENSSL_INCDIR=/usr/include/openssl11/ CFLAGS="-DHAVE_EVP_KDF_CTX=1"
...
What am I doing wrong?

You are probably overriding your luarocks config which sets CFLAGS to e.g. -fPIC.

[karai17]

Having the same issue trying to build a Docker image, unsure how to resolve this.

Edit: RUN luarocks install luaossl CFLAGS="-O2 -fPIC -DHAVE_EVP_KDF_CTX=1" seems to have solved the problem!

Edit 2: Building works but when I try to run my app I get /usr/local/openresty/luajit/lib/lua/5.1/_openssl.so: undefined symbol: EVP_KDF_size, version OPENSSL_1_1_1b

Edit 3: I was able to solve the issue by also using openresty's openssl-devel rpm module and building Luaossl against that:

RUN yum -y install openresty-openssl-devel
RUN luarocks install lapis CRYPTO_DIR=/usr/local/openresty/openssl CRYPTO_INCDIR=/usr/local/openresty/openssl/include OPENSSL_DIR=/usr/local/openresty/openssl OPENSSL_INCDIR=/usr/local/openresty/openssl/include

It would seem to me, however, that the ideal solution here would be for Luaossl to check if the OpenSSL lib it is being built against already has EVP_KDF available instead of assuming it does not.

[anishp55]

is there a planned fix for this? or at least get the patches from the fedora folks, they have the luaossl lib in their repos.

[mna]

FWIW I have pushed the 20200709 luaossl version in my LuaRocks namespace with the following patch:

    ["kdf.diff"] = [[
diff --git a/src/openssl.c b/src/openssl.c
index 1f0f5e4..b7aa862 100644
--- a/src/openssl.c
+++ b/src/openssl.c
@@ -86,6 +86,11 @@
 #include <lualib.h>
 #include <lauxlib.h>

+#if __has_include(<openssl/kdf.h>)
+#include <openssl/kdf.h>
+#define HAVE_EVP_KDF_CTX 1
+#endif
+
 #if LUA_VERSION_NUM < 503
 #include "../vendor/compat53/c-api/compat-5.3.h"
 #endif
]];

Assuming your C compiler understands the __has_include extension (clang and gcc do), you should be able to install it with luarocks install mna/luaossl. I don't intend to maintain that for too long, hopefully either Fedora or luaossl will take care of this in the future, but in the meantime, if that can help other people...

[mna]

Hmmm while that works on Fedora, it does not work on Ubuntu. It's probably not the right way to check, sorry about that.

[MagicD3VIL]

Edit: RUN luarocks install luaossl CFLAGS="-O2 -fPIC -DHAVE_EVP_KDF_CTX=1" seems to have solved the problem!

This fixed it for me on Mageia 8. Thanks, @karai17 !

[guicassolato]

set HAVE_EVP_KDF_CTX=1

I'm afraid this proposed solution will now cause a different issue with OpenSSL 1.1.1.

Since #199 (20220711):

luaossl/src/openssl.c

Lines 624 to 626 in 247c0d9

	#if HAVE_EVP_KDF_CTX
	#include <openssl/core_names.h>
	#endif

and because core_names.h does not exist in OpenSSL 1.1.1, compiling will fail with a openssl/core_names.h: No such file or directory.

[LeRatierBretonnien]

Hi
Any fix for OpenSSL 1.1.1 ? Core_names.h is missing, as said above. Any fix in the pipe ?
Thanks !

[LeRatierBretonnien]

Yes, is there any fix for that ? All solutions above are not working (Opensuse 15.3, openssl 1.1.1). Any other hints or ideas ?