Callables that do not get quality metadata, discussion/tracking issue

[MagielBruntink]

Below we describe various reasons why there can be mismatches between Lizard's output and callables in the metadata DB.

Super class constructors

Callables are created by OPAL for a class' super class constructors and inserted into the DB. Their line numbers match the class' signature header. Such callables are never analyzed by Lizard, since they don't exist in code.

Example: /com.github.tomakehurst.wiremock.admin.tasks/FindRequestsTask.%3Cinit%3E()%2Fjava.lang%2FVoidType has line_start and line_end of 30 in wiremock-standalone-2.27.2-sources/com/github/tomakehurst/wiremock/admin/tasks/FindRequestsTask.java (download jar).

27: import static com.github.tomakehurst.wiremock.client.ResponseDefinitionBuilder.responseDefinition;
28: import static java.net.HttpURLConnection.HTTP_OK;
29: 
30: public class FindRequestsTask implements AdminTask {
31:
32:    @Override
33:    public ResponseDefinition execute(Admin admin, Request request, PathParams pathParams) {
34:        RequestPattern requestPattern = Json.read(request.getBodyAsString(), RequestPattern.class);
35:        FindRequestsResult result = admin.findRequestsMatching(requestPattern);
36:
37:        return responseDefinition()
38:                .withStatus(HTTP_OK)
39:                .withBody(Json.write(result))
40:                .withHeader("Content-Type", "application/json")
41:                .build();
42:    }
43:}

The execute method in this example has proper quality metadata:

{
    "access": "public",
    "defined": true,
    "quality": {
        "metrics": {
            "nloc": 9,
            "length": 10,
            "complexity": 1,
            "token_count": 72,
            "parameter_count": 3
        },
        "callable_name": "FindRequestsTask::execute",
        "callable_long_name": "FindRequestsTask::execute( Admin admin , Request request , PathParams pathParams)",
        "rapid_plugin_version": "1.2.0",
        "quality_analyzer_name": "Lizard",
        "quality_analyzer_version": "1.17.7",
        "quality_analysis_timestamp": "1612360517.528548"
    }
}

Lamdas

Lamdba functions do not get metadata. OPAL produces weird line numbers. Are lambdas analyzed by Lizard at all?

Examples from wiremock-standalone-2.27.2-sources/com/github/tomakehurst/wiremock/admin/tasks/FindRequestsTask.java (download jar).
/com.github.tomakehurst.wiremock.admin/AdminRoutes$1.%3Cinit%3E(AdminRoutes)%2Fjava.lang%2FVoidType (lines 130-130 in DB)
/com.github.tomakehurst.wiremock.admin/AdminRoutes$1.apply(%2Fjava.util%2FMap$Entry)AdminTask (lines 130-130 in DB)

124:    public AdminTask taskFor(final RequestMethod method, final String path) {
125:        return tryFind(routes.entrySet(), new Predicate<Map.Entry<RequestSpec, AdminTask>>() {
126:            @Override
127:            public boolean apply(Map.Entry<RequestSpec, AdminTask> entry) {
128:                return entry.getKey().matches(method, path);
129:            }
130:        }).transform(new Function<Map.Entry<RequestSpec, AdminTask>, AdminTask>() {
131:            @Override
132:            public AdminTask apply(Map.Entry<RequestSpec, AdminTask> input) {
133:                return input.getValue();
134:            }
135:        }).or(new NotFoundAdminTask());
136:    }

The enclosing method taskFor /com.github.tomakehurst.wiremock.admin/AdminRoutes.taskFor(%2Fcom.github.tomakehurst.wiremock.http%2FRequestMethod,%2Fjava.lang%2FString)AdminTask has proper quality metadata.

{
    "access": "public",
    "defined": true,
    "quality": {
        "metrics": {
            "nloc": 13,
            "length": 13,
            "complexity": 1,
            "token_count": 126,
            "parameter_count": 2
        },
        "callable_name": "AdminRoutes::taskFor",
        "callable_long_name": "AdminRoutes::taskFor( final RequestMethod method , final String path)",
        "rapid_plugin_version": "1.2.0",
        "quality_analyzer_name": "Lizard",
        "quality_analyzer_version": "1.17.7",
        "quality_analysis_timestamp": "1612360517.528548"
    }
}

Code not present in sources jar

In some cases there are callables in the DB that do not appear to be included in the downloaded sources jar.

Example: /com.flipkart.zjsonpatch/CompatibilityFlags.%3Cclinit%3E()%2Fjava.lang%2FVoidType (and many others from com.flipkart.zjsonpath). The jar specified by sourcesUrl does not include those files and callables.

Files of a product not present in DB

In some cases lizard analyzed source code files that are not part of bytecode of JAR.

Shared (fat) sources.jar

Artifacts of the same Group can share (fat) sources.jar, hence product under analyzed including files not part of product's bytecode.

org.suigeneris:jrcs.rcs:0.4.2 and org.suigeneris:jrcs.diff:0.4.2 have the same sources.jar including source code for both products.

Shaded JAR

Source code of shaded library could present in the dependent's sources.jar, however shaded JAR has not been included in the bytecode, and not analyzed by OPAL.

io.netty:netty-common:4.1.47.Final shades org.jctools. into io.netty.util.internal.shaded.org.jctools..

org.glassfish.jersey.bundles.repackaged:jersey-guava:2.25 shades com.google.common and com.google.thirdparty.

Code present in sources jars but in the meta-data DB

There are cases where the sources jars contain more code than actually managed by Maven, leading to our quality analyzer to report errors when putting meta data into the DB (because the files do not exist in the DB). An example is discussed in #38.