Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong metadata when a large number of requests take place #58

Open
SkanderMansouri opened this issue May 19, 2020 · 1 comment
Open

Comments

@SkanderMansouri
Copy link

SkanderMansouri commented May 19, 2020

Hi, I have an app that connects to the MQTT, I want to publish 1200 devices with the id of each device as a metadata. the following is the code

"use-strict"
const RSVP = require('rsvp');
const Mqtt = require('mqtt');
const cls = require('cls-hooked');

const namespace = "firstName";
let clsNamespace;

let client = Mqtt.connect("alis://test.mosquitto.org");


if (!client) {
    logger.Error("Test", 'Init', 'No mqtt client provided');
    throw new extError('No mqtt client created');
}

client.on('connect', async () => {
    console.log("Connected");
    try {
        clsNamespace = cls.createNamespace(namespace);
        main();
    } catch (error) {
        console.log(error);
    }
});


function main() {
    var devices = [];
    for (var i = 0; i < 1200; i++) {
        devices.push({ "id": i });

    }
    RSVP.all(devices.map(async (item) => await updateDevice(item)));
}

async function updateDevice(device) {
    try {
        return await wrapContext(clsNamespace, async () => {
            setContext({ device: device.id });
            console.log("update " + device.id + " metadata =" + JSON.stringify(__getMetadata()));
            return publish("message", device.id);
        });
    } catch (error) {
        console.log(error);
    }

}

function setContext(context) {
    try {
        let ctxKeys = clsNamespace.get('contextKeys') ? clsNamespace.get('contextKeys') : [];
        for (const key in context) {
            clsNamespace.set(key, context[key]);
            if (ctxKeys.indexOf(key) === -1) {
                ctxKeys.push(key);
            }
        }
        clsNamespace.set('contextKeys', ctxKeys);
    } catch (error) {
        console.error(error);
        console.log('cannot set context', context);
        throw error;
    }
}

function publish(message, deviceId) {
    return new RSVP.Promise((resolve, reject) => {
        try {
            client.publish(message,
                deviceId,
                (error) => {
                    if (error) {
                        console.log("error")
                        reject(error);
                    } else {
                        console.log("publish " + deviceId + " metadata" + JSON.stringify(__getMetadata()));
                        resolve();
                    }
                });
        } catch (error) {
            console.log(error);
        }

    });
}

async function wrapContext(cls, callback) {
    let defer = RSVP.defer();
    let context = await cls.run(async (contextObj) => {
        try {
            let result = await callback(contextObj);
            defer.resolve(result);
        } catch (error) {
            defer.reject(error);
        }
    });
    return defer.promise;
};

function __getMetadata() {
    const metadata = {};
    let contextData = {};
    for (const key of clsNamespace.get('contextKeys') || []) {
        contextData[key] = clsNamespace.get(key);
    }

    for (const key in contextData) {
        metadata[key] = contextData[key];
    }
    return metadata;
}

the output is the following:

update 0 metadata ={"device":0}
publish 0 metadata{"device":0}
update 1 metadata ={"device":1}
publish 1 metadata{"device":1}
... (same thing for 1165 devices)
update 1166 metadata ={"device":1166}
update 1167 metadata ={"device":1167}
update 1168 metadata ={"device":1168}
update 1169 metadata ={"device":1169}
... (same thing until 1199)
update 1199 metadata ={"device":1199}
publish 1166 metadata{"device":1199}
publish 1167 metadata{"device":1199}
publish 1168 metadata{"device":1199}
... (same thing until 1199)

As you can see, the metadata is correct for the 1165 first publish log but once there's an interruption in the iteration and the function become asychnronous, the metadata of the first publish will be missmatched.

When debugging with DEBUG_CLS_HOOKED, I found out that after the 1169th update, the context becomes missing and then destroyed. thus, the currentUid changes and makes the 1166th publish gets the wrong context. (attached the debug logs)
TestDebugCls.log

Is there a way to fix this?

@MarioArriaga92
Copy link

Any update on this? My team is considering using this lib for production use, should we be worried about this issue? and if so, is there any workaround or suggestion on a library fit for large number of requests taking place?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants