Question related to Kafka trigger and azure functions(NodeJs)?

[MuhamedSalihSeyedIbrahim]

Hi Team,
I am new to kafka extension nodejs azure functions Triggers.

I Want to be clarified before using it in our application,

I am having these following questions:-

1. Is this kafka trigger extension for(nodejs) azure funtion prod ready?
2. Max data holding capacity of kafka?
3. Function app kafka extension - how much data can it receive and process in (nodejs) function app ?
4. How many parallel execution of function app can be achieved if we deployed the app in premium plan?
5. Does kafka trigger extention support listening to kafka service that hosted in third party environment (other kafka as a service providers/ other kafka server)?
6. Does this kafka trigger extension uses socket?
7. Do we expected do some socket configuration in dashboard of azure function app? If yes , then what are they and how to do it ?
8. If we use kafka trigger extension in azure function does it continuously run by listening sockets?
9. If the above question is yes then what are the billing cost changes we end up with?
10. Does the application work if kafka trigger extension delivers function app(nodejs), more than or equal 20MB of json data? Can function app process it?

[TsuyoshiUshio]

Sorry for the delay.

1. Is this kafka trigger extension for(nodejs) azure funtion prod ready?
   -> Yes. Kafka extension is already G.A.

2. Max data holding capacity of kafka?
   -> It's depend on your Kafka broker.

3. Function app kafka extension - how much data can it receive and process in (nodejs) function app ?
   -> It depends on plan or your application. Could you try test it on your environment?

4. How many parallel execution of function app can be achieved if we deployed the app in premium plan?
   -> Maximum the same number of the partition. You can refer to https://tsuyoshiushio.medium.com/azure-functions-kafka-trigger-performance-tips-concept-and-architecture-ec94a31d8b93

5. Does kafka trigger extention support listening to kafka service that hosted in third party environment (other kafka as a service providers/ other kafka server)?
   -> Yes.

6. Does this kafka trigger extension uses socket?
   -> Yes. Kafka uses a binary protocol over TCP. https://kafka.apache.org/0100/protocol.html#protocol_network

7. Do we expected do some socket configuration in dashboard of azure function app? If yes , then what are they and how to do it ?
   No. All the configuration is reside in the function.json or host.json. refer to https://github.com/Azure/azure-functions-kafka-extension#configuration-settings for more information.

8. If we use kafka trigger extension in azure function does it continuously run by listening sockets?
   Yes.

9. If the above question is yes then what are the billing cost changes we end up with?
   It depends on your plan of the functions app. We support Premium Functions and AppS Servcie Plan. Refer to https://azure.microsoft.com/en-us/pricing/details/functions/ for pricing.

10. Does the application work if kafka trigger extension delivers function app(nodejs), more than or equal 20MB of json data? Can function app process it?

No. By default 1048576 bytes. It is configured with fetch.message.max.bytes. However, recommended approach is keep the message is small.
If you want to send a big message, you can index it and upload to blob storage for example, then send the id as the message.
For more information, refer to https://stackoverflow.com/questions/21020347/how-can-i-send-large-messages-with-kafka-over-15mb

[MuhamedSalihSeyedIbrahim]

@TsuyoshiUshio 3. Could you try test it on your environment? - No, it's not deployed and checked it but I have did a POC in local its work great.

Thanks @TsuyoshiUshio