High Memory Utilization on Single Node, other_ets Consumption

[iamhritik]

Describe the bug

Hey everyone,

We've encountered an unexpected issue with one of our nodes experiencing a significant spike in memory utilization. Upon investigation, we've found that the majority of the memory is being consumed by other_ets
Any insights or advice on resolution are appreciated.

Environment:
RabbitMQ: 3.11.5
Erlang: 25.3.2.10

queue type: quorum

rabbitmq conf

listeners.tcp.1 = 0.0.0.0:5672
mqtt.listeners.tcp.1 = 0.0.0.0:1443
mqtt.tcp_listen_options.backlog = 9182
mqtt.tcp_listen_options.recbuf  = 191072
mqtt.tcp_listen_options.sndbuf  = 191072
mqtt.tcp_listen_options.keepalive = true
mqtt.tcp_listen_options.nodelay   = true
mqtt.tcp_listen_options.exit_on_close = true
mqtt.tcp_listen_options.send_timeout  = 120
mqtt.allow_anonymous  = false
mqtt.default_user     = guest
mqtt.default_pass     = guest
mqtt.vhost            = /
mqtt.exchange         = amq.topic
mqtt.subscription_ttl = 86400000
management.listener.port = 15672
management.listener.ip   = 0.0.0.0
log.file.level = info
vm_memory_high_watermark.relative = 0.6
handshake_timeout = 20000
tcp_listen_options.backlog = 9182
tcp_listen_options.recbuf  = 191072
tcp_listen_options.sndbuf  = 191072
tcp_listen_options.keepalive = true

Reproduction steps

1. Start RabbitMQ node with provided configurations.
2. Trigger memory spike by publishing a large number of messages or simulating high processing activity.
3. Monitor memory usage, noting other_ets process consuming excessive memory.

...

Expected behavior

Expecting it to decrease after a while.

Additional context

No response

[mkuratczyk]

3.11.x is out of community support: https://www.rabbitmq.com/release-information

Given you seem to be using RabbitMQ for MQTT, the issue is likely addressed in more recent versions - MQTT support has been completely redesigned in 3.12 and is much more efficient: https://www.rabbitmq.com/blog/2023/03/21/native-mqtt

If you observe the same problem after the upgrade, feel free to open an issue but provide actual steps to reproduce it - code or commands (using any of the MQTT test clients like https://github.com/emqx/emqtt-bench, https://github.com/rabbitmq/omq or whatever you prefer) that trigger the issue for you - "Trigger memory spike by publishing a large number of messages or simulating high processing activity" is not a reproduction step.