we’d like to use snmp4j to scale to 1000s (potentially 10000s) parallel SNMP sessions. Each sessions fetches a rather large number of (mainly repeater) OIDs - e.g. MIB-2 ifTable.
We’re already on JDK21, so virtual threads are available. We use them in our application to schedule SNMP polls and process results.
When it comes to snmp4j, I’m aware of two areas where threading is involved for our use case:
TransportMapping (we use DefaultUdpTransportMapping)
Timers (for retransmission / timeout handling in sessions)
Current assumption is, that also using virtual threads for transport mappings and timers further enhances scaling capabilities.
A quick test with Thread factories using virtual threads instead of platform threads (using SNMP4JSettings.setThreadFactory and SNMP4JSettings.setTimerFactory) led to some locking issues. On first sight it seems like all platform threads acting as carrier threads (Fork-Join pool) are exhausted due to blocking I/O operations in synchronized blocks.
Does this seem like a reasonable explanation for SNMPSession.get() / SNMPSession.walk() not returning in finite time?
Are there any plans to make snmp4j “virtual thread ready” by using ReentrantLock or the like instead of synchronized locking?
Very good and valid question To be honest I took it for granted that a snmp4j session maps to a single target, since a few wrapper classes in our app made that assumption. Should have questioned that to start with.
That’s good news, thanks. And I totally agree that virtual threads aren’t a silver bullet. They’re just good in waiting for blocking things to happen.
Since I’m a network guy I like the QoS analogy: QoS doesn’t create bandwidth, it’s just managed unfairness
Will happily go ahead and refactor some wrapper classes …
Are you willing to share anything about your ability to leverage virtual threads to collect data from 1000s or 10000s of targets using SNMP4J? We are jus beginning to think about attempting to leverage virtual threads in an existing snmp-based management tool and would love to hear about your experience. Were you successful in being able to scale to concurrent collection from 1000s of targets?
sorry for the late response. Seems I missed the notification mail when we switched providers
Concerning scaling of SNMP polling using snmp4j using virtual threads:
Internal lab testing looks quite promising with a few hundred simulated devices (using snmpsim-lextudio).
What means promising? Low resource usage (CPU), no thread contention, low total poll times.
Current setup:
a single snmp4j UDP socket (DefaultUdpTransportMapping) served by DefaultThreadFactory
snmp4j timers served by DefaultTimerFactory
one virtual thread per polled devices to do actual polling. Processing of result is injected using Consumer (typically put in a BlockingQueue processed by another thread)
one virtual thread per polled devices to schedule next poll
jdk 25 (avoid carrier thread pinning in sync blocks which was a potential issue in jdk < 24)
In the real world we’re currently facing timing issues at a pilot customers site. We’re in the middle of debugging, so at the moment I cannot really judge if the root cause is in our implementation or the specific environment (massive IP fragmentation, huge poll results, …)
Can keep you updated if you’re interested.
This is really helpful and promising, as you said.
I would love to hear what your troubleshooting reveals and if you are able to resolve any issues with your implementation.
I wasn’t aware of the thread pinning issues prior to idk 25 so that was really helpful to know.
It will likely be late in 2026 or early 2027 before we can upgrade our application to idk 25 but I am hoping to find some time to experiment before then.
to follow up on this one with some field experience gathered:
So far, our polling daemon scales really well (currently 1000+ polled agents) utilizing: * (home grown) pipeline architecture for flexible and type-safe wiring of processing steps
* virtual threads for scheduling poll tasks and polling individual SNMP agents
The biggest struggle we faced were (receive) UDP socket drops with the default implementation of org.snmp4j.MessageDispatcher (single threaded). A lot of processing was done in the context of the thread serving the UDP socket, which led to quite a lot of drops since the thread couldn’t keep up with the load. We implemented a multithreaded variant of MessageDispatcher (using the decorator pattern in org.snmp4j.util.MultiThreadedMessageDispatcher) and also use very short lived virtual threads for message dispatching now. We didn’t observe any socket drops since then. Key metrics (e.g. socket stats, snmp4j stats) are monitored using opentelemetry metric instruments which was quite helpful during troubleshooting.
To be honest I took it for granted that a snmp4j session maps to a single target, since a few wrapper classes in our app made that assumption. Should have questioned that to start with.
