SNMPV3 Trap Receiver stops receiver after few minutes

SNMPV3 Trap Receiver stops receiver after few minutes.
This receiver receives the traps but if there is an idle time, then it stops receiving messages even when the application is still running. This is a V3 trap receiver with Auth and Priv enabled. Engine id is configured. Here is the code -

{
OctetString localEngineIDOctet = OctetString.fromString(EngineID, 16);
threadPool = ThreadPool.create(“Trap”, THREAD_POOL_SIZE);
dispatcher = new MultiThreadedMessageDispatcher(threadPool,
new MessageDispatcherImpl());

    //TRANSPORT
    listenAddress = GenericAddress.parse(System.getProperty(
            "snmp4j.listenAddress", "udp:0.0.0.0/162"));  //SET THIS
    TransportMapping<?> transport;
    if (listenAddress instanceof UdpAddress) {
        DefaultUdpTransportMapping defaultUdpTransportMapping = new DefaultUdpTransportMapping((UdpAddress) listenAddress);
        // This ensures that socket is open infinite time and traps are not lost.
        defaultUdpTransportMapping.setSocketTimeout(0);
        transport = defaultUdpTransportMapping;
    } else {
        transport = new DefaultTcpTransportMapping((TcpAddress) listenAddress);
    }

    snmp = new Snmp(dispatcher, transport);
    snmp.getMessageDispatcher().addMessageProcessingModel(new MPv1());
    snmp.getMessageDispatcher().addMessageProcessingModel(new MPv2c());

    // V3 security
    boolean authRequired = NOAUTH_NOPRIV != SECURITY_LEVEL;
    boolean privRequired = AUTH_PRIV == SECURITY_LEVEL;

    SecurityProtocols securityProtocols = SecurityProtocols.getInstance();
    // add privacy protocol
    if (privRequired) {
        securityProtocols.addPrivacyProtocol(PRIVACY_PROTOCOL);
    }
    // add authenticate protocol
    if (authRequired) {
        securityProtocols.addAuthenticationProtocol(AUTH_PROTOCOL);
    }
    // USM user
    USM usm = new USM(securityProtocols.addDefaultProtocols(), localEngineIDOctet, 0);
    usm.setEngineDiscoveryEnabled(true);
    SecurityModels.getInstance().addSecurityModel(usm);
    snmp.getMessageDispatcher().addMessageProcessingModel(new MPv3(usm));
    snmp.getUSM().addUser(
            new OctetString(AUTH_USERNAME), localEngineIDOctet,
            getUser(AUTH_PROTOCOL, PRIVACY_PROTOCOL, authRequired, privRequired)
    );
    snmp.setLocalEngine(localEngineIDOctet.getValue(), 0, 0);

    snmp.setContextEngineIdDiscoveryDisabled(false);
    log.info("Starting Receiver");

    snmp.listen();
    //transport.listen();

    /*try
    {
        synchronized (this) {
            this.wait();
        }
    }
    catch (InterruptedException ex)
    {
        Thread.currentThread().interrupt();
    }*/
}

Hi
Possible causes for this issue are (in descending likelihood):

  1. Garbage collector finalizing the Snmp instance and stopping associated message processing.
  2. Trap sender is sending incorrect engine time/boots values.

Best regards
Frank

The process which is listening is still alive and so does show all related objects.
My test sender is sending 2 traps every 15 seconds with engine boots always set as 0.

The trap from my program is meant to be sent to any NMS that is configured. How does one find apt engine boots to pass? Currently value 0 works with snmp4j based receiver if the process doesn’t stop.

if the sender pauses for around a min, it stops receiving anything.

Thanks,
Shilpi

EngineBoots counter must be incremented each time you start the trap sender. While it is running engineTime needs to reflect the uptime of the trap sender.

For details, please see the USM RFC.

I enabled logging for SNMP and discovered that it was throwing error -
received message outside time window (authoritative)

Now looking at the UsmTimeTable this error is coming for the authoritative engine when now + time diff - time received is greater than 150.

As in my case, the sender is generating a unique Engine id in combination with enterprise Id. and the Receiver will have access to it and can configure it. Is the code I share below ok?

The trap sender I have will be bundled with a product that will be configured to send trap messages on events to an already configured NMS at the customer’s end.

Here is what I added before each SNMP.send call and this makes the receiver and sender work with engine boots 0.

        int currentTimeInSeconds = (int) TimeUnit.NANOSECONDS.toSeconds(System.nanoTime());
        snmp.setLocalEngine(localEngineIdOctet.getValue(), 0, currentTimeInSeconds);

Appreciate help in this.

I enabled logging and was able to find it was failing for error ->
“received message outside time window (authoritative)”

When I looked into the code for UsmTableEntry for Authoritative Engine (When receiver and sender have same engine id configured with USM), It only checks for Engine time diff to be not greater than 150.

So, If I set the Current time before each trap is sent, then even on restart of Sender, Traps are received.

Want to confirm if this behaviour is ok. The trap sender I am configuring is going to be bundled inside as a product and customers will receive engine id to configure on their already running NMS which could be from any provider.

The extra code I added -
Before calling snmp.send

Although this “works somehow”, it is a misuse of the SNMPv3 standard and it will most likely fail with a standard conforming SNMPv3 notifications receiver with strict security enabled, because that will block/ignore any traps from unknown engine IDs.

Instead, use a fixed but unique engine ID and increase engineBoots by one on any reboot and engineTime according to the local time of the sender as required by the SNMPv3 standard. Anything else is not interoperable and against the SNMPv3 USM RFC.

The engine-id is generated for the sender only once and persisted across restarts.

With snmp4j-3.4.4, I see the following in UsmTimeTable class

//In case of authoritative

if ((localTime.getEngineBoots() == 2147483647) ||
(localTime.getEngineBoots() != entry.getEngineBoots()) ||
(Math.abs(now + localTime.getTimeDiff() - entry.getLatestReceivedTime())
> USM_MAX_TIME_DIFFERENCE_HUNDREDS))

When I increase the engineBoots in the trap-sender (on sender restart),
the check above causes the receiver to discard the trap with the following log message.

DEBUG CheckTime: received message outside time window (authoritative):engineBoots differ 0!=1

//In case of non-authoritative

if ((entry.getEngineBoots() > time.getEngineBoots()) ||
((entry.getEngineBoots() == time.getEngineBoots()) &&
(entry.getLatestReceivedTime() > time.getLatestReceivedTime())))

the condition looks correct and IMO, would not fail.

Thank you frank for your help here. I am little confused now -

Engine ID is set to a unique value generated per Engine. I am setting Enterprise ID as SNMP4JSettings and then invoke MPV3.getLocalEngineId and save it for all future use.

So each PDU and SNMP object itself has a local engine id set.

Is that not what is expected?

Authoritative engines are more secure, isn’t it?

The only thing I am not updating is EngineBoots.

EngineTime is also being set to current time now, which is time since engine boot always.

The trap sender is the „authoritative engine“, I might be that you mixed that up in your analysis?

Engine ID is set to a unique value generated per Engine. I am setting Enterprise ID as SNMP4JSettings and then invoke MPV3.getLocalEngineId and save it for all future use.

So each PDU and SNMP object itself has a local engine id set.

Is that not what is expected?

Authoritative engines are more secure, isn’t it?

The only thing I am not updating is EngineBoots.

In the above statements there are a lot of misunderstandings. Please read the USM RFC to sort them out. Note that a device/application can have more than one engine (ID). The authoritative engine ID is then that engine that validates authorization on SNMPv3 message level.

Please verify too, if the generated engine ID is really unique in your domain. The SNMP4J helper methods to create an engine ID cannot ensure this! It is a common problem of SNMP beginners/developers not ensuring that.

Can you confirm on the above message regarding engineBoots and engineTime behavior change?

As I mentioned if I increase engine boot every time I restart the trap sender then the receiver goes in the flow of authoritative Engine and throws an error -

DEBUG CheckTime: received message outside time window (authoritative):engineBoots differ 0!=1

It seems that you increase the engine boots counter on the receiver site and not on the sender. But the sender is authoritative for traps. This way it has to fail.

But I have to admit, that I do not understand your setup and what you are trying to do. Therefore it would be probably better to stop here instead of confusing others and yourself more. Please read the USM RFC and things will get clearer. If we only see short snippets of a solution it is more guessing than knowing what the real problem is.