Saturday , December 28 2024

how to solve a problem of hanging alarms in Huawei U2000

Let’s assume that you have U2000 NMS server to monitor Huawei devices. We can manage these devices in 2 ways: outband or inband management. Outband management means that you have a separate DCN network to manage devices. It is commonly used for critical nodes, for example for backbone routers. Unlike to backbone network, it is difficult to implement DCN for mobile backhaul networks, where the number of devices reaches hundreds or even thousands. In such situation inband management is implemented to reduce cost. Then the decision how to send SNMP packets to the NMS server is based on routing protocols. The packets travel through the monitored network and are susceptible to all turbulences, which can appear in the network. This may lead to the fact that some SNMP packets may be lost by the network.

Let’s imagine such case. A link between a router and NMS is “DOWN”. No redundant link is established. The router sends SNMP trap to the NMS server but the server is not available. The SNMP packet is lost. Then the link is going to “UP” state and the router send SNMP trap to U2000. This trap is then dropped by U2000 because there is not related “DOWN” trap, which was lost before.

And what’s next?

U2000 synchronizes alarms with devices every 30 minutes and NMS server receives “DOWN” trap from the router, which was lost earlier. As the clearing trap was dropped, this “DOWN” alarm will not be cleared anymore. Then we have “DOWN” hanging (not cleared) alarm in U2000.

How to cope with this problem?

The first solution is to implement inform mode for SNMP packets:

snmp-agent target-host inform …

Managed devices require an acknowledgement from the NM server, after sending inform packets. If a managed device does not receive the acknowledgement, it will resend the inform packet to the NM station and generate alarm logs. If the managed device does not receive an acknowledgement from the NM station, it will store the inform packets in its memory. Anyway using inform mode may consume lots of system resources.

The next solution is to configure private-netmanager option to trap mode:

snmp-agent target-host trap address udp-domain ip-address [ udp-port port-number | { public-net | vpn-instance vpn-instance-name } ] * params securityname security-string [ { v3 [ authentication | privacy ] | v2c | v1 } | notify-filter-profile profile-name | private-netmanager | ext-vb ] *

When a Huawei NMS is deployed and this parameter is configured, a trap message, sent to the NMS, contains more information, such as the trap type, sequence of the trap, and sending time.
In such situation, even U2000 receives a trap from synchronization, it will compare the sequence of the trap and sending time. Then we avoid problems of not cleared alarms in network management system. We have implemented such solution in our customer’s network and it works really fine.

Read More »

Huawei cheat sheet – ACL applications

Our blog topics touched access lists ACLs, used on Huawei devices. We talked about ACLs in traffic policies, ACLs and their matching order, ACLs in PBR and so on and so forth.

As ACLs are widely used, I decided to prepare a simple cheat sheet, that describes three kinds of ACLs (the most often used) and their usage scenarios.

Additionally a new page has been opened. I will upload new cheat sheets there, when they are ready.

Have a suggestion for a cheat sheet? Create your own? Let us know in comments.

Read More »

ACL in traffic policy on Huawei device

We have to remember that traffic policy consists of 3 parts:

  • Classifier
  • Behavior
  • Traffic-policy

In brief, to configure a traffic policy:

  • define traffic class
  • define action to be applied to the traffic class
  • associate traffic classifiers and behaviors
  • apply the traffic policy to an interface.

Let’s start from ACL.

We have possibility to configure many rules in an ACL. If the ACL is specified in if-match clause, then a packet is matched against multiple rules. If the packet matches a rule in the ACL, then it stops checking against the next rules.

  • In a case of DENY action in the ACL, the matched packet is denied, regardless of what traffic behavior defines.
  • When PERMIT action is defined in the ACL, then traffic behavior is applied to the matched packet.

Let’s focus on if-match clauses now and matching order between them.

We have 2 possibilities to configure a traffic classifier:

  • With logic OR relationship between multiple matching rules
  • With logic AND relationship between multiple matching rules.

In the flowchart you can see traffic policy implementation with logic OR:

 

Notice that traffic behavior is applied for the packets that match any one of the if-match clauses. In this situation, packet is matched against If-match clauses, in the order of the If-match clauses configuration. This kind of traffic classifier is commonly used.

In logic AND implementation, the device combines all if-match clauses and then processes the combination of if-match clauses according to the procedure of logic OR. If one of the if-match clauses is applied with ACL, each rule of the ACL is combined with all of the other if-match clauses. In this situation, the order of if-match clauses does not impact on the final matching result.

As you know, one or more classifiers and behaviors can be configured in a traffic policy. A packet is matched against traffic classifiers in the order, in which those classifiers have been configured. If the packet matches a traffic classifier, no further match operation is performed. If not, the packet is matched against the following traffic classifiers one by one. If the packet does not match traffic classifiers at all, the packet is forwarded with no traffic policy executed.

Anyway, the order of traffic classifier can be changed by the following command:

classifier classifier-name behavior behavior-name precedence precedence

Let’s try to configure a traffic policy as follows:

  • packets coming from source IP address 10.1.1.0/24, with DSCP 18, are then remarked as AF31
  • packets coming from source IP address 172.16.1.0/24, with DSCP 18, are discarded
  • other packets are forwarded directly.
#
acl number 3000  
 rule 5 permit ip source 10.1.1.0 0.0.0.255 dscp af21 
 rule 10 deny ip source 172.16.1.0 0.0.0.255 dscp af21 
#
traffic classifier labnario operator or
 if-match acl 3000
#
traffic behavior labnario
 remark dscp af31
#
traffic policy labnario
 classifier labnario behavior labnario
#
interface GigabitEthernet0/0/0
 traffic-policy labnario inbound

I have a home task for you. Try to configure the same policy but using logic AND. Your traffic policy must do the same like the one configured.

Unfortunately, awards are not provided 😉

I will publish the solution soon on Facebook.

All comments (solutions) are welcome.

Read More »

rate limiting of ARP packets on Huawei switch

How to protect Huawei switches against ARP flood attack?

DoS attack is an attempt to make a network resources unavailable to its intended users. There are several different types of DoS attacks, but most of them rely on spoofing and flooding techniques. Some of these attacks can be hard to defend against, because DoS packets may look exactly like normal packets.

One common method of attack involves saturating the target device with a flood of request packets, so that this device cannot respond to a legitimate traffic or responds so slowly, as to be unavailable.

Attackers often use ARP protocol to attack network devices, because it is easy to use and has no security mechanisms built in. Flooding a network device with ARP request packets can lead to insufficient CPU resources to process other services, when processing a large number of ARP packets. To protect the device, ARP rate limiting mechanism can be used. On Huawei switches this feature can be implemented in the following ways:

  • Limiting the rate of ARP packets globally, in a VLAN or on an interface
  • Limiting the rate of ARP packets based on the source MAC address
  • Limiting the rate of ARP packets based on the source IP address

limiting the rate of ARP packets globally, in a VLAN or on an interface

To enable ARP packets rate limiter globally (all ARP request packets processed by the system will be limited), the following command has to be configured:

<labnarioSW>arp anti-attack rate-limit enable

By default this feature is disabled. Let’s see if it is enabled now:

[labnarioSW]dis arp anti-attack config arp-rate-limit 
ARP rate-limit configuration:
-------------------------------------------------------------------------------
Global configuration:
    arp anti-attack rate-limit enable
Interface configuration:
Vlan configuration:
-------------------------------------------------------------------------------

When enabled, packets will be rate limited to 100 per second, by default. To change this value, use the command:

[labnarioSW]arp anti-attack rate-limit 90 ?
  INTEGER  Set interval value (second)

[labnarioSW]arp anti-attack rate-limit 90 1

As you see, a time interval of rate limit can also be changed in the range of 1-86400 seconds.

To enable the alarm function for discarded ARP packets, use the command:

[labnarioSW]arp anti-attack rate-limit alarm enable

To enable ARP packets rate limiter in a VLAN or on an interface, use the same commands as above, in a VLAN or an interface view respectively:

[labnarioSW]vlan 10
[labnarioSW-vlan10]arp anti-attack rate-limit enable
[labnarioSW-vlan10]arp anti-attack rate-limit 20 1

[labnarioSW]int g0/0/1
[labnarioSW-GigabitEthernet0/0/1]arp anti-attack rate-limit enable
[labnarioSW-GigabitEthernet0/0/1]arp anti-attack rate-limit 10 1 block timer 60

In the interface view, it is possible to block all the traffic, coming from the specific source, when defined rate limit is exceeded. Block timer can be set in the range of 5-864000 seconds. Alarm function of discarded ARP packets can be enabled in VLAN and interface views as well.

Let’s see our final configuration:

[labnarioSW-vlan10]dis arp anti-attack config arp-rate-limit
ARP rate-limit configuration:
-------------------------------------------------------------------------------
Global configuration:
    arp anti-attack rate-limit enable
    arp anti-attack rate-limit 90 1
    arp anti-attack rate-limit alarm enable
Interface configuration:
  GigabitEthernet0/0/1 :
    arp anti-attack rate-limit enable
    arp arp anti-attack rate-limit 10 1 block timer 60
Vlan configuration:
  Vlan10 :
    arp anti-attack rate-limit enable
    arp anti-attack rate-limit 20 1
-------------------------------------------------------------------------------

limiting the rate of ARP packets based on the source MAC address

<labnario>system-view
[labnarioSW]arp speed-limit source-mac ?
  H-H-H    The source Mac address
  maximum  Input the speed-limit value

There are two options, let’s choose the first one. This option limits the number of ARP packets with specific source MAC address. The range of this values can be set as packets per second.

Let’s configure limiter to allow no more than 50 ARP packet per second, sourced from MAC address aaaa-bbbb-cccc:

[labnarioSW]arp speed-limit source-mac aaaa-bbbb-cccc maximum 50

If no MAC address is specified (second option), then limiter is applied to all ARP packets:

[labnarioSW]arp speed-limit source-mac maximum ?
  INTEGER  The range of speed-limit value(in packets/second)

[labnarioSW]arp speed-limit source-mac maximum 100
[labnarioSW]

By default, the rate limit of ARP packets is set to 0, which means that ARP packets are not limited.

To display ARP rate limit configuration, use the command as below:

[labnarioSW]display arp anti-attack configuration arp-speed-limit 
ARP speed-limit for source-MAC configuration:
MAC-address         suppress-rate(pps)(rate=0 means function disabled)
-------------------------------------------------------------------------------
aaaa-bbbb-cccc      50
Others              100
-------------------------------------------------------------------------------
The number of configured specified MAC address(es) is 1, spec is 512.

As you see, the rate limiter is now configured to limit all ARP packets, sourced from any MAC address, to no more than 100 packets per second, and to limit ARP packets, sourced from MAC of aaaa-bbbb-cccc, to no more than 50 packets per second.

limiting the rate of ARP packets based on the source IP address

To configure ARP packet rate limiter to limit packets, based on the source IP address, use the command as below:

[labnarioSW]arp speed-limit source-ip 10.11.12.100 maximum 5

To limit all ARP packets, use the following command:

[labnarioSW]arp speed-limit source-ip maximum 50

Let’s check our configuration:

[labnarioSW]dis arp anti-attack config arp-speed-limit 
ARP speed-limit for source-MAC configuration:
MAC-address         suppress-rate(pps)(rate=0 means function disabled)
-------------------------------------------------------------------------------
aaaa-bbbb-cccc      50
Others              100
-------------------------------------------------------------------------------
The number of configured specified MAC address(es) is 1, spec is 512.

ARP speed-limit for source-IP configuration:
IP-address          suppress-rate(pps)(rate=0 means function disabled)
-------------------------------------------------------------------------------
10.11.12.100        5
Others              50
-------------------------------------------------------------------------------
The number of configured specified IP address(es) is 2, spec is 512.

When both commands are configured, ARP packets sourced from host 10.11.12.100 will be rate limited by our switch to no more than 5 packets per second, all the other packets sourced from any IP address will be limited to no more than 50 packets per second at the same time.

By default, the rate limit of ARP packets with the same source IP address is set to 0, which means that ARP packets are not limited by the switch.

Rate limiting of ARP packets is not a perfect solution against ARP flood attack. When configured, both malicious and legitimate ARP packets will be affected by this feature. Nonetheless we have to remember, that this feature can help our switch to survive attack, while still forwarding traffic in a network.

Read More »

ACL matching order on Huawei device

The first what a device has to do is to check if the ACL exists. If it does, the device matches packets against rules, according to the rule ID. We can configure rule IDs manually or they are automatically allocated. In case of automatically allocated rules, there is a certain space between two rule IDs. The size of the space depends on ACL step. By default it is 5 but we can change it by command. In this manner, we can add a rule before the first rule or between rules. ACL rules are displayed in ascending order of rule IDs, not in the order of configuration.

ACL rules can be arranged in two modes: configuration and auto.

In the configuration mode (default mode), we decide which rule should be first, which second and so on and so forth. In this mode, the device matches rules in ascending order of rule IDs. Anytime we can configure an additional rule with smaller rule ID. In such case, later configured rule may be matched earlier. We make such a decision, not the system.

In the auto mode, unlike in the configuration mode, the system automatically allocates rule IDs. We don’t have possibility to specify rule ID. The most precise rule is placed at the beginning of ACL.

When can we use it?

For example, if we filter a wide range of packets and want to allow some packets (from this wide range) to pass, it is enough to define a specific rule, without rules reordering. This rule will be placed first, as it is more specific.

For basic ACL rules, the source address wildcards are compared. If they are the same, then the configuration order is taken into account.

For advanced ACL rules, more factors are compared, like wildcards of source and destination and protocol ranges of source and destination.

Let’s configure the same ACL with 2 modes:

[Huawei]acl number 3000  match-order auto
[Huawei-acl-adv-3000]rule perm ip source 10.1.1.0 0.0.0.255 destination 172.16.1.0 0.0.0.255
[Huawei-acl-adv-3000]rule permit ip source 150.20.0.0 0.0.0.255 destination 10.1.0.0 0.0.0.255
[Huawei-acl-adv-3000]rule deny ip sou 10.1.1.10 0.0.0.0 destination 172.16.1.0 0.0.0.255
[Huawei-acl-adv-3000]rule den ip source 150.20.0.0 0.0.0.255 destination 10.1.0.15 0.0.0.0

[Huawei-acl-adv-3000]dis this
#
acl number 3000  match-order auto
 rule 5 deny ip source 10.1.1.10 0 destination 172.16.1.0 0.0.0.255 
 rule 10 deny ip source 150.20.0.0 0.0.0.255 destination 10.1.0.15 0 
 rule 15 permit ip source 10.1.1.0 0.0.0.255 destination 172.16.1.0 0.0.0.255 
 rule 20 permit ip source 150.20.0.0 0.0.0.255 destination 10.1.0.0 0.0.0.255 

[Huawei]acl 3000 match-order config
[Huawei-acl-adv-3000]rule perm ip source 10.1.1.0 0.0.0.255 destination 172.16.1 .0 0.0.0.255
[Huawei-acl-adv-3000]rule permit ip source 150.20.0.0 0.0.0.255 destination 10.1 .0.0 0.0.0.255
[Huawei-acl-adv-3000]rule deny ip sou 10.1.1.10 0.0.0.0 destination 172.16.1.0.0.255
[Huawei-acl-adv-3000]rule den ip source 150.20.0.0 0.0.0.255 destination 10.1.0. 15 0.0.0.0

[Huawei-acl-adv-3000]dis this
#
acl number 3000  
 rule 5 permit ip source 10.1.1.0 0.0.0.255 destination 172.16.1.0 0.0.0.255 
 rule 10 permit ip source 150.20.0.0 0.0.0.255 destination 10.1.0.0 0.0.0.255 
 rule 15 deny ip source 10.1.1.10 0 destination 172.16.1.0 0.0.0.255 
 rule 20 deny ip source 150.20.0.0 0.0.0.255 destination 10.1.0.15 0

Look at the order of these 2 ACLs. Although the order of rules was the same in both cases, the final ACLs look different.

Read More »