A. Testbed Topology and specifications
As shown in Fig. 6, our testbed consists of five machines (IPv4 client, B4, AFTR, IPv4 server and attacker). They were all based on VMware workstation VMs, and those machines were built upon CentOS-7 images. Every machine had the following specifications:
• RAM: 3 GB
• Hard disk: 20 GB
• CPU: 1 core
The host PC that hosted the whole testbed had the below specifications:
• OS: Windows 10 Pro.
• RAM: 16 GB.
• Hard disk: 1TB.
• CPU: Intel-Core i7, 4 physical cores.
IPIP6 tunnel was built between B4 and AFTR which took care of encapsulating IPv4 packet inside IPv6 tunnel then de-capsulating it. As for the NAT process on AFTR side, iptables rule was configured to masquerade the source IP address to the AFTR’s network interface (ens35).
We note that the above setup contains an important simplification compared to Table 1: the Softwire-ID is missing from the connection tracking table. This change did not influence our experiments, whereas it simplified the setup of our testbed: we could use iptables instead of a real AFTR implementation.
B. Attack scenarios
i. Spoofing Attack from within VMnet-11 (an inside job)
The proposed attack as shown in Fig. 6 is Spoofing the ingress tunnel end point (ens39), which is the softwire tunnel interface of the B4 router, then sniffing / logging traffic or perform MitM attack against egress tunnel end point (AFTR).
The attack was based on the “thc-ipv6” toolkit [38], which has several tools within it. One powerful tool was chosen to perform the attack (four2six). What it actually does is manually creating a crafted packet of IPv4 inside IPv6 (encapsulated packet) and send it over the channel to the victim (AFTR).
This crafted packet is actually a spoofed one imitating that it was sent by the B4 router, where it has the same specifications as of a genuine B4 originated packet:
• IPv6 Src: 2001:db8:0:1::2
• IPv6 Dst: 2001:db8:0:1::1
• IPv4 Src: 10.0.0.2
• IPv4 Dst: 198.51.100.2
In fact, the spoofed attack went through AFTR then to the end receiver (IPv4 server) and series of ICMP echo requests and replies had been exchanged between the attacker and victim. In more details, what happened is:
• The AFTR received the malicious packet.
• The AFTR saw the packet as a 4in6 packet, therefore, it then decapsulated the packet.
• The AFTR then forwarded the extracted IPv4 packet with source address of 198.51.100.1 towards the IPv4 server (198.51.100.2).
• IPv4 server sent ICMPv4 echo reply back.
• The AFTR saw the later packet and encapsulated it in IPv6 packet.
• The echo reply from the AFTR, which is an IPv6 packet with IPv4 packet in its payload, is sent back to its destination IPv6 of address 2001:db8:0:1::2.
• Figure 7 shows three different terminals, the top one is the incoming packets at attacker interface, the one in the middle is the B4 incoming traffic from the AFTR side, and at the bottom is the incoming traffic at IPv4 client.
• B4 received the IPv6 packet carrying an IPv4 packet with ICMP echo reply because it is the legitimate owner of the 2001:db8:0:1::1 IPv6 destination address.
• The Attacker saw the same reply because its NIC runs in promiscuous mode.
• The interesting thing here is illustrated by IPv4 client receiving an ICMP4 echo reply for a request that he didn’t request in the first place, which has been issued by the attacker initially.
• Echo replies in Fig. 7 are circulated in red.
ii. Attack from within VMnet-10 (compromised B4 network)
It is also worth mentioning that B4 by design is also capable of routing native IPv6 packets, which takes us to Fig. 8, where we added IPv6 address to interface ens38 at the B4 machine in order to process and forward incoming IPv6 packets.
A real-life spoofing scenario could also be stopped before reaching our internal network of AFTR (VMnet-11 in our example). Therefore, another attacking scenario was proposed in Fig. 8, where the attacker is not sitting directly in AFTR network, but outside it. The idea behind such an attack is an attacker having access to our home router network, which gives him direct contact with the CPE. This access allows the attacker to send a crafted packet with spoofed IPv6 address of a B4 router or any other IPv6 address from the internal network of AFTR (VMnet-11 in our example).
Fortunately for the user, the attack didn’t go through, B4 router was always dropping any incoming packet from the attacker-2 machine that has an IP address (a spoofed one) from the pool of (VMnet-11). It seems like an internal security feature within the machine.
iii. Source port exhaustion attack from within VMnet-10
At first, the attack was conducted using the same platform, regular specs windows-based computer, however, the results were not stable and not reproducible due to limited cores on the host PC. Therefore, we made the decision to use the resources of NICT StarBED, Japan. A “P” series node [39] was used, which is a Dell PowerEdge 430 server with the following specifications: Two Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.1 GHz CPUs having 16 cores each and 348 GB 2400MHz DDR4 SDRAM.
Windows 10 Pro operating system was installed, and the same process was repeated by building the testbed using VMware workstation player and CentOS-7 based virtual machines. The specifications for each machine were as follow:
• RAM: 4 GB
• Cores: 4 cores, except for AFTR having 6 cores.
• Disk: 20 GB.
Before disclosing the attack details, it is important to explain the mathematical sense behind it. In networking world, ports are counted based on 16 bits standard. The 16-bits was chosen when the TCP and UDP standards were designed [40]. As a result, the maximum number of ports equals 216 = 65,536. This number applies to TCP and UDP ports. In our experiment, we focus on the UDP ports and how to exhaust them by sending too many queries (UDP requests) in short period of time. However, not all ports on range are usable for establishing new connection, because ports between 1 and 1024 are called well-known ports and they are reserved for specific functions such as FTP, HTTP, DNS, etc. Therefore, the total range of ports that NAPT (Network Address and Port Translation) device can use as UDP source port numbers equals 65,536–1024 = 64,512 ports.
Figure 9 shows the attacking scenario-3 by sending too many AAAA queries from “IPv4 Client-1” and “IPv4 Client-2” machines. In order for the attack to function, few files must be correctly configured, such as:
• /sys/module/nf_conntrack/parameters/hashsize
The HASHSIZE is actually the size of hash table storing the lists of conntrack entries [41], which is better of as base of 2, which means that hashsize could be 214 = 16,384.
• /proc/sys/net/netfilter/nf_conntrack_max
It represents the maximum number of allowed conntrack entries that netfilter can keep running simultaneously. It was set to hashize*8 = 16,384*8 = 131,072. This number will have a significant impact when we run our attack later, so that our NAT table doesn’t get full easily.
• /proc/sys/net/netfilter/nf_conntrack_udp_timeout
This timeout means that after 30 seconds, the already used UDP ports are ready to be re-assigned by the kernel, we left it out as the default value of 30 seconds.
• /etc/security/limits.conf
This file controls the number of maximum open files at the same time, where we added the below line:
# root hard nofile 1000000
To perform this kind of attack, another tool can be used, where a huge number of DNS queries can be sent to the target machine (AFTR), which is called “dns64perf++” [42] (described in [43]). When all parameters (on the attacking machine side) are set correctly (number of requests, delay between requests, etc.), the AFTR pool of source ports will be exhausted, and the machine will not be able to process incoming packets anymore till the UDP timeout (port allocation timeout in AFTR machine) is up. The below command was used on each of our attacking clients:
/dns64perf + + v4 198.51.100.2 53 0.0.0.0/5 60000 1 1 60000 400000 0.1
What this command really does is sending 60,000 queries from each client. For two clients, this number reaches 120,000 queries, which is still under 131,072 (nf_conntrack_max). As a result, our NAT table will not be overfilled with packets that exceed its capacity. Another interesting fact in the command is the delay of 400,000 nanoseconds between queries, which means 2,500 queries/s, which means, for two clients, we are sending a total of 5,000 queries/s.
The attack was executed by an automated script from client 1, which does the followings:
• Accesses AFTR remotely (via ssh) and runs tshark (traffic monitoring software) and captures the traffic at ens35 interface.
• Runs the attack from IPv4-client − 1 (the same machine).
• Accesses IPv4 client-2 remotely (via ssh) and runs the same attack with the same parameters.
• Waits till attacks are finished on both clients.
• Accesses AFTR remotely (via ssh) and stops tshark.
• Sends the tshark results as pcap file to the host machine, so it can be read in the Wireshark software.
Figure 10 shows the exhaustion of the source ports clearly. To prove that we must have only 64,512 sent queries, which comes from subtracting the well-known 1024 ports from the total of 65,536 ports. Figure 10 shows that AFTR processed only 64,536 packets at ens35 interface. After digging deeper, we found out that there are more packets embedded within Wireshark results such as 20 ICMP and 4 ARP packets. As a result, the reaming packets are 64,536–20 – 4 = 64,512, which is exactly the number we were anticipating.
In principle, we sent 60,000 queries from each client (120,000 in total), AFTR was able to process no more than 64,512 packets and it ran out of ports in around 14 seconds. This failure of service supposed to last until UDP session timeout at AFTR is over and new source ports are available to be re-allocated.
C. Mitigation method
Several IDS & IPS (Intrusion Detection & Prevention systems) were tested in order to identify and drop the spoofed packet from the legitimate one. The process proved to be quite hard to implement, especially when our attacker machine is located inside the access network of the ISP (VMnet-11). Therefore, a workaround had been put in place and iptables rules were emplaced to do the job.
1) First attack mitigation: -
An ip6tables rule was configured at AFTR in order to allow the incoming traffic at ens34 interface of the AFTR only under the condition that the incoming packet’s source MAC address belongs to B4 ens39 interface (see Fig. 6).
ip6tables -A INPUT -i ens34 -s 2001:db8:0:1::2 -m mac ! --mac-source 00:0c:29:67:14:14 -j DROP
2) Second attack mitigation: -
Despite the fact that this attack didn’t go through, and spoofed packet had been dropped, a proposed solution is presented here in case of building the topology using another machine (different Linux distribution for instance), where such attack could be prevented at B4 router side with the following rules:
iptables -A FORWARD -i ens38 -s 10.0.0.0/24 -j ACCEPT
iptables -A FORWARD -i ens38 -j DROP
The first rule accepts packets with source address from our private IPv4 address network (10.0.0.0/24), while the second one drops every packet that approaches ens38 interface and needs to be forwarded. Those two rules take care of a malicious packet as it reaches the B4 side and before it heads towards the AFTR (see Fig. 8).
3) Third attack mitigation method: -
One of the ways to mitigate this attack or at least reducing its impact, is by adding two (or more) IP addresses to AFTR ens35 interface (see Fig. 9), where the whole calculation will be double (or triple, depends on how many IP addresses we add) due to the fact that extended NAT works by binding IP / port pair together. The fact that ens35 has two IP addresses now, gives the AFTR more flexibility by having one additional pool of exactly the same port numbers of 64,512 ports, but they will be coupled with another source IP address. In case that two IP addresses were not enough to mitigate the attack, system administrator can easily add third, fourth, and more IP addresses as needed. On the practical side, we added two extra IP addresses on ens35 interface on AFTR. As a result, the AFTR was able to process all incoming 120,000 packets, unlike last time when it stopped after 64,512 packets.
Another mitigation method is possible through rate limiting process which limits the number of packets per second that can be sent and received by the NIC (Network Interface Card) [44], it also enables the administrator to assign bandwidth restriction on specific traffic such as ICMP, UDP, etc. Some research work [45] and [46] have proposed the rate limiting mitigation mechanism after noticing the difference and the asymmetry between incoming and outgoing traffic in the designated network.