Router stops routing after changing MAC Address

["Greg Scott" <GregScott@InfraSupportEtc.com>]

Hello - This feels like a kernel issue.  I spent hours and hours and
hours looking for documentation and archives around this but did not
find anything.  

I have a Linux router and I need the ability to swap hardware without
causing downtime.  The problem, of course, is ARPs.  The NICs in the
replacement system need the same MAC Addresses as the NICs in the
original system.  I'd like this all to be in the kernel and not depend
on a daemon process that can die.

How to change MAC addresses is documented well enough - and it works -
but when I change MAC addresses, my router stops routing.  From the
router, I can see the systems on both sides - but the router just
refuses to forward packets.  Here are my little test scripts to change
MAC Addresses.

First - ip-fudge-mac.sh
[root@test-fw2 gregs]# more ip-fudge-mac.sh
ip link set eth0 down
ip link set eth0 address 01:02:03:04:05:06
ip link set eth0 up

ip link set eth1 down
ip link set eth1 address 17:20:16:01:60:03
ip link set eth1 up

echo "1" > /proc/sys/net/ipv4/ip_forward



Now original-mac.sh

[root@test-fw2 gregs]# more original-mac.sh
ifdown eth0
ifconfig eth0 hw ether 00:c1:28:01:d8:07
ifup eth0

ifdown eth1
ifconfig eth1 hw ether 00:50:da:90:e4:aa
ifup eth1

echo "1" > /proc/sys/net/ipv4/ip_forward

I have systems both on the left and right side of my test router.  Here
is some output from the router with tcpdump showing what happens.  I
replaced the first 3 real public IP Address octects with "1.2.3".  The
first set of tcpdump records shows it forarding with the original
hardware MAC Addreses are set.  We see round trips from the left side to
the right side and back with echo request and reply packets.

The second set shows what happens after changing MAC Addresses.  We only
see packets come in on the left side - but nothing happening on the
right side.  

Packet forwarding must somehow depend on MAC Addresses but I cannot find
anything anywhere that tells me how this works.  

I reproduced this problem on at least two different Linux routers - one
running 2.4.27, the other running 2.6.11-1.  Am I asking the kernel to
do something bad?  What would it take to put together a patch to change
this behavior?


[root@test-fw2 gregs]# ./original-mac.sh
[root@test-fw2 gregs]# /usr/sbin/tcpdump -i eth1 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
17:14:51.010439 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 479
17:14:51.010537 IP 1.2.3.49 > 172.16.16.1: icmp 64: echo reply seq 479
17:14:52.010448 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 480
17:14:52.010621 IP 1.2.3.49 > 172.16.16.1: icmp 64: echo reply seq 480
17:14:53.010531 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 481
17:14:53.010696 IP 1.2.3.49 > 172.16.16.1: icmp 64: echo reply seq 481
17:14:54.010716 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 482
17:14:54.010882 IP 1.2.3.49 > 172.16.16.1: icmp 64: echo reply seq 482

8 packets captured
8 packets received by filter
0 packets dropped by kernel
[root@test-fw2 gregs]# ./ip-fudge-mac.sh
[root@test-fw2 gregs]# /usr/sbin/tcpdump -i eth1 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
17:15:10.031945 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 498
17:15:11.031980 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 499
17:15:11.806487 fe80::1520:16ff:fe01:6003 > ff02::2: icmp6: router
solicitation 
17:15:12.032062 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 500
17:15:13.032154 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 501
17:15:14.032222 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 502
17:15:15.032305 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 503
17:15:15.805873 fe80::1520:16ff:fe01:6003 > ff02::2: icmp6: router
solicitation 
17:15:16.032394 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 504
17:15:17.032465 IP 172.16.16.1 > 1.2.3.49: icmp 64: echo request seq 505

10 packets captured
10 packets received by filter
0 packets dropped by kernel
[root@test-fw2 gregs]# 
[root@test-fw2 gregs]# /usr/sbin/tcpdump -i eth0 -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol
decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes

0 packets captured
0 packets received by filter
0 packets dropped by kernel
[root@test-fw2 gregs]# 


Thanks

- Greg Scott
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/