Blog: switch

We recently deployed VRF groups on our core switches and there are a few key changes to troubleshooting/configuration.
Ping:
Standard way: ping 192.168.1.1
w/ VRFs: ping vrf NAME 192.168.1.1 (where NAME is the name of the vrf group)

Traceroute
Standard way: traceroute 192.168.1.1
w/ VRFs: traceroute 192.168.1.1 /vrf NAME

If you are trying to make a connection to a destination that resides in a VRF from the core switches you will most likely have to include some sort of VRF tagging.  This even applies to services like RADIUS, NTP, DHCP Relay, and TACACS.

Recently, I was working on our Cisco 3560E switch. I needed to create a route-map and apply it to an interface for some changes we were planning to make. I was able to create the route-map, but it wouldn’t allow me to apply it to the interface with the “ip policy route-map” command. After doing some research I realized to apply a route-map to an interface for policy-based routing our switch had to be licensed for “IP Services.” The command wasn’t even available, which makes sense being that it wasn’t supported. While upgrading the license; I tried to apply the “ip policy route-map” command again. I did a “sh run int vlan 1” to see if the command was showing up in the config and to my surprise it wasn’t listed!  I went back to the documentation and found the route-map command “set ip default next-hop” was not supported at all on the 3560/3750 switch platforms. I removed this command from my route-map and applied the route-map to the interface and everything seemed to apply correctly. Unfortunately, our whole plan revolved around the ability to use the “set ip default next-hop” command.

So when you are working with Cisco equipment there are at least 3 ways they let you know a command isn’t supported in the IOS:[more]

1. The logical way:  The command isn’t present in IOS and it can’t be used.
2. The illogical way: Allow you to apply a command, but doesn’t prompt you with an error if another “child” command is not supported.  This can only be discovered if you review the configuration and see that the command you entered is nowhere to be found.
3. And what I like to call “The Cisco Way”:  Include the command in the IOS to lead users to believe that the command is supported and works with that IOS/platform all the while not supporting the command in any variation of the IOS/Platform.

After further review, the documentation did have a note that stated the command we needed wasn’t supported on these platforms.  In summary, it is a good idea to fully read any and all documentation on supported/unsupported commands for a platform.  

The switch ports on the Cisco routers don’t appear to be as robust as a standard switch.  All clients at a clients motor bank went down the other morning.  After travelling on site, it turned out the problem was a bad cable end on one client.  All seven devices connected to the same 8-port switch on the branch router, and this one bad cable took down all seven devices.  I could move the faulty device from the router’s switch port to an external switch, and everything would work.  I went ahead and replaced the bad cable end, and reconnected all devices to the router’s switchports.

The setup documentation, on the CD that is in the box and the website, for the Linksys SRW2024 switch instructs users to use a baud rate of 19,200 to connect to the console. Upon connecting at the rate everything in the console looks like gibberish. Connecting at 38,400 opens the telnet session properly and allows access to the configuration. I verified in the settings that the baud rate is set to 38,400. Luckily I did not follow my initial thought that the firmware was corrupt.

I've recently been trying out the PuTTY Connection Manager and I think it's a very useful tool. What I find most useful is the ability to store the connection information for all of the routers/switches that you connect to regularly similar to VissionApp or RoyalTS does for terminal servers. It is currently a work in progress but the beta version is pretty stable. You can download it for free here: http://puttycm.free.fr/

I recently started reevaluating how we do port security as a result of a recent customer's information security audit.  We normally turn on port security and set the maximum MAC addresses to 1 (the default) or 2 (if there is an IP phone connected).  The default behavior is to disable the port when the MAC changes or if the number of concurrent MAC’s exceeds the maximum.

However during testing I discovered this didn’t work exactly like I expected.  Port security was enforced as long as a device stayed connected to the port.  If the port was disconnected, the switch would remove the pre-existing MAC’s and ANY new device could connect, as long as the maximum was not exceeded.  While this prevents unauthorized hubs and switches, it doesn’t prevent someone from unplugging a device and plugging in a different unauthorized device.

The solution to this is to use the sticky option on the port security interface command: [more]

• switchport port-security – enables port security, optional “maximum <n>” to set the max greater than 1
• switchport port-security mac-address sticky – turns on the sticky MAC feature

After enabling, you will notice the currently connected MAC address(es) will appear in the running config:

• switchport port-security
• switchport port-security mac-address sticky
• switchport port-security mac-address sticky 0080.6433.xxxx

This will stay in the config until the switch is rebooted, so it’s important to write the config.

Other related commands:

• show port-security address – lists all the learned MAC addresses by interface
• show port-security interface fa0/1 – shows the detailed port security settings for an interface, including enable/disable status
• clear port-security sticky interface fa0/1 – clears the learned sticky MAC addresses, must be done prior to a shut/no shut to re-enable a port disabled due to port security

When you use sticky MAC addresses you'll want to make sure that the MAC addresses are cleared off of a switch when a device is moved.  We had a laptop that was moved from one client location to another and one of the distribution switches was thinking the device was plugged into the old switch and the other distribution switch thought it was plugged ito the new switch.  This created a situation where some network traffic was reaching the laptop and some was going into a black hole.  After clearing the the sticky MAC addresses on the old switch the problem was resolved.

Update:  You might also be interested in a couple stick MAC address tips.

We have had recently had a problem with a Cisco 3560 PoE switch and a Toshiba IP phone. The issue is that when a call is made from a Toshiba IP phone (problem spanned phone and phone/firmware versions), the conversion was one way. The party on the remote end could not hear the party in private banking. We have this switch deployed several other places with the same version of IOS and don’t see the issue. Finally the mystery has been solved. After running capture on the links and doing troubleshooting, I was able to determine that the "switchport voice vlan 1" command on each interface that was used for a phone was the magic bullet. For some reason, in its given setup, this switch needs this command to make everything work whereas it isn’t needed in other places. Cisco is current still looking into the issue and last I heard it is at TAC level 4.

When adding portchannels to a switch configuration that span more than one stacked switch, it is advisable to upgrade to the latest IOS version.  The most current is:  12.2(25)SEE3