Blog: Networking

If you want something wired right, sometimes you have to do it yourself.

This story started out with a new T1 connection being installed for a customer to connect to a new router.  Before I arrived on location, it was understood that the T1 connection was ran by AT&T and ready to go.  Upon arrival, I checked the networking closet where the new router was going.  I looked for the usual biscuit jack to connect the T1 to the router.  After being unable to find it, I called AT&T and had them tell me where it was.  The smartjack was outside the building with no extension on the line to the inside.  This required an electrician to come out and extend the T1 into the building. 

After the electrician came out, we had the customer connect the cat5 cable into the biscuit jack and router’s T1 card.  The electrician had already left by this time.  The serial connection showed down and further tests concluded that AT&T was not seeing the router.  At the other end of the T1 connection (point to point T1), it was showing looped when testing from the problem end.  The electrician said that he saw “fire” on the line and refused to return saying that it was a problem with AT&T.

Later, AT&T returns back onsite to confirm that the smartjack is working and places a loopback on the biscuit jack.  They concluded that there were some wiring problems with the biscuit jack and had fixed the problem.  AT&T says they were able to loop back to their test equipment, but the router was still not coming up.   At this point, we go to Cisco TAC and get them to send us a new T1 card to try thinking that the current one is faulty.

I receive the new T1 card and arrive onsite.  I install the new T1 card and check the interface to find it still down.  On the card, the alarm light was lit amber and the CD (carrier detect) was off.  At this point, I’m thinking that the card isn’t the problem, but the problem is still with the wiring.  The kicker here is that I put my loopback plug on the biscuit jack and verified that the other end’s router showed the circuit as up and looped.  I put the original card back in the router and installed the new card into the other available slot so that there were now both T1 cards in the router.

I went outside and opened the box to the network enclosure.  Instead of finding a nice simple smartjack, I find this: [more]

An RJ-48 connector from the smart jack feeds these other 4 connectors into the posts.  There were two sets of 4 posts.  The top set of four said T1, the bottom four said "Data".  As a note, T1 only requires 4 copper wires to connect.  Later examinations showed that only the T1 wires were even connected and nothing went to the bottom four pins for data (further wire tracing was necessary to determine this).

After one of our other network engineers and I began looking at the diagrams for the wiring enclosure and the biscuit jack pin outs, we could tell that something was still not quite right.  I removed the biscuit jack which had been wired in as white/blue, blue, empty, white green, green, empty, brown, white/brown.  We know that T1 should be using pinouts of 1,2,4,5.  From the picture above, we can see that green and blue pairs are being connected to the T1 posts.  The brown and orange wires aren’t doing anything here.

I cut off the biscuit jack and started putting RJ-45 plugs on the cable, and we tried switching the green and blue pairs around.  This time when I plugged the cable into the router, the CD light lit up and the alarm went off.  We could see the circuit as up, but it was going up and down constantly.  We tried changing the wires around in different orders leaving the greens on 1 and 2, and blues on 4 and 5.  After finding more diagram information on the enclosure and tracing wires, we came to a conclusion that we had it correctly set at White/green, green, empty, white/blue, blue, empty, empty, empty.

The circuit was still going up /down so I then decided to try moving the cable into the new card that had been put in the other slot on the router.  This move fixed the circuit problem going up and down, and it stayed up. 

The next challenge was testing pings across the line, and there were a few packets being dropped.  We changed the clock rate source on the serial interface to be internal and this fixed the dropped packets.

Again, I wanted to point out that the loopback plug worked by showing the connection up, but it did not appear that the wires necessarily had to be in the correct order.


 

At a customer site, running a Baracuda firewall, iPhone users were unable to check for devices updates. Everytime the user would have iTunes check for device updates it would fail. One of the rules in the Barracuda is to allow iTunes updates.  The rule to allow iTunes updates was enabled, but the traffic was still being blocked. I monitored the weblog and found the iPhone version check was being blocked by a streaming media filter. I added a URL pattern to allow access to iTunes.apple.com and iPhone update traffic could get through the Barracuda.


 

I was troubleshooting an application called QuickFile which uses FTP to be able to transmit certain Tax data at one of our customer sites. When you launched the application or when you tried to send the data it would give you an error saying that it could not connect to the FTP server. I worked on allowing outbound ftp access from this computer but it was still not working. After watching the logs on the ASA I could see the FTP connection was successfully being established. I then called the vendor and they sent me a breakdown of what the program is trying to do at that point and I found that it had to modify some files in the QuickFile program directory during the process. I then gave the user rights to that directory and it started to work again. So be careful not to completely trust the error messages that applications display as they can sometimes be misleading.


 

The time would not stay synchronized on my Windows 7 machine.  It looked like it was configured to sync with DC3, but it was not working.  After some research I found a hotfix, http://support.microsoft.com/kb/978714, that fixed it.  I used this command to configure the time synchronization:

w32tm /config /manualpeerlist:time-a.timefreq.bldrdoc.gov /update /syncfromflags:manual

See http://support.microsoft.com/kb/223184 for additional settings in the registry.  This KB article is for Windows 2000.  I could not find a KB article for Windows 7, but I think most of these settings still work.  There are other documents from Microsoft about this, but I just liked this KB article best.


 

Secunia is one of the many security firms who maintain teams of researchers looking for vulnerabilities in software applications.  I have seen their name credited on several vulnerability notices from CERT and SANS.  They offer a software vulnerability tool called Secunia Personal Software Inspector that is free for personal use.  It scans your system looking for all executable files and then compares them to their database of current software versions/vulnerabilities.  I have used it on a couple of systems that I believed to be current and found at least half a dozen out-of-date or vulnerable apps.  Apart from the security benefits, it can also be an easy way to see if there have new releases for any of your software.  For example, Secunia PSI informed me that a new version of Wireshark was available for my home computer even though it didn't find any security vulnerabilities for the version I was using.  This can be much easier than individually opening each app and clicking on "check for updates", or even worse, having to go to the app's website to see if a new version is available.


 

In IIS6 you allowed large file uploads to an ASP.NET website by adding the following key to the web.config:

<httpRuntime maxRequestLength="102400"
     useFullyQualifiedRedirectUrl="true"
     executionTimeout="900" />

You would just set the maxRequestLength to a number of kilobytes that is greater than the size of the largest file you plan to upload.  We recently moved a site from a server running IIS6 to IIS7.  After the move, when our website users were having problems uploading large files.  They would submit a form, the progress bar would move for a while, then they would get a 404 page not found error.  The form was posting back to itself, so the page did exists.  Turns out we needed to add an additional web.config setting that’s new to IIS7. [more]

    <system.webServer>
        <security>
            <requestFiltering>
                <requestLimits maxAllowedContentLength=" 104857600"/>
            </requestFiltering>
        </security>
    </system.webServer>

The maxAllowedContentLength in this settings is specified in bytes.

Note: These settings exist to help revent denial of service attacks, so keep that in mind when you pick your limits.


 

During an IT security audit this week, I had the chance to test the Cisco AnyConnect VPN client from a VPN-hostile network.  The bank has implemented multiple layers of Internet filtering including web content filtering, outbound port filtering, and inbound IP address/port filtering.  As expected, the old Cisco IPSEC VPN client could not connect.  The AnyConnect client, however, connected on the first try without having to ask the bank to modify any of their access controls.

FYI, the AnyConnect client dynamically determines if it can also use a Datagram Transport Layer Security (DTLS) tunnel via UDP in addition to the SSL tunnel.  If the DTLS tunnel connects and is “healthy” the client will use both the SSL and DTLS tunnels to transmit data.  If the DTLS tunnel cannot connect or is unreliable, the client will dynamically switch to using only the SSL tunnel.  You can read more about it on Cisco’s AnyConnect FAQ page.


 

While debugging a problem that required uploading files to a website, one of the files started getting an error at the start of the upload instead of at the end.  This didn’t make sense because the problem being debugged was after the file was uploaded to the website.  This looked like another problem instead of the original problem we were hunting.   We suspected that the problem was connected to the file being uploaded instead of the website.  When we looked for the file on the Desktop it wasn’t there.  This was odd.  Went back to Internet Explorer and the file was displayed in Internet Explorer’s  file dialog.   But the file was not on the Desktop.  We tried closing Internet Explorer and restarting, but Internet Explorer still showed the phantom file. 

It turns out this is an artifact of Internet Explorer’s sandbox implementation on Vista or Windows 7.  Under certain conditions Internet Explorer writes to a virtualized Desktop folder located on the file system at: C:\Users\xxx\AppData\Local\Microsoft\Windows\Temporary Internet Files\Virtualized\C\Users\xxx\Desktop.  If you have need to move or remove the files in the “Virtualized” directory, the easiest solution is to open the file browser dialog in Internet Explorer and then move or delete the files from inside that dialog. [more]

More information is available in a article by Mark Russinovich: http://blogs.technet.com/markrussinovich/archive/2009/02/03/3174194.aspx


 

We've had issues with cached credentials not updating when a user’s password expires while he or she is away from the office. The only connection into the network is through terminal services (non-VPN) and the password is changed on the terminal server.  The problem is that the cached credentials on the user’s laptop are not updated, even after the user connects via VPN for a while.  Here is the easiest way I've found to force cached credentials to update to the new password.  While connected via VPN, have the user lock their laptop (Win+L) and then unlock the laptop using the new password. This procedure forces the laptop to check in with the domain controller and authenticate using the new password.