CoNetrix

Blog: iLO

Upon rebooting a Terminal Server that had resource issues, we could not log back into the server through RDP.  We could log in through iLO, and it was apparent that the logins were working but they were very slow.  Upon examining the services, we could see that the IPSEC service was not started. 

Trying to manually start the service gave the following popup: "Could not start the IPSEC Services service on Local Computer.  Error 2: The system cannot find the file specified."  The event logs also showed that TCP/IP was in blocking mode. 

Disabling the service and rebooting restored all network communication, but trying to start the service would drop all connectivity again and slow down the server.  I found another article that said that IPSEC may need to be rebuilt.  When I looked for the registry keys for IPSEC, they were not there.  After I ran the following commands, the registry keys were populated, and IPSEC was able to run properly.

To rebuild IPSEC, follow these steps: [more]

  1. Click Start, click Run, type regedit, and then click OK.
  2. In Registry Editor, locate and then click the following subkey: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\IPsec\Policy\Local.  (In my case, the server’s registry ended before IPsec.  If this is the case, skip to step 6.)
  3. On the Edit menu, click Delete.
  4. Click Yes to confirm that you want to delete the subkey
  5. Quit Registry Editor
  6. Click Start, click Run, type regsvr32 polstore.dll, and then click OK.
Networking, Terminal Server, iLO, IPSEC,
 

I learned the reason that VMware suggests having service consoles for ESX hosts on at least two distinct networks last week. I was troubleshooting intermittent backup issues with Veeam on a customer network and couldn’t really find any pattern to the failures. Two or three backups in a row would run successfully, then 5 in row might fail. The behavior was very random. However, the failures were always on Virtual Machines associated with a specific ESX host. At first I thought the host was healthy, but after watching the VI client for an extended period of time, I noticed that the ESX host would drop offline (showing disconnected in the VI client) and then come back online again.  This indicated the problem wasn’t just affecting the management/backup server. [more]

In order to level set my troubleshooting efforts, I decided to reboot this ESX host. However, after the reboot, I could not connect to it with the VI client. I could ping the IP assigned to the service console, but couldn’t SSH or connect via the VI client. I logged in via iLO and found that an ifconfig at the command line returned IP = 0.0.0.0…..interesting. So what is responding to my pings. I checked the arp cache on one of the switches and found that a thin client had been plugged in that had the same IP as my LAN service console. What is really odd is the MAC address for the thin client was all zeros AND the IP I was using for the LAN service console is not even available to be distributed by DHCP. I was not able to connect to the thin client to see how it was configured, but I was able to connect to ESX host via a second service console port that I placed on the iSCSI network. The management/backup server has a connection to the iSCSI network to do backups to disk so I was able to change the LAN-facing service console IP to another IP and everything started working fine. The backup issue was obviously being caused by changes in the arp entries on the backup server between the thin client and the ESX host. So, be aware that at boot-time, if ESX determines that the IP it is using for a service console is already in use, it just rips it out of the configuration and continues to boot with NO WARNINGS or ERRORS on the console.

Networking, Veeam, iSCSI, iLO, VMware, ESX, backup,
 

I had been troubleshooting a DL380 server and replaced bad memory.  I had the server powered off and connected to it using iLO.  I used iLO to send power signal to the server so that I could watch it boot up.  For some reason right after I powered on the server, I lost connectivity to iLO and the blue UID light on the server remained on.

Once Windows came up, I checked the HP System Management software, and it did not list an iLO management processor like the other servers did.  I figured that iLO was hung, so I shutdown the server.  Still no response from iLO, so I checked BIOS with a physical monitor and keyboard.  Upon boot, it did not show the message to press a function key to configure iLO.[more]

In order to reset iLO, I had to completely unplug both power supplies from the back of the server.  After powering the system on again, I then saw the option to configure iLO.  I saw that it had an IP address but I still could not connect.  The blue UID light was off though.  After the system came back up, I had to reset the iLO interface through the HP System Management software before it would work again correctly.

Networking, iLO, HP, Windows 2008 Server,