Blog: Exchange

I was installing Exchange 2007 SP2 Update Rollup 4 the other day at one of our network support client's sites. This particular customer has 6 exchange servers that needed the update. The first couple of servers took forever to install the update rollup. It really shouldn’t take 30 minutes to install a 50 MB download. After two servers the other guys working the maintenance window were already waiting on me so I had to make up some ground. After some searching (I didn’t have to look far…its posted on the “how to install exchange updates” page -> http://technet.microsoft.com/en-us/library/ee221147%28EXCHG.80%29.aspx) I found that during the install, if setup can’t connect to the CRL web site, the installation takes an abnormally long time to finish.

The reason is that each time the installer compiles an assembly, it has to check the code signing certificate used to sign the assembly against the CRL. If that connection can’t be made, each attempt must time out before moving on to the next assembly. Ok, so why can’t the CRL be downloaded? At this particular customer location, the problem was due to a Barracuda web filter that requires authentication. The attempts to download the CRL come across as anonymous and are blocked. It could also happen if an ISA server is in place and only certain groups of users are allowed internet access via security group membership. Whatever the reason, the work around is to turn off “Check for publisher’s certificate revocation” option in Internet Explorer. There is a registry key you can change, but I found the option in IE.  [more]

1. Start IE
2. Go to Tools -> Internet Options
3. Click on Advanced -> Security
4. Click to clear the “Check for publisher’s certificate revocation” check box
5. After the update is installed, reverse your change

Troubleshooting ActiveSync without access to a users smart phone is difficult. Using an device emulator is possible but time consuming. I recently ran across a nice site from Microsoft which is useful in checking several different items associated with Exchange remote connectivity.

https://www.testexchangeconnectivity.com    [more]

The Microsoft Exchange team is an interesting group. For years, it seemed like they didn’t listen to any user feedback about the product. The GUI was way too complicated and automation procedures were difficult because there was not a convenient CLI for the product. When Exchange 2007 came out, even though the GUI lacked a little functionality, the product as a whole was way better and it incorporated a lot of feature requests that Exchange admins have been asking for. Most notably, the LCR, CCR, and SCR features that allow local HA and DR for Exchange with a lot less complexity than past versions.

Now, enter Exchange 2010. Exchange 2010 takes everything that we were introduced to in Exchange 2007 on the availability side (LCR,CCR, SCR) and removes it. Yep, removes it. All the availability features have been merged into one technology called Database Availability Groups (DAGs). DAGs have a really nice feature set on paper…I say on paper because I haven’t really implemented them in practice, but the one deal killer at least for most of our customers is that DAGs REQUIRE Enterprise Edition OS licenses. [more]The reason is that DAGs still depend on pieces of Microsoft Cluster Services. This kinda stinks because in Exchange 2007 you could implement database HA with LCR and DR with SCR and never buy any Enterprise licenses. Like I said, the Exchange team is an interesting group….I wonder if they discussed this and decided...”that’s no big deal…doesn’t everyone have Enterprise volume licensing”.

When setting up a public folder don’t forget to allow contributor access for anonymous users so that users external to your company will be able to send emails to the public folder.

If you track the undelivered messages you will see this error if anonymous users do not have contributor access: 550 5.2.0 STOREDRV.Deliver. To fix this you can use the following Exchange Shell command:

Add-PublicFolderClientPermission -Identity \"public folder name" -User anonymous -AccessRights contributor

While investigating why only some Outlook events showed up on a users iPhone calendar. I found that items created by the user or accept as meeting invitations were on both the iPhone and Outlook. But if it was created by another organizer, such as an assistant, on behalf of the user the event did not show on the iPhone calendar.  [more]

The user was running an older firmware version 3.0.1, which was before the calendar invitation fix. I downloaded and installed the latest firmware version 3.1.2 and installed it on the iPhone. Once installed the iPhone resynchronized and all calendar events including those created by an organizer were present on the iPhone and Outlook.

So… installing firmware 3.1 or newer, breaks the free tethering option that can be installed on the iPhone. Not installing the latest firmware does not allow for the new calendar invitation options. Therefore a choice must be made.

A customer’s Outlook Active Sync stopped working for their phones. I connected to their 64-Bit Exchange 2007 server and found that nothing in IIS was working.

Looking at the event logs found where .NET 1.1 had been installed right before IIS stopped working.

IIS 6.0 supports both 32-bit and 64-bit. However IIS 6.0 does not support running both at the same time. ASP.NET 1.1 runs only in 32-bit mode. ASP.NET 2.0 runs in 32-bit mode or in 64-bit mode. Therefore, if you want to run ASP.NET 1.1 and ASP.NET 2.0 at the same time, you must run IIS in 32-bit mode. However Microsoft Exchange Server 2007 only supports Microsoft.NET 2.0, 64-bit version.

The problem was that Microsoft .NET Framework 1.1 was installed on the Exchange server and broke IIS since it is running in 64-bit mode for use with Exchange 2007.

• I uninstalled .NET 1.1
• Went to a command prompt
• Used the following command to disable the 32-bit mode:
  cscript %SYSTEMDRIVE%\inetpub\adminscripts\adsutil.vbs SET W3SVC/AppPools/Enable32bitAppOnWin64 0
• Then used the following command to install the version of ASP.NET 2.0 and to install the script maps at the IIS root and under:
  %SYSTEMROOT%\Microsoft.NET\Framework64\v2.0.50727\aspnet_regiis.exe -i
• Restarted IIS, World Wide Web Publishing Service, and HTTP SSL

[more]After completing the steps above everything began functioning as it should.

I found that the reason .NET 1.1 was installed, was because all the management server sessions were being used and access to the VM host was needed. An installation of Virtual Infrastructure  Client 2.0 was started but canceled on the Exchange server. VI Client 2.0 requires .NET 1.1 which was automatically installed before the VI installation was canceled.

Progress. Innovation. One small step for man. Call it what you will but the advancement of software usually comes at a price with some bumps along the way. Apple, while a good company that puts out a good product, is mortal like the rest of us and as such is subject to the same development bumps and bruises. That was my experience this week when an executive assistant at one of our clients came to me in a panicked state saying “Help! I just sent out a meeting invite to over 30 executives and it keeps sending the invitation over and over and over again! People are getting upset!” [more]

Immediately I put together a lineup of potential offenders and began working my way through:

1. Exchange message queues
2. Online spam filter reinjection
3. Notification of meeting change/update
4. Corrupt/Malformed meeting event
5. Possible wrong address in the list (we’ve seen this happen before) 
6. MAPI profile/client issues.

Troubleshooting:

1. An inspection of the Exchange queues revealed nothing out of the ordinary, and the Exchange logs showed that each repeated meeting request appeared to be a new/separate message that was being received (and dutifully sent out) by the Exchange server. Nope, that’s not it.
2. Online spam filter reinjection into Exchange was not a possibility since every recipient was internal… the spam service never saw the message. Innocent.
3. Since the same meeting was supposedly being re-sent, I thought that it may be possible that the meeting would re-send whenever a user would update/respond to/propose a new meeting time for the calendar event. After looking at the executive assistant’s sent items as well as the inboxes of several attendees, none of this was true… the meetings were actually being re-sent. Strike three. 
4. Thinking that there may be some oddity in the meeting event such as a reoccurring event, I had the user delete the meeting then recreate it while I watched. I noticed the user used a distribution list when inviting attendees.
5. Some of our engineers have seen some quirks when using a distribution list with incorrect/invalid email addresses. I had the user recreate the distribution list from scratch, populating it only by clicking on addresses in the Global Address List. Re-created the meeting with new distribution list. Same behavior.
6. Thinking that the problem may be a MAPI profile issue due to the Exchange logs indicating that each message was a separate submission from the client, I went to the user’s office to rebuild their MAPI profile. In doing so I realized that the user was on a thin client. Before building her Terminal Server MAPI profile I asked the user what time she had left the previous day. She said she left right at 5:00pm and had logged off of the Terminal Server at that time. The last meeting that was resent went out at 5:14pm. Hmmm…

Solution:

At this point I had seemingly ruled out the client aspect as well as the server aspect of the problem, what could be left? Blackberry! I asked the executive assistant if she had a Blackberry, thinking that surely the Blackberry Enterprise Server was the guilty party since the problem was happening when she was logged out. “No, I don’t have a Blackberry… a couple of months ago I got an iPhone instead.” At this point I was getting desperate so I asked her to power off her phone for the remaining 6 hours of the workday. Magically not a single meeting invite was sent out. After that I asked her to power it on. Immediately a repeated meeting invite was sent! I asked her if anything had changed on her phone recently to which she replied ,”actually, I just upgraded my phone this weekend to the new 3.0 iPhone OS”. A quick Google confirmed that other users who had upgraded to the 3.0 Apple iPhone OS and had sent meeting requests to a distribution groups had experienced the same problem. A call to Apple support yielded no help as a “Product Specialist” (referred to as “iPhone Ninjas” by Apple Tier 1 support, no joke) told me that they don’t have any record of that happening to anyone else, call Microsoft since it’s an Exchange account.  So, until iPhone OS 3.1, it looks like users will not be able to use distribution groups when creating meeting requests. Isn’t there an App for that?

If your Entourage Mac for Exchange Server stops synchronizing new e-mail from your Exchange server it might be caused by a corrupt inbox cache.  A customer was having a problem where their Mac was not synchronizing with their Exchange server and the inbox cache was the culprit for them.  The solution is to go to the properties of the mail folder and select to clear the cache.  After doing so, synchronization will download all email from the Exchange Server.  I found the solution for this problem posted on the following site: www.macwindows.com/entourage

We recently had a customer that switched Internet service providers.  After making the switch, users were getting “sending delayed” messages from the Exchange server.  The problem turned out to be caused by mis-configured DNS settings in the Exchange 2003 SMTP service.  Some e-mails appeared to be going through fine, while others were delayed and eventually dropped.  Sometimes, messages would go through, the other person would respond, and then they’d get a delayed notification for the originally sent message.  After some e-mails that were sent to CoNetrix were completely dropped, I started looking more closely at the SMTP service configuration.  I went through every setting and eventually found some entries for the old ISP's DNS servers buried in the following location: [more]

Servers -> SERVERNAME -> Protocols -> SMTP -> Default SMTP Virtual Server -> Properties -> Delivery tab -> Advanced -> Configure

This configure allows you to specify “external DNS servers”.  Apparently, based on the name, someone thought the real “external” DNS servers should be used (instead of the local DNS service that uses the external servers as forwarders).  I removed the old ISP server entries and replaced them with the external DNS servers of the new ISP as a test.  Once I did this, e-mail started sending immediately.  I then changed the entry to the local IP of the server (so the local DNS service would be used).  Things continued to work.  Setting those DNS entries is not part of our normal server setup procedures, so I'm not sure where the DNS entries originally came from.  They may have been populated by some "wizard", so keep the SMTP DNS settings in mind if you ever have a similar problem.