Blog: Exchange 2007

I had a user that was trying to access OWA from home.  The user had the correct website and the credentials were being entered correctly, however they kept getting an error message about insufficient access.  This error was preventing the user from using OWA at all.  I could see the user account showed that they had logged in successfully by looking at the timestamps on the Active Directory User object. 
 
The problem turned out to be caused from non-inherited permissions in Active Directory.
 
The following information explaining why this happened was found from a Technet forum thread.

If your Exchange 2007 OWA is failing for a user after the mailbox is migrated from Exchange 2003 to Exchange 2007, the user account should be checked on the security tab under advanced to see if it has "Allow inheritable permissions from the parent to propagate to this object and all child objects. Include these with entries explicitly defined here."

1. Open up Active Directory Users and Computers
2. Go to the View menu, Advanced.
3. Locate the user in AD, right click, properties.  Jump to the security tab.
4. Click "Advanced" next to the "For special permissions or for advanced settings, click Advanced.
5. Click "Allow inheritable permissions from the parent to propagate to this object and all child objects. Include these with entries explicitly defined here." Check box and apply.
6. Click OK and OK again.

Once changed and replicated OWA works. This is checked by default but is turned off for accounts with administrative privileges.
 
So how does this get turned off? Well if the account is an administrative account or was ever an administrative account previously. It will be turned off automatically.
 
Reference the following.
XADM: Do Not Assign Mailboxes to Administrative Accounts http://support.microsoft.com/kb/328753 which says
 
By not assigning mailboxes to accounts with administrative permissions, you avoid security issues related to "elevation of privilege" attacks. For example, in an elevation of privilege attack, a security hole exists in which Group X is made a member of the Domain Administrators group, and access control lists (ACLs) exist on Group X that permit Group Y to modify Group X. In this situation, members of Group Y can make themselves members of Group X and so become a member of the Domain Administrators group.
 
To help guard against such security issues, the Administrator account and accounts that are members of these security groups are not permitted to inherit permissions. On the Security tab of the group or account's properties page, you can see that the Allow inheritable permissions from parent to propagate to this object check box is not selected. Moreover, if you click to select this check box, a Microsoft Windows 2000 system task soon clears it automatically. Clearing the check box is a function of Windows 2000 intended to prevent hackers from playing with security and inappropriately increasing their permissions to the level of administrator.
 
As a side effect of this inheritance setting, if you do try to use a mailbox assigned to an administrative account, you may not be able to log on to or resolve the mailbox. Also, in Exchange System Manager, although the Administrator account can have an Exchange 2000 alias and an Exchange 2000 mailbox, it does not have e-mail addresses. The Recipient Update Service, which updates the e-mail addresses and several other attributes, does not have the authority to update objects if the Allow inheritable permissions from parent to propagate to this object check box is not selected.

I set up an distribution group on our Exchange server for when we need to communicate with a new customer.  Email to the group from the inside worked fine, but any external mail sent to the address would bounce with a generic "Unknown recipient".  I asked a couple of our other network engineers to look at it, and one of them found the RequireSenderAuthenticationEnabled attribute was set to true, so he changed it with this command: [more]

"Set-DistributionGroup "groupname" -RequireSenderAuthenticationEnabled $false".

I was installing Exchange 2007 SP2 Update Rollup 4 the other day at one of our network support client's sites. This particular customer has 6 exchange servers that needed the update. The first couple of servers took forever to install the update rollup. It really shouldn’t take 30 minutes to install a 50 MB download. After two servers the other guys working the maintenance window were already waiting on me so I had to make up some ground. After some searching (I didn’t have to look far…its posted on the “how to install exchange updates” page -> http://technet.microsoft.com/en-us/library/ee221147%28EXCHG.80%29.aspx) I found that during the install, if setup can’t connect to the CRL web site, the installation takes an abnormally long time to finish.

The reason is that each time the installer compiles an assembly, it has to check the code signing certificate used to sign the assembly against the CRL. If that connection can’t be made, each attempt must time out before moving on to the next assembly. Ok, so why can’t the CRL be downloaded? At this particular customer location, the problem was due to a Barracuda web filter that requires authentication. The attempts to download the CRL come across as anonymous and are blocked. It could also happen if an ISA server is in place and only certain groups of users are allowed internet access via security group membership. Whatever the reason, the work around is to turn off “Check for publisher’s certificate revocation” option in Internet Explorer. There is a registry key you can change, but I found the option in IE.  [more]

1. Start IE
2. Go to Tools -> Internet Options
3. Click on Advanced -> Security
4. Click to clear the “Check for publisher’s certificate revocation” check box
5. After the update is installed, reverse your change

The Microsoft Exchange team is an interesting group. For years, it seemed like they didn’t listen to any user feedback about the product. The GUI was way too complicated and automation procedures were difficult because there was not a convenient CLI for the product. When Exchange 2007 came out, even though the GUI lacked a little functionality, the product as a whole was way better and it incorporated a lot of feature requests that Exchange admins have been asking for. Most notably, the LCR, CCR, and SCR features that allow local HA and DR for Exchange with a lot less complexity than past versions.

Now, enter Exchange 2010. Exchange 2010 takes everything that we were introduced to in Exchange 2007 on the availability side (LCR,CCR, SCR) and removes it. Yep, removes it. All the availability features have been merged into one technology called Database Availability Groups (DAGs). DAGs have a really nice feature set on paper…I say on paper because I haven’t really implemented them in practice, but the one deal killer at least for most of our customers is that DAGs REQUIRE Enterprise Edition OS licenses. [more]The reason is that DAGs still depend on pieces of Microsoft Cluster Services. This kinda stinks because in Exchange 2007 you could implement database HA with LCR and DR with SCR and never buy any Enterprise licenses. Like I said, the Exchange team is an interesting group….I wonder if they discussed this and decided...”that’s no big deal…doesn’t everyone have Enterprise volume licensing”.

When setting up a public folder don’t forget to allow contributor access for anonymous users so that users external to your company will be able to send emails to the public folder.

If you track the undelivered messages you will see this error if anonymous users do not have contributor access: 550 5.2.0 STOREDRV.Deliver. To fix this you can use the following Exchange Shell command:

Add-PublicFolderClientPermission -Identity \"public folder name" -User anonymous -AccessRights contributor

Public folder permissions can be challenge with Exchange 2007, primarily because Microsoft removed the options from the GUI and replaced them with Powershell commands.  It’s particularly difficult to make mass changes because there’s not a single command to propagate permissions to all subfolders.  However Microsoft does provide Powershell scripts to help with this.  By default they are saved at C:\Program Files\Microsoft\Exchange Server\Scripts.  [more]

More information about these scripts is available at http://technet.microsoft.com/en-us/library/aa997966%28EXCHG.80%29.aspx.

A customer’s Outlook Active Sync stopped working for their phones. I connected to their 64-Bit Exchange 2007 server and found that nothing in IIS was working.

Looking at the event logs found where .NET 1.1 had been installed right before IIS stopped working.

IIS 6.0 supports both 32-bit and 64-bit. However IIS 6.0 does not support running both at the same time. ASP.NET 1.1 runs only in 32-bit mode. ASP.NET 2.0 runs in 32-bit mode or in 64-bit mode. Therefore, if you want to run ASP.NET 1.1 and ASP.NET 2.0 at the same time, you must run IIS in 32-bit mode. However Microsoft Exchange Server 2007 only supports Microsoft.NET 2.0, 64-bit version.

The problem was that Microsoft .NET Framework 1.1 was installed on the Exchange server and broke IIS since it is running in 64-bit mode for use with Exchange 2007.

• I uninstalled .NET 1.1
• Went to a command prompt
• Used the following command to disable the 32-bit mode:
  cscript %SYSTEMDRIVE%\inetpub\adminscripts\adsutil.vbs SET W3SVC/AppPools/Enable32bitAppOnWin64 0
• Then used the following command to install the version of ASP.NET 2.0 and to install the script maps at the IIS root and under:
  %SYSTEMROOT%\Microsoft.NET\Framework64\v2.0.50727\aspnet_regiis.exe -i
• Restarted IIS, World Wide Web Publishing Service, and HTTP SSL

[more]After completing the steps above everything began functioning as it should.

I found that the reason .NET 1.1 was installed, was because all the management server sessions were being used and access to the VM host was needed. An installation of Virtual Infrastructure  Client 2.0 was started but canceled on the Exchange server. VI Client 2.0 requires .NET 1.1 which was automatically installed before the VI installation was canceled.

I was working on a Symantec Mail Security appliance the other day that was passing a bunch of spam through. The problem should have been easy to fix; the license had expired. All that was necessary was to upload the new license file. I emailed it to myself and opened OWA from a server on the customer network to download the file. When I tried to install it, I kept getting an invalid license error. I called Symantec tech support and they regenerated the license and emailed it to me thinking the license was corrupt. That one worked fine. Hmmm…curious, I tried opening the license file I had downloaded from OWA. No surprise why it wouldn’t work, it was an empty text file. I did some testing emailing that file (unzipped) back and forth from account to account. It wasn’t a spam/AV filter problem. Turns out the problem was that I downloaded the file via OWA having sent it unzipped. With Exchange 2007 SP1 prior to update rollup 6, if a file that contains XML data is attached to a message, the XML content in files is removed when you open or save the attachment by using OWA. The license file was XML content…the one from Symantec support worked because it was zipped. http://support.microsoft.com/kb/950675

One of our clients has an Exchange 2007 environment that has been in production about a year.  Recently they have started to get some complaints about performance. From time to time, users will see the pop-up noting Outlook is waiting on Exchange server. I began troubleshooting using the Exchange performance troubleshooting tools that are packaged with the Exchange Management GUI. Results showed that the server was experiencing extremely high RPC/MAPI traffic. I began to look for a tool that I had used several times in Exchange 2003 called ExMon, which is a real-time MAPI connection monitor. I found references to it online, but the download was nowhere to be found. It turns out you have to call Microsoft to get it for Exchange 2007…it isn’t available as a download on the Microsoft.com site. Using the ExMon tool and a lot of google searching led me to the root cause of the issue: Blackberry Enterprise Server.

Turns out that a lot of people fight this exact problem. BES enabled users generate between 4x and 16x the amount of MAPI traffic a regular “high usage” outlook user would generate…its even documented in BES admin guide that you should plan for each BES user to be equivalent to 3.6 users. And this is extremely conservative. [more]From the performance numbers I have gathered, in the case of our client the number is more like 6-6.5x. There are reports online from BES admins noting over 10x in there environments. BES requires a special type of mapi dll in order to function. That is why you have to install the Exchange 2003 Management tools on the BES server. It abuses the mapi protocol using combinations of mailbox notifications and full mailbox scans to implement its functionality. The load increases exponentially as mailbox sizes grow. It just makes sense that the BES enabled users would be the ones with the largest mailboxes. In this case there were a handful of users with > 1GB mailboxes that are BES users…bad combination. Bottom line, if BES will be used enterprise wide, planning should included the increased load BES will create…most importantly the IOPs on the disk subsystem. MAPI calls are expensive disk operations.