Blog: Exchange Server

A while back when I was setting up a new Exchange 2010 environment, I had just finished getting the new Unified Messaging server online and had begun testing. One of the issues I ran into was when I tried to administratively reset someone’s PIN. As usual, I’d go into the EMC, find the mailbox, right-click, and choose Reset PIN. Then I’d let it auto-generate one and send the email. Except it wouldn’t send the email. In the application log, I would see something like:

"E:\Program Files\Microsoft\Exchange Server\V14\UnifiedMessaging\voicemail\048696bb-3475-41d8-b497-b839c9e1daa8.txt". Error details: "Microsoft.Exchange.UM.UMCore.SmtpSubmissionException: Submission to the Hub Transport server failed. The operation will be retried. ---> Microsoft.Exchange.Net.ExSmtpClient.UnexpectedSmtpServerResponseException: Unexpected SMTP server response. Expected: 220, actual: 500, whole response: 500 5.3.3 Unrecognized command

In the end user’s mailbox, they would not receive an email in the inbox with their new PIN; however, it occasionally would appear in the Drafts folder as if Exchange had composed the email, but had forgotten to hit Send. [more]

They say hindsight is 20/20. I believe it.

Turns out, none of the receive connectors in Exchange allowed for Exchange Server Authentication. Whoops!


 

I came across a problem with OWA where it kept redirecting the external server address to the internal server address (Ex.  Mail.public.com -> servername.domain.dom).
 
After doing some thorough research, I discovered that there is a property in the IIS Metabase file that controls whether or not the server name is always used.  Microsoft KB article 834141 says that “The UseHostName property will instruct IIS to always replace the SERVER_NAME variable with the fully qualified machine name.”  This property is a Boolean value.  Setting this property to false stopped the automatic redirects and kept the external server name in the web browser. [more]
 
In order to edit the metabase, you must use commands with adsutil.vbs.  You must have the site ID of the website you want to edit the value for.  I show in the screen shot below how you can view the log file name in order to determine the site ID number.  I then show the commands used to get and set the UseHostName property for the website.


 

I had a user that was trying to access OWA from home.  The user had the correct website and the credentials were being entered correctly, however they kept getting an error message about insufficient access.  This error was preventing the user from using OWA at all.  I could see the user account showed that they had logged in successfully by looking at the timestamps on the Active Directory User object. 
 
The problem turned out to be caused from non-inherited permissions in Active Directory.
 
The following information explaining why this happened was found from a Technet forum thread.

If your Exchange 2007 OWA is failing for a user after the mailbox is migrated from Exchange 2003 to Exchange 2007, the user account should be checked on the security tab under advanced to see if it has "Allow inheritable permissions from the parent to propagate to this object and all child objects. Include these with entries explicitly defined here."

  1. Open up Active Directory Users and Computers
  2. Go to the View menu, Advanced.
  3. Locate the user in AD, right click, properties.  Jump to the security tab.
  4. Click "Advanced" next to the "For special permissions or for advanced settings, click Advanced.
  5. Click "Allow inheritable permissions from the parent to propagate to this object and all child objects. Include these with entries explicitly defined here." Check box and apply.
  6. Click OK and OK again.

Once changed and replicated OWA works. This is checked by default but is turned off for accounts with administrative privileges.
 
So how does this get turned off? Well if the account is an administrative account or was ever an administrative account previously. It will be turned off automatically.
 
Reference the following.
XADM: Do Not Assign Mailboxes to Administrative Accounts
http://support.microsoft.com/kb/328753 which says
 
By not assigning mailboxes to accounts with administrative permissions, you avoid security issues related to "elevation of privilege" attacks. For example, in an elevation of privilege attack, a security hole exists in which Group X is made a member of the Domain Administrators group, and access control lists (ACLs) exist on Group X that permit Group Y to modify Group X. In this situation, members of Group Y can make themselves members of Group X and so become a member of the Domain Administrators group.
 
To help guard against such security issues, the Administrator account and accounts that are members of these security groups are not permitted to inherit permissions. On the Security tab of the group or account's properties page, you can see that the Allow inheritable permissions from parent to propagate to this object check box is not selected. Moreover, if you click to select this check box, a Microsoft Windows 2000 system task soon clears it automatically. Clearing the check box is a function of Windows 2000 intended to prevent hackers from playing with security and inappropriately increasing their permissions to the level of administrator.
 
As a side effect of this inheritance setting, if you do try to use a mailbox assigned to an administrative account, you may not be able to log on to or resolve the mailbox. Also, in Exchange System Manager, although the Administrator account can have an Exchange 2000 alias and an Exchange 2000 mailbox, it does not have e-mail addresses. The Recipient Update Service, which updates the e-mail addresses and several other attributes, does not have the authority to update objects if the Allow inheritable permissions from parent to propagate to this object check box is not selected.


 

I am working on a large upgrade of Exchange 2007 to Exchange 2010 at one of our customers. One of the “to-do” items is to check and make sure that all of the exchange web services are healthy. The issue that I ran across was a discrepancy between what the powershell command for testing the EWS returned and what Outlook returned. The powershell command to test all EWS is the following: [more]
 
[PS] D:\Documents and Settings\cnx_user\Desktop>Test-OutlookWebServices | fl
Id      : 1003
Type    : Information
Message : About to test AutoDiscover with the e-mail address cnx_user@mydomain.com.
Id      : 1006
Type    : Information
Message : The Autodiscover service was contacted at https://server.domain.dom/Autodiscover/Autodiscover.xml.
Id      : 1016
Type    : Success
Message : [EXCH]-Successfully contacted the AS service at https://server.domain.dom/EWS/Exchange.asmx. The elapsed time was 296 milliseconds.
Id      : 1015
Type    : Success
Message : [EXCH]-Successfully contacted the OAB service at https://server.domain.dom/EWS/Exchange.asmx. The elapsed time was 0 milliseconds.
Id      : 1014
Type    : Success
Message : [EXCH]-Successfully contacted the UM service at https://server.domain.dom/UnifiedMessaging/Service.asmx. The elapsed time was 171 milliseconds.
Id      : 1006
Type    : Success
Message : The Autodiscover service was tested successfully.
 
Ok, so it looks like everything was working as expected, until I tried testing from Outlook under my account. Outlook hides the “Test Email AutoConfiguration” option well. You have to hold down CTRL and right-click the Outlook icon in the system tray to see the option. This option basically testing the EWS. Uncheck the “Use Guesssmart” and “Secure Guesssmart Authenication” options, put in your password and click Test. I kept getting this error on the log tab.
 
Autodiscover to https://server.domain.dom/Autodiscover/autodiscover.xml FAILED (0x800C8203)
 
I spent a good half hour trying to figure out what might be wrong when I finally realized that the EWS testing tool was using my username (cnx_user) followed by the AD domain. See below:

Well, it just so happens that this customer does not deploy two aliases per user mailbox (user@ad_domain & user@public_domain) as is the default for Exchange. Their default address policy only configures the public-facing alias on all mailboxes which is cnx_user@mydomain.com. The Outlook EWS tools was using an alias that didn’t exist... changing it to cnx_user@mydomain.com instead of cnx_user@domain.dom solved the problem.


 

If you have any programs that use Outlook to script objects in Exchange, you need to  be aware that the folder name for root public folder changed with Outlook 2010.  In previous versions of Outlook the root public folder is named “\Public Folders”.  Starting with Outlook 2010 the root public folder is named “\Public Folders – test@example.com” where test@example.com is the connected user’s primary email address.  Fortunately you can use the GetDefaultFolder() method on the Outlook NameSpace object to get common folders without hard coding the name of it.  For example you could call GetDefaultFolder(olPublicFoldersAllPublicFolders) to get the "All Public Folders" folder in the Exchange Public Folders store if you're using Exchange.


 

I set up an distribution group on our Exchange server for when we need to communicate with a new customer.  Email to the group from the inside worked fine, but any external mail sent to the address would bounce with a generic "Unknown recipient".  I asked a couple of our other network engineers to look at it, and one of them found the RequireSenderAuthenticationEnabled attribute was set to true, so he changed it with this command: [more]

"Set-DistributionGroup "groupname" -RequireSenderAuthenticationEnabled $false".


 

One of our clients has an Exchange 2007 environment that has been in production about a year.  Recently they have started to get some complaints about performance. From time to time, users will see the pop-up noting Outlook is waiting on Exchange server. I began troubleshooting using the Exchange performance troubleshooting tools that are packaged with the Exchange Management GUI. Results showed that the server was experiencing extremely high RPC/MAPI traffic. I began to look for a tool that I had used several times in Exchange 2003 called ExMon, which is a real-time MAPI connection monitor. I found references to it online, but the download was nowhere to be found. It turns out you have to call Microsoft to get it for Exchange 2007…it isn’t available as a download on the Microsoft.com site. Using the ExMon tool and a lot of google searching led me to the root cause of the issue: Blackberry Enterprise Server.

Turns out that a lot of people fight this exact problem. BES enabled users generate between 4x and 16x the amount of MAPI traffic a regular “high usage” outlook user would generate…its even documented in BES admin guide that you should plan for each BES user to be equivalent to 3.6 users. And this is extremely conservative. [more]From the performance numbers I have gathered, in the case of our client the number is more like 6-6.5x. There are reports online from BES admins noting over 10x in there environments. BES requires a special type of mapi dll in order to function. That is why you have to install the Exchange 2003 Management tools on the BES server. It abuses the mapi protocol using combinations of mailbox notifications and full mailbox scans to implement its functionality. The load increases exponentially as mailbox sizes grow. It just makes sense that the BES enabled users would be the ones with the largest mailboxes. In this case there were a handful of users with > 1GB mailboxes that are BES users…bad combination. Bottom line, if BES will be used enterprise wide, planning should included the increased load BES will create…most importantly the IOPs on the disk subsystem. MAPI calls are expensive disk operations.


 

I was experimenting with options for iPhone passwords - those enforced from the Exchange server.  I created a custom mailbox policy that required alphanumeric passwords.  I fiddled with it a while to see what the options meant and then went back to the original policy that just required a 4-digit PIN.  However, I was unable to go back to a numeric PIN (it kept requiring 4 characters including a special character) until I Reset All the iPhone settings (which erased my e-mail setup, network settings - including the WPA key, etc.).

So, if you're tempted to test a longer and/or alphanumeric and/or complex password on your iPhone and may want to go back to what you originally had, be prepared to Reset the phone (you don't lose applications or data but you lose all your custom settings).

 

I worked on a problem the other day in which a user with Outlook 2003 would search for an item in her inbox using a keyword in the subject of an email and the search results would come back empty even though you could clearly see items that should have been returned in her inbox. At first, I thought she just had a setting set incorrectly in her Outlook client, but after trying it with my mailbox, I saw the same behavior. I found the following powershell command VERY useful.

Test-ExchangeSearch –Identity <username>

This command inserts a test message containing a GUID into your mailbox and then searches for it to test the full text catalog. Both with accounts failed this test. Next, I tried to rebuild the index using using the following command.

ResetSearchIndex.ps1 <dbname> [more]

This command stops the MSExchangeSearch service, deletes the full text catalog and restarts the services (which then rebuilds the catalog). I found it interesting that the catalog folders and files were recreated, but the index files never grew. The catalogs for other databases/storage groups were gigs and this one was only 145 kilobytes after 2 hours which should have been plenty of time for the service to at least start crawling the database. I had tried everything I could find from google searches so I started reading through release notes for the Exchange 2k7 SP1 Update Rollup patches (there are currently 5 for SP1 and 7 for pre-SP1). I found fixes for indexing in three of them so I installed the latest one, SP1 Update Rollup 5 at the DR site and moved my mailbox. Everything worked. Moved it back and it was broken again. Pretty convincing. I installed the patch on all production Exchange servers and started another index rebuild. But, I saw behavior differently from I had seen before. When running the Test-ExchangeSearch command with no identity/username, it worked. When I used a real account it didn’t work. I also saw that the event logs said the index service was rebuilding the catalogs, but they were not increasing in size. Very strange…turns out that during a backup operation (which was running at the time when I did the second index rebuild operation), the search service will NOT start a database crawl. It waits until after the backup is done. The next day everything was running normal. Lesson learned…patch and be patient.

 

The iPhone currently supports the following security policies from Exchange (note: you must have Exchange Server 2003 SP2 or Exchange Server 2007 SP1 or greater):

  • Remote Wipe
  • Enforce password on device
  • Minimum password length
  • Require alphanumeric password
  • Require complex password
  • Inactivity time in mimutes

When you perform the remote wipe from Exchange, it restores your iPhone to factory default (note: this could take up to an hour).  The gotcha – after you perform remote wipe, be sure to “remove mobile device partnership” with your iPhone; otherwise, the next time you try to sync with Exchange it will wipe (restore to factory default) again . . .