One of our clients has an Exchange 2007 environment that has been in production about a year. Recently they have started to get some complaints about performance. From time to time, users will see the pop-up noting Outlook is waiting on Exchange server. I began troubleshooting using the Exchange performance troubleshooting tools that are packaged with the Exchange Management GUI. Results showed that the server was experiencing extremely high RPC/MAPI traffic. I began to look for a tool that I had used several times in Exchange 2003 called ExMon, which is a real-time MAPI connection monitor. I found references to it online, but the download was nowhere to be found. It turns out you have to call Microsoft to get it for Exchange 2007…it isn’t available as a download on the Microsoft.com site. Using the ExMon tool and a lot of google searching led me to the root cause of the issue: Blackberry Enterprise Server.
Turns out that a lot of people fight this exact problem. BES enabled users generate between 4x and 16x the amount of MAPI traffic a regular “high usage” outlook user would generate…its even documented in BES admin guide that you should plan for each BES user to be equivalent to 3.6 users. And this is extremely conservative. [more]From the performance numbers I have gathered, in the case of our client the number is more like 6-6.5x. There are reports online from BES admins noting over 10x in there environments. BES requires a special type of mapi dll in order to function. That is why you have to install the Exchange 2003 Management tools on the BES server. It abuses the mapi protocol using combinations of mailbox notifications and full mailbox scans to implement its functionality. The load increases exponentially as mailbox sizes grow. It just makes sense that the BES enabled users would be the ones with the largest mailboxes. In this case there were a handful of users with > 1GB mailboxes that are BES users…bad combination. Bottom line, if BES will be used enterprise wide, planning should included the increased load BES will create…most importantly the IOPs on the disk subsystem. MAPI calls are expensive disk operations.