Chrysjan's Messaging Blog

The Exchange Team has released a new version of the Exchange 2010 Mailbox Requirements Calculator. This version includes the following improvements and new features:

For more information, check the source article

Download the calculator here

Early 2008 we have posted a blog entry with a VB script that generates some pre-canned reports that are based on message tracking logs. The script has proven to be useful in understanding Microsoft’s Exchange work load and guide some design decision for Exchange 2010. This script was developed by Todd Luttinen, Principal Program Manager at Microsoft.

During the development of Exchange 2010, we needed to extended our log analysis beyond just message tracking and to answer a variety of questions that assist with design decisions. This exposed a bottle neck with having a single script that has all the parsing and analyzers bundled together.

This resulted in the creation of ExLogAnalyzer by Victor Boctor, Principal Architect at Microsoft. ExLogAnalyzer was developed in C# with the following goals:

• Separation of syntax and semantics.
• Multi-Server support (process log files that span multiple servers). Log events across servers are processed in chronological order.
• Multi-Log Type support (process / cross reference logs of different log types to produce a single report). Log events across log types are processed in chronological order.
• Provide an extensibility model to support rapid development and distribution of extensions (to support new log types) and analyzers (to encapsulate reporting logic).
• Ability for the community to develop their own analyzers or even extensions.
• Support for Exchange 2007 / 2010 log types.

The main shift in this model, compared to the previous script, is that ExLogAnalyzer is built as a framework that can be used to analyze Exchange as well as possibly any other log format. New log types are supported via plugins called “extensions”. Extensions are responsible for doing all the parsing and converting of log lines into events, where each event triggers a method and passes all the pre-parsed information as the event arguments. The specific reports are also implemented as plugins known as “analyzers”, where each analyzer handles the events it is interested in and does the appropriate accounting and report generation (typically in CSV format). Implementing each analyzer in isolation (rather than one script that answers multiple questions) makes it much simpler to develop, understand and distribute such analyzers. Such extensions and analyzers can also be easily shared given the plugin model. The following simple diagram summarizes the architecture of this tool:

The ExLogAnalyzer is now released to the community with the following extensions / analyzers available out of the box:

• Message Tracking Log
• MsgTrkTopSendersByDeliverLogAnalyzer – Generates the top 1000 senders based on mailbox deliveries. Messages to the internet are not counted.
• MsgTrkTopSendersBySubmitLogAnalyzer – Provides an analysis of the sender load distribution based on number of messages sent from their mailboxes.
• MsgTrkTopRecipientLogAnalyzer – Generates the top 1000 recipients based on mailbox deliveries. Messages to the internet are not counted.
• MsgTrkMessageSizeDistributionLogAnalyzer – Provides an understanding of the message size distribution.
• MsgTrkRecipientNotFoundLogAnalyzer – Discover and summarize recipients for which “Recipient Not Found” error was generated.
• MsgTrkMailflowVisualizerLogAnalyzer – Generates a directed graph showing the server being analyzed and all the inbound / outbound mail flow paths.
• MsgTrkComponentLatencyPercentileLogAnalyzer (E14) – Analyzes the latencies of the different components and determines the latencies experienced by the specified percentiles of messages.
• MsgTrkDuplicateDeliveryLogAnalyzer – Analyzes the sources for duplicate deliveries to Store. Note that end users don’t see such duplicates.
• MsgTrkEventFrequencyLogAnalyzer – Provides an understanding of the distribution of the event + source combinations.
• MsgTrkEventTimeDistributionLogAnalyzer – Provides an understanding of the event distribution over time with a per hour resolution.
• MsgTrkExpandLogAnalyzer – Analyzes the distribution list expansion load on the system.
• MsgTrkReceiveLogAnalyzer – Analyzes the distribution of the sources for the messages received by a server or a set of servers.
• Smtp Receive Log
• SmtpReceiveWorkLoadLogAnalyzer – Analyzes the SMTP receive work load over time while tracking tarpitting, client time outs, etc.
• SmtpReceiveDelayedAckLogAnalyzer (E14) – Analysis of delayed ack performance over time. This report provides an overview of the redundancy that is achieved for legacy systems via delayed ack.
• SmtpReceiveFormatterLogAnalyzer – Re-writes the logs with each session in a separate file, it also reformats the log so that the common session information is included in the header, hence, making the session details more readable.
• SmtpReceiveSeparatorLogAnalyzer – Re-writes the logs with each session in a separate file while maintaining the exact log format.
• SmtpReceiveSessionIndexLogAnalyzer – Provides a summary of all sessions processed within the provided logs.
• Connectivity Log
• ConnectivityWorkLoadLogAnalyzer – An analyzer that samples the connections over time. This analyzer generates a CSV file per source (e.g. SMTP or MAPI).
• ConnectivityStatsLogAnalyzer – An analyzer that provides the frequency of sessions, failed and DNS failures per source + destination combination.
• ConnectivityFormatterLogAnalyzer – Re-writes the sessions as a file per session, moved all the common session information to the header to make the sessions more readable.

Sample Reports

Following are some samples to provide a feel of the outputs of some of these analyzers.

Mail Flow Visualizer (demonstrated possible visualization using directed graphs):

Message Size Distribution:

SmtpReceiveFormatterLog (log re-writing for splitting sessions and making them more readable):

# Session Id: 08CBDCECE3DDF231
# Start Time (local): 2009-07-28T11:07:46.922
# End Time (local): 2009-07-28T11:07:46.953
# Start Time (UTC): 2009-07-28T18:07:46.922Z
# End Time (UTC): 2009-07-28T18:07:46.953Z
# Disconnect Type: Local
# Connector Id: MyServer\MyServer_CrossForest
# Local End Point: 157.54.7.153:25
# Remote End Point: 157.54.71.39:4183

0000000,+,,
0000000,*,None,Set Session Permissions
0000000,*,SMTPSubmit SMTPAcceptAnyRecipient SMTPAcceptAuthenticationFlag SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender BypassAntiSpam BypassMessageSizeLimit SMTPAcceptEXCH50 AcceptRoutingHeaders AcceptForestHeaders AcceptOrganizationHeaders SMTPAcceptXShadow,Set Session Permissions
0000000,>,220 MyServer E14 Cross Forest,
0000000,<,EHLO otherhost.otherforest.microsoft.com,
0000000,>,250-MyServer.redmond.corp.contoso.com Hello [157.54.71.39],
0000000,>,250-SIZE 10485760,
0000000,>,250-PIPELINING,
0000000,>,250-DSN,
0000000,>,250-ENHANCEDSTATUSCODES,
0000000,>,250-AUTH,
0000000,>,250-8BITMIME,
0000000,>,250-BINARYMIME,
0000000,>,250-CHUNKING,
0000000,>,250-XEXCH50,
0000000,>,250 XSHADOW,
0000000,<,XSHADOW 3333YTkxYjEtYzE1OC00NDcxLWI4OTktMDA2NDI5YmVmZWRlQFRLNUVYMTRNTFRXNjUxLndpbmdyb3VwLndpbmRlcGxveS5udGRldi5taWNyb3NvZnQuY39t,
0000000,>,250 q7rdaFIdKk3NNRTbjRsjrQ==,
0000000,<,MAIL FROM:<sender@contoso.com> SIZE=25477 XSHADOW=70136df4-c89b-4700-9654-b642c4eb78bb,
0000000,*,08CBDCECE3DDF231;2009-07-28T18:07:46.922Z;1,receiving message
0000000,<,RCPT TO:<receiver@contoso.com> ORCPT=rfc822;receiver2@contoso.com,
0000000,>,250 2.1.0 Sender OK,
0000000,>,250 2.1.5 Recipient OK,
0000000,<,XEXCH50 1136 2,
0000000,>,354 Send binary data,
0000015,>,250 2.0.0 XEXCH50 OK,
0000015,<,BDAT 25477 LAST,
0000031,>,250 2.6.0 <DB82FD8C490D4F43ACE766C04B23A7050F0F12@someserver.otherforest.contoso.com> [InternalId=16796908] Queued mail for delivery,
0000031,<,XQDISCARD 50,
0000031,>,251 OK, no discard events,
0000031,<,QUIT,
0000031,>,221 2.0.0 Service closing transmission channel,
0000031,-,,Local

Top Senders by Submit (analysis yielding CSV – full report has top 1000):

Sender Distribution by Submit (analysis yielding CSV):

Distribution Group Expansion Analyzer (analysis yielding CSV):

Getting started

• Download ExLogAnalyzer from here.
• Checkout the Power Point slide deck included in the download for more details about ExLogAnalyzer.
• Use ExLogAnalyzer and its distributed sample analyzers to analyze your logs.
• Develop your own analyzers. Visual Studio and Visual C# Express Edition are the recommended tools. However, you can use Notepad or your favorite editor, given that ExLogAnalyzer detects and compiles the analyzer CSharp files at runtime.
• Provide us with feedback about ExLogAnalyzer, sample analyzers or the development process.
• Share your analyzers or ideas for useful new analyzers with the Exchange community.

Source: MSExchangeTeam

It is common for Exchange organizations to include different versions of Exchange. Interoperability scenarios present challenges to administrators for maintaining the different tool sets required to manage different versions of recipient and configuration objects in the organization. It is important to understand the implications of using Exchange 2010 management tools in these situations. Some of this information may change in the future.

Exchange Object Versions

Exchange management tools generally take some action against objects that are stored in the Active Directory (AD) directory. These objects can be recipient objects (Mailbox enabled accounts, mail enabled accounts, mail contacts, etc…) or they can be Exchange configuration objects (Exchange servers, mailbox databases, connectors, etc…)

Exchange uses a property that tags an object as “belonging” to one specific version of Exchange. The property in AD is msExchangeVersion. To quickly check the Exchange version of an object, from the Exchange Management Shell, use the “Get” cmdlet for that object type, and issue a command like the following which gets the ExchangeVersion property value for a mailbox enabled user:

[PS] C:\> Get-Mailbox user1 | Format-List ExchangeVersion
ExchangeVersion : 0.10 (14.0.100.0)

In this example, the version is 14.0.100. This is the version of Exchange that was used to create this recipient object. For a management tool to be able to manage this object, it must be for version 14.0.100 or higher.

All Exchange objects are stamped with an ExchangeVersion value, and it is in this way that management tool interoperability is controlled.

Exchange 2003 Interoperability

The Exchange System Manager (ESM) is the primary management interface for Exchange 2003. The ESM does not have a direct interface for managing recipient objects, instead relying on the Active Directory Users and Computers (ADUC) to be installed on the same machine to expose property pages for recipient objects. The following limitations apply to management tool interoperability:

• Exchange 2010 management tools can be used to view Exchange 2003 recipient objects, but not the other way around.
• The ESM cannot be used to manage any configuration objects for Exchange 2010.
• The Exchange 2010 management tools can be used to view certain Exchange 2003 configuration objects, but not make any modifications.

Exchange 2007 Interoperability

Both Exchange 2007 and Exchange 2010 use their own versions of EMC and EMS. Interoperability limitations between Exchange 2007 and Exchange 2010 management tools can be more difficult to understand because of the similarity between the tools.

The following matrix will help to determine what can be done from each tool.

EMC Management Tool Interoperability:

Additional Scenarios

Move Mailbox

• Exchange Server 2007 EMC cannot move an Exchange Server 2007 mailbox to an Exchange Server 2010 server, or move an Exchange Server 2010 mailbox to an Exchange Server 2010/Exchange Server 2007 server.
• Exchange Server 2010 EMC can move an Exchange Server 2010/Exchange Server 2007 mailbox to an Exchange Server 2010/Exchange Server 2007 server.
  • When an Exchange Server 2007 mailbox is moved to an Exchange Server 2010 sever, the mailbox is upgraded to Exchange Server 2010 version. After the version upgrade, the mailbox should still work with the associated policies (ActiveSync, OWA, UM, etc) that are in a down-version. (Open Issue)
  • When moved back to an Exchange Server 2007 server from an Exchange Server 2010 server, the mailbox is downgraded to Exchange Server 2007.

Move OAB

• Exchange Server 2010 EMC can move an Exchange Server 2007/Exchange 2003 OAB to an Exchange Server 2010/Exchange Server 2007/Exchange 2003 server.
• Exchange Server 2010 EMC can move an Exchange Server 2010 OAB to an Exchange Server 2007/Exchange 2003 server.

Send Connector Source/Send Server

• Exchange Server 2007 EMC can set either an Exchange Server 2007 or Exchange Server 2010 transport server as the source server of a send connector.
• Exchange Server 2010 EMC can set either an Exchange Server 2010 or Exchange Server 2007 transport server as the source server of a send connector.

Queue Viewer

• Exchange Server 2007 Queue Viewer snap-in cannot connect to an Exchange Server 2010 server to view Exchange Server 2010 queues or messages.
• Exchange Server 2010 Queue Viewer snap-in cannot connect to an Exchange Server 2007 server to view Exchange Server 2007 queues or messages.

The limitations that are imposed in the EMC can be translated to limitations that also exist in EMS when using the same underlying PowerShell commands. For example, when viewing the properties for a mailbox in EMC, the Get-Mailbox command is used in the background to populate the property information. If a change is made, the Set-Mailbox command is used to make the change to the mailbox object. In some situations it may be possible to view information for an object that was created with one tool version, but not make any changes.

The limitations depend on specific conditions and can be related to the configuration management levels exposed in EMC: Organization Configuration, Server Configuration and Recipient Configuration.

Organization Configuration

Organization objects are global in nature and can be managed using both versions of management tools with some specific exceptions.

Exchange 2010 mailbox and public folder databases only appear under the Organization Configuration node in the Exchange 2010 EMC. Exchange 2007 mailbox and public folder databases appear under the Server Configuration node in Exchange 2007 EMC.

Messaging Records Management features for Exchange 2010 (MRM v2) only appear under the Organization Configuration node in the Exchange 2010 EMC. MRM v1 features appear in both versions and can be managed from both tools.

Server Configuration

Server management is very clear cut in EMC interoperability scenarios. Exchange 2007 servers do not appear in the result pane of the Exchange 2010 EMC and vice versa.

EMS commands for server configuration can be used from either version of EMS as long as the cmdlet does not depend on a specific configuration object version on the target. A message is generated and no changes are made if the cmdlet is not compatible.

Recipient Configuration

Recipient Objects that are created with Exchange 2007 management tools can be managed using both tool sets. Recipient Objects that are created using Exchange 2010 management tools can only be managed using Exchange 2010 management tools. Exchange 2007 SP2 management tools have logic to validate actions based on object type. When attempting to manage an Exchange 2007 object, an informational message is seen similar to this:

After clicking the OK button the properties are not activated for change but can be viewed. A similar message is generated with trying to modify an Exchange 2010 recipient object using an Exchange 2007 EMS command as shown in the following example where Set-Mailbox is used in Exchange 2007 EMS to attempt a change to an Exchange 2010 mailbox:

[PS] C:\>set-mailbox user2 -CustomAttribute1 “some value”
Set-Mailbox : Object is read only because it was created by a future version of Exchange: 0.10 (14.0.100.0). Current supported version is 0.1 (8.0.535.0).
At line:1 char:12
+ set-mailbox <<<< user2 -CustomAttribute1 “some value”
+ CategoryInfo : NotSpecified: (0:Int32) [Set-Mailbox], InvalidADObjectOperationException + FullyQualifiedErrorId : 855E94FC,Microsoft.Exchange.Management.RecipientTasks.SetMailbox

Using Get-Mailbox in this instance works as expected because the command only retrieves information but does not make any changes.

Side-by-Side Management Tools

Exchange 2010 and Exchange 2007 Service Pack 2 (SP2) make it possible to deploy the different versions of Exchange Management Shell (EMS) and Exchange Management Console (EMC) from both products on the same workstation in a configuration called Side-by-Side Deployment. Both sets of management tools are then available from the Programs menu as shown:

Both versions of EMC and EMS can be opened and used independently from the other version. The following shot shows both versions of EMC open on the same machine. Alternatively, the snap-in for Exchange 2007 EMC and Exchange 2010 EMC can both be added to the same console.

Prerequisites for Side-by-Side Deployment

The following Windows operating systems are supported for the Exchange Management Tool role:

• Windows Vista SP2 x64 bit
• Windows 2008 SP2 x64 bit

The prerequisites for installing Exchange 2007 and Exchange 2010 Management tools in a side-by-side deployment are:

• .Net Framework 3.5 and 3.5 SP1
• Windows Remote Management v2.0
• Windows PowerShell v2.0
• Windows Installer version 4.5 is required to install Exchange 2007 SP2 on Vista SP2, but is not required on Windows Server 2008 SP2 because it is already included.
• Internet Information Services (IIS) features IIS 6 Metabase Compatibility (Web-Metabase) and IIS 6 Management Console (Web-Lgcy-Mgmt-Console).

Both sets of management tools use similarly named files. To limit confusion between file sets, the Exchange 2007 files and Exchange 2010 files are stored in different locations. The Exchange 2007 files by default are stored in folders directly under the Program Files\Microsoft\Exchange Server directory while the Exchange 2010 files are stored in the V14 folder in the same directory as shown:

Source: MsExchangeTeam