You may wonder, “why a deep dive into transport queues, is there so much to tell about it?” Well, that’s the first thing I thought too. But after some investigation and writing down some things I discovered that there is much to tell about them. In this article, I will discuss which transport queues exist and how they work together to deliver message(s) to a mailbox or another mail server. After that, I will explain which tools you can use to manage the transport queues.

Let’s start with the definition of a queue. What does the dictionary tells us about it? Queue means to form a line, and to wait for services.

In Exchange, the queues are used to temporarily store messages waiting until they are passed through to the next stage. Exchange contains five types of queues: submission queue, mailbox delivery queue, remote delivery queue, poison message queue, and the unreachable queue.
Depending on the routing of messages, they are placed in a specific queue.

What are the queues used for? Below is an overview:

• Submission queue, all messages received will always first be placed in the submission queue. The submission queue receives the messages via SMTP-receive, the Pickup directory, or the store driver. Once placed in this queue, the categorizer will scan messages. Once scanned the categorizer will place the message in the correct queue. This can be the delivery queue or the undeliverable queue.  The categorizer makes this decision by checking the location of the recipient. The submissions queue can be found both on Edge and Hub servers, each Transport server can only have one submissions queue.
• Mailbox delivery queue, messages in this queue are waiting to be delivered to a mailbox server in the same site. The mailbox delivery queue can only be found on a Hub server. Unlike the submission queue a Hub server can have multiple mailbox delivery queues.
• Remote delivery queue, as the name already says this queue is used for delivering messages to remote servers by using SMTP. This queue can be both seen on an Edge and Hub server and both servers may have multiple queues.  Queues will be automatically created when needed and will be deleted when there will be no message in the specific queue for 3 minutes. Messages with the same destination will be placed in the same queue. When having an Edge server this are external domains or external send connectors. When you don’t have an Edge server these can also be seen on the Hub server. Besides this the Hub server contains queues for messages that need to be delivered to other Hub servers in other site(s).
• Poison message queue, Exchange will move messages that may screw up your Exchange environment after a server failure. All messages which are placed in this queue will be permanently suspended so they will not be automatically be processed further. An administrator can check this queue and may decide to process the message further, this is a manual task.  If an administrator thinks the message may screw up his environment he may delete the message from the queue. Messages which will end up in this queue may be harmful in their content and/or format. Another reason can be an agent which caused the transport engine to fail when it processed the message.  When this queue is empty you won’t see it when using the queue viewer or the get-queue command.
• Undeliverable queue, this queue contains messages that can’t be routed to their destinations. This can happen in several cases: the mail server responsible for receiving the mail is not available, the internet-connection has dropped, etc.

Figure 1. Overview of the transport pipeline

In both Exchange 2007 and Exchange 2010 the queues are stored in an ESE database.  The default location of the database is C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue.

By default, this directory will contain the following files:

As all ESE databases this database has also a log file and a checkpoint file. Changes are written to the log files and memory first before they are committed to the database. The reason for this is to improve the performance. After the log file is flushed to the database Exchange will update the Checkpoint file to contain the name of the latest log file flushed to the database. If a server fails the Checkpoint file is used for recovery purposes to determine where to start the actual recovery. In fact, this principle is the same as for the ‘normal’ Exchange database and its log files.

There is only one difference between ESE databases used for mailboxes and the ESE database used for the queue. Circular logging is on by default, on mailbox databases this is not recommend in most cases but can be enabled in certain circumstances  for example during a migration of a large environment.

This has one disadvantage on the queue database, it can’t replay log files which are restored from a backup to recover the transport database.

Transport queue management

For managing the queues you can use the following tools:

• Queue viewer
• PowerShell, the Exchange Management Shell contains several commands to manage the queue: get-queue, suspend-queue, retry-queue, get-message, resume-message, remove-message and export-message.

Queue viewer

Let’s start with the queue viewer  the tool can be opened by performing the following steps:

• Open the Exchange Management Console
• Select the toolbox
• In the right piece of the screen select the Queue Viewer

The queue viewer will open the local transport database by default if available, but you can also open the transport databases located on other HUB Transport servers in the same Exchange organization.  What the GUI does is executing PowerShell cmdlets to retrieve the information, in case of getting an overview of the queue’s the get-queue cmdlet is executed. One exception to this is the queue viewer that is opened on the Edge Transport server, this one cannot connect to other server but can only view the transport database of the local server.

Figure 2. The Queue Viewer as part of the “Tools” option in Exchange Management Console

There are two tabs: queues which will display the current queues  and messages which will display the messages that are in a queue on the specific server.

When the queue tab is selected you may see a lot of queues which may make it a little bit difficult to have a good overview.  For this issue you can use the filter option which will let create a filter to display only the information you would like to see. Follow the steps below to create a filter:

• Select a queue property, for example next hop domain
• Select a comparison operator, for example =
• Enter a value , for example Microsoft.com

This filter will only display queues which have as next hop domain microsoft.com.

Another option is to create a filter which only filters queues which have a specific status. This can be useful if you only want to see queues that contain messages which are not delivered yet.

To create an overview of messages in a specific queue create a filter manually or just right click the queue and select the option view messages. This will open a new tab and will automatically apply a filter which selects only messages from the specific queue.

Using the queue viewer it’s possible to suspend the queue. This will freeze all messages that are current in the queue and will pause processing them until you resume the queue.

In some cases a queue will have the status retry, in this case the Exchange server wasn’t yet able to deliver the message(s) to the next hop mail server. If a queue is in retry state it will try to deliver the message(s) after one minute specified time. If you do not want to wait before it’s the retry counter has expired just select the queue and select the action retry. This will immediately retry to send the messages.

Besides the filter that can be used for the queues, you can also create a filter for the messages tab. With this filter will make it possible to filter out messages with specific SCL value, source IP-address, subject, etc.

The current status can be seen when getting the properties of the message, just as a queue a message can have several states:

• Active, if in the delivery queue the message is being deliver to the next hop mail server, if in the submission queue the categorizer is processing the message.
• Pending remove, the administrator choose to remove the message from the queue but the message is already in the delivery queue. A message will be deleted if it reenters the queue because of an error but will be delivered if no error occurs.
• Pending suspend, the administrator choose to suspend the message in a queue but the message is already in the delivery queue. The behavior is the same as with the pending remove status.
• Ready, the message is waiting to be processed.
• Retry, Exchange couldn’t deliver the message the last time it tried it, Exchange will try to deliver the message again if the retry counter expires.
• Suspended, the processing of the message has been suspended and the message will not be processed further till the administrator resumes the message.

As an administrator you will have a few actions when right clicking a message: suspend, resume and remove messages with or without NDR.

Exchange Management Shell

Via the Exchange Management Shell you can perform the same actions. There’s one thing you can do with the Shell which you can’t do via the GUI, export messages.

But let’s start with the commands you can use to manage the transport queues. To view the current queues you will need to use the command get-queue, for example:

Figure 3. Queue Information using the Exchange Management Shell

Above you see an example of the output of get-queue |fl command. In this case only the submission queue exists and has zero messages in it.

As with using the queue a filter can be applied to the queue, for this we will need to add some extra parameters:

The above example will display all queues which have more than 100 messages in it and which has the status of retry.

To change the queue status we can use the following commands: suspend-queue, resume-queue and retry-queue. The first will freeze the queue and prevents messages from being processed further, resume-queue will start processing the messages again and last but not least retry will retry to send the messages in the queue instead of waiting till the retry counter has expired.

Below some examples of the commands:

Will suspend all queues which have more than 100 messages in it and have the status retry.

Will resume the queue with messages send to contacts at Microsoft and have the status suspended.

Will resume all queues which contain messages that are send to contacts at Microsoft and have the status retry.

There is only one exception for the retry-queue command, it can’t be used with the parameter
-Resubmit $true on the poison queue. If you wish to like to reprocess a message which is in the poison queue you will need to use the resume-message which is explained later.

When you want to view messages in specific queues you can use the command get-message:

Just like with the queues you can also suspend, resume and remove messages from the queue. As said earlier there is one extra feature which can only performed by using PowerShell, exporting messages.

But let’s start from with the first command: suspend-message.

The above command will suspend the message with ID 3 which is in the queue called unreachable located on hub.

This command will suspend all messages which contain the word Newsletter in the subject and will suspend the message(s) until the administrator will resume the message(s) by using the resume-message command.

Applies the same filter as the suspend-message command only with the resume-message the message will be processed further if the current state is suspend.

Earlier we suspended all messages with the word Newsletter in the subject, by using the resume-message it will start processing the message again.

The last command is the export-message command, this will let you export a specific message or a complete queue to eml files which can be opened via Outlook.

The above command will export the message 1234 that’s located in the Microsoft.com queue on the Hub01. It will export the message to a file called export.eml and will be placed in the Microsoft Export directory.

Before running this command you will need to meet a few prerequisites:

• The directory where the files will be placed in must exist
• You must have write permissions to the directory

The above example was an easy one so let’s give an example of a more difficult one:

Quite a large command with all those parameters but what does it do, let’s start from the beginning:

• First it will retrieve all messages in the queue which have as senderdomain microsoft.com and are in a queue on Hub01.
• Then for each message it will find that meets the criteria it performs the action between {}, I will describe this step by step.
• First the $Temp parameter is set to the directory where we would like to store the exported messages, in this case c:\Microsoft Export\  and we will use the value of the InternetMessageID as the filename for the exported message. Because the InternetMessageID contains < and > brackets and this are characters which may not be used in filenames we will replace them by _
• Next step if to perform the export itself, the $_.Identity will be automatically filled with the results of the get-message command. After that we tell the command to convert the binary data so we can save the message content in the location specified after the -Path parameter.

For now this was the first part of the article. In the next part of this article we will dive deeper and will explain how you can change configuration settings related to transport queues.