Recently we began working with a client on their transition from an on-premises Exchange 2010 deployment towards Exchange Online using the Hybrid migration method.  During the discovery phase of an engagement, we generate a point in time Mailbox Report that provides statistics on, among other things – the size and item count for each Primary and Archive Mailbox along with their corresponding Recoverable Items folders.  For the purposes of this article, we will focus on the following components:

  1. How much (total) data is to be migrated (Primary + Archive)
    1. The migration team focuses on these numbers along with other information specifics gathered during the Discovery phases such as available internet bandwidth to perform preliminary modeling around total migration duration.  This allows us to forecast and plan our migration schedule.
  2. Flag mailboxes with high item counts
    1. Mailboxes with hundreds of thousands of items impact the timeline of a respective Move Request.  The more items in a mailbox, the longer it takes to migrate
  3. Highlight what Microsoft considers large mailboxes i.e. Mailboxes over 50 or 100GB in size
    1. Depending on the O365 Subscription Plan purchased by the organization, these numbers and features vary.  You can find information about Plan limits here: Exchange Online Limits

Upon analysis of the Mailbox Report for this respective client, we quickly realized that out of 2500 mailboxes we had approximately 100 that had an Archive size of greater than or equal to 100GB.  Fortunately this was 4% of the total number of mailboxes however, sizes ranged from 100GB – 200GB.  Using the Pre-flight Tool (of which you can find my previous write up on here) to generate a readiness report on mailbox migrations, a failure is logged if the source mailbox is 100GB in size or greater:

 primarySMTPAddress status errorMessage
alexis.levin@domain.com Fail Archive size 180 GB (193 275 660 107 bytes) exceeds target quota 100 GB (107 374 182 400 bytes).
bob.wood@domain.com Fail Archive size 180.7 GB (194 27 854 253 bytes) exceeds target quota 100 GB (107 374 182 400 bytes)

 

Based on the errors in the above table, we experienced two scenarios if you attempt to migrate a mailbox with an on-premises Archive over 100GB in size:

  1. The migration will not begin and fail with the “exceeds target quota” error
  2. If the mailbox is slightly below the quota of 100GB, it will begin and may even reach a “Synced” state.  However due to expansion, the Cloud size could breach the 100GB quota limit and fail with the same error message when you finalize the move

Prior to engaging us as a partner, our client had logged a ticket with Microsoft Support on this issue and were notified that they would have to reduce the size offending mailboxes by:

  1. Exporting the contents of the Archive Mailbox to PST
  2. Reducing the size of the mailbox bringing it to under 100GB by deleting or moving content
  3. Importing the PST into the Cloud Archive Mailbox

Litigation Hold

Due to the nature of our client’s business, which mandates specific regulatory compliance, all mailboxes had been placed under Litigation Hold.  In order to purge content, the Hold would need to be lifted on the offending mailboxes thereby exposing a compliance risk to the business.  In addition, we would need to contend with the following factors:

  1. Exporting large PST files are time and resource intensive tasks
    1. The total PST volume aggregate for 100 mailboxes totaled 15TB.  This data would then need to be ingested into the target mailboxes in Exchange Online
  2. PST files introduce an inherent security due to their portability
  3. PST files are prone to corruption due to the complex nature of their file structure especially when extracted and copied over the network

*Unlimited Archiving

Microsoft had always documented that (if your subscription plan allowed for it) archiving was “unlimited.”  However, Microsoft has since recently announced that Archiving is in fact limited to 1TB.  While this may be technically true, the context here relates to mailboxes that already exist in the Cloud and does not apply to a migration scenario.  When it comes to migration, the 100GB limit is enforced at the mailbox level even when it comes to Archiving.  Microsoft achieves the 1TB of total storage for Archive mailboxes using a process known as auto-expansion.  This process is well documented here and is worth the read but here are the key points:

  1. The Main Archive mailbox is where the quota is set.  The threshold is 90GB by default and if auto-expansion is enabled, additional storage is provisioned once the quota threshold is breached
    1. If auto-expanding archiving is enabled for the mailbox – the threshold is increased by 10GB to 100GB.  The hard limit is increased by 10GB to 110GB.
  2. Data is copied from the Main Archive mailbox to the additional storage known as the Auxiliary archive.  This is documented at 50% which means in theory, when the mailbox breaches the 90GB threshold, 45GB of data would be copied from the Main Archive to the Auxiliary archive
  3. Once the data is copied to the Auxiliary archive, data from the Primary Archive is flushed reclaiming that space
  4. The process can take up to 30 days
  5. Once the Auxilliary archive quota is reached (again 100GB), the process repeats by provisioning a secondary Auxilliary archive and continues until presumably the total aggregate of all archive data hits 1TB
  6. The 100GB limit per mailbox is imposed to keep mailboxes within manageable parameters to ensure stability

Checking for Auto-expansion

Once you attempt to migrate a large archive and hit the transition threshold, you can check on the auto-expansion process.

Here is the size of the Archive mailbox for a user On-Premises Exchange.  We can see that the Archive mailbox is 183GB in size.

After migrating the mailbox to Exchange Online, connect to Exchange Online PowerShell and run the following command to retrieve the GUIDs of the Archive mailboxes:

Get-Mailbox <User> | select -ExpandProperty MailboxLocations

We can see in the screenshot below that there are 2 Archive mailboxes provisioned.  The Main Archive and an Aux Archive that has been created as the result of auto-expansion:

Running the additional command:

Get-MailboxStatistics <MainArchiveGUID> |select TotalItemSize,TotalDeletedItemSize

We see the size of the Main Archive has been reduced as a result of auto-expansion:

Running the same command against the Aux Archive mailbox displays it’s current size:

We see that approximately 46GB has been moved to the Aux Archive from the Main Archive.  And that the Main Archive has flushed the copied data.  As more data is populated into the Main Archive the auto-expansion will repeat the process.

We quickly learned that, even if we were able to migrate a large Archive mailbox we would still need to content with the auto-expansion process which would add months to our project timelines.  The only way to migrate data without compromising fidelity and project timelines was to request a temporary quota increase.  We proceeded to do the following:

  1. Submitted a request with Microsoft Premier Support to increase the Archive Quota:
    1. Typically, Microsoft support can in fact increase the Archive quota to up to 130GB (Cloud Size).  Keep in mind if migrating from older versions of Exchange or using 3rd party migration tools, the Archive size at the source platform may be under the target quota, however due to the conversion process you can expect an expansion at the target.  For example, migrating from Exchange 2010 to Exchange Online during the writing of this article, we experienced an expansion of up to 30% post migration.
  1. Microsoft Support does not have the ability to increase the quota past 130GB. In order to get a quota increase past this, Microsoft Support will need to request the quota increase on your behalf with Microsoft engineering.  You will need to do the following:

Prepare a report of all mailboxes over 90GB by running the following in PowerShell connected to on-premises Exchange:

$CSV = Import-CSV C:\temp\import.csv

foreach($Mailbox in $CSV){

$UPN=Get-MailBox -Identity $Mailbox.User | select UserPrincipalName,DisplayName

$Primary=Get-MailboxStatistics -Identity $Mailbox.User | select *Item*

$Archive=Get-MailboxStatistics -Identity $Mailbox.User -Archive| select *Item*

$report = New-Object psobject

$report | Add-Member -MemberType NoteProperty -name UPN -Value $UPN.UserPrincipalName

$report | Add-Member -MemberType NoteProperty -name DisplayName -Value $UPN.DisplayName

$report | Add-Member -MemberType NoteProperty -name PrimaryItemCount -Value $Primary.ItemCount

$report | Add-Member -MemberType NoteProperty -name PrimaryTotalItemSize -Value $Primary.TotalItemSize

$report | Add-Member -MemberType NoteProperty -name PrimaryDeletedItemCount -Value $Primary.DeletedItemCount

$report | Add-Member -MemberType NoteProperty -name PrimaryTotalDeletedItemSize -Value $Primary.TotalDeletedItemSize

$report | Add-Member -MemberType NoteProperty -name PrimaryAssociatedItemCount -Value $Primary.AssociatedItemCount

$report | Add-Member -MemberType NoteProperty -name ArchiveItemCount -Value $Archive.ItemCount

$report | Add-Member -MemberType NoteProperty -name ArchiveTotalItemSize -Value $Archive.TotalItemSize

$report | Add-Member -MemberType NoteProperty -name ArchiveDeletedItemCount -Value $Archive.DeletedItemCount

$report | Add-Member -MemberType NoteProperty -name ArchiveTotalDeletedItemSize -Value $Archive.TotalDeletedItemSize

$report | Add-Member -MemberType NoteProperty -name ArchiveAssociatedItemCount -Value $Archive.AssociatedItemCount

$report | Export-Csv -Path C:\temp\export.csv -NoTypeInformation -Append

}

  • Respectfully request a quota increase and supply the report generated above for Microsoft engineering via Support to analyze. If approved, Microsoft engineering will increase quotas at the mailbox level and provide a report stating the new quotas
  • Exclude mailboxes in the report from any Archive policies to prevent growth of the Archive mailbox
  • Proceed to migrate mailboxes as quickly as possible
  • Connected to Exchange Online PowerShell run Start-ManagedFolderAssistant against all mailboxes in the report generated above once finalized.  This will initiate the auto-expansion process and reduce the Main archive mailbox size behind the scenes while provisioning the additional storage as required.  This is a back-end non-intrusive process

The following figure depicts the properties of an Archive mailbox that had the quota increased and where the auto-expansion process is in effect:

Other Options

While Microsoft was gracious enough to increase quotas in our scenario, we cannot guarantee that they would extend this courtesy to every organization looking to migrate Archives larger than 100GB into Exchange Online.  To be fair, this particular scenario is not well documented and Microsoft does need to address it.  Better yet, provide feasible options for migration such as a temporary quota increase.  After all, the goal is to get data into the Exchange Online service as quickly as possible while maintaining fidelity.  If the service subscription includes “unlimited” Archiving, you should be able to migrate up to 1TB of your data into the Archive mailbox without having to wait it out through the auto-expansion process.  Alternatively, the following options also exist:

  1. 3rd party tools are another option that will allow you to migrate data from a source platform to Exchange Online however you would still hit the limitations imposed on the mailbox.  You can migrate data and once you hit the transition threshold, the auto-expansion process would initiate however, you would still have to wait up to 30 days in order to free space from the Main Archive and then resume the migration until completion.
  2. The PST Export/Import process is still a viable solution.  If you are fortunate enough to have a small number of mailboxes, this may not be as painful as if you had a large number.  Otherwise, for a large mailbox you would need to break the mailbox apart by exporting it into multiple (up to) 20BG PST files.  Using the Network Upload from the Security & Compliance Center you would then need to upload the PST’s for ingestion to Exchange Online mailbox.  Microsoft Engineering can manually provision the Aux Archive for the ingestion directly.
  3. Moving data from the Archive mailbox into the Primary mailbox on-premises is another option although you would need to monitor the size of the Primary mailbox so that it does not exceed 100GB an also consider Litigation Hold and the impacts of lifting it

Increasing quotas on Archive mailboxes was a lifesaver.  It would have taken months to complete Archive Mailbox migrations due to the auto-expansion process especially with mailboxes up to 200GB.  In preparation for migration, knowing the current limitations and options available can help you plan and strategize for the best possible outcome.