RSM McGladrey talks about its migration

March 28^th 4:30 - 6:30 PM
Cost: Free, as always!
Location: RSM McGladrey, just off 494 & France in Bloomington

The straight truth: RSM McGladrey talks about their mail migration to Exchange

"What will it take to switch from Lotus to Exchange?" Mary Lou LaBore has an answer, not a sales pitch, industry propaganda, or a lesson on how not to do it, but a real example, and she's promised/warned that she'll be frank about it.

Microsoft's case study on RSM McGladrey's migration to Microsoft Exchange generated some buzz on various Lotus-centric blogs (e.g. here). Their migration was not I would have preferred, of course; as Ed Brill had noted there were political/organizational forces at work too, such as their parent company's MS infrastructure, and also the timing was such that IBM's story on Notes was still murky at that time. Whether they made the best decision will be debatable for years but as someone who knows the company and even used to be an employee for five years, I can say they were more than ready for something more, either Java or .NET, and Microsoft won this one.

Either way, that die was cast many months ago... and then comes the migration, starting with email.

I suppose, when I tell you that the MN Lotus User Group asked Mary Lou LaBore, the head administrator at RSM McGladrey and long time member of our user group, to speak about their email migration experience, you might expect to hear propaganda? A horror story highlighting the poor design flaws of MS Exchange that reared up and incited mass revolt?

The truth is: Their email migration was pretty smooth. Theirs is a story of what it takes to migrate, and we asked Mary Lou LaBore to speak because it was a rare chance to hear truth instead of hype.

Note: Mary Lou Labore was originally scheduled for September of 2006 but had to cancel due to a family emergency. March was the soonest we could reschedule her because of the cyclical nature of her job. (Winter is the most hectic time for IT staff of an accounting company.)

Meeting Notes

20 people attended representing 15 companies.

Mary Lou LaBore: RSM McGladrey's migration from Lotus Notes to Exchange
Chad & Kim: Binary Tree's migration tools from Exchange to Lotus Domino (an vice versa)
Tom Hillebrand: IBM's Direction and what you already have in Lotus Domino

Mary Lou LaBore of RSM McGladrey presented to the Minnesota Lotus Notes User Group discussing RSM McGladrey's experience with their move from Lotus Notes R5 to Microsoft Exchange and other technology. Click here to download her presentation. (ppt, 8.6Mb)

Mary Lou has been with RSM McGladrey for 25 years. She has managed RSM McGladrey's email since 1994 when the firm started using CC:Mail. CC:Mail was a new technology in the world at that time and it was promoted by the highest ranking people in the company which made it a big success. In 1998 the firm moved to Lotus Notes mail and began using Lotus Notes for collaboration with such tools as EA Systems, a tool to automate and streamline audit engagements. She has been the administrator since. She currently is the Enterprise Messaging Team Lead for Lotus Domino application servers, Exchange and other associated messaging servers.

RSM McGladrey has about 7,000 employees. "Notes has served us very well", Mary Lou said. In 1998 RSM McGladrey started with Lotus Notes R3 for an application that documented and organize electronic manuals for audit and tax consultants. They moved everyone from CC:Mail to Lotus Notes R4 mail and business critical apps.

Some of RSM McGladrey's Lotus Notes based tool include:

Auditor Assistant - a tool RSM McGladrey uses for conducting and reviewing internal audits they do for companies.
Engagement Management System - organizes work-papers and audits that RSM McGladrey does for clients (used by over 4,000 people).
STP - an application that consultants use to audit companies.
RSM McGladrey currently has over 40,000 notes databases (many are for the various customer audits they do)

The ability of Lotus Notes to rapid deploy application allowed us to have what we have today. January, 2005 they did an analysis of how people worked. They sent out observers with the RSM McGladrey auditors to experience what they do day-in and day-out. 85% of RSM McGladrey workforce are mobile. All users utilized local replicas of mail files instead of working on-line with the Lotus Notes server. Some of RSM McGladrey's challenges were with how mobile users could connect to the RSM McGladrey home server from customer locations in order to replicate their mail and other Lotus Notes database. RSM McGladrey did not support web mail. The client’s security people would not allow port 80 or the Notes port via the internet to allow mobile users to replicate and they had on-going issues with their VPN solution. RSM McGladrey is still on Lotus Notes R5.0.10 on their servers but are planning on upgrading the server to R6 because R5 hasn't been supported by IBM for a couple year. Their users are still on R5.0.8 of the Lotus Notes client. Since the move to Exchange, all new development in Lotus Notes has been halted, however on-going application maintenance and enhancements continue to move forward.

Some of RSM McGladrey's reasons for their move to Exchange were:

H&R Block, RSM McGladrey's parent company and an Exchange user, pressured RSM McGladrey to migrate to Exchange
New generations of people (recruits) knew MS products but had not had experience with Lotus Notes
Concern about "hooks" into applications that Outlook would support vs. Domino because they were considering a document management system by Interwoven which has specific integration points into Outlook
IT studied other products. They rejected the new releases of Notes and decided upon Exchange

Mary Lou said, "Notes helped us to be the successfully company that we are today."

The study began in January 2005 and by June RSM McGladrey decided to move to MS Exchange 2003, MS Live Communication server, MS SharePoint, Win2003 servers, Active Directory (required to use Exchange), Microsoft Identity Integration Server (MIIS). MS Internet Security and acceleration (ISA), application-layer firewall, VPN and Web cache solution, Outlook Web access, Outlook mobile access. Since the release of Outlook 2003 RSM McGladrey can run Outlook in cached mode, similar to how they worked with local replicas in Lotus Notes (Outlook 2003 was the first version of Outlook to provide this capability). Instead of using Intellisync of Good technology for mobile devices as they had previously done, they're using MSFG for security of their mobile devices. Their roll-out of the support for mobile devices such at smart phones and PDA is "kind of successful...when you bite into the first generation, you get what you get".

They had a Microsoft Consultant on their team to come in and do a complete assessment of RSM McGladrey's environment in order to architect a solution.

June of 2005

Scope of the project was...mail, calendar scheduling support mobile devices. Mary Lou explained that "Public folders in Exchange are Microsoft's attempt to emulate what Notes does and we knew to stay away from that". RSM McGladrey now uses public folders for "resources" such as scheduling conference rooms. RSM McGladrey now uses MS Sharepoint instead of public folders. RSM McGladrey had already been paying for MS Outlook licenses for year as part of their MS Office bundle. They also decided to continue to use their Tumbleweed solution for Anti-virus and anti-SPAM. They also worked on their Active Directory infrastructure since you can't use Exchange without Active Directory.

RSM McGladrey has critical business applications that required mail routing (Lotus Notes and non-notes application), so they bought a product by GeniiSoft called CoexLinks (which converts Notes links to links that work in Outlook and launch the documents in Lotus Notes just like it works in Lotus Notes). Also GreyDuck's Mail Doc Link/Mail Database Link agents were incorporate into many Lotus Notes applications to allow doclinks to be sent to Non-Lotus Notes mail recipients.

RSMi.com internet mail was supporting thousands of web mail users. It was highly important the RSM could co-exist between their Exchange and Lotus Notes messaging environments. They were able to accomplish this with the use of "Smart Hosts" which is a feature in Lotus Notes and in Exchange. It allows your server to "pass on" messages that the server doesn't have an Email address for, to another server called a "Smart Host" and that server will deal with the message in its own way. There were many other technologies that also were considered and needed to be migrated as part of this project. Add-on tools such as: Blackberry server. A SOX application. Symantec AV, Tivoli Storage Manager (used for all backups). Unified messaging and VoIP. Make sure a fax solution would work with Outlook. Needed to Synch all the Domino address books. Used MIIS to keep these in synch. RSM McGladrey decided on Avya for mobile messaging (MS didn't have as good of a solution).

With all of the team analysis and due-diligence, they were approved to move forward on the project in mid-August 2005. They were given less than 3 months to accomplish their mission to migrate 7,000 user to Exchange, but first the team was instructed to migrate a newly acquired company of 2,500 people that had been using a Lotus Notes infrastructure for messaging and application. This new company would be the first to use the new Exchange environment and the team was given 6 weeks to plan and execute this project without anyone from the newly acquired company knowing about the project or about the fact that they were being acquired by RSM McGladrey.

The team took their Microsoft consultant and the crew and went to the location where the servers were in Iowa and lived on Redbull and candy for 6 weeks in order to execute this project. The Team consisted of 8 RSM McGladrey people 3 people from MS and a team from Binary Tree.

This 1^st 6-week phase to migrate the 2,500 people running Lotus Notes from the newly acquired company started the end of August and ended October 2005 with a go-live date of October 3, 2005.

First the team needed to issue them New Lotus Notes ID's so that their mail could be moved to RSM McGladrey server. They also decided to migrated only the last 30 days of Email because this company had no mail policies and there were individual mail boxes with up to 5 Gigabyte of mail. RSM McGladrey had an 80 Mb mail quota on their individual files. It was a "soft quota" enforced only by the fact that your department was charged monthly for every Megabyte each mail file was over the quota. They also needed to migrate each persons list of contacts to the new system. They used the Lotus Tool to migrate their local address books. They also needed to migrate groups distribution lists to Active Directory. They needed to get the new company to join their AD forest but this cause many challenges. Luckily the new company was on Lotus Notes R5.0.12 and that fit well into RSM McGladrey R5 environment. RSM McGladrey converted their mail databases and user ID's to RSM McGladrey so they could still get to their old Notes db's. The people in this acquired company were the first people at RSM McGladrey to be on Exchange.

Some of the challenges they faced were:

The team could not do a dry run of the migration for the new company because of legal issues, no one could know this merger was going to happen
They had to physically move old Lotus Notes servers from Pheonix (which is in a different time zone, but that was a whole separate issue) to the Davenport location on the migration weekend

The migration of this 2,500 person company was a success. When the employees arrived at work on 10/3/05 they had new Email and we told they had been acquired. Most of the support issues of the new week or so were related to Active Directory and how the 'forests' were configured. They had about a week of support calls to the help desk and then the calls tapered off. RSM McGladrey had put together a "Train me" web site that people could use. Mary Lou stated that in regards to users using Outlook, "there was a learning curve for people to learn to use Outlook. Training was important".

The 2^nd 6-week phase project began 10/3/05 to Migrate the 5,000 RSM McGladrey employees to Outlook with a go-live date of 11/11/05. The time estimate for THIS project was 2,400 hours of work. Mary Lou said that they had a great team full of 'problem solvers' that could get together, hash-out, and solve issues as they arose.

RSM McGladrey's Lotus Notes Messaging Environment consisted of:

2 SMTP dedicated servers (for inbound and outbound SMTP mail) 2 Domino clustered servers (backed by a SAN). 2,500 people on each server but also clustered with the other server.
5,000 users, no web access to mail, no mobile, no security interface, using local replicas of their mail, no mail quotas.

The new Exchange environment consists of:

2 SMTP dedicated servers (for inbound and outbound SMTP mail)
4 active mail servers and 1 passive server (backed by a SAN)
8,500 users with a 150 Mb hard quota
3-Front-end server (Outlook Web Access, Outlook Mobile, RPC over HTTP)
ISA server

For each Exchange server you calculate how many mail stores you can create. You need to calculate this compared to Notes where you just throw a mail box on whatever server you want. For each mail store you can apply different rules which is why you want to group people into different mail stores. MIIS provisions people within Active Directory and creates a Notes ID and an Outlook mailbox for them.

How scalability was MIIS? it may be scalable, but the challenge was finding someone that knows how to work with it. "There is probably more we can do with it if someone knew more about it."

Every location has a domain controller with the Global Address List. When a user hits their mail quota their option is to move the mail to a local PST file [which is similar to a local archive database in Lotus Notes]. The problem with PST files is that there is not security around them and because they're on the user's computer they are not backed up. RSM McGladrey is currently looking into a storage and discover solution for Archiving.

Hardware Costs for the new environment was $245,500. The overall costs for this project was about $214 per user. It cost about $1.5 million to migrate the users...not just to migrate but also to live in co-existence between Lotus Notes and Exchange because of 40,000 Lotus Notes databases they still have and use today. Some of the challenges were:

Lotus Notes mail-in database and continuing to send/receive Email from Lotus Notes applications Migrating personal address books (personal NAB import/export tool)
Blackberry
Lotus Notes DocLinks
Distribution lists and generic mail-boxes were a challenge to figure out how to grant people rights to these mail stores in Active Directory
RSM McGladrey had links to specific business reference documentation in everyone's Lotus Notes client. They wanted to link to this documentation in each person's Outlook client, but haven't been able do it
In Lotus Notes RSM McGladrey has specific branding messages on all outbound SMTP mail. They didn't have time to do it in Outlook
They had an icon in Notes that would identify internet mail when a user looked at their lists of messages. Haven't been able to do this Outlook
In Lotus Notes they had an indicator in each user's mailbox showing the size of their mail file. Haven't been able to do this Outlook

Coexistence: RSM McGladrey does coexist between Notes and Outlook. Mary Lou indicated she doesn't have any issues working in Outlook and Notes. The GeniiSoft CoexLinks and special processing written by GreyDuck and BinaryTree were very useful. Workers are happy. They have Outlook Web Access and support for mobile devices. The Global Address List (GAL) contains everyone that they need and it contains many details about each person because MIIS is pulling lots of info from PeopleSoft into the GAL. Outlook has Soft deletes so if a user deletes a message they can retrieve the message within 7 days without calling IT for a restore. [Something Lotus Notes has been doing for years]. In Outlook you can drag and drop Emails into other MS-Applications.

The Exchange 2003 servers seem to be running fine on Windows 2003. Performance is good...a couple hiccups here and there. Users are working in cache mode so that help off-load the server. You can create "Query-based lists" if you have accurate information in your GAL...Thanks to PeopleSoft. These lists are only for mail distribution lists and not for security. Mary Lou would like to use these lists for security purposes in Sharepoint because in Lotus Notes she was able to use Groups or OU's for security to Lotus Notes applications. Mary Lou had heard horror stories about restoring mail. Things like, it would take forever because Exchange was a single object store that could be terabytes in size and a restore would take forever. RSM McGladrey averted these issues and purchased "Kroll Ontrack Powercontrols Tools" which they can use for Email discovery by mailbox store.

What Mary Lou misses is: Auditability. If a change is made to an object in AD there isn't much of an audit trail. In Notes you could see who modified a document and the last modfied date. She also had nice tools in Lotus Notes for maintaining distributions lists that she misses.

What would she do differently - pay more attention to mail distribution lists when migrating companies. They are more important to people than we thought.

Where is RSM McGladrey Today?

Still using Lotus Notes applications. Everyone has the Notes client for applications and an Outlook client for mail.
The long term goal is to move these application off of Notes. They plan to move the auditing applications to a product called "Caseware".
They plan to move to .NET based solutions
The use MS SharePoint for their intranet with displays the employee directory and pictures of people. We had this in Notes but we never put it on the web.
Some of the Sharepoint application are back-ended by Notes. All the data resides in Notes, it's just exposed via SharePoint.
They have halted new development in Notes but continue to maintain and enhance their existing Lotus Notes applications as needed for the business.
RSM McGladrey has Lotus Notes server licenses and plans to upgrade their servers.
They no longer pay for Lotus Notes client licenses and will not upgrade beyond R5.
They use Lotus Sametime for instant messaging and are working on moving to the federation piece of LiveConference server when it's ready so that they can use this for instant messaging instead of Lotus Sametime, however they need to find a way to store chat transcripts and conduct discover to maintain SOX compliance. [Something Sametime does now].
The project cost approximately $1.5 million and took at least 2,400 person hours to complete over 11 months.
The estimated cost for moving RSM McGladrey's Notes apps to another technology is upwards of $1.5 million. Projects are currently underway planning a new architecture and how to accomplish this.

Chad Holznagel & Kim Armstrong from Binary Tree spent 25 minutes explaining their software processes and tool for migrating from Exchange to Lotus Domino (and vice versa).

BinaryTree has been a Lotus/IBM business partner for many years. Their primary experience has been with moving people from Exchange to Notes. The discussed how they analyze Domino databases and mail environments for planning and then they help customers migrate. They have the Common Migration Tool (CMT for Notes) which migrates Exchange/Outlook to Notes. They have another tool that migrates Exchange Public folders to Notes. They also have coexistence tools. Something that does DocLink translation so Outlook clients can use it. They do planning and assist with migrations. Have migrated over 4,000 customers to Notes.

Why do people migrate:

Strategic: someone is trying to get somewhere
Technological: consolidation purposes
Political: this seems to be the most popular reason
Shiny: Which product is more shiny. The vendor that came in most recently
Total Cost of Ownership: This isn't as important to companies as the vendor may think

The types of migration they deal with are:

Rip & Replace
Mergers and Acquisitions
Restructuring

People ask us....Which one is better [Notes vs Exchange]?
There isn't a very good answer for this. The move is from hierarchical to relational or vice versa...each has their strengths. But why try to fit a circle into a square. In other words, when people move from on messaging platform to another they try to recreate the same functionality they had in the old system in the new environment. This doesn't always make sense since you are coming from a very different paradigm (hierarchical vs. relational). Instead of try to duplicate what you HAD been doing, why not figure out how to accomplish a similar goal leveraging the strengths of your new environment.

When Binary Tree assisted RSM McGladrey with the migration of the 2,500 users from Lotus Notes to Exchange is was the most data they've ever moved in that period of time. They had 30 computers setup and doing the processing to migrate all the information.

Tom Hillebrand of GreyDuck took the last 20 minutes to present IBM's Direction and Dedication to Lotus Domino, What we have in Lotus Domino today and what we will have later in 2007.
You can download Tom's presentation below entitled MNNotes411.ppt

Tom stated how IBM has been leading the personal productivity and Groupware markets since the first DOS based PC's were introduced with Lotus 123 spreadsheets. IBM produced one of the first PC based Email packages, "cc:Mail". IBM has been there from the start and is still going strong with Lotus Domino Release 7. IBM's strategy is to move forward in to the new way people work..beyond groupware into Activity Centric computing, composite applications (mashups). IBM is not sitting back and simply promoting workflow, they are analyzing current trend and looking forward to how people work TODAY. Lotus created the groupware collaboration market and they will be the leader in software that supports the new paradigms of activity centric computing.

Tom acknowledged that there are companies unhappy with their current version of Lotus Notes and they look to other technologies to solve the perceived short comings of the software. He contends that, in cases similar to RSM McGladrey, people working on old version of Lotus Notes (release 5 in RSM McGladrey case), SHOULD be unhappy with the product. The R5 software is no longer supported. It was released around the year 2000. There are 2 major releases ahead of it and the newest major release is due out in 2007. If you were still running Windows '95 on your PC you wouldn't be very happy with Microsoft Windows, would you? It would be slow, clunky and not have any of the new features or user interface enhancements. Technology moves quickly and Lotus provides tremendous enhancements to every release of their software.

Lotus Domino provides very efficient ways to roll-out upgrades to your user community. When you configure Lotus SmartUpgrade on your Domino server, you can configure it so that the users' Notes clients are upgraded automatically, or you can have them 'accept' the upgrade when prompted.

Tom extolled the virtues of what we have today with Lotus Domino, Lotus Sametime and Lotus Quickplace and he briefly highlighted what new features can be expected in 2007 with the new releases. Lotus has always been backward compatible. You can still run R2 version of Lotus Notes applications in R7! As a result, when new features are released you are not forced to use them even implement them. What this means is that quite often the you have the functionality to do what you need to do, but you may not know it. For example, many companies are looking to 3rd party solutions monitor their Lotus Domino environment. Since R6 the Lotus Domino servers have been able to monitor themselves and in R7 even better features have been implemented that allow you to analyze your server resources and Lotus Domino will make recommendation to you, such as adding memory, or your utilization of disk space. (If you are interested in some of the features you may not be aware of in R7, please read this presentation entitled "Domino7.ppt")

In Tom Hillebrand's opinion, if you think that MS Exchange, or other MS products, have a feature that doesn't exist in Lotus Domino or IBM products, then you're not looking hard enough, and chances are that IBM/Lotus have been doing it much longer than Microsoft...it's "old news" to us...we don't talk about it any more...we've moved on to the next thing. Case in point: The ability for Outlook to work in cached mode is a feature that Lotus Notes has had from the beginning.

Other Valuable Resources