can I use a signed SSL certificate home : internet : mail : toaster : FAQ : migrate to new server multilog date stamps

How do I migrate my Mail::Toaster from one server to another?

The easiest way is to simply remove the hot swap SCSI drive from the old server and stick it in the new one. If you are using RAID, move the RAID card too, along with all the drives. Grin, and pat yourself on the back for thinking so far ahead and buying good equipment. :)

Assuming you can't use the above instructions for whatever reasons, the following info might prove helpful. Your users account information is stored in several places, all of which need to be moved to the new server. Here are the critical directories:

  • /var/qmail/control
  • /var/qmail/users
  • ~vpopmail
  • /var/db/mysql

A thorough checklist would look something like this:

  • 1. Upgrade exiting Mail::Toaster to latest version
  • 2. Wait a day or two (make sure nothing broke)
  • 3. Build the new Mail::Toaster
  • 4. Stop all the services on both systems.
  • 5. Copy the directories above from old to new server
  • 6. Start up the new system

If there is mail on the old systems queue, give it a new IP and leave it online until it's finished emptying it's queue (up to a week).

This assumes the new server will inherit the IP and hostname from the old one. If that's not the case, then you have DNS entries to update, config files to alter, etc.

How do I migrate from a non-toaster to a Mail::Toaster?

  • Hi listers,
  • I'm expecting a server move/reinstall at the end of the year and thought that I for once should plan the move more than 3 days in advance and maybe get it going smoothly for a change. So, I'll be migrating a number of users with Maildirs and all to the new box, which will be based on Matt's splendid Mail::Toaster, The old server is currently running sendmail+spamassassin, a few scripts The fact that we lost a week's worth of email the last time we had a move and that I sort of forced everybody to set up everything twice is a strong motivation factor to keep the setup around and then offer alternatives and let people decide if they want to use something else.

Losing mail in an upgrade simply means that you failed to plan the migration adequately. During migrations and upgrades, I consider three things to be essential for the transition to be a success:

  • 1. zero customer impact
  • 2. zero customer impact
  • 3. zero customer impact

The first one is kind of hard to achieve, and the second two are harder, but it's always possible. My very first mail migration was switching from sendmail to qmail, about a decade ago. I've done hundreds since then, but I'll use that first case as an example of how to do it The Right Way[TM]. I had to do it in stages:

Stage 1

  • replace sendmail's local delivery with a hacked version of procmail that delivered to Maildirs.
  • convert all mbox files to Maildir
  • replace pop3 daemon

With a little care and planning, that can be done with zero customer impact. You install the new pop3 daemon on a different port, you convert a test mailbox, and you test your conversion scripts. You make sure that the message status in mbox is taken into consideration so that users don't end up with duplicates in their inbox. Once you are certain that the new system is working and your conversion scripts are working, you throw the switch, watch the logs, and ready yourself to fix the unforeseen problems that appear.

After a day or two, the dust from your last changes will have settled and you can begin the next stage of the migration.

Stage 2

  • replace the sendmail mta with qmail

Once again, you install qmail and set it up to listen on a different port for testing. It's pretty trivial to do this and send test messages to make sure it works as you expect. Once you've tested, tested and tested some more, you throw the switch and watch the mail logs to make sure it's doing The Right Thing. Since you still have the old sendmail, you can always revert back quickly should you discover any major hurdles that you can't fix in a matter of minutes.

  • So. I need to move all Maildirs to the new server

Why?

If it's really necessary, how? I have a script I wrote that'll take a huge file system tree (ie, a ~vpopmail/domains) directory and busts it into little buckets (ie, one domain at a time) and uses rsync to copy the files over to the new server. It's quite slick. I wrote it's predecessor when I needed to migrate 20,000 domains from a data center in Seattle to a data center in Atlanta. By doing an initial sync before the maintenance window, you only have to migrate the messages that have changed since your last sync. I got really wild and crazy and found that if I busted the rsync processes up into little syncs and spawned 15 concurrent processes, I could migrate the changes about 100 times faster.

So, you'll want to come up with something like this if you're migrating to a new box.

Also, if you move to a new box, you want to redirect connections from the old server to the new one. There are a variety of methods for this:

  • put the IP from the old server on the new one
  • use your router/load balancer/firewall to forward the packets
  • install proxy applications to proxy connection from old to new server

I have used all three techniques, and each one has it's advantages and disadvantages. Using proxy servers is REALLY helpful when you need to only redirect one type of traffic (say http, or smtp) from one server to another.

  • and should ideally find a way to merge all "virtual" (sendmail-virtual) users and (virtual) domains from a system-users setup to a vpopmail setup, while still allowing users with multiple domain names to receive all mail for all domains in one consolidated mailbox. This is where I expect the most trouble, really.

This shouldn't be any problem. We had to do something like this when removing sendmail from 400 servers and moving all the accounts over to a qmail cluster I built. Basically you just need to write a script that parses your aliases file and generates the appropriate alias/forwards within each domain. Example:

aliases

webmaster@x.com bobsmith

webmaster@y.com bobsmith

In sendmail, you probably deliver to local users (as you said) so you'll probably need to either set up a default domain (isp.com) or use an ipmap so that incoming authentications for bobsmith get mapped to bobsmith@isp.com. Then, you'll create each domain (x.com and y.com) and add aliases for webmaster that point to bobsmith@isp.com (the consolidated mailbox). It's a little bit kludgy at first but it cleanly converts from a very kludgy host based system to a very nice virtually managed solution.

  • I must admit I'm not too intimate with the innards of vpopmail and qmailadmin yet to see an easy way to fix this.
  • So what say ye? I assume it is possible, what I'm really after is sort of a indicator of how difficult it will be and how much pain it will cause me in the process.

That's all up to you. :-) Seriously. I could do it quite easily. How long it would take would depend on how clean your sendmail set up is, how many mailboxes need to be migrated, how fast your hardware is, etc. So, it's very possible, and exactly how much pain is involved depends on how familiar you are with the tools you are using.

  • Any pointers to existing pain measures and/or howtos/faq's highly appreciated.

I'd recommend building yourself a Mail::Toaster now and play with it extensively for a while. Set up your vanity domains on it and get used to it. Get to know how to do the stuff you'll need to do in the migration on your development system. Then you'll have a big leg up on it when you go into production.

Or just hire someone to do it for you. ;-)

Last modified on 6/4/05.