Migrating a crap ton of data
At work we have an old server. Like, 5 years old, not a name brand, some custom thing by a place called @XI. Nice server, don't get me wrong, but way more than we are making use of. The thing has four six-core procs and 64GB of ram, of which our one virtual Windows 2012 server is using 2 cores and 16GB. Total waste. There is one system partition and one 3TB data partition, which is full.
So I bought a new server with 10TB of data space. The trick is how to migrate the data over.
I've heard of XCopy, and that seemed like a good place to start, but when I first tried it on the smaller Corporate files, it didn't take the permissions over, and i had to keep restarting it. Probably my fault for not understanding the switches, but it didn't go well, and it was a lot of cleanup on Monday.
But it was only about 100GB, not 3TB. 3TB is a ton of data.
So with a bit more research, I discovered ROBOCOPY. I love just the sound of that name. I'm planning on moving this data during the maintenance on Saturday night, and I know that it's going to take awhile. Do I have to stay for the entire thing?
Well, while researching exactly what flags I need to use, I discover that Robocopy can do some amazing things. Like, it can move data, and if the files are open, it goes to the shadowcopy and pulls it from there so it's not interrupted. It runs at a default of 8 threads (max 128), and can move data really fast. I'm liking this program more and more. And I can get it to move just the stuff that has changed since a specified date and time.
Now I'm excited. Instead of waiting for the maintenance, I start a copy. Thankfully I'm not deleting the data off the old server for a little while to ensure that the new stuff works. I have a plan. Move the data tonight, do the maintenance, and copy just the stuff that changed tomorrow. Nice and easy, super fast.
I am curious, how fast can you move 3TB of data on a LAN connection?
I open up a data transfer calculator, click a couple of the buttons, and see what it says.
1TB @ 100Mb = 2583 hours.
What the ever living nightmarish hell did I just do?
I check the connection on my servers, praying to the machine gods that I didn't just sign up for a copy session that is going to last... three and a half months.
They're both running at 1Gb. Time: 2.5 hrs /TB. Oh, thank you thank you thank you. So it should be done when I wake up.
Robocopy... It's Xcopy on steroids.
Update: This morning it was still running. must have been only moving small files (of which there are apparently a bigger crap ton). I restarted it, telling it to ignore existing files (don't overwrite) and it's moved a TB in the last couple hours, so good. Should be done by tonite.
Posts
