{"id":6297,"date":"2017-01-22T20:23:26","date_gmt":"2017-01-22T20:23:26","guid":{"rendered":"https:\/\/stevepedwards.today\/DebianAdmin\/?p=6297"},"modified":"2017-01-22T20:23:26","modified_gmt":"2017-01-22T20:23:26","slug":"dd-and-cloning-windows-system-drives","status":"publish","type":"post","link":"https:\/\/stevepedwards.today\/DebianAdmin\/dd-and-cloning-windows-system-drives\/","title":{"rendered":"DD and Cloning Windows System Drives"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Be careful with the newer \u00a0\"free Refresh\/Install\" fix Win10 ISO from MS - it still tries to conceal the wifi-sense section, off the screen bottom - in the custom install section! Devious bastards!<\/strong><\/span><\/p>\n

DON'T use the Express option - all defaults are ON.<\/span><\/p>\n

----<\/p>\n

I have had an older SATA caddy that has served me well from years back, so you can use this style along with the prior Post mains powered type to try and clone hard drives provided the one you are copying to is the same size or larger obviously - although you may get away with a smaller drive if there is no data at the end of the last partition and your source drive is defragmented nicely - see the page bottom tests for that success:<\/p>\n

\"\"<\/a><\/p>\n

Just make sure you copy the right drive to the other!<\/p>\n

Mint makes this easy in the GUI, as the mouse hovering tell you the device ID of the Windows drive and blank drive, with their partition sizes and free space:<\/p>\n

The Win10 Upgrade drive is \/dev\/sdd:<\/p>\n

\"\"<\/a><\/p>\n

The blank drive is the 320GB \/dev\/sde<\/p>\n

\"\"<\/a><\/p>\n

The beauty of DD is it can clone it's own linux system drive when running! It doesn't stop at directories like \/proc or \/sys that stop tools like cp or rsync. So mounted or unmounted drives don't bother it - it just wants an input and output device, set at it's default block size - which is 512B - and other useful things to know:<\/p>\n

man dd<\/span><\/p>\n

ibs=BYTES<\/span>
\n read up to BYTES bytes at a time (default: 512)<\/span><\/p>\n

noerror
\ncontinue after read errors<\/span><\/p>\n

obs=BYTES<\/span>
\n write BYTES bytes at a time (default: 512)<\/span><\/p>\n

Sending a USR1 signal to a running 'dd' process makes it print I\/O sta<\/span>
\n tistics to standard error and then resume copying.<\/span><\/p>\n

$ dd if=\/dev\/zero of=\/dev\/null& pid=$!<\/span>
\n $ kill -USR1 $pid; sleep 1; kill $<\/span><\/p>\n

pid18335302+0 records in 18335302+0 records out 9387674624 bytes
\n(9.4 GB) copied, 34.6279 seconds, 271 MB\/s<\/span><\/p>\n

Start Cloning...<\/strong><\/p>\n

stevee@T3400 ~ $ sudo dd conv=noerror if=\/dev\/sdd of=\/dev\/sde<\/span>
\n[sudo] password for stevee:<\/span><\/p>\n

Unfortunately it gives no feedback until it completes successfully or errors, so you just have to let it go, but the man page shows how you can read progress seen below later.<\/p>\n

As the Win10 drive is 80GB in total, there is plenty of space on the 320GB.<\/p>\n

DD copies from bit 1 at the start of the MBR (by default but can start or end anywhere you define) right to the last byte of the drive which is why it can clone an exact copy of a drive or any portion of it - an MBR for example - see the man page.<\/p>\n

The issues later may be that different drive geometries mess up the exact data block\/sector boundaries (why 512B is a generally safe fit default for most file systems), which are usually re-ordered by Windows chkdsk at boot when it finds the problem.<\/p>\n

Linux ext4 file systems are journalled so are better at recovering the data order for cloned linux drives at boot in general - but why bother? Just re-install, it's much quicker.<\/p>\n

You won't know if it has worked until you put the clone back in the Windows PC the original drive came from and boot it...for this 80GB drive, I'd allow about 3 hours...it's 18.50..<\/p>\n

At least the new mains gizmo has a flashing light on data transfer so you know it's doing something...<\/p>\n

If you really want to know how far along you are, you can interrupt the terminal process using a neat trick from an old Post\/man page:<\/p>\n

https:\/\/stevepedwards.today\/DebianAdmin\/cool-command-1-dd-progress\/<\/a><\/span><\/p>\n

To get a progress report while\u00a0dd\u00a0is running, you need to open another virtual terminal, and then send a special USR1 signal to the dd\u00a0process.<\/em><\/p>\n

First, find out the process id of the\u00a0dd\u00a0process by running the following in the new virtual terminal.<\/em><\/p>\n

stevee@T3400 ~ $ pgrep -l '^dd$'<\/span>
\n4259 dd<\/span><\/p>\n

In a 2nd terminal\u00a0send the USR1 signal to the\u00a0dd\u00a0process:<\/em><\/p>\n

stevee@T3400 ~ $ sudo kill -USR1 4259<\/span>
\n[sudo] password for stevee:<\/span><\/p>\n

Note that as soon as the USR1 signal is detected,\u00a0dd\u00a0will print out the current statistics to its STDERR in it's original terminal.<\/em><\/p>\n

29796945+0 records in<\/span>
\n29796945+0 records out<\/span>
\n15256035840 bytes (15 GB) copied, 1981.41 s, 7.7 MB\/s<\/span><\/strong><\/p>\n

You may also view a running PID with:<\/p>\n

ps -ef | grep if=
\nroot 3479 2949 0 13:42 pts\/2 00:00:00 sudo dd conv=noerror if=\/dev\/sda of=\/dev\/sdd<\/span>
\nroot 3480 3479 5 13:42 pts\/2 00:11:04 dd conv=noerror if=\/dev\/sda of=\/dev\/sdd<\/span>
\nstevee 4823 4737 0 17:18 pts\/8 00:00:00 grep --colour=auto if=<\/span><\/span><\/p>\n

Because the dd process is relatively resource heavy on a PC doing nothing else, it may show near the top of top when the<\/span>\u00a0m or c\u00a0keys are pressed to show %mem\/cpu use<\/span><\/span>:<\/p>\n

PID USER PR NI VIRT RES SHR S %CPU %MEM<\/strong> TIME+ COMMAND <\/span>
\n 3480 root 20 0 5932 580 428 D 8.0 0.0<\/strong> 11:38.88 dd conv=noerror if=\/dev\/sda of=\/dev\/sdd<\/span><\/p>\n

PID USER PR NI VIRT RES SHR S %CPU %MEM<\/strong> TIME+ COMMAND <\/span>
\n 3480 root 20 0 5932 576 424 D 6.0 0.0<\/strong> 11:48.56 dd<\/span><\/p>\n

Looks like my old caddy is USB1 only - max 12MB\/s - as it's only copying at 7.7MB\/s<\/p>\n

So it has done about 15GB in 33\u00a0mins = 30GB\/hr - so looking at about 3hrs\u00a0total...<\/p>\n

After reporting the status,\u00a0dd\u00a0will resume copying. You can repeat the above\u00a0kill\u00a0command any time you want to see the interim statistics. Alternatively, you can use the\u00a0watch\u00a0command to execute kill\u00a0at a set seconds interval.<\/em><\/p>\n

stevee@T3400 ~ $ sudo watch -n 10 kill -USR1 4259<\/span><\/p>\n

checking each 1\/2 hr would be -n 1800<\/span><\/p>\n

\"\"<\/a><\/p>\n

Ctrl-C to stop the watch or DD commands.<\/p>\n

21:35 Finished. 2hrs, 54 mins<\/strong><\/p>\n

156301488+0 records in<\/span>
\n156301488+0 records out<\/span>
\n80026361856 bytes (80 GB) copied,<\/strong> 10421.5 s, 7.7 MB\/s<\/span><\/p>\n

Windows needs to boot it and run chkdsk now to sort itself out - if possible...so it won't be mountable in Linux in this confused state.<\/p>\n

First, Gparted will re-scan all hardware to get a look at it's general state - and it has a check option, but I'm not going to do that with Gparted until Windows has tried to do it first:<\/p>\n

\"\"<\/a><\/p>\n

At least the system partition shows a boot flag in GP, so a good start...let's try it in the Win PC...<\/p>\n

Can you believe it? The disk just died! I'll redo it another day...pisser.<\/p>\n

Ok, another 4 hr dd session later, but I have a perfect W10 clone viewed remotely via rdesktop<\/span>, that booted first time with not even an apparent file system correction! Due no doubt, to me changing the disk to dynamic before I re-cloned - forgot about that...<\/p>\n

rdesktop -g 90% dellw7<\/span><\/p>\n

\"\"<\/a><\/p>\n

So can Windows now extend into the full drive size? Yes - despite the warning, it immediately filled up the remaining space:<\/p>\n

\"\"<\/a><\/p>\n

Can I shrink it back so I have some spare data space? 150GB should be good..<\/p>\n

\"\"<\/a><\/p>\n

Now the new partition can be formatted and shared.<\/p>\n

Seems the way to go here is convert Windows system drives to dynamic disks BEFORE you clone them with DD. Excellent.<\/p>\n

So what's the story with Win10 basic system drives for cloning? It appeared to work ok yesterday but the disk failed before I booted it.<\/p>\n

I'll try on this 160GB to 160GB layout:<\/p>\n

\"\"<\/a><\/p>\n

sudo dd if=\/dev\/sde of=\/dev\/sdd<\/span><\/p>\n

8hrs later...doesn't look good initially...but..<\/p>\n

153095455232 bytes (153 GB) copied, 27110.2 s, 5.6 MB\/s
\ndd: writing to \u02dc\/dev\/sdd,: No space left on device<\/strong>
\n312581809+0 records in
\n312581808+0 records out<\/span><\/p>\n

160041885696 bytes (160 GB) copied, 28419.4 s, 5.6 MB\/s<\/span><\/p>\n

GP reads only an unallocated drive still because the linux partition table is not updated until a reboot.<\/p>\n

\"\"<\/a><\/p>\n

It was slightly too small by 4GB, though marked at 160GB also, so all you can do\u00a0is put it in the\u00a0Win pc\u00a0and\u00a0try...<\/p>\n

It booted no problem! Didn't need to repair either.<\/p>\n

\"\"<\/a><\/p>\n

This proves\u00a0that linux, so GP, doesn't give an accurate state reading of newly resized clones until at least a linux kernel reboot, so a seeming fail here is not necessarily so. The proof is a successful boot to allow Windows to organize the FS.<\/p>\n

So, cloning Windows with DD can be with dynamic or basic drives - it makes no odds, as you would expect for byte by byte data transfer.<\/p>\n

If there is only empty space at the end of a larger, defrag'd source drive, it will probably still clone ok.<\/p>\n

The time taken to clone becomes the drawback over a fresh install as drives may get too big to be worth the wait against whether what is on the drive (special Apps etc.) can be re-installed later, in less time overall with a new Win install - or not at all - maybe due to lost software keys or tech settings that are now unknown and cannot be re-entered manually etc.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Be careful with the newer \u00a0\"free Refresh\/Install\" fix Win10 ISO from MS - it still tries to conceal the wifi-sense section, off the screen bottom - in the custom install section! Devious bastards! DON'T use the Express option - all defaults are ON. ---- I have had an older SATA caddy that has served me ...\u00a0 DD and Cloning Windows System Drives<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-6297","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"a3_pvc":{"activated":true,"total_views":3,"today_views":0},"_links":{"self":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/6297","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/comments?post=6297"}],"version-history":[{"count":0,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/6297\/revisions"}],"wp:attachment":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/media?parent=6297"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/categories?post=6297"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/tags?post=6297"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}