{"id":4838,"date":"2016-09-17T16:39:57","date_gmt":"2016-09-17T15:39:57","guid":{"rendered":"https:\/\/stevepedwards.today\/DebianAdmin\/?p=4838"},"modified":"2016-09-17T16:39:57","modified_gmt":"2016-09-17T15:39:57","slug":"save-gigabytes-of-failed-rsync-video-transfers-on-a-drive","status":"publish","type":"post","link":"https:\/\/stevepedwards.today\/DebianAdmin\/save-gigabytes-of-failed-rsync-video-transfers-on-a-drive\/","title":{"rendered":"Save Gigabytes of Failed Rsync Video Transfers on a Drive"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

If you watch and keep a lot of YouTube or other video files, and backup regularly from your main laptop to other PCs, you may accumulate a lot of partial videos from interrupted copies and \u00a0rsync transfers that have extensions of the type .crdload or .MOV_0002.mp4.B8ObyK which can accumulate to a surprising size.<\/p>\n

Running find for mp4 files of these types on my 1.5TB backup drive I found many which I could then approximately sum (ignoring k values as most are in MB, but if you want accuracy, drop the -h to show 1K-blocks) via:<\/p>\n

sudo find \/ -name *.mp4.* -exec du -h {} +; | awk '{sum +=$1} {print $1, sum}'<\/span>
\n221M 221<\/span>
\n571M 792<\/span>
\n539M 1331<\/span>
\n91M 1422<\/span>
\n....<\/span>
\n221M 7218.3
\n580M 7798.3
\n422M 8220.3
\n37M 8257.3<\/strong><\/span><\/p>\n

This is about 8GB of useless video files!<\/p>\n

Run:<\/p>\n

sudo find \/ -name *.mp4.*<\/span><\/p>\n

Make sure you check the files before you\u00a0remove them with -exec rm<\/strong>!!<\/p>\n

sudo find \/ -name *.mp4.* -exec rm --i=no {} +;<\/span><\/p>\n

Because if you ran the same line for mp3s you may delete important non MP3 files such as:<\/p>\n

100K \/Quadra\/www\/wp\/wp-includes\/ID3\/module.audio.mp3.php<\/strong><\/span><\/p>\n

Out of interest, as I have a large vid collection, I could measure it approximately as 177GB!:<\/p>\n

sudo find \/Quadra\/ -name *.mp4 -exec du -h {} +; | awk '{sum +=$1} {print $1, sum}'<\/span><\/p>\n

175M 176824<\/span>
\n382M 177206<\/strong><\/span><\/p>\n

OR as MB:<\/p>\n

sudo find \/Quadra\/ -name *.mp4 -exec du -Bm {} +; | awk '{sum +=$1} {print $1, sum \"M\"}'<\/span><\/p>\n

382M 188597M<\/strong><\/span><\/p>\n

If you want MB accuracy converted to GB:<\/p>\n

sudo find \/Quadra\/ -name *.mp4 -exec du -Bm {} +; | awk '{sum +=$1} {print $1MB, sum \"MB\", sum\/1024 \"GB\"}'<\/span><\/p>\n

175M 157937MB 154.235GB<\/span>
\n382M 158319MB 154.608GB<\/span><\/p>\n

or<\/p>\n

sudo find \/Quadra\/ -name *.mp3 -exec du -Bk {} +; | awk '{sum +=$1} {print $1KB, sum \"KB\", sum\/1024 \"MB\", sum\/(1024*1024)\"GB\"}'<\/span><\/p>\n

If you test what find actually calculates when used by default without with a glob or specific name etc. with -exec du, it includes the parent directory size also.<\/p>\n

As a test, create 2 empty text files with touch in an empty dir Public, then add a character as in the Bits to Gb in 200 years Post, which we know is 1 byte in size, but uses a 4k block space on disk:<\/p>\n

stevee@T3400 ~\/Pictures $ cd<\/span>
\nstevee@T3400 ~ $ cd Public\/
\nstevee@T3400 ~\/Public $ ls<\/span>
\nstevee@T3400 ~\/Public $ touch test1.txt test2.txt<\/span>
\nstevee@T3400 ~\/Public $ ls -l<\/span>
\ntotal 0
\n-rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test1.txt
\n-rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test2.txt
\nstevee@T3400 ~\/Public $ ls -las<\/span>
\ntotal 8
\n4 drwxr-xr-x 2 stevee stevee 4096 Feb 8 22:24 .
\n4 drwxr-xr-x 39 stevee stevee 4096 Feb 8 19:40 ..
\n0 -rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test1.txt
\n0 -rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test2.txt
\nstevee@T3400 ~\/Public $ echo -n \"A\" >> test1.txt<\/span>
\nstevee@T3400 ~\/Public $ ls -las<\/span>
\ntotal 12
\n4 drwxr-xr-x 2 stevee stevee 4096 Feb 8 22:24 .
\n4 drwxr-xr-x 39 stevee stevee 4096 Feb 8 19:40 ..
\n4 -rw-r--r-- 1 stevee stevee 1 Feb 8 22:25 test1.txt<\/strong><\/span>
\n0 -rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test2.txt<\/strong><\/span>
\nstevee@T3400 ~\/Public $ echo -n \"A\" >> test2.txt<\/span>
\nstevee@T3400 ~\/Public $ ls -las<\/span>
\ntotal 16
\n4 drwxr-xr-x 2 stevee stevee 4096 Feb 8 22:24 .
\n4 drwxr-xr-x 39 stevee stevee 4096 Feb 8 19:40 ..
\n4 -rw-r--r-- 1 stevee stevee 1 Feb 8 22:25 test1.txt<\/span>
\n4 -rw-r--r-- 1 stevee stevee 1 Feb 8 22:25 test2.txt<\/span>
\nstevee@T3400 ~\/Public $ find -exec du -Bk {} +; | awk '{sum +=$1} {print $1, sum \"K\"}'<\/span>
\n12K 12K<\/span><\/strong>
\n4K 16K<\/span><\/strong>
\n4K 20K<\/span><\/strong><\/p>\n

The text files only add to 8k but the total shown by find\/exec\/du includes the parent dir size.<\/p>\n

Without the specific file type search, you get parent dir sizes also:<\/p>\n

stevee@T3400 ~\/Public $ find *txt -exec du -Bk {} +; | awk '{sum +=$1} {print $1, sum \"K\"}'<\/span>
\n4K 4K<\/span>
\n4K 8K<\/span>
\nstevee@T3400 ~\/Public $ find . -exec du -Bk {} +; | awk '{sum +=$1} {print $1 \"k\", sum \"k\"}'<\/span>
\n12Kk 12k<\/span>
\n4Kk 16k<\/span>
\n4Kk 20k\u00a0<\/strong><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

If you watch and keep a lot of YouTube or other video files, and backup regularly from your main laptop to other PCs, you may accumulate a lot of partial videos from interrupted copies and \u00a0rsync transfers that have extensions of the type .crdload or .MOV_0002.mp4.B8ObyK which can accumulate to a surprising size. Running find ...\u00a0 Save Gigabytes of Failed Rsync Video Transfers on a Drive<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-4838","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"a3_pvc":{"activated":true,"total_views":1,"today_views":0},"_links":{"self":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4838","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/comments?post=4838"}],"version-history":[{"count":0,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4838\/revisions"}],"wp:attachment":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/media?parent=4838"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/categories?post=4838"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/tags?post=4838"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}