{"id":3831,"date":"2016-08-06T17:50:12","date_gmt":"2016-08-06T16:50:12","guid":{"rendered":"https:\/\/stevepedwards.today\/DebianAdmin\/?p=3831"},"modified":"2025-07-03T17:32:11","modified_gmt":"2025-07-03T16:32:11","slug":"an-intro-to-metachar-search-methods-wreck-the-mp3s","status":"publish","type":"post","link":"https:\/\/stevepedwards.today\/DebianAdmin\/an-intro-to-metachar-search-methods-wreck-the-mp3s\/","title":{"rendered":"An Intro to Metachar Search Methods, Wreck the MP3s"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Common Metach\u00adara\u00adcters<\/h3>\n
\n\n\n\n\n\n\n
\n
^<\/strong><\/span><\/div>\n<\/td>\n
\n
[<\/strong><\/span><\/div>\n<\/td>\n
\n
.<\/strong><\/span><\/div>\n<\/td>\n
\n
$<\/strong><\/span><\/div>\n<\/td>\n<\/tr>\n
\n
{<\/strong><\/span><\/div>\n<\/td>\n
\n
*<\/strong><\/span><\/div>\n<\/td>\n
\n
(<\/strong><\/span><\/div>\n<\/td>\n
\n
\\<\/strong><\/span><\/div>\n<\/td>\n<\/tr>\n
\n
+<\/strong><\/span><\/div>\n<\/td>\n
\n
)<\/strong><\/span><\/div>\n<\/td>\n
\n
|<\/strong><\/span><\/div>\n<\/td>\n
\n
?<\/strong><\/span><\/div>\n<\/td>\n<\/tr>\n
\n
<<\/strong><\/span><\/div>\n<\/td>\n
\n
><\/strong><\/span><\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\n
The escape character is usually \\<\/strong><\/span><\/div>\n
<\/div>\n<\/div>\n<\/div>\n

This Post evolved due to the masochistic desire improve my knowledge of regexs and metachars, possibly using them to tidy up MP3 album\/song titles that can be horrendous looking on the command line, so a good place to practise regex ops.<\/p>\n

ls PorcupineTree\\ -\\ Up\\ the\\ Downstair\/Unknown\\ Album\\ \\(e2118a0e\\)\\ -\\ Track\\<\/span><\/p>\n

Space removal in file names is the first step, started and covered in the Find with -Exec Post, as there are different ways to do this; sed, tr, awk and rename for a start, examples of which have been covered in that Post, plus the PDF eBook Creation Post where it was necessary for the task to work.<\/p>\n

cat file.txt <\/span><\/p>\n

this has one or more spaces in it<\/span><\/p>\n

sed 's\/ \/\/g' < file.txt<\/span><\/p>\n

thishasoneormorespacesinit<\/span><\/p>\n

cat file.txt | tr -d \" \"<\/span><\/p>\n

thishasoneormorespacesinit<\/span><\/p>\n

awk 'gsub(\" \",\"\")' < file.txt <\/span><\/p>\n

thishasoneormorespacesinit<\/span><\/p>\n

touch file\\ name\\ with\\ spaces.txt<\/span><\/p>\n

rename 's\/ \/\/g' file\\ name\\ with\\ spaces.txt<\/span><\/p>\n

filenamewithspaces.txt<\/span><\/p>\n

Note the original file content is not altered by any of them but rename DOES rename the file unless it already exists - that's it's job!<\/p>\n

Important to understand for the addition of -exec rename \u00a6 to the find command later.<\/p>\n

I'm fascinated by the ability of linux to manipulate text the way it does, but filters\/metachars are a massive and complex subject that needs to be studied and practised - on replaceable\/unimportant files ideally - to get experience and an idea of the weirdness of it; my large MP3 collection being ideal for this due to the complex names, and as I have backups for the inevitable deletions that can occur when you just wanted a name change!<\/p>\n

What follows are some ideas on ways to approach the topic.<\/p>\n

First, set yourself a problem to solve that has probable future re-use benefits - start with the whitespace removal examples I have covered below.<\/p>\n

This will always be useful to know on a Linux system, and gives a good starting position to alter commands from so build complexity, so understanding.<\/p>\n

It's a good idea to know how many files and directories your user owns compared to those found (so you know you can change them) to start, so for my MP3 directory:<\/p>\n

sudo find -type d | wc -l<\/span><\/p>\n

559<\/span><\/p>\n

find -type d -user stevee | wc -l<\/span><\/p>\n

559<\/span><\/p>\n

sudo find -type f | wc -l<\/span><\/p>\n

7248<\/span><\/p>\n

find -type f -user stevee | wc -l<\/span><\/p>\n

7248<\/span><\/p>\n

That is fine. But you need to work on a small sample first this is far too many to keep track of. Also get rid on any none mp3 files like art if not necessary. This gets your brain in find command mode also.<\/p>\n

There are many .jpg in here so they can just go:<\/span><\/p>\n

find -name '*jpg' | wc -l<\/span><\/p>\n

422<\/span><\/p>\n

find -name '*jpg' -exec rm -v {} +;<\/span><\/p>\n

Some Thumbs.db and desktop.ini files can go too\u00a6<\/p>\n

Now there is<\/p>\n

find -name '*' | wc -l<\/span><\/p>\n

6992<\/span><\/p>\n

Cd into one directory and work out as many generic regexs as you can in each for various file name types before applying them globally if you value your data make SURE you have backups of it all!<\/p>\n

The white space find and removal started with just the find part for either directories or files, but these have mostly already had white space removed by me before:<\/p>\n

stevee@dellmint \/Quadra\/MP3 $ find -type d -name \"*[[:blank:]]*\"<\/span><\/p>\n

.\/Steves Greatest Misses<\/span><\/p>\n

stevee@dellmint \/Quadra\/MP3 $ find -type f -name \"*[[:blank:]]*\"<\/span><\/p>\n

.\/All_Found_MP3s\/Love Philter.mp3<\/span><\/p>\n

.\/All_Found_MP3s\/Lovely Brasilia.mp3<\/span><\/p>\n

.\/All_Found_MP3s\/Boo Bass Riff.mp3<\/span><\/p>\n

.\/All_Found_MP3s\/IVETE SANGALO & BABADO NOVO. Amor Perfeito (Ax\u00c3\u00a9) (Ao Vivo) \u00a9GaliaBsAs.mp3\u00a6.<\/span><\/p>\n

Run the first command for directories only from above you know what happens with rename if a non-white space file of the same name already exists:<\/p>\n

find -type d -name \"*[[:blank:]]*\" -exec rename \"s\/ \/\/g\" {} +;<\/span><\/p>\n

.\/Steves Greatest Misses not renamed: .\/StevesGreatestMisses already exists<\/span><\/p>\n

Now amend it for files:<\/span><\/p>\n

find -type f -name \"*[[:blank:]]*\" -exec rename \"s\/ \/\/g\" {} +;<\/span><\/p>\n

.\/All_Found_MP3s\/Love Philter.mp3 not renamed: .\/All_Found_MP3s\/LovePhilter.mp3 already exists<\/span><\/p>\n

.\/All_Found_MP3s\/IVETE SANGALO & BABADO NOVO. Amor Perfeito (Ax\u00c3\u00a9) (Ao Vivo) \u00a9GaliaBsAs.mp3 not renamed:\u00a6. <\/span><\/p>\n

As these already exist, they can be removed by changing the -exec rename for remove note that rm -vr will recurse into that directory so remove any files also that DO NOT have white space be careful! Use rm -v for directories ONLY unless sure you want that!<\/span><\/p>\n

find -type d -name \"*[[:blank:]]*\" -exec rm -vr {} +;<\/span><\/p>\n

removed<\/strong> \u02dc.\/Steves Greatest Misses\/LovePhilterDance.mp3,<\/span><\/p>\n

removed<\/strong> \u02dc.\/Steves Greatest Misses\/Driving In Rain.mp3,<\/span><\/p>\n

\u00a6.<\/span><\/p>\n

find -type f -name \"*[[:blank:]]*\" -exec rm -v {} +;<\/span><\/p>\n

removed \u02dc.\/All_Found_MP3s\/Love Philter.mp3,<\/span><\/p>\n

removed \u02dc.\/IveteSangaloSonia\/IVETE SANGALO & BABADO NOVO. Amor Perfeito (Ax\u00c3\u00a9) (Ao Vivo) \u00a9GaliaBsAs.mp3,\u00a6..<\/span><\/p>\n

Now on to the next step say, tidying the files that contain chars in () brackets first e.g:<\/span><\/p>\n

find -regex '.*[^-_.\/0-9a-zA-Z].*'<\/span><\/p>\n

.\/MarinaElali\/AlbumArt_{560D258D-8C82-455E-9968-4B33457C283A}_Large.jpg<\/span><\/p>\n

.\/MarinaElali\/04-MarinaElali-OneLastCry(Paginasdavidainternacional).mp3<\/span><\/p>\n

.\/SugababesBest\/UnknownAlbum(db0da20f)-Track08.mp3\u00a6.<\/span><\/p>\n

find -regex '.*[^-_.\/0-9a-zA-Z].*' | wc -l<\/span><\/p>\n

2854<\/span><\/p>\n

Alphanums inside () brackets is a bit too ambitious at first try numbers inside brackets first. How?<\/span><\/p>\n

What do I already know? From Shott's PDF, the metachars are: <\/span><\/p>\n

Regular expression metacharacters usage consist of the following:<\/span><\/span><\/span><\/p>\n

^ $ . [ ] { } - ? * + ( ) | \\<\/span><\/span><\/span><\/p>\n

How do you isolate them? The () don't appear explicitly in that regex as they are special shell recognised chars in themselves, so have appeared in file names as a by product of the surrounding catchall asterisk * expansion.<\/span><\/p>\n

Experiment by simplifying that regex to just a numbers contained in () maybe?<\/span><\/p>\n

find -regex '.*([0-9]).*' | wc -l<\/span><\/p>\n

79<\/span><\/p>\n

Analyse them by directing the list into a home dir file and read it with less to check if each entry has numbers within () and chars either side, but nothing else inside the () it seems so:<\/span><\/p>\n

find -regex '.*([0-9]).*' > ~\/numberbrackets.txt<\/span><\/p>\n

.\/All_Found_MP3s\/11Track11(2).mp3<\/span><\/p>\n

.\/All_Found_MP3s\/36Track36(2).mp3<\/span><\/p>\n

\u00a6.<\/span><\/p>\n

Can just these parts of the titles be removed? <\/span><\/p>\n

From above, the SED global search and replace methods use a forward slash to delimit chars, as does the format for the rename cmd; white space removal was achieved by searching the found file name for ONE white space after the s\/, then replacing it with NO char between the next two \/ delimiters, and to do this globally (g) for EVERY file found by find.<\/span><\/p>\n

rename \"s\/ \/\/g\"<\/span><\/p>\n

If the same logic is used to replace the numbers using a range [0-9] it may leave the actual () behind with nothing inside or not eh? Let's check on one file only first. Note the testing problem here no DRY-RUN option that I know of or have researched for either find or exec..?<\/span><\/p>\n

Let's try it on the last file in the list and gain insight using ls. This shows that using TAB completion does not expand the first ( after Track43, so needs to be escaped before it will TAB complete:<\/span><\/p>\n

ls All_Found_MP3s\/43Track43\\(<\/span><\/p>\n

43Track43(2).mp3 43Track43.mp3<\/span><\/p>\n

This is a clue to removal for the rename string e.g. rename \"s\/<\/span>\\([0-9]\\)<\/span>\/\/g\"<\/span><\/span><\/p>\n

As the ( is a metachar, it needs to be escaped with a \\ to be read literally.<\/span><\/p>\n

What this also highlights is that in prior copy operations (e.g. Windows merge), a file with the same name existed, so that is one reason for the current (2) version. <\/span><\/p>\n

If so, I can't rename the file with the test, as it exists already. <\/span><\/p>\n

find -name '*43Track43*'<\/span><\/p>\n

.\/All_Found_MP3s\/43Track43.mp3<\/span><\/p>\n

.\/All_Found_MP3s\/43Track43(2).mp3<\/span><\/p>\n

It discerns both tracks with and without (), so now to add the rename regex part to show it works as required, but rename can't overwrite the existing file:<\/span><\/p>\n

find -name '*43Track43*' -exec rename \"s\/\\([0-9]\\)\/\/g\" {} +;<\/span><\/p>\n

.\/All_Found_MP3s\/43Track43(2).mp3 not renamed: .\/All_Found_MP3s\/43Track43.mp3 already exists<\/span><\/p>\n

Wow! It seems it would have worked exactly as required by removing the () also, but for the already existing file! Progress! All I can do here is substitute (2) for something nicer that tells me I did the change, such as an \u00a3 for clarity. First be sure there are no files already with a \u00a3 present:<\/span><\/p>\n

find -name '*\u00a3*' | wc -l<\/span><\/p>\n

0<\/span><\/p>\n

find -name '*43Track43*' -exec rename \"s\/\\([0-9]\\)\/\u00a3\/g\" {} +;<\/span><\/p>\n

Great! Now I get a renamed track with the original unchanged:<\/span><\/p>\n

find -name '*43Track43*'<\/span><\/p>\n

.\/All_Found_MP3s\/43Track43.mp3<\/span><\/p>\n

.\/All_Found_MP3s\/43Track43\u00a3.mp3<\/span><\/p>\n

.\/All_Found_MP3s\/43Track43(2).mp3<\/span><\/p>\n

As above for the white space files, once renamed the original can be deleted now or later using this same regex now I know it works. Depends if you want to prove this example works en-masse or not later. If not, remove it now.<\/span><\/p>\n

rm -v All_Found_MP3s\/43Track43\\(2\\).mp3<\/span><\/p>\n

The output can consist of much StdError, so it's difficult to see what's changed or not, which can be shown because wc -l shows no files output if no file changes occur:<\/span><\/p>\n

find -regex '.*([0-9]).*' -exec rename \"s\/\\([0-9]\\)\/\/g\" {} +; | wc -l<\/span><\/p>\n

.\/SteveStevens\/SS-Track02(8).mp3 not renamed: .\/SteveStevens\/SS-Track02.mp3 already exists<\/span><\/p>\n

.\/SteveStevens\/SS-Track02(4).mp3 not renamed: .\/SteveStevens\/SS-Track02.mp3 already exists\u00a6<\/span><\/p>\n

0<\/span><\/p>\n

Proved if the screen output is sent to \/dev\/null there is no StdOut:<\/span><\/p>\n

find -regex '.*([0-9]).*' -exec rename \"s\/\\([0-9]\\)\/\/g\" {} +; 2> \/dev\/null<\/span><\/p>\n

If confident of your results and you do the changes globally, then the deletes globally, and all has gone well, then the files changed will equal the files deleted and your total file count will be the same.<\/span><\/p>\n

If you really persisted you may eventually create a regex that renames many of your awful file names as you like, to a generic, tidier format within the bounds of sanity at least. If so save it and publish it to the web for all to use! There is a script example for MP3s in Server Hacks 1, but I'm not typing it here..!<\/span><\/p>\n

So what would be next in the horrendous looking category? Those names with long strings inside {} probably all of these are jpgs actually, but for the exercise:<\/span><\/p>\n

find -name '*{*'<\/span><\/p>\n

.\/AlbumArt_{866F7EC7-163A-4791-BB41-8081A3C9DF2B\u00a3_Small.jpg<\/span><\/p>\n

.\/AlbumArt_{866F7EC7-163A-4791-BB41-8081A3C9DF2B\u00a3_Large.jpg<\/span><\/p>\n

You can fight with that one...<\/span>but as an aside here's a great little music char that popped up during this:<\/span><\/p>\n

\u00e2,\u00ab<\/span><\/p>\n

Try you backup drive with a find for that! Unless you've been to Brasil and know Ivette Sangalo...<\/p>\n

find -name *\u00e2,\u00ab*<\/span><\/p>\n

.\/IveteSangalo-MTVAoVivo\/PERER\u00c3\u0160lyrics(IveteSangalo)\u00e2,\u00ab_files<\/span>
\n.\/IveteSangalo-MTVAoVivo\/PERER\u00c3\u0160lyrics(IveteSangalo)\u00e2,\u00ab_files\/PERER\u00c3\u0160lyrics(IveteSangalo)\u00e2,\u00ab.htm<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Common Metach\u00adara\u00adcters ^ [ . $ { * ( \\ + ) | ? < > The escape character is usually \\ This Post evolved due to the masochistic desire improve my knowledge of regexs and metachars, possibly using them to tidy up MP3 album\/song titles that can be horrendous looking on the command line, ...\u00a0 An Intro to Metachar Search Methods, Wreck the MP3s<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-3831","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"a3_pvc":{"activated":true,"total_views":1,"today_views":0},"_links":{"self":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/3831","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/comments?post=3831"}],"version-history":[{"count":2,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/3831\/revisions"}],"predecessor-version":[{"id":10566,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/3831\/revisions\/10566"}],"wp:attachment":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/media?parent=3831"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/categories?post=3831"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/tags?post=3831"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}