{"id":8890,"date":"2016-07-22T00:18:09","date_gmt":"2016-07-21T23:18:09","guid":{"rendered":"http:\/\/probook\/DebianAdmin\/?p=3527"},"modified":"2016-07-22T00:18:09","modified_gmt":"2016-07-21T23:18:09","slug":"exploring-find-cmd-options-continuing-with-metacharacters-2","status":"publish","type":"post","link":"https:\/\/stevepedwards.today\/DebianAdmin\/exploring-find-cmd-options-continuing-with-metacharacters-2\/","title":{"rendered":"Exploring Find Cmd Options \u2013 Continuing With Metacharacters"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Regular expression metacharacters usage consist of the following:<\/p>\n

^ $ . [ ] { } - ? * + ( ) | \\<\/p>\n

These cannot be read about using the man pages for each symbol except the [ -test\u201d chars e.g:<\/p>\n

man ^<\/span><\/p>\n

No manual entry for ^<\/span><\/p>\n

These metacharacters can be read and expanded by the shell as special characters, or -escaped\u201d by prefixing with the backslash -\\\u201d or quoted in -\u201d to prevent expansion.<\/p>\n

For the first example to show the difference between a literal or unescaped metacharacter, starting with probably the most commonly used metacharacter \u2013 the *;<\/p>\n

Using the find command for example, an erroneous usage search attempt for any files that match ANY other character at the start of the file name:<\/p>\n

find Videos\/ -name *<\/span><\/p>\n

find: paths must precede expression: chapter1.txt<\/span><\/p>\n

Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]<\/span><\/p>\n

This fails with stderror output as the shell attempts to expand the * to output all the files in the current directory (ABOVE Videos) so fails as a search path order conflict. <\/span><\/p>\n

But, if the asterisk is quoted, all files in the Video dir are listed, as they all start with a character of some sort.<\/span><\/p>\n

find Videos\/ -name '*'<\/span><\/span><\/p>\n

Videos\/Martial Law 9-11- Rise Of The Police State.mp4<\/span><\/p>\n

Videos\/What Happened on the Moon.mp4<\/span><\/p>\n

Videos\/The Untold Secrets of NASA - Unbelievable Mars - Space Documentary(2015).mp4<\/span><\/p>\n

Videos\/Peter Joseph's 'Where are we going'.mp4\u2026..etc.<\/span><\/p>\n

So, escaping does the same:<\/span><\/p>\n

find Videos\/ -name <\/span>\\<\/span>*<\/span><\/span><\/p>\n

so gives the same result as above \u2013 all files in Videos are found and listed.<\/span><\/p>\n

If the asterisk is prefixed or appended by another character, then it is not escaped so it can do the job it is intended for. To find all files starting with a \"0\u201d<\/span><\/p>\n

find Videos\/ -name 0*<\/span><\/p>\n

Videos\/027 Richard Dolan Montreal - ModernKnowledge @ CapricornRadioTV.mp4<\/span><\/p>\n

What if you want to find all files starting with any number? You can use a range defined as:<\/span><\/p>\n

find Videos\/ -name '[0-9]*'<\/span><\/p>\n

Videos\/9-11- Decade of Deception (Full Film NEW 2015).mp4<\/span><\/p>\n

Videos\/027 Richard Dolan Montreal - ModernKnowledge @ CapricornRadioTV.mp4<\/span><\/p>\n

but, the equivalent numbers alone also\u00a0show files beginning with a 0 or a 9:<\/span><\/p>\n

find Videos\/ -name '[09]*'<\/span><\/p>\n

Videos\/9-11- Decade of Deception (Full Film NEW 2015).mp4<\/span><\/p>\n

Videos\/027 Richard Dolan Montreal - ModernKnowledge @ CapricornRadioTV.mp4<\/span><\/p>\n

The lesson here is: Don't assume your command structure is correct globally on the basis of 1 result set! It just happens that this directory happens only to have files beginning with a 0 and a 9 so give the same result as 2 very different search conditions!<\/span><\/p>\n

So, in English, you understand [0-9]\u00a0as -find any file starting with a 0 OR a 1 OR a 2...9.\u201d<\/span><\/p>\n

To do the opposite \u2013 find all files NOT beginning with numbers:<\/span><\/p>\n

find Videos\/ -name '[!0-9]*'<\/span><\/p>\n

Videos\/Martial Law 9-11- Rise Of The Police State.mp4<\/span><\/p>\n

Videos\/What Happened on the Moon.mp4<\/span><\/p>\n

Videos\/The Untold Secrets of NASA - Unbelievable Mars - Space Documentary(2015).mp4<\/span><\/p>\n

Videos\/Peter Joseph's 'Where are we going'.mp4\u2026..etc.<\/span><\/p>\n

Note the shebang ! Has to be INSIDE the brackets as outside is interpreted by find as being the start of a file name, not a logical NOT parameter:<\/span><\/p>\n

find Videos\/ -name '![0-9]*'<\/span><\/p>\n

(no files found as none begin with a -!\u201d)<\/span><\/p>\n

For NOT in a find command context, it would be a space delimited operator, separated from and before the file name option, to find all files NOT beginning with a number:<\/span><\/p>\n

find Videos\/ ! -name '[0-9]*'<\/span><\/p>\n

Videos\/Martial Law 9-11- Rise Of The Police State.mp4<\/span><\/p>\n

Videos\/What Happened on the Moon.mp4<\/span><\/p>\n

Videos\/The Untold Secrets of NASA - Unbelievable Mars - Space Documentary(2015).mp4<\/span><\/p>\n

Videos\/Peter Joseph's 'Where are we going'.mp4\u2026..<\/span>etc.<\/span> <\/span><\/p>\n

What about file name order relating to case and given by a range? <\/span><\/p>\n

It's complex due to the POSIX or other ASCII standard your PC is set to, AND how find lists it's results depending on inode order also. This is why you get some seemingly weird results for alphabetical listings.<\/span><\/p>\n

First \u2013 so I know how many files I have totally in Videos:<\/span><\/p>\n

find Videos\/ -name '*' | wc -l<\/span><\/p>\n

137<\/span><\/p>\n

Will Shott's TLCL.pdf gives examples for grep on p273 that show ranges the likes of:<\/span><\/p>\n

find Videos\/ -name '[ABCDEFGHIJKLMNOPQRSTUVWXZY]*' | wc -l<\/span><\/p>\n

131<\/span><\/p>\n

So I know I am not seeing ALL files, as 2 begin with numbers,<\/span><\/p>\n

find Videos\/ -name '[0-9]*' | wc -l<\/span><\/p>\n

2<\/span><\/p>\n

and 3 with lowercase e.g.<\/span><\/p>\n

find Videos\/ -name '[abcdefghijklmnopqrstuvwxyz]*' | wc -l<\/span><\/p>\n

3<\/span><\/p>\n

This does not account for all 137 files, so check these can be found using a number range [0-9], a lowercase range [a-z] and an uppercase range [A-Z] together to be sure as:<\/span><\/p>\n

find Videos\/ -name '[ABCDEFGHIJKLMNOPQRSTUVWXZYabcdefghijklmnopqrstuvwxyz]*' | wc -l<\/span><\/p>\n

134<\/span><\/p>\n

then:<\/span><\/p>\n

find Videos\/ -name '[ABCDEFGHIJKLMNOPQRSTUVWXZYabcdefghijklmnopqrstuvwxyz0123456789]*' | wc -l<\/span><\/p>\n

136<\/span><\/p>\n

BUT that's NOT exactly correct \u2013 I'm missing 1 file!? <\/span><\/p>\n

How can I find it??<\/span><\/p>\n

Use the NOT shebang ! with the find option...? Aha! A file that begins with a quote \" ' \u201d<\/span><\/p>\n

find Videos\/ ! -name '[0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXZY]*'<\/span><\/p>\n

Videos\/'Why in the World are They Spraying' <\/b>Documentary HD (multiple language subtitles).mp4<\/span><\/p>\n

I found that file by elimination logic in the find command, but how would you escape the quote character to find that file \u2013 assuming you knew it existed? It cannot be escaped in the range box \u2013 it is seen as a delimiter itself so expects input!<\/span><\/p>\n

find Videos\/ ! -name '[\\'0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXZY]*'<\/span><\/p>\n

><\/span><\/p>\n

Going back to basics as at the start above using the \\ escape does it:<\/span><\/p>\n

find Videos\/ -name \\'*<\/span><\/p>\n

Videos\/'Why in the World are They Spraying' Documentary HD (multiple language subtitles).mp4<\/span><\/p>\n

Because the ASCII range of keys is keymap dependent, but POSIX lists the historcal map numerically as Shott states:<\/span><\/p>\n

-Back when Unix was first developed, it only knew about ASCII characters, and this fea-<\/i><\/span><\/p>\n

ture reflects that fact. In ASCII, the first 32 characters (numbers 0-31) are control codes<\/i><\/span><\/p>\n

(things like tabs, backspaces, and carriage returns). The next 32 (32-63) contain printable<\/i><\/span><\/p>\n

characters, including most punctuation characters and the numerals zero through nine.<\/i><\/span><\/p>\n

The next 32 (numbers 64-95) contain the uppercase letters and a few more punctuation<\/i><\/span><\/p>\n

symbols. The final 31 (numbers 96-127) contain the lowercase letters and yet more punc-<\/i><\/span><\/p>\n

tuation symbols. Based on this arrangement, systems using ASCII used a collation order<\/i><\/span><\/p>\n

that looked like this:<\/i><\/span><\/p>\n

ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz<\/i><\/span><\/p>\n

This differs from proper dictionary order, which is like this:<\/i><\/span><\/p>\n

aAbBcCdDeEfFgGhHiIjJkKlLmMnNoOpPqQrRsStTuUvVwWxXyYzZ...To support this ability, the POSIX standards introduced a concept called a locale, which<\/i><\/span><\/p>\n

could be adjusted to select the character set needed for a particular location. We can see<\/i><\/span><\/p>\n

the language setting of our system using this command:<\/i><\/span><\/p>\n

[me@linuxbox ~]$ echo $LANG<\/i><\/span><\/p>\n

en_US.UTF-8<\/i><\/span><\/p>\n

With this setting, POSIX compliant applications will use a dictionary collation order<\/i><\/span><\/p>\n

rather than ASCII order. This explains the behavior of the commands above. A character<\/b><\/i><\/span><\/p>\n

range of [A-Z] when interpreted in dictionary order includes all of the alphabetic char-<\/b><\/i><\/span><\/p>\n

acters except the lowercase -a\u201d<\/b>, hence our results\u2026To partially work around this problem, the POSIX standard includes a number of character classes which provide useful ranges of characters.\u201d <\/i><\/span><\/p>\n

I have only 1 file that starts with a lowercase -a\u201d:<\/span><\/p>\n

find Videos\/ -name a*<\/span><\/p>\n

Videos\/antigravity<\/b> hutchison effect.mp4<\/span><\/p>\n

So, if my $LANG variable is POSIX compliant:<\/span><\/p>\n

$LANG <\/span><\/p>\n

en_GB.UTF-8: command not found<\/span><\/p>\n

I should NOT find that one file with Shott's [A-Z] \u2013 <\/b><\/i>and I don't \u2013 only the string\u00a0\"anti\" <\/i><\/span><\/p>\n

find Videos\/ -name '[A-Z]*' | grep anti<\/span><\/p>\n

Videos\/ILLUMINATI SECRETS - The New Atlanti<\/strong>s - FEATURE FILM.mp4<\/span><\/p>\n

find Videos\/ -name '[A-Z]*' | wc -l<\/span>
\n131<\/span>
\nfind Videos\/ -name '[a-Z]*' | wc -l<\/span>
\n134<\/span><\/p>\n

Knowing this, AND that the find command also sorts in a combo of inode order, it can be understood why there is apparent illogical alphabetical listing order; RRC; in the output, such as part of all the files here:<\/span><\/p>\n

find Videos\/ -name '*'<\/span><\/p>\n

Videos\/Richplanet 2016 UK Tour - PART 2 OF 3.mp4<\/span><\/p>\n

Videos\/RP EP26 PT1.mp4<\/span><\/p>\n

Videos\/Crop Circles- The Hidden Truth - Part 4.mp4<\/span><\/p>\n

Table 19-2: POSIX Character Classes<\/b><\/i><\/span><\/p>\n

Character Class Description<\/b><\/i><\/span><\/p>\n

[:alnum:] The alphanumeric characters. In ASCII, equivalent to:<\/i><\/span><\/p>\n

[A-Za-z0-9]<\/i><\/span><\/p>\n

[:word:] The same as [:alnum:], with the addition of the underscore<\/i><\/span><\/p>\n

(_) character.<\/i><\/span><\/p>\n

[:alpha:] The alphabetic characters. In ASCII, equivalent to:<\/i><\/span><\/p>\n

[A-Za-z]<\/i><\/span><\/p>\n

[:blank:] Includes the space and tab characters.<\/i><\/span><\/p>\n

[:cntrl:] The ASCII control codes. Includes the ASCII characters 0<\/i><\/span><\/p>\n

through 31 and 127.<\/i><\/span><\/p>\n

[:digit:] The numerals zero through nine.<\/i><\/span><\/p>\n

[:graph:] The visible characters. In ASCII, it includes characters 33<\/i><\/span><\/p>\n

through 126.<\/i><\/span><\/p>\n

[:lower:] The lowercase letters.<\/i><\/span><\/p>\n

[:punct:] The punctuation characters. In ASCII, equivalent to:<\/i><\/span><\/p>\n

[-!\"#$%&'()*+,.\/:;<=>?@[\\\\\\]_`{|}~]<\/i><\/span><\/p>\n

[:print:] The printable characters. All the characters in [:graph:]<\/i><\/span><\/p>\n

plus the space character.<\/i><\/span><\/p>\n

[:space:] The whitespace characters including space, tab, carriage<\/i><\/span><\/p>\n

return, newline, vertical tab, and form feed. In ASCII,<\/i><\/span><\/p>\n

equivalent to:<\/i><\/span><\/p>\n

[ \\t\\r\\n\\v\\f]<\/i><\/span><\/p>\n

[:upper:] The uppercase characters.<\/i><\/span><\/p>\n

[:xdigit:] Characters used to express hexadecimal numbers. In ASCII,<\/i><\/span><\/p>\n

equivalent to:<\/i><\/span><\/p>\n

[0-9A-Fa-f]<\/i><\/span><\/p>\n

Remember, however, that this is not an example of a regular expression, rather it is the<\/i><\/span><\/p>\n

shell performing pathname expansion\u2026.POSIX Basic Vs. Extended Regular Expressions<\/i><\/span><\/p>\n

Just when we thought this couldn\u2019t get any more confusing, we discover that POSIX also<\/i><\/span><\/p>\n

splits regular expression implementations into two kinds: basic regular expressions<\/i><\/span><\/p>\n

(BRE) and extended regular expressions (ERE). The features we have covered so far are<\/i><\/span><\/p>\n

supported by any application that is POSIX compliant and implements BRE. Our grep<\/i><\/span><\/p>\n

program is one such program.<\/i><\/span><\/p>\n

What\u2019s the difference between BRE and ERE? It\u2019s a matter of metacharacters. With BRE,<\/i><\/span><\/p>\n

the following metacharacters are recognized:<\/i><\/span><\/p>\n

^ $ . [ ] *<\/i><\/span><\/p>\n

All other characters are considered literals. With ERE, the following metacharacters (and<\/i><\/span><\/p>\n

their associated functions) are added:<\/i><\/span><\/p>\n

( ) { } ? + |<\/i><\/span><\/p>\n

However (and this is the fun part), the -(\u201d, -)\u201d, -{\u201d, and -}\u201d characters are treated as<\/i><\/span><\/p>\n

metacharacters in BRE if they are escaped with a backslash, whereas with ERE, preced-<\/i><\/span><\/p>\n

ing any metacharacter with a backslash causes it to be treated as a literal. Any weirdness<\/i><\/span><\/p>\n

that comes along will be covered in the discussions that follow.\u201d<\/i><\/span><\/p>\n

ls \/usr\/sbin\/[[:upper:]]*<\/span><\/p>\n

\/usr\/sbin\/ModemManager \/usr\/sbin\/VBoxControl<\/span><\/p>\n

\/usr\/sbin\/NetworkManager \/usr\/sbin\/VBoxService<\/span><\/p>\n

Find can be used similarly. Find all files starting with a lower case letter:<\/span><\/p>\n

find Videos\/ -name '[[:lower:]]*'<\/span><\/p>\n

Videos\/screencasts<\/span><\/p>\n

Videos\/hutchison effect wiki never before seen footage 3 25 2011.mp4<\/span><\/p>\n

Videos\/antigravity hutchison effect.mp4<\/span><\/p>\n

Find all files NOT beginning with letters: <\/span><\/p>\n

find Videos\/ ! -name '[[:alpha:]]*'<\/span><\/p>\n

Videos\/'Why in the World are They Spraying' Documentary HD (multiple language subtitles).mp4<\/span><\/p>\n

Videos\/9-11- Decade of Deception (Full Film NEW 2015).mp4<\/span><\/p>\n

Videos\/027 Richard Dolan Montreal - ModernKnowledge @ CapricornRadioTV.mp4<\/span><\/p>\n

Same as [0-9] above:<\/span><\/p>\n

find Videos\/ -name '[[:digit:]]*'<\/span><\/p>\n

Videos\/9-11- Decade of Deception (Full Film NEW 2015).mp4<\/span><\/p>\n

Videos\/027 Richard Dolan Montreal - ModernKnowledge @ CapricornRadioTV.mp4<\/span><\/p>\n

Note complexity is required to NOT find either letters OR numbers = my\u00a0\"quote\" started file name:<\/span><\/p>\n

find Videos\/ ! -name '[[:alpha:]]*' ! -name '[[:digit:]]*<\/span>'<\/span><\/p>\n

Videos\/'Why in the World are They Spraying' Documentary HD (multiple language subtitles).mp4<\/span><\/p>\n

This can be simplified to a NOT alphanumeric range:<\/span><\/p>\n

find Videos\/ ! -name '[0-9a-Z]*'<\/span><\/p>\n

find Videos\/ -name '[!0-9a-Z]*'<\/span><\/p>\n

Videos\/'Why in the World are They Spraying' Documentary HD (multiple language subtitles).mp4<\/span><\/p>\n

Find all files without white space anywhere in the name:<\/span><\/p>\n

find Videos\/ ! -name '*[[:blank:]]*'<\/span><\/p>\n

Videos\/<\/span><\/p>\n

Videos\/Irrefutable.mp4<\/span><\/p>\n

Videos\/RichDLoydPye.mp4<\/span><\/p>\n

Videos\/screencasts<\/span><\/p>\n

A really handy addition to the above for Linux systems is to rename all white spaced files without white space, say, for an MP3 collection, that show horrible looking Album\/Track names on the command line often;\u00a0using the -exec addition seen already in Cool Commands Posts:<\/p>\n

BEFORE:<\/strong><\/p>\n

ls \/Storebird\/MP3\/Candy\\ Dulfer\\ -\\ Sax-A-Go-Go\\ \\(74321_111812\\)\/Candy\\ Dulfer\\ \\ \\ -\\ 2\\ Funky.mp3<\/span><\/p>\n

run it for directories first:<\/p>\n

find \/Storebird\/MP3\/ -type d -name '*[[:blank:]]*' -exec rename \"s\/ \/\/g\" {} +;<\/span><\/p>\n

AFTER:<\/strong><\/p>\n

ls \/Storebird\/MP3\/CandyDulfer-Sax-A-Go-Go\\(74321_111812\\)\/<\/span><\/p>\n

Re-run it for files:<\/p>\n

find \/Storebird\/MP3\/ -type f -name '*[[:blank:]]*' -exec rename \"s\/ \/\/g\" {} +;<\/span><\/p>\n

ls \/Storebird\/MP3\/CandyDulfer-Sax-A-Go-Go\\(74321_111812\\)\/CandyDulfer-2Funky.mp3<\/span><\/p>\n

You could then edit\u00a0and append these two lines to a simple shell script or alias to run on any directory you cd into e.g.:<\/p>\n

find -type d -name \"*[[:blank:]]*\" -exec rename \"s\/ \/\/g\" {} +; <\/span><\/p>\n

find -type f -name \"*[[:blank:]]*\" -exec rename \"s\/ \/\/g\" {} +;<\/span><\/p>\n

vi ~\/rmspaces.sh<\/span><\/p>\n

\"rmspaces.png\"<\/a><\/p>\n