This find command exploration nearly got me sectioned! It is the most un-intuitive, cryptic, annoying command I have come across. If you get to the end of this Post doing similar tests you may understand why...if you want some quick simple examples (not all work in Mint) look here:
The find command has many complex options, so I wanted to look at some, but with a useful common task in mind. With the VIM editor reading man find, use the / search function to search for a parameter you want to explore, e.g: /-mount using the n key to find the next occurance of that search term.
Using find generally, the target to be found follows the path to be searched:
find [-H] [-L] [-P] [-D debugopts] [-Olevel] [path...] [expression]
Doing this type of exercise will expose you to some interesting specific options by reading man find, and help to think about and understand more complex system parameters like ASCII key sort order, so (hopefully!) how to use them in building commands to find what you need.
The man page examples are not very clear or comprehensive but a start¦
The most basic find option would probably be just a whole or part of a file name that you know exists somewhere, so a good catchall is that name with it's file type if you know it e.g. mp4, docx, pdf etc. and then filter out from there as your memory is jogged (or not!).
Find's defaults include recursive directory search, so no switch required for that, great - but in Unix tradition - it DOES NOT recognise mixed upper/lower case, so you need to specify correct case in the file name unless you use the -iname switch e.g.
Example 1 Find file by name and/or doc type
it will find:
find Videos/ -name UFO*
Videos/UFOs - Nazis and Roswell to the 21st Century with Richard Dolan.mp4
Videos/UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4
find Videos/ -name *mp4
(finds all mp4's)
but NOT:
find Videos/ -name ufo*
find Videos/ -name *MP4
(finds nothing as none contain lower case ufo or upper case MP4)
find Videos/ -iname ufo*
Videos/Ufo - The Secret Nasa Transmissions Haunebu Vril.mp4
Videos/UFOs - Nazis and Roswell to the 21st Century with Richard Dolan.mp4
Videos/UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4
If you know only the mid part of a file name, you need wildcards front and back as *CV alone won't work i.e:
find . -name *CV*
./LSADocs/Steve_Edwards_CV_2Page.docx
./Desktop/Steve_Edwards_CV_2Page.docx
./Desktop/Steve_Edwards_IT_CV_3p.docx
Another important default listing output is the ASCII char number order, which may confuse you with alphabetical file names if you don't realise this, which is also distribution dependent.
For example, create a test directory for this example, cd into it, create files, then ls the content:
~/test $ touch file{a..c}
~/test $ touch file{A..C}
~/test $ ls -l file*
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:08 filea
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:09 fileA
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:08 fileb
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:09 fileB
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:08 filec
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:09 fileC
Find seems to list the same ASCII way:
find file*
filea
fileA
fileb
fileB
filec
fileC
This accounts for file names seemingly out of alphabetical order at times due to capitalisation, but it is due to a more technical reason explained later:
find Videos/ -name *mp4 | head -3
Videos/David Keith Presents Solar Geoengineering Lecture at Stanford U..mp4
Videos/Masonic Switzerland - Home of the Pharaohs (TPS).mp4
Videos/ALDEBARAN MYSTERY 2 - NAZI UFO SECRETS, THULE, VRIL & THE BLACK SUN.mp4
BUT! If a file begins with a number, it is listed before letters:
~/test $ find *txt
1.txt
2.txt
3.txt
file1.txt
File1.txt
Another important switch to know at first is -mount or -xdev, which stops find searching non local file systems, which could runaway across mounted networked PCs or attached backup drives etc.
DESCRIPTION
This manual page documents the GNU version of find. GNU find searches
the directory tree rooted at each given file name by evaluating the
given expression from left to right, according to the rules of prece
dence (see section OPERATORS), until the outcome is known (the left
hand side is false for and operations, true for or), at which point
find moves on to the next file name.
Remember that you need sudo if you are finding files other than your own e.g. on / dir.
Example 2 Find most recent file added/changed in a system/dir?
I DLd some you tube vids to Videos, which should be the most recent files known in this directory, as a known quantity check, then ran the below googled find cmd to get a starting point for some options to investigate, which did indeed find the currently downloading file as the most recent in this directory, so how?
It sorted by find's modification time option %T, which is the longest time in seconds since the start of Unix Time.
https://stevepedwards.today/DebianAdmin/it-related-numberphile-vids/
find Videos/ -type f -printf "%T@ %p\n"| sort -r | head -3
1468387319.9033764330 Videos/The Human Brain (full documentary) HD.mp4.crdownload
1468386234.9634008040 Videos/RichPlanet TV- Agendas Of The Global Elite.mp4
1468385210.6674238120 Videos/RichPlanet TV- The Falklands Conflict Cover Up.mp4
-type c
File is of type c:
f regular file
-printf format
True; print format on the standard output, interpreting `\' escapes and `%' directives.
Field widths and precisions can be specified as with the `printf' C function. Please note
that many of the fields are printed as %s rather than %d, and this may mean that flags
don't work as you might expect. This also means that the `-' flag does work (it forces
fields to be left-aligned). Unlike -print, -printf does not add a newline at the end of
the string.
\n Newline.
%Tk File's last modification time in the format specified by k, which is the same as
for %A.
%p File's name.
Note the file name by %p decides the find cmd sort order first which would be files beginning with R before T, so the reverse sort by number is used to get the highest time first in secs since Jan 1970 Unix Time. Without %p you just get the last mod time:
find Videos/ -type f -printf "%T@ \n" | sort -r | head -3
1468387319.9033764330
1468386234.9634008040
1468385210.6674238120
If you check this time using the date command you do get the time the download completed shown by ls -lt = 6.21:59am:
date --date='@1468387319'
Wed Jul 13 06:21:59 BST 2016
ls -lt Videos/ | head -3
total 44030732
-rw-r--r-- 1 stevee stevee 376983001 Jul 13 06:21 The Human Brain (full documentary) HD.mp4
-rw-r--r-- 1 stevee stevee 134365402 Jul 13 06:03 RichPlanet TV- Agendas Of The Global Elite.mp4
As the system is dynamic, you will always have recent file changes for many processes as shown by the changes in the last minute (-1) of browser session for example, during this download (imagine what the system/kernel is doing per nanosecond!). This also shows another find default of current directory (pwd) WITH hidden . files shown if path not specified.
find -cmin -1
./.config/google-chrome/Default
./.config/google-chrome/Default/Top Sites
./.config/google-chrome/Default/History
./.config/google-chrome/Default/History-journal
./.config/google-chrome/Default/Top Sites-journal
./.config/google-chrome/Default/Preferences
./.config/google-chrome/Default/Current Session
./Videos/The Human Brain (full documentary) HD.mp4.crdownload
-cmin n
File's status was last changed n minutes ago.
This is the ongoing DL status as bytes are added
-mmin n
File's data was last modified n minutes ago.
This is the final name change from .crdownload to .mp4
Example 3 Largest file in a system/dir?
Before experimenting, use the GUI to tell you what IS the biggest file by clicking Size column 1.8GB in this case:
Lots of ways to do this other than find no doubt, (like alias = ducks in Notepad page ), but using find..?
For these types of problems I have to ask what do I know already about a file size's attributes, via what commands, and how do I get to use those to show me what I want?
I know from the encapsulation Post:
https://stevepedwards.today/DebianAdmin/encapsulation-from-bits-to-gigabytes-in-200-years-2/
that ls -ls shows the ext4 block size and IO block size reserved for a file even if it does not completely fill it, so is an indicator of total file size, within 7 block size accuracy, as there are 8 x 512k hard drive block sizes to an ext4 IO sector of 4 x 1024k = 4096k.
ls -ls Videos/ | sort -nr | head -5
1731532 -rw-r--r-- 1 stevee stevee 1773080965 Nov 9 2015 Masonic Switzerland - Home of the Pharaohs (TPS).mp4
1716976 -rw-r--r-- 1 stevee stevee 1758175907 Nov 4 2015 The illuminati Exposed (The Movie).mp4
1669104 -rw-r--r-- 1 stevee stevee 1709156184 Dec 1 2015 UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4
1539972 -rw-r--r-- 1 stevee stevee 1576922999 Apr 28 01:56 9-11- Decade of Deception (Full Film NEW 2015).mp4
1487004 -rw-r--r-- 1 stevee stevee 1522684968 Dec 9 2015 The Great Culling- Our Water Official Full Movie.mp4
Seems the GUI uses ls similarly also seen above as it gives the same order?
Does that field have potential use in any of find's options or switches? Yes:
%s File's size in bytes.
Substituting this into Ex 1 above to sort on the file size instead of mod Time, but still print each file's %p name on a new line \n:
find Videos/ -type f -printf "%s%p\n" | sort -nr | head -5
1773080965Videos/Masonic Switzerland - Home of the Pharaohs (TPS).mp4
1758175907Videos/The illuminati Exposed (The Movie).mp4
1709156184Videos/UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4
1576922999Videos/9-11- Decade of Deception (Full Film NEW 2015).mp4
1522684968Videos/The Great Culling- Our Water Official Full Movie.mp4
Similarly to above example for mod time, without the %p file name you just get the IO sector size:
1773080965
1758175907
1709156184
1576922999
1522684968
ls -als Videos/The\ Great\ Culling-\ Our\ Water\ Official\ Full\ Movie.mp4
1487004 -rw-r--r-- 1 stevee stevee 1522684968 Dec 9 2015 Videos/The Great Culling- Our Water Official Full Movie.mp4
Do I even need the file -type f? No. Same result.
This is almost identical to ls -ls Videos/ | sort -nr | head -5 but as bs not IO sectors. (1522684968/1487004=1024).
Example 4 finding files set guid +a=s for security reasons
The book 100 Linux Server Hacks mentions disabling binaries that are set uid = s for commands that you probably don't want all users to have root perms to when they run it which is the reason for this s permission historically; as for passwd, it is good for users to be able to change their own at any time without admin permission so best retained as is.
For example, the perm s replaces the x in the User perms section, so is run as root as he is the file owner, and looks like :
ls -l /usr/bin/passwd
-rwsr-xr-x 1 root root 47032 Jan 27 00:50 /usr/bin/passwd
but could also be in the Group or Others depending. You would remove it with chmod u-s.
Some of these can be found in the /bin, /usr/bin and /usr/sbin dirs using the -perms option in 2 forms, and can all be set as search paths at once:
sudo find /bin/ /usr/bin /usr/sbin/ -perm +a=s
/bin/su
/bin/ping6
/bin/umount
/bin/fusermount
/bin/ping
/bin/mount
/usr/bin/chfn
/usr/bin/passwd
/usr/bin/chage
/usr/bin/newgrp
/usr/bin/mail-unlock
/usr/bin/ssh-agent
/usr/bin/crontab
/usr/bin/mail-lock
/usr/bin/dotlockfile
/usr/bin/wall
/usr/bin/pkexec
/usr/bin/traceroute6.iputils
/usr/bin/X
/usr/bin/gpasswd
/usr/bin/mlocate
/usr/bin/lppasswd
/usr/bin/bsd-write
/usr/bin/expiry
/usr/bin/mail-touchlock
/usr/bin/mtr
/usr/bin/chsh
/usr/bin/sudo
/usr/sbin/pppoe
/usr/sbin/uuidd
/usr/sbin/pppd
You could send all these to a text file as usual:
sudo find / -mount -perm +a=s > setuid.txt
Note the list order though¦try and work out why it's like that yourself...and let me know! It's different between PCs so I assume it's to do with inode numbers or drive state or some such...?
Not what I thought in example 1 re ASCII format! Is it mod/creation time?
As I've said before, this is why you have to experiment yourself!
Example 5 Find a User's files but skipping their /home dir (Mindbending!)
You may want to find all files belonging to a user that are outside /home say, to delete after an account removal if you forgot to purge fully with deluser -remove-all-files.
This gave me extreme brainache! First, AN answer - but it is not recursing into /750GB - then the how I got to it:
sudo find /* home -prune -user stevee
/750GB
/metest
/Quadra
Good web examples here to understand the AND/OR logical operation:
https://www.theunixschool.com/2012/07/find-command-15-examples-to-exclude.html
The cmd order is critical in Mint, because:
The find command works like this: It starts finding files from the path provided in the command which in this case is the current directory(.). From here, it traverses through all the files in the entire tree and prints those files matching the criteria specified. -prune will not allow the find command to descend into the file any further if it is a directory. Hence, when find starts with the current directory, prune does not allow it to descend the current directory since it itself is a directory, and hence only the current directory gets printed, not the files within the directory. The print happens here because it is the default functionality of find to print anything which is true.
You have to work hard to understand the weirdness..!
A missing * can make all the difference, as can an incorrect logical operator like a ! (NOT) , or a -prune -o (OR), because the final command must evaluate TRUE to print out. This takes time to get your head around and I still don't get it when constructing a cmd...hmm why I'm a shit programmer also I guess...
Find starts relative to the dir that follows, which is . if not otherwise defined, so if at / root to start:
find /
...whole root dir contents
find .
...whole current dir contents
find . -name . -prune
.
find . ! -name . -prune
./lib32
./mnt
./sys
./proc
./usr
./home
./initrd.img
./lib64
./boot
./var
./lost+found
./opt
./cdrom
./.rpmdb
./run
./root
./750GB
./media
./Share
./sbin
./lib
./dev
./srv
./etc
./PiImg
./tmp
./vmlinuz
./Quadra
./bobtest.txt
./bin
Also note there is no ASCII alphabetical order here either. The reason is at the Post end.
There is a big difference between:
find .
find /
and
find /*
later, depending what follows¦like logical operator ! {NOT}
So, find . ! -name . -prune means find in . any file NOT named . itself, AND don't recurse (prune) into . (or print out .) so prints all the files in . but not . itself.
If you leave out -prune, the whole tree is printed except . alone!
This is proved by grepping the output for "." at the line start, with only 1 or no characters after it, which there is not for "." if it is listed, as all other output has more than ./xxx after it:
Anchoring
The caret ^ and the dollar sign $ are meta-characters that respectively
match the empty string at the beginning and end of a line.
Repetition
A regular expression may be followed by one of several repetition
operators:
? The preceding item is optional and matched at most once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{n} The preceding item is matched exactly n times.
{n,} The preceding item is matched n or more times.
{,m} The preceding item is matched at most m times. This is a GNU
extension.
{n,m} The preceding item is matched at least n times, but not more
than m times.
find . ! -name . -prune | grep ^.?
or
find . ! -name . -prune | grep ^.$
no output here so "." is NOT listed.
Soooo¦by continued logic, to find all the files in the pwd root . dir, or /, that only belong to user bob, (as . belongs to root) without searching his home directory (as it obviously has bob's files in it):
sudo find . ! -name . -prune ! -name home -user bob
./bobtest.txt
This is correct ownership as shown by:
ls -al ./bobtest.txt
-rw-r--r-- 1 bob root 0 Jul 14 00:21 ./bobtest.txt
and the home dir was omitted.
You would not have understood the workings the same if the last command was shown more simply as you would expect it to work from the man page for user file ownership as:
-user uname
File is owned by user uname (numeric user ID allowed).
sudo find -user bob
./bobtest.txt
find: ˜./proc/9683/task/9683/fd/5,: No such file or directory
find: ˜./proc/9683/task/9683/fdinfo/5,: No such file or directory
find: ˜./proc/9683/fd/5,: No such file or directory
find: ˜./proc/9683/fdinfo/5,: No such file or directory
./home/bob
./home/bob/.config¦.etc.
This is correct for finding ALL files belonging to bob, but along with unwanted StdError, too many lines get shown, not just the desired result of the single known file in root; /bobtest.txt, and his home dir files (because home was NOT pruned!) - so the print out is noisy and confusing as . was not pruned, but if it had been, then /home would not have been recursed either. Confused yet?
It showes 73 lines inc. Stderror
You could try to remove the Stderror to clean it up except for bob's files, with:
sudo find -user bob 2> /dev/null
The more intuitive command to show all bob's files under /root, except those in his home directory is:
sudo find /* home -prune -user bob
/bobtest.txt
Let's go through that successful cmd again in English:
sudo find /* home -prune -user bob
sudo required to enter / root dir
find all files/dirs under /, except in /home, that only belong to user bob.
So, you should be able to apply that to other users, and include (by default) any other network or external file systems that are mounted on root / correct...? Wrong!! It does not recurse even though /root was not pruned! This command does my head in...!
What happens then?
I have an internal partition on my laptop that does not mount at boot, that has a backup of my /stevee directory, owned by stevee; and backup of root directories, owned by root; on it, so I'll mount it:
stevee@AMD / $ sudo mount -t ext4 /dev/sda3 750GB/
stevee@AMD / $ ls /750GB/ -al
total 16
drwxr-xr-x 4 stevee stevee 4096 Jul 14 16:24 .
drwxr-xr-x 29 root root 4096 Jul 14 00:21 ..
drwxr-xr-x 22 root root 4096 Jul 11 20:41 BURoot
drwxr-xr-x 9 stevee stevee 4096 Jul 8 00:21 stevee
If I run the same command for user stevee what happens? No recursion into them! Why not? / was not pruned...??
Is it because the search started from the files under /root, not itself...?
sudo find /* home -prune -user stevee
./750GB
./Quadra
so find shows these dirs of stevee, but not his home dir as required proved with grep. OK so far.
ls -al ./ | grep stevee
drwxr-xr-x 4 stevee stevee 4096 Jul 14 16:24 750GB
drwxrwxr-x 2 stevee root 4096 Jan 18 19:38 Quadra
So how do you get the recursion to happen for these and show just more sub files of these stevee's dirs? First what is on this drive? 2 dirs and one test file.
ls /750GB/ -l
total 8
drwxr-xr-x 22 root root 4096 Jul 11 20:41 BURoot
-rw-r--r-- 1 stevee stevee 0 Jul 14 20:22 metest
drwxr-xr-x 9 stevee stevee 4096 Jul 8 00:21 stevee
I give find the new root reference directory. This works, but it is not clear what is being pruned at first, as no dir name was specified, so apparently nothing was a candidate for pruning leaving only files/dir owned by stevee??:
Maybe it means prune any dir owned by stevee without being specifically named, then print what files are left that are owned by stevee...?
find /750GB/* -prune -user stevee
/750GB/metest
/750GB/stevee
This command is so annoying! I can't see how you would ever know for sure if you got what you need from this command at all...it just does not seem worth the time and effort to study...but to finish I found out how to exclude the stevee dir AND the root owned dir by naming it's full path for pruning, then adding the -user option to find stevee's remaining files. It is about the order of options position...dirs to the left, options to the right in this case...
sudo find /750GB/* /750GB/stevee/ -prune -user stevee
/750GB/metest
/750GB/stevee
/750GB/stevee/
Yet, confusingly, this next seemingly "logical" construction does the opposite of what you may think by still printing all the files belonging to stevee, from ALL the sub dirs I "thought" I told it not to:
sudo find /750GB/* ! -name /750GB/stevee/ -user stevee
/750GB/stevee/Documents/AmpDocs
/750GB/stevee/Documents/AmpDocs/AllValveAmpPosts.pdf....etc.
because it actually reads: Find all files/dirs under /750GB NOT named /750GB/stevee, so prints everything belonging to stevee except the dir /750GB/stevee.
But, this alternative still acts the same way though the name switch is not involved! It's like the logic in Mint is reversed!
sudo find /750GB/* -not \( -path 750GB/stevee/* \) -user stevee
/750GB/stevee/Documents/AmpDocs
/750GB/stevee/Documents/AmpDocs/AllValveAmpPosts.pdf....etc.
The fix is to name the directory completely explicitly using quotes so that NO files that begin with /750GB/stevee at all are printed, so just the dir is:
sudo find /750GB/* ! -wholename "/750GB/stevee/*" -user stevee
/750GB/metest
/750GB/stevee
If you want to omit the directory also, trim the end slash:
sudo find /750GB/* ! -wholename "/750GB/stevee*" -user stevee
/750GB/metest
or use the path switch:
sudo find /750GB/* ! -path "/750GB/stevee*" -user stevee
/750GB/metest
Or from the complete top of / not recursing stevee's dirs:
sudo find /* ! -path home -prune ! -path /750GB -prune -user stevee
/metest
If you want to find dirs/files in the root dir / that belong to a user other than and/or ALONG WITH root:
sudo find /* ! -path home -user stevee
...../proc/8539/setgroups
/proc/8539/timers
/proc/8539/timerslack_ns...
/var/www/wpFileSystem/wp/wp-activate.php
/var/www/wpFileSystem/wp/.htaccess
/var/lib/lightdm-data/stevee
What a complete brain ache...
https://stackoverflow.com/questions/4210042/exclude-directory-from-find-command
There is clearly some confusion here as to what the preferred syntax for skipping a directory should be. (Really!?)
GNU Opinion
To ignore a directory and the files under it, use -prune
From the GNU find man page
Reasoning
-prune stops find from descending into a directory. Just specifying -not -path will still descend into the skipped directory, but -not -path will be false whenever find tests each file.
Issues with -prune
-prune does what it's intended to, but are still some things you have to take care of when using it.
find prints the pruned directory.
TRUE That's intended behaviour, it just doesn't descend into it. To avoid printing the directory altogether, use a syntax that logically omits it.
-prune only works with -print and no other actions.
NOT TRUE. -prune works with any action except -delete. Why doesn't it work with delete? For -delete to work, find needs to traverse the directory in DFS order, since -delete will first delete the leaves, then the parents of the leaves, etc... But for specifying -prune to make sense, find needs to hit a directory and stop descending it, which clearly makes no sense with -depth or -delete on.
Find command list order. So what is the reason that find appeared to show lists alphabetically initially, then not in the later examples?
From UnixDB book:
"..find does not display a sorted list...info on file characteristics is obtained from the inode as well as the directory entry..sequence..displayed depends on the internal organization of the file system"