{"id":4339,"date":"2016-08-21T21:16:07","date_gmt":"2016-08-21T20:16:07","guid":{"rendered":"https:\/\/stevepedwards.today\/DebianAdmin\/?page_id=4339"},"modified":"2025-06-08T15:27:35","modified_gmt":"2025-06-08T14:27:35","slug":"awk-lines-and-info","status":"publish","type":"page","link":"https:\/\/stevepedwards.today\/DebianAdmin\/awk-lines-and-info\/","title":{"rendered":"Awk\/Perl\/SED Notepad"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Create yourself a copy of the awk\/sed man page - it's easier to read:<\/p>\n

man -t awk | ps2pdf - > awkman.pdf<\/span><\/p>\n

man -t sed\u00a0| ps2pdf - > sedman.pdf<\/span><\/p>\n

From sed & awk by OReilly - older kernel\/system example problems<\/i><\/b>...examples don't copy in lines, but columns often, and major parts of examples are assumed knowledge\/steps, so missing...etc...so you'll have to work it out as you go...<\/p>\n

A better intro is Shott's:<\/p>\n

https:\/\/linuxcommand.org\/lc3_adv_awk.php<\/a><\/span><\/p>\n

VIM - select all and delete is :%d \u00a0 \u00a0<\/strong> select all and copy is\u00a0:%y \u00a0 <\/strong>this goes into vim\u00a0mem so can be pasted to another vim doc with Shift P.<\/em><\/span><\/p>\n

FS: field separator, default = 1 whitespace<\/em><\/span><\/p>\n

OFS: output FS,\u00a0default = 1 whitespace<\/em><\/span><\/p>\n

NF: no. of fields<\/em><\/span><\/p>\n

NR: no. of records<\/em><\/span><\/p>\n

RS: record separator, default = newline<\/em><\/span><\/p>\n

ORS: output record separator, default = newline<\/em><\/span><\/p>\n

FNR: filename record<\/em><\/span><\/p>\n

-e program-text\u00a0<\/span>--source program-text<\/span>
\nUse program-text as AWK program source code. This option allows the easy inter<\/span>mixing of library functions (used via the -f and --file options) with source code\u00a0<\/span>entered on the command line.<\/span><\/p>\n

Pattern\/Action...<\/p>\n

\n

By default, awk runs a loop. It can have parameters set before and\/or after the loop body with BEGIN and END:<\/p>\n

\"awkloop.png\"<\/a><\/p>\n

\"Of these three parts, the main input loop or \"what happens during processing\" is where most of the work<\/em><\/span>
\ngets done. Inside the main input loop, your instructions are written as a series of pattern\/action<\/em><\/span>
\nprocedures. A pattern is a rule for testing the input line to determine whether or not the action should be<\/em><\/span>
\napplied to it. The actions, as we shall see, can be quite complex, consisting of statements, functions, and<\/em><\/span>
\nexpressions.<\/em><\/span>
\nThe main thing to remember is that each pattern\/action procedure sits in the main input loop, which<\/em><\/span>
\ntakes care of reading the input line. The procedures that you write will be applied to each input line, one<\/em><\/span>
\nline at a time.\"<\/em><\/span><\/p>\n

So, you need to KNOW re the awk loop above to complete the first line scan...if you want number of chars in a line e.g. by setting the FS (Field Separator) to null, when there are no white spaces between chars:<\/p>\n

cat hashes.txt<\/span><\/p>\n

The number of hashes won't be read correctly for the first line, only the second by default loop. This could be a prob in a program eh? A headscratcher debugging it!<\/p>\n

awk 'FS = \"\"; {print NF}' hashes.txt<\/span>
\n0<\/span>
\n60<\/span><\/p>\n

You get round this by using a BEGIN function before the main awk loop body starts to allow the first line to be read before the loop body:<\/p>\n

awk 'BEGIN {FS=\"\"} {print NF }' hashes.txt<\/span>
\n60<\/span>
\n60<\/span><\/p>\n

Awk seems powerful as it is compact, quite easy to read, and generally gets a lot done in a relatively short script, compared to other langs, e,g. C. For example, run Shott's loop here to see:<\/p>\n

ls -l \/ | awk '<\/span>
\n$1 ~ \/^-\/ {t[\"Regular Files\"]++}<\/span>
\n$1 ~ \/^d\/ {t[\"Directories\"]++}<\/span>
\n$1 ~ \/^l\/ {t[\"Symbolic Links\"]++}<\/span>
\nEND {for (i in t) print i \":\\t\" t[i]}<\/span>
\n'<\/span>
\nSymbolic Links: 2<\/span>
\nDirectories: 24<\/span>
\nRegular Files: 1<\/span><\/p>\n

Basic Behaviour<\/strong><\/p>\n

All\u00a0awk variables are initialized to zero so this script works immediately, as 0 is inc'd to 1 at start:<\/p>\n

BEGIN {<\/span>
\ndo {<\/span>
\n++x<\/span><\/p>\n

print x<\/span>
\n} while ( x <= 4 )<\/span>
\n}<\/span><\/p>\n

awk -f do.awk<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span><\/p>\n

awk -e 'BEGIN {do {++x; print x} while ( x <= 4 )}'<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span><\/p>\n

Count Blank Lines (bad idea to make 6 blanks and 6 text, as seen later!!)<\/p>\n

cat -n text<\/span>
\n1 <\/span>
\n2 not a blank line<\/span>
\n3 <\/span>
\n4 not a blank line<\/span>
\n5 <\/span>
\n6 not a blank line<\/span>
\n7 <\/span>
\n8 not a blank line<\/span>
\n9 <\/span>
\n10 not a blank line<\/span>
\n11 not a blank line<\/span>
\n12<\/span><\/p>\n

awk '{print}' text<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span>
\nnot a blank line<\/span><\/p>\n

awk '{ print NR }' text<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span>
\n6<\/span>
\n7<\/span>
\n8<\/span>
\n9<\/span>
\n10<\/span>
\n11<\/span>
\n12<\/span><\/p>\n

awk '{ print NR, $0 }' text<\/span>
\n1 <\/span>
\n2 not a blank line<\/span>
\n3 <\/span>
\n4 not a blank line<\/span>
\n5 <\/span>
\n6 not a blank line<\/span>
\n7 <\/span>
\n8 not a blank line<\/span>
\n9 <\/span>
\n10 not a blank line<\/span>
\n11 not a blank line<\/span>
\n12<\/span><\/p>\n

awk '{ print NF, $0 }' text<\/span>
\n0 <\/span>
\n4 not a blank line<\/span>
\n0 <\/span>
\n4 not a blank line<\/span>
\n0 <\/span>
\n4 not a blank line<\/span>
\n0 <\/span>
\n4 not a blank line<\/span>
\n0 <\/span>
\n4 not a blank line<\/span>
\n4 not a blank line<\/span>
\n0<\/span><\/p>\n

vi awk.scr<\/span><\/p>\n

# test for integer,<\/span>
\n#\/[0-9]+\/{ print \"That is an integer\" }<\/span>
\n#\/[A-Za-z]+\/ { print \"This is a string\" }<\/span>
\n\/^$\/ { print \"This is a blank line.\" }<\/strong><\/span>
\n#\/^$\/ {print x += 1}<\/span><\/p>\n

awk -f awk.scr text<\/span>
\nThis is a blank line.<\/span>
\nThis is a blank line.<\/span>
\nThis is a blank line.<\/span>
\nThis is a blank line.<\/span>
\nThis is a blank line.<\/span>
\nThis is a blank line.<\/span><\/p>\n

awk '\/^$\/ { print NR, \"This is a blank line.\" }' text<\/span>
\n1 This is a blank line.<\/span>
\n3 This is a blank line.<\/span>
\n5 This is a blank line.<\/span>
\n7 This is a blank line.<\/span>
\n9 This is a blank line.<\/span>
\n12 This is a blank line.<\/span><\/p>\n

This matches printable chars between start and end of line and counts each matching line processed<\/p>\n

awk '\/^$<\/strong>\/ {print x += 1}' text<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span>
\n6<\/span><\/p>\n

This matches each line start so counts all lines inc. blanks, same as NR.<\/p>\n

awk '\/^\/<\/strong> {print x += 1}' text;<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span>
\n6<\/span>
\n7<\/span>
\n8<\/span>
\n9<\/span>
\n10<\/span>
\n11<\/span>
\n12<\/span><\/p>\n

This counts NF, so states \u00a0which have no fields, so show blank lines = 0 fields<\/p>\n

awk ' {print NF} ' text;<\/span>
\n0<\/span>
\n4<\/span>
\n0<\/span>
\n4<\/span>
\n0<\/span>
\n4<\/span>
\n0<\/span>
\n4<\/span>
\n0<\/span>
\n4<\/span>
\n4<\/span>
\n0<\/span><\/p>\n

Or, by much opposite logic, non blank lines using Shotts POSIX (or \/[:alnum]\/ ;\u00a0\/[:ascii]\/ \/[:print]\/ \/[:space]\/ etc.)<\/p>\n

awk '\/[:alpha]\/ { print NR, \"This is a string\" }' text<\/span>
\n2 This is a string<\/span>
\n4 This is a string<\/span>
\n6 This is a string<\/span>
\n8 This is a string<\/span>
\n10 This is a string<\/span>
\n11 This is a string<\/span><\/p>\n

This can be confusing; you may think blank lines can have a newline at the start\/end (they do!), but \/^$\/ above, and [:ctrl] below doesn't show that, yet Libre shows they do - 12 in all, so you MUST put the ^$ in quotes to see them, as newlines or ctrl chars don't work fully to show all 12:<\/p>\n

\"newlines.png\"<\/a><\/p>\n

awk '\/'^$'\/ {print \" record found\" }' text \u00a0 \u00a0same as<\/span>\u00a0 \u00a0 \u00a0awk '\/^\/ {print \" record found\" }' text<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span>
\nrecord found<\/span><\/p>\n

awk '\/\/' text<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span><\/p>\n

not a blank line<\/span>
\nnot a blank line<\/span><\/p>\n

awk '\/ \/' text<\/span>
\nnot a blank line<\/span>
\nnot a blank line<\/span>
\nnot a blank line<\/span>
\nnot a blank line<\/span>
\nnot a blank line<\/span>
\nnot a blank line<\/span><\/p>\n

blank lines only:<\/strong><\/p>\n

awk '\/^$\/ {print NR \" record found\" }' text<\/span>
\n1 record found<\/span>
\n3 record found<\/span>
\n5 record found<\/span>
\n7 record found<\/span>
\n9 record found<\/span>
\n12 record found<\/span><\/p>\n

non blanks IDd by new line char :<\/p>\n

awk '\/'\\n'\/ { print NR \" This is a string\" }' text<\/span>
\n2 This is a string<\/span>
\n4 This is a string<\/span>
\n6 This is a string<\/span>
\n8 This is a string<\/span>
\n10 This is a string<\/span>
\n11 This is a string<\/span><\/p>\n

non blanks IDd by non printable Ctrl char\u00a0:<\/p>\n

awk '\/[:ctrl]\/ { print NR \" This is a string\" }' text<\/span>
\n2 This is a string<\/span>
\n4 This is a string<\/span>
\n6 This is a string<\/span>
\n8 This is a string<\/span>
\n10 This is a string<\/span>
\n11 This is a string<\/span><\/p>\n

via a regex:<\/p>\n

awk '\/[0-9a-zA-Z]\/ { print NR \" This is a string\" }' text<\/span>
\n2 This is a string<\/span>
\n4 This is a string<\/span>
\n6 This is a string<\/span>
\n8 This is a string<\/span>
\n10 This is a string<\/span>
\n11 This is a string<\/span><\/p>\n

blank line\u00a0removal (not necessarily the same as white space removal and you need to be careful about what else may be removed - like } in code!!):<\/p>\n

awk 'gsub(\" \",\"\")' < text<\/span>
\nnotablankline<\/span>
\nnotablankline<\/span>
\nnotablankline<\/span>
\nnotablankline<\/span>
\nnotablankline<\/span>
\nnotablankline<\/span><\/p>\n

Part of a CSS style file with blank lines and {} :<\/p>\n

cat style.css\u00a0<\/span><\/span><\/p>\n

text-indent: -9999px;<\/span><\/p>\n

background: url(images\/menu-indicator-right.png) no-repeat;<\/span><\/p>\n

\/*border: solid 1px red;*\/<\/span><\/strong><\/p>\n

}<\/span><\/strong><\/p>\n

Piping it thru gosub\u00a0as above removes blank lines but also the closing }\u00a0 from each section !!<\/p>\n

cat style.css | awk 'gsub(\" \",\"\")'<\/span><\/p>\n

.menu1ulul.sf-sub-indicator{<\/span>
\nposition:absolute;<\/span>
\nwidth:7px;<\/span>
\nheight:15px;<\/span>
\ntop:8px;<\/span>
\nright:15px;<\/span>
\ntext-indent:-9999px;<\/span>
\nbackground:url(images\/menu-indicator-right.png)no-repeat;<\/span><\/strong>
\n\/*border:solid1pxred;*\/<\/span><\/strong><\/p>\n

Can use :\u00a0cat style.css | sed '\/^[[:space:]]*$\/d' OR<\/span> \u00a0cat style.css | sed '\/^.$\/d'<\/span><\/p>\n

.menu1 ul ul .sf-sub-indicator {<\/span>
\nposition: absolute;<\/span>
\nwidth: 7px;<\/span>
\nheight: 15px;<\/span>
\ntop: 8px;<\/span>
\nright: 15px;<\/span>
\ntext-indent: -9999px;<\/span>
\nbackground: url(images\/menu-indicator-right.png) no-repeat;<\/span>
\n\/*border: solid 1px red;*\/<\/span>
\n}<\/span><\/p>\n

sed removes whitespace between words, but NOT blank lines with this:<\/p>\n

sed 's\/ \/\/g' < text<\/span><\/p>\n

notablankline<\/span><\/p>\n

notablankline<\/span><\/p>\n

notablankline<\/span><\/p>\n

notablankline<\/span><\/p>\n

notablankline<\/span>
\nnotablankline<\/span><\/p>\n

Negation - a real world problem. Dir full of .c files with their binaries that are not +x so cannot run due to a past chmod error. ID all those with a .c extension, then negate the term to show only none .c files:<\/strong><\/p>\n

ls<\/span><\/p>\n

floating sevendebug vars<\/span>
\nfloating.c sevendebug.c vars.c<\/span>
\nfull sidefx vowels<\/span>
\nfull1 sidefx.c vowels.c<\/span>
\nfull1.c sieve vsphere<\/span>
\nfull.c sieve1.py vsphere.c<\/span>
\ngraph sieve.c zero<\/span>
\ngraph.c Sieve.c zero.c<\/span>
\nguess SievefromWeb<\/span><\/p>\n

shows only the .c files:<\/p>\n

ls | awk '\/.c$\/'<\/span><\/p>\n

triang.c<\/span>
\ntri.c<\/span>
\ntrisub.c<\/span>
\ntwiceterm.c<\/span>
\ntwo.c<\/span>
\nvars.c<\/span>
\nvowels.c<\/span>
\nvsphere.c<\/span>
\nzero.c<\/span><\/p>\n

now negate it to show only the exe files that need chmod +x<\/p>\n

ls | awk '!\/.c$\/'<\/span><\/p>\n

triang2<\/span>
\ntrisub<\/span>
\ntwiceterm<\/span>
\ntwo<\/span>
\nUnitsConverter<\/span>
\nvars<\/span>
\nvowels<\/span>
\nvsphere<\/span>
\nzero<\/span><\/p>\n

Now they can all be +x:<\/p>\n

chmod +x `ls | awk '!\/.c$\/' `<\/span><\/p>\n

\"cexes.png\"<\/a><\/p>\n

\n

dstat -tc | awk '{if(NR>3) print $0 fflush()}' | tee dstatlive.txt (used with\u00a0<\/span><\/span>watch -n 1 gnuplot livedstat.gnu)<\/span><\/p>\n

sadf -p -P 0,1,2,3 | grep user | grep cpu0 | awk '{print $3,$4,$8}'<\/span><\/p>\n

cat data1.dat | awk '{print $3}' | sed ':a;N;$!ba;s\/\\n\/; \/g' \u00a0(creates 1 long line - comma delimits each line by removing \\n, except last line )<\/span><\/span><\/p>\n

sadf -p -P 0,1,2,3 | grep user | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu; xdg-open userCPU.png (see gnu live data Posts)<\/span><\/span><\/p>\n

sadf -p -P 0,1,2,3 | grep idle | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu; mv -v userCPU.png idleCPU.png ; xdg-open idleCPU.png\u00a0(see gnu live data Posts)<\/span><\/span><\/p>\n

awk 'gsub(\" \",\"\")' < file.txt (removes whitespace)<\/span><\/span><\/p>\n

cat 1_5.txt<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span><\/p>\n

AWK Subs<\/strong><\/p>\n

Sub a function for text:<\/p>\n

awk 'gsub(\"blank\",sqrt(2))' text<\/span>
\nnot a 1.41421 line<\/span>
\nnot a 1.41421 line<\/span>
\nnot a 1.41421 line<\/span>
\nnot a 1.41421 line<\/span>
\nnot a 1.41421 line<\/span>
\nnot a 1.41421 line<\/span><\/p>\n

Replace all numbers with a char:<\/p>\n

awk 'gsub(\"[0-9]\",\"=\")' 1_5.txt<\/span>
\n=<\/span>
\n=<\/span>
\n=<\/span>
\n=<\/span>
\n=<\/span><\/p>\n

cat ABABAB.txt<\/span>
\nABABAB<\/span><\/p>\n

awk -e '{if (gsub(\/A\/,\"B\")) {print}}' < ABABAB.txt<\/span>
\nBBBBBB<\/span><\/p>\n

awk -e '{if (gsub(\/B\/,\" \")) {print}}' < ABABAB.txt<\/span>
\nA A A<\/span><\/p>\n

awk -e '{if (gsub(\/B\/,\"\")) {print}}' < ABABAB.txt<\/span>
\nAAA<\/span><\/p>\n

Subs one char for the whole record\/string ...? Shows looping behaviour of awk. Useful? Maybe:<\/p>\n

awk -e '{if (gsub(\/B\/,tolower($0) )) {print}}' < ABABAB.txt<\/span>
\nAabababAabababAababab<\/span><\/p>\n

OR<\/p>\n

echo \"ABABAB\" | awk -e '{if (gsub(\/B\/,tolower($0) )) {print}}'<\/span>
\nAabababAabababAababab<\/span><\/p>\n

awk -e '{print tolower($0)}' < ABABAB.txt<\/span>
\nababab<\/span><\/p>\n

Again, note the loop effects and first line issue without a BEGIN:<\/strong><\/p>\n

awk -e '{print $0\"\\n\",tolower($0)}' < ABABAB.txt<\/span>
\nABABAB<\/span>
\n\u00a0 ababab<\/span><\/p>\n

awk -e '{print $0\"\\n\", tolower($0)}' < ABABAB.txt | awk -e '{print $0\"\\n\", toupper($0)}'<\/span><\/strong>
\nABABAB<\/span><\/strong>
\n\u00a0 \u00a0ABABAB<\/span><\/strong>
\n\u00a0 \u00a0ababab<\/strong><\/span>
\n\u00a0 \u00a0 \u00a0ABABAB<\/span><\/strong><\/p>\n

Nothing from the first awk gets printed in the 2nd version..like the loop does not complete for 1st awk ?<\/p>\n

cat ABABAB.txt | awk -e '{print tolower($0)}'<\/span>
\nababab<\/p>\n

cat ABABAB.txt | awk \u00a0'{print tolower($0)}' | awk \u00a0'{print toupper($0)}'<\/span>
\nABABAB<\/span><\/p>\n

awk -e '{if (gsub(\/B\/,\"\\v\")) {print}}' < ABABAB.txt<\/span>
\nA<\/span>
\n\u00a0 A<\/span>
\n\u00a0 \u00a0 A<\/span><\/p>\n

awk -e '{if (gsub(\/[A-Z]\/, \"1\")) {print}}' < ABABAB.txt<\/span>
\n111111<\/span><\/p>\n

\"Sed offers capabilities that seem a natural extension of interactive text editing. For instance, it offers a<\/em><\/span>
\nsearch-and-replace facility that can be applied globally to a single file or a group of files. While you<\/em><\/span>
\nwould not typically use sed to change a term that appears once in a particular file, you will find it very<\/em><\/span>
\nuseful to make a series of changes across a number of files. Think about making 20 different edits in<\/em><\/span>
\nover 100 files in a matter of minutes, and you get an idea of how powerful sed can be.\" <\/span>E.G. Subs \"In Place\" for a mysql DB to change you web name for home server name globally - BACK IT UP first!!<\/span>\u00a0<\/strong><\/em><\/span><\/p>\n

sed -i 's\/https:\/http:\/g' NEO.sql<\/p>\n

Or tr and perl are better\u00a0to use when learning concepts, as sed and awk don't work in the same simple search and replace format for \"\\n\" type chars in Mint.<\/p>\n

This seems to be what paste -s uses anyway, so you could also use 1 white space \" \" instead of a tab \\t, also:<\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | tr \"\\n\" \"\\t\"<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | perl -pe 's\/\\n\/\\t\/g'<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

sed and awk fail for this simple format requiring more cryptic parameters:<\/strong><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | sed 's\/\\n\/\\t\/g'<\/strong><\/span>
\n192.168.1.2<\/span>
\n192.168.1.7...<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | sed '{:q;N;s\/\\n\/ \/g;t q}'<\/strong><\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | awk 'gsub(\"\\n\",\"\\t\")'<\/span><\/p>\n

no OP<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | awk '{printf \"%s \",$0} END {print \"\"}'<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

OR<\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | awk 'BEGIN {RS=\"\"}{gsub(\/\\n\/,\" \",$0); print $0}'<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

Perl evaluation\/substitution of integers:<\/p>\n

perl -pe 's\/(\\d+)$\/\"=\"x$1\/e' 1_5.txt<\/span>
\n=<\/span>
\n==<\/span>
\n===<\/span>
\n====<\/span>
\n=====<\/span><\/p>\n

Oh look! It was right in front of me all the time above, adding print!! Dick...<\/p>\n

perl -pe 's\/(\\d+)$\/\"=\"x$1\/e; print \"$1 \"' 1_5.txt<\/span>
\n1 =<\/span>
\n2 ==<\/span>
\n3 ===<\/span>
\n4 ====<\/span>
\n5 =====<\/span><\/p>\n

Does it work for diff values on diff lines..? Yep!<\/p>\n

cat 6_1.txt<\/span>
\n6<\/span>
\n5<\/span>
\n4<\/span>
\n3<\/span>
\n2<\/span>
\n1<\/span><\/p>\n

perl -pe 's\/(\\d+)$\/\"=\"x$1\/e; print \"$1 \"' 6_1.txt<\/span>
\n6 ======<\/span>
\n5 =====<\/span>
\n4 ====<\/span>
\n3 ===<\/span>
\n2 ==<\/span>
\n1 =<\/span><\/p>\n

perl -pe 's\/(\\d+)$\/\"=\"x$1\/e; print \"$1 \"' random.txt<\/span>
\n90 ======<\/span>
\n87 ===<\/span>
\n74 ==================<\/span>
\n40 ============<\/span>
\n90 ======<\/span>
\n70 ==============<\/span>
\n46 ==================<\/span>
\n10 ==========<\/span>
\n35 =======<\/span>
\n08 ========<\/span>
\n1 =<\/span>
\n0<\/span><\/p>\n

Remember - load avg is divided by cores to get value per core (avg of 4 of a localhost core = 4\/4=100% utilization per core - PC blows up and you die...)<\/p>\n

uptime<\/span>
\n23:56:19 up 9:20, 5 users, load average: 0.88,<\/strong><\/span> 0.95, 0.81<\/p>\n

perl -e 'while(1) {`uptime` =~ \/average: ([\\d.]+)\/<\/strong>; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)); sleep 3 }'<\/span><\/p>\n

0.87 ##<\/span>
\n0.87 ##<\/span>
\n0.88 ##<\/span>
\n0.97 <\/span>
\n0.97 <\/span>
\n1.21 <\/span>
\n1.21 <\/span>
\n1.11 ##<\/span>
\n1.10 ##<\/span>
\n1.10 ##<\/span>
\n1.10 ##<\/span>
\n1.10 ##<\/span>
\n1.01 #<\/span>
\n0.93 <\/span>
\n0.93 <\/span>
\n0.85 ##<\/span>
\n0.85 ##<\/span><\/p>\n

For these types of single line histos, increase the multiplier for clearer proportions, accounting for decimal:<\/p>\n

perl -e 'while(1) {`uptime` =~ \/average: ([\\d.]++)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 100)); sleep 3 }'<\/span>
\n0.91 #<\/span>
\n0.92 ##<\/span>
\n0.92 ##<\/span>
\n0.84 <\/span>
\n0.84 <\/span>
\n0.78 <\/span>
\n0.71 ##<\/span><\/p>\n

Perl matches:<\/p>\n

\".\" =~ \/.\/ # Match<\/em><\/p>\n

\" \" =~ \/\\h\/ # Match, space is horizontal whitespace.<\/em><\/p>\n

\\d Match a decimal digit character.<\/strong><\/em>
\n\\D Match a non-decimal-digit character.<\/em>
\n\\w Match a \"word\" character.<\/em>
\n\\W Match a non-\"word\" character.<\/em>
\n\\s Match a whitespace character.<\/strong><\/em>
\n\\S Match a non-whitespace character.<\/em>
\n\\h Match a horizontal whitespace character.<\/em>
\n\\H Match a character that isn't horizontal whitespace.<\/em>
\n\\v Match a vertical whitespace character.<\/em>
\n\\V Match a character that isn't vertical whitespace.<\/em>
\n\\N Match a character that isn't a newline.<\/em>
\n\\pP, \\p{Prop} Match a character that has the given Unicode property.<\/em>
\n\\PP, \\P{Prop} Match a character that doesn't have the Unicode property<\/em><\/p>\n

Trying to work out this cryptic perl lang...from the above, which takes a number from a matching column, from ONE line only from each loop, and multiplies the digit value by 10, then prints that number of # chars...So...<\/p>\n

For a generic data file with 1 line only<\/strong>, without the while\/sleep to match digits, amend the \u00a0\"=~\u00a0\/avg: ([\\d.])\/\" ; <\/strong>to read the correct column digits. The\u00a0$1, \"#\" x ($1 * 10) \u00a0<\/strong>- reads\/prints column 1 (digit), then multiplies by 10, the \"#\" char, by the actual digit value, ignoring the decimal fractions:<\/p>\n

cat load.txt<\/span>
\nload average: 1.15<\/strong>, 0.92, 0.57<\/span><\/p>\n

perl -e '{`cat load.txt` =~ \/([\\d.]+)\/<\/strong>; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)) }'<\/span>
\n1.15<\/strong> ##<\/span><\/p>\n

e.g:<\/p>\n

perl -e '{`cat load.txt` =~ \/$x ([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)) }'<\/span>
\n1.15 ##<\/span><\/p>\n

perl -pe '{}' 1_5.txt<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n4<\/span>
\n5<\/span><\/p>\n

Can't get it to do more than one line of the 5...unlike sed\/awk, perl does not process multilines it seems...without while(<>)<\/p>\n

awk '{print}' 1_5.txt | perl -e '{\/^\/ =~ \/([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)) }'<\/span>
\n1 #<\/span><\/p>\n

Aah...awk histo for cmd history - nice....<\/p>\n

history | awk '{h[$2]++}END{for(i in h){print h[i],i|\"sort -rn|head -20\"}}' |awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span>
\nsudo 194
\nls 151 ##
\ncat 97
\nfortune 57
\nperl 47
\nawk 41 #
\nfind 38
\ndu 35 ##
\nvi 25 ##
\nrsync 23 ##
\ncd 23 ##
\nman 19
\ncrontab 17
\nrm 16 ##
\nprintf 16 ##
\nmplayer 14 ##
\ndf 13 ##
\nfor 11 #
\nalias 10 #
\nx11vnc 9<\/p>\n

awk '{print}' 1_5.txt | perl -e 'while(<>){ \/^\/=~\/(\\d)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10))}'<\/span>
\n1 #<\/span>
\n1 #<\/span>
\n1 #<\/span>
\n1 #<\/span>
\n1 #<\/span><\/p>\n

\n

grades.txt<\/p>\n

john 85 92 78 94 88
\nandrea 89 90 75 90 86
\njasper 84 88 80 92 84<\/p>\n

awk.scr<\/p>\n

# average five grades<\/span>
\n{ total = $2 + $3 + $4 + $5 + $6<\/span>
\navg = total \/ 5<\/span>
\nprint $1, avg }<\/span><\/p>\n

awk -f awkscr grades.txt<\/span>
\njohn 87.4
\nandrea 86
\njasper 85.6<\/p>\n

change:<\/p>\n

# average five grades
\n{print NR \".\", $1, avg }<\/span>
\nEND {<\/span>
\nprint \"\"<\/span>
\nprint NR, \"records processed.\" }<\/span><\/p>\n

gives:<\/p>\n

awk -f awkscr grades.txt<\/span>
\n1. john <\/span>
\n2. andrea <\/span>
\n3. jasper<\/span><\/p>\n

3 records processed.<\/span><\/p>\n

make OFS a tab:<\/p>\n

BEGIN { OFS = \"\\t\" }
\n{print NR \".\", $1, avg }
\nEND {
\nprint \"\"
\nprint NR, \"records processed.\" }<\/p>\n

awk -f awkscr grades.txt<\/span>
\n1. \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0john<\/span>
\n2.\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0andrea<\/span>
\n3. \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0jasper<\/span><\/p>\n

3 records processed.<\/span><\/p>\n

combine all grades file options:<\/p>\n

BEGIN { OFS = \"\\t\" }
\n{total = $2 + $3 + $4 + $5 + $6
\navg = total \/ 5
\nprint NR \".\", $1, avg }
\nEND {
\nprint \"\"
\nprint NR, \"records processed.\" }<\/p>\n

awk -f awkscr grades.txt<\/span><\/p>\n

1. \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0john 87.4<\/span>
\n2.\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0andrea 86<\/span>
\n3. \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0jasper 85.6<\/span><\/p>\n

3 records processed.<\/span><\/p>\n

For file presidents.txt of text:<\/p>\n

John Kennedy
\nLyndon B. Johnson
\nRichard Milhouse Nixon
\nGerald R. Ford
\nJimmy Carter
\nRonald Reagan
\nGeorge Bush
\nBill Clinton<\/p>\n

If above awkscr is run:<\/p>\n

awk -f awkscr presidents.txt<\/span>
\n1. John 0<\/span>
\n2. Lyndon 0<\/span>
\n3. Richard 0<\/span>
\n4. Gerald 0<\/span>
\n5. Jimmy 0<\/span>
\n6. Ronald 0<\/span>
\n7. George 0<\/span>
\n8. Bill 0<\/span><\/p>\n

8 records processed.<\/span><\/p>\n

Amending for removal of avgs and finding OFS\u00a0number of fields:<\/p>\n

BEGIN { OFS = \"\\t\"<\/strong> }<\/span>
\nEND {<\/span>
\nprint NF, \"fields present.\"<\/span>
\nprint NR, \"records processed.\" }<\/span><\/p>\n

awk -f awkscr presidents.txt
\n2 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0fields present.<\/span>
\n8 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0records processed.<\/span><\/p>\n

But are there only 2 real fields? No, as for the spaces between Lyndon B. Johnson, Ford and Milhouse - OFS has set the field, not the text content:<\/p>\n

awk ' {print NF, $0}' presidents.txt<\/p>\n

2 John Kennedy
\n3 Lyndon B. Johnson
\n3 Richard Milhouse Nixon
\n3 Gerald R. Ford
\n2 Jimmy Carter
\n2 Ronald Reagan
\n2 George Bush
\n2 Bill Clinton<\/span><\/p>\n

You can find the text's no of fields for each record (which tells of a middle name or letter), with the last name (from the last field) with a total of records processed with just:<\/p>\n

{print $NF
\nprint NF, \"fields present.\"
\nprint NR, \"records processed.\" }<\/p>\n

awk -f awkscr presidents.txt
\nKennedy<\/span>
\n2 fields present.<\/span>
\n1 records processed.<\/span>
\nJohnson<\/span><\/strong>
\n3 fields present.<\/span><\/strong>
\n2 records processed.<\/span><\/strong>
\nNixon<\/span><\/strong>
\n3 fields present.<\/span><\/strong>
\n3 records processed.<\/span><\/strong>
\nFord<\/span><\/strong>
\n3 fields present.<\/span><\/strong>
\n4 records processed.<\/span><\/strong>
\nCarter<\/span>
\n2 fields present.<\/span>
\n5 records processed.<\/span>
\nReagan<\/span>
\n2 fields present.<\/span>
\n6 records processed.<\/span>
\nBush<\/span>
\n2 fields present.<\/span>
\n7 records processed.<\/span>
\nClinton<\/span>
\n2 fields present.<\/span>
\n8 records processed.<\/span><\/p>\n

Or for explicit clarity:<\/p>\n

awk ' {print $0\" \" NF \" fields present\" ; print NR, \"records processed.\" }' presidents.txt<\/span>
\nJohn Kennedy 2 fields present<\/span>
\n1 records processed.<\/span>
\nLyndon B. Johnson 3 fields present<\/span>
\n2 records processed.<\/span>
\nRichard Milhouse Nixon 3 fields present<\/span>
\n3 records processed.<\/span>
\nGerald R. Ford 3 fields present<\/span>
\n4 records processed.<\/span>
\nJimmy Carter 2 fields present<\/span>
\n5 records processed.<\/span>
\nRonald Reagan 2 fields present<\/span>
\n6 records processed.<\/span>
\nGeorge Bush 2 fields present<\/span>
\n7 records processed.<\/span>
\nBill Clinton 2 fields present<\/span>
\n8 records processed.<\/span><\/p>\n

Instead of having all the information on one line, the person's\u00a0name is on one line, followed by the company's name on the next line and so on. Here's a sample record:
\nJohn Robinson
\nKoren Inc.
\n978 Commonwealth Ave.
\nBoston
\nMA 01760
\n696-0987
\nThis file has 6 records, or you can look at it as a single user record that has six fields where a blank line separates each field.\u00a0To process this data, we can specify a multi line record by defining the field separator to be a newline,\u00a0represented as \"\\n\", and set the record separator to the empty string (RS = \"\") which stands for a blank line. print gives the 1st and last records ($NF=$6 here).
\nBEGIN { FS = \"\\n\"; RS = \"\" }
\nWe can print the first and last fields using the following script:
\n# block.awk - print first and last fields
\n# $1 = name; $NF = phone number<\/p>\n

BEGIN { FS = \"\\n\"; RS = \"\" }<\/span>
\n{ print $1, $NF }<\/span><\/p>\n

vi multiline_record.txt<\/span><\/p>\n

John Robinson
\nKoren Inc.
\n978 Commonwealth Ave.
\nBoston
\nMA 01760
\n696-0987<\/p>\n

vi block.awk<\/span><\/p>\n

# block.awk - print first and last fields
\n# $1 = name; $NF = phone number<\/p>\n

BEGIN { FS = \"\\n\"; RS = \"\" }
\n{ print $1, $NF }<\/p>\n

awk -f block.awk multiline_record.txt \u00a0 \u00a0<\/span><\/p>\n

BEGIN { FS = \"\\n\"; RS = \"\" }
\n{ print $1, $NF }<\/span><\/p>\n

John Robinson 696-0987<\/span><\/p>\n

awk -e '{ FS = \"\\n\"; ORS = \" \"; print }' multiline_record.txt<\/span>
\nJohn Robinson Koren Inc. 978 Commonwealth Ave. Boston MA 01760 696-0987 stevee@AMDA8 ~\/Awk $<\/span><\/p>\n

\n

A key ability of awk is to set field separators of your choice, which then determine how many fields the file is read as. If a file (names.txt) contains:<\/p>\n

John Rink,Boob Inc,2 Tities street,Boston,MA 666-555-1111<\/span>
\nJohn Robinson,Koren Inc.,978 4th Ave.,Boston,MA 01760,696-0987<\/span>
\nPhyllis Chapman,GVE Corp.,34 Sea Drive,Amesbury,MA 01881,879-0900<\/span>
\nPhyllis Man,GVE Corp.,34 Sea Drive,Amesbury,MA 707-724-0000<\/span>
\nPhil Thap,GVE Corp.,34 Sea Drive,Amesbury,MA (707) 724-0000<\/span>
\n(Phyllis Chap,GVE Corp.,34 Sea Drive,Amesbury,MA (707)724-0000<\/span>
\nPhyllis Apeman,GVE Corp.,34 Sea Drive,Amesbury,MA 1-707-724-0000<\/span>
\nPhyllis Chipmunk,GVE Corp.,34 Sea Drive,Amesbury,MA1 707-724-0000<\/span>
\nPhyllis Shaman,GVE Corp.,34 Sea Drive,Amesbury,MA 1(707)724-0000<\/span><\/p>\n

You could choose to separate on say, commas, TABS (\\t), single letters, any number range [0-9] etc.<\/p>\n

This decides which column number would need stating to print out by amending the script column $x, say for script\u00a0phonelist.awk, with prior commented out examples, but now separates by a single space \" \" so \"creates\" up to 7 columns, depending on record (line) content that print $6 will show from the \"names\" data file:<\/p>\n

vi phonelist.awk<\/span><\/p>\n

# phonelist.awk -- print name and phone number.
\n# input file -- name, company, street, city, state and zip,phone
\n#BEGIN { FS = \",\" }
\n# comma-delimited fields
\n#{ print $1 \", \" $6 }
\n#$5 !~ \/MA\/ { print $1 \", \" $6 }
\n#$6 !~ \/1?(-| )?\\(?[0-9]+\\)?( |-)?[0-9]+-[0-9]+\/<\/p>\n

BEGIN { FS = \" \" } # delimited fields<\/span>
\n{ print $1,\": \" $NF }<\/span>
\nEND {<\/span>
\nprint \"\"<\/span>
\nprint NR, \"records processed. \\n\" }<\/span><\/p>\n

using phonelist.awk, becomes:<\/p>\n

awk -f phonelist.awk names.txt<\/span>
\nJohn : 666-555-1111<\/span>
\nJohn : 01760,696-0987<\/span>
\nPhyllis : 879-0900<\/span>
\nPhyllis : 707-724-0000<\/span>
\nPhil : 724-0000<\/span>
\n(Phyllis : (707)724-0000<\/span>
\nPhyllis : 1-707-724-0000<\/span>
\nPhyllis : 1707-724-0000<\/span>
\nPhyllis : 1(707)724-0000<\/span><\/p>\n

9 records processed.<\/span><\/p>\n

PRINT FIRST\/LAST RECORD (line) ONLY:<\/p>\n

efficient - doesn't read whole file:<\/p>\n

awk 'NR==1 {print; exit}' presidents.txt<\/span><\/p>\n

John Kennedy<\/p>\n

awk '{if(NR<=1) print}' presidents.txt<\/span>
\nJohn Kennedy<\/p>\n

awk '{if(NR<2) print}' presidents.txt<\/span>
\nJohn Kennedy<\/p>\n

awk '{if(NR>7) print}' presidents.txt<\/span>
\nBill Clinton<\/p>\n

awk '{if(NR>=8) print}' presidents.txt<\/span>
\nBill Clinton<\/p>\n

awk 'END{print}' presidents.txt<\/span>
\nBill Clinton<\/p>\n

awk -f phonelist.awk names.txt<\/span>
\nJohn, 666-555-1111
\nJohn, 01760,696-0987
\nPhyllis, 01881,879-0900
\nPhyllis, 707-724-0000
\nPhil, (707)
\n(Phyllis, (707)724-0000
\nPhyllis, 1-707-724-0000
\nPhyllis, 707-724-0000
\nPhyllis, 1(707)724-0000<\/p>\n

9 records processed.<\/p>\n

It you chose a TAB (\\t) then only 2 columns can be created for output as the phone numbers are tab delimited in this file:<\/p>\n

BEGIN { FS = \"\\t\"<\/strong> } # comma-delimited fields<\/span>
\n{ print $2<\/strong> }<\/span>
\nEND {<\/span>
\nprint \"\"<\/span>
\nprint NR, \"records processed.\" }<\/span><\/p>\n

awk -f phonelist.awk names<\/span>
\n666-555-1111
\n01760,696-0987
\n01881,879-0900
\n707-724-0000
\n(707) 724-0000
\n(707)724-0000
\n1-707-724-0000
\n707-724-0000
\n1(707)724-0000<\/p>\n

9 records processed.<\/p>\n

For the example of names.txt, which has FS = \",\" a comma, find how many fields each record has (count the commas and add 1 eg 5 commas = 6 columns ) :<\/p>\n

cat names.txt<\/span>
\nJohn Rink,Boob Inc,2 Tities street,Boston,MA,666-555-1111
\nJohn Robinson,Koren Inc.,978 4th Ave.,Boston,MA,01760,696-0987
\nPhyllis Chapman,GVE Corp.,34 Sea Drive,Amesbury,MA,01881,879-0900
\nPhyllis Man,GVE Corp.,34 Sea Drive,Amesbury,MA,707-724-0000
\nPhil Thap,GVE Corp.,34 Sea Drive,Amesbury,MA,(707) 724-0000
\n(Phyllis Chap,GVE Corp.,34 Sea Drive,Amesbury,MA,(707)724-0000
\nPhyllis Apeman,GVE Corp.,34 Sea Drive,Amesbury,MA,1-707-724-0000
\nPhyllis Chipmunk,GVE Corp.,34 Sea Drive,Amesbury,MA,1707-724-0000
\nPhyllis Shaman,GVE Corp.,34 Sea Drive,Amesbury,MA,1(707)724-0000<\/p>\n

Combining presidents example from above and showing no. of\u00a0space separated fields (space is default):<\/p>\n

awk ' {print $0\"\\n\" NF \" fields present\" ; print NR, \"records processed.\" }' names.txt<\/span>
\nJohn Rink,Boob Inc,2 Tities street,Boston,MA,666-555-1111
\n5 fields present<\/strong>
\n1 records processed.
\nJohn Robinson,Koren Inc.,978 4th Ave.,Boston,MA,01760,696-0987
\n5 fields present<\/strong>
\n2 records processed.
\nPhyllis Chapman,GVE Corp.,34 Sea Drive,Amesbury,MA,01881,879-0900
\n5 fields present
\n3 records processed.
\nPhyllis Man,GVE Corp.,34 Sea Drive,Amesbury,MA,707-724-0000
\n5 fields present
\n4 records processed.
\nPhil Thap,GVE Corp.,34 Sea Drive,Amesbury,MA,(707) 724-0000
\n6 fields present<\/strong>
\n5 records processed.
\n(Phyllis Chap,GVE Corp.,34 Sea Drive,Amesbury,MA,(707)724-0000
\n5 fields present
\n6 records processed.
\nPhyllis Apeman,GVE Corp.,34 Sea Drive,Amesbury,MA,1-707-724-0000
\n5 fields present
\n7 records processed.
\nPhyllis Chipmunk,GVE Corp.,34 Sea Drive,Amesbury,MA,1707-724-0000
\n5 fields present
\n8 records processed.
\nPhyllis Shaman,GVE Corp.,34 Sea Drive,Amesbury,MA,1(707)724-0000
\n5 fields present
\n9 records processed.<\/p>\n

Made clearer if I\u00a0replace commas with comma and space for first record and try again:<\/p>\n

John Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111<\/p>\n

awk ' {print $0\"\\n\" NF \" fields present\" ; print NR, \"records processed.\" }' names.txt | head -3<\/span>
\nJohn Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111<\/span>
\n10 fields present<\/span><\/strong>
\n1 records processed.<\/span><\/p>\n

Lesson there is keep clear records with a \", \" or \": \" or similar for ease of NR. Names will be auto split to first and last name fields by a space anyway too.\u00a0For CSV files, need to set FS or OFS =\",\"<\/p>\n

awk ' {FS=\",\" ;<\/strong> print $0\"\\n\" NF \" fields present\" ; print NR, \"records processed.\" }' names.txt<\/span>
\nJohn Robinson,Koren Inc.,978 4th Ave.,Boston,MA,01760,696-0987
\n7 fields present<\/strong><\/p>\n

Add lines from file1 to file 2 example.<\/p>\n

- remember - need to feed to > newfile.txt for a new copy, and remove the n from cat to remove the numbers:<\/p>\n

awk 'NR%2 == 1<\/strong> {getline f2 < \"presidents.txt\"; print f2} 1' names.txt | cat -n<\/span><\/p>\n

1 John Kennedy<\/span>
\n2 John Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111<\/span>
\n3 John Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987<\/span>
\n4 Phyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900<\/span>
\n5 Phyllis Man, GVE Corp., 34 Sea Drive, Amesbury, MA, 707-724-0000<\/span>
\n6 Phil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000<\/span>
\n7 (Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000<\/span>
\n8 Phyllis Apeman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1-707-724-0000<\/span>
\n9 Phyllis Chipmunk, GVE Corp., 34 Sea Drive, Amesbury, MA, 1707-724-0000<\/span>
\n10 Lyndon B. Johnson<\/span>
\n11 Phyllis Shaman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1(707)724-0000<\/span><\/p>\n

You have to supply the minimum NR = 2, so line record 1 gets read. This is proved by setting it to 1, as it does not complete line 1 read, so feeds nothing to file 2:<\/p>\n

awk 'NR%1\u00a0== 1<\/strong> {getline f2 < \"presidents.txt\"; print f2} 1' names.txt | cat -n<\/span>
\n1 John Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111<\/span>
\n2 John Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987....<\/span><\/p>\n

~\u00a0Matches - <\/strong>search for a string in a specific field and if it matches, print xxx..<\/p>\n

awk ' { print $0\"\\n\" NF \" fields present\" }' names.txt<\/span>
\nJohn Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111
\n10 fields present
\nJohn Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987
\n10 fields present
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\n11 fields present
\nPhyllis Man, GVE Corp., 34 Sea Drive, Amesbury, MA, 707-724-0000
\n10 fields present
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000
\n11 fields present
\n(Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000
\n10 fields present
\nPhyllis Apeman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1-707-724-0000
\n10 fields present
\nPhyllis Chipmunk, GVE Corp., 34 Sea Drive, Amesbury, MA, 1707-724-0000
\n10 fields present
\nPhyllis Shaman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1(707)724-0000
\n10 fields present<\/p>\n

awk '$4 ~\/Inc\/ {print $1,$2,$4 \" \" $10}' names.txt<\/span>
\nJohn Rink, Inc, 666-555-1111<\/span>
\nJohn Robinson, Inc., 01760,696-0987<\/span><\/p>\n

grep Inc names.txt<\/span>
\nJohn Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111<\/span>
\nJohn Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987<\/span><\/p>\n

\n

The cheque book deduction example - from above, one liners start getting too long, so you need to program with scripts, and comment addition is compulsory as for any script. It does a lot with just 2 functions:<\/p>\n

cat cheques.awk<\/span>
\n# checkbook.awk
\nBEGIN { FS = \"\\t\" }
\n#1 Expect the first record to have the starting balance.
\nNR == 1 { print \"Beginning Balance: \\t\" $1
\nbalance = $1
\nnext
\n# get next record and start over
\n}
\n#2 Apply to each check record, subtracting amount from balance
\n{
\nprint $1, $2, $3
\nprint balance -= $3
\n}<\/p>\n

cat spent.txt<\/span>
\n1000
\n125 Market 125.45
\n126 Hardware Store 34.95
\n127 Video Store 7.45
\n128 Book Store 14.32
\n129 Gasoline 16.10<\/p>\n

This deducts the spends from a 1000 balance:<\/p>\n

awk -f cheques.awk spent.txt<\/span>
\nBeginning Balance: 1000<\/span>
\n125 Market 125.45<\/span>
\n874.55<\/span>
\n126 Hardware Store 34.95<\/span>
\n839.6<\/span>
\n127 Video Store 7.45<\/span>
\n832.15<\/span>
\n128 Book Store 14.32<\/span>
\n817.83<\/span>
\n129 Gasoline 16.10<\/span>
\n801.73<\/span><\/p>\n

This can be tidied by making the current balance clearer in line with the start balance:<\/p>\n

# checkbook.awk<\/span>
\nBEGIN { FS = \"\\t\" }<\/span>
\n#1 Expect the first record to have the starting balance.<\/span>
\nNR == 1 { print \"Beginning Balance: \\t\" $1<\/span>
\nbalance = $1<\/span>
\nnext<\/span>
\n# get next record and start over<\/span>
\n}<\/span>
\n#2 Apply to each check record, subtracting amount from balance<\/span>
\n{<\/span>
\nprint $1, $2, $3<\/span>
\nprint \"Current Balance: \\t\", balance -= $3<\/strong><\/span>
\n}<\/span><\/p>\n

awk -f cheques.awk spent.txt<\/span>
\nBeginning Balance: 1000<\/span>
\n125 Market 125.45<\/span>
\nCurrent Balance: 874.55<\/span>
\n126 Hardware Store 34.95<\/span>
\nCurrent Balance: 839.6<\/span>
\n127 Video Store 7.45<\/span>
\nCurrent Balance: 832.15<\/span>
\n128 Book Store 14.32<\/span>
\nCurrent Balance: 817.83<\/span>
\n129 Gasoline 16.10<\/span>
\nCurrent Balance: 801.73<\/span><\/p>\n

awk '{if (NR>0) print}' names.txt<\/span>
\nJohn Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111
\nJohn Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\nPhyllis Man, GVE Corp., 34 Sea Drive, Amesbury, MA, 707-724-0000
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000
\n(Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000
\nPhyllis Apeman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1-707-724-0000
\nPhyllis Chipmunk, GVE Corp., 34 Sea Drive, Amesbury, MA, 1707-724-0000
\nPhyllis Shaman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1(707)724-0000<\/p>\n

You can print last record only now you know how many there are (9):<\/p>\n

awk '{if (NR>8) print}' names.txt<\/span>
\nPhyllis Shaman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1(707)724-0000<\/p>\n

Operator Descriptions<\/strong>
\n||\u00a0Logical OR
\n&&\u00a0Logical AND
\n!\u00a0Logical NOT<\/p>\n

awk '{if (NF == 10 && NR > 0) print}' names.txt<\/span>
\nJohn Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111
\nJohn Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987
\nPhyllis Man, GVE Corp., 34 Sea Drive, Amesbury, MA, 707-724-0000
\n(Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000
\nPhyllis Apeman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1-707-724-0000
\nPhyllis Chipmunk, GVE Corp., 34 Sea Drive, Amesbury, MA, 1707-724-0000
\nPhyllis Shaman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1(707)724-0000<\/p>\n

awk '{if (NF == 11 && NR > 0) print}' names.txt<\/span>
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000<\/p>\n

Precedence and ~ Matches<\/strong><\/p>\n

The names.txt file records have 10 or 11 fields:<\/p>\n

awk '{print NF}' names.txt<\/span>
\n10
\n10
\n11
\n10
\n11
\n10
\n10
\n10
\n10<\/p>\n

The first && option will find records with 11 or more fields - which there are, so TRUE - so prints anyway:<\/p>\n

awk '{if (NR > 0 && NF >= 11) print}' names.txt<\/span>
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000<\/p>\n

The next option is as above, because OR is false - no records start with an X:<\/p>\n

awk '{if ((NR > 0 && NF >= 11) || $1 ~ \/X\/) print}' names.txt<\/span>
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000<\/p>\n

BUT...if both AND and OR conditions are TRUE,\u00a0then those files that fit either condition are printed - the file starting with a \"(<\/span>\" also, which doesn't begin with a capital and has only 10 fields. The first 2 lines have 11 fields so are also TRUE.<\/p>\n

awk '{if ((NR > 0 && NF >= 11) || $1 ~ \/\\(\/) print}' names.txt<\/span>
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000
\n(<\/span>Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000<\/p>\n

For the OR operator to be precedent, either one or both of the && conditions must be false, and the OR true. The first letter of a record starts P (TRUE) && the 2nd letter starts with Q (FALSE) so first condition is FALSE. The 2nd condition (OR) is TRUE, because it's the only file that starts with the \"(\"<\/p>\n

awk '{if ($1 ~ \/P\/ && $2 ~ \/Q\/ || $1 ~ \/\\(\/) print }' names.txt<\/span>
\n(Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000<\/p>\n

If neither side are TRUE, nothing is printed:<\/p>\n

awk '{if ($1 ~ \/Z\/ && $2 ~ \/Z\/ || $1 ~ \/Z\/) print }' names.txt<\/span><\/p>\n

The logic is negated by the \"!\" so all records are printed:<\/p>\n

awk '{if ($1 !~ \/Z\/ && $2 !~ \/Z\/ || $1 !~ \/Z\/) print }' names.txt<\/span>
\nJohn Rink, Boob Inc, 2 Tities street, Boston, MA, 666-555-1111
\nJohn Robinson, Koren Inc., 978, 4th Ave., Boston, MA, 01760,696-0987
\nPhyllis Chapman, GVE Corp., 34 Sea Drive, Amesbury, MA, 01881 879-0900
\nPhyllis Man, GVE Corp., 34 Sea Drive, Amesbury, MA, 707-724-0000
\nPhil Thap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707) 724-0000
\n(Phyllis Chap, GVE Corp., 34 Sea Drive, Amesbury, MA, (707)724-0000
\nPhyllis Apeman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1-707-724-0000
\nPhyllis Chipmunk, GVE Corp., 34 Sea Drive, Amesbury, MA, 1707-724-0000
\nPhyllis Shaman, GVE Corp., 34 Sea Drive, Amesbury, MA, 1(707)724-0000<\/p>\n

ls -l<\/span>
\n-rw-r--r-- 1 stevee stevee 512 Aug 24 06:11 awk.scr
\n-rw-r--r-- 1 stevee stevee 157 Aug 23 19:28 block.awk
\n-rw-r--r-- 1 stevee stevee 240 Aug 21 20:29 blocklist.awk
\n-rw-r--r-- 1 stevee stevee 293 Aug 23 19:28 cheques.awk
\n-rw-r--r-- 1 stevee stevee 64 Aug 22 20:52 grades.txt
\n-rw-r--r-- 1 stevee stevee 72 Aug 23 17:28 multiline_record.txt
\n-rw-r--r-- 1 stevee stevee 612 Aug 24 08:17 names.txt
\n-rw-r--r-- 1 stevee stevee 368 Aug 22 23:36 phonelist.awk
\n-rw-r--r-- 1 stevee stevee 121 Aug 23 16:40 presidents.txt
\n-rw-r--r-- 1 stevee stevee 109 Aug 23 19:26 spent.txt
\n-rw-r--r-- 1 stevee stevee 144 Aug 24 07:24 text<\/p>\n

Creating running total of file bytes from ls cmd using script \"fls.awk\", making it exe and starting with;<\/strong><\/p>\n

ls -l $* | awk '{print $5, \"\\t\", $9}'<\/span><\/p>\n

512 awk.scr
\n157 block.awk
\n240 blocklist.awk
\n293 cheques.awk
\n64 grades.txt
\n72 multiline_record.txt
\n612 names.txt
\n368 phonelist.awk
\n121 presidents.txt
\n109 spent.txt
\n144 text<\/p>\n

.\/fls.awk<\/span>
\n512 awk.scr
\n157 block.awk
\n240 blocklist.awk
\n293 cheques.awk
\n38 fls.awk
\n64 grades.txt
\n72 multiline_record.txt
\n612 names.txt
\n368 phonelist.awk
\n121 presidents.txt
\n109 spent.txt
\n144 text<\/p>\n

.\/fls.awk *txt<\/span>
\n64 grades.txt
\n72 multiline_record.txt
\n612 names.txt
\n121 presidents.txt
\n109 spent.txt<\/p>\n

Adding to script fls.awk to become:<\/p>\n

ls -l $* | awk '{print $5, \"\\t\", $9}'<\/span>
\n{<\/span>
\nsum += $5<\/span>
\n++filenum<\/span>
\nprint $5, \"\\t\", $9<\/span>
\n}<\/span>
\nEND { print \"Total: \", sum, \"bytes (\" filenum \" files)\" }<\/span><\/p>\n

This is an example of the bad proof reading - you work it out for yourself - I got it to work as:<\/p>\n

where fls.awk is:
\nBEGIN { print \"BYTES\", \"\\t\", \"FILE\" }<\/span>
\n{print $5, \"\\t\", $9<\/span>
\nsum += $5<\/span>
\n+filenum}<\/span>
\nEND { print \"Total: \", sum, \"bytes (\" filenum NR \" files)\" }<\/span><\/p>\n

ls -l | awk -f fls.awk<\/span>
\nBYTES FILE<\/p>\n

512 awk.scr<\/span>
\n157 block.awk<\/span>
\n240 blocklist.awk<\/span>
\n293 cheques.awk<\/span>
\n154 fls.awk<\/span>
\n64 grades.txt<\/span>
\n72 multiline_record.txt<\/span>
\n612 names.txt<\/span>
\n368 phonelist.awk<\/span>
\n121 presidents.txt<\/span>
\n109 spent.txt<\/span>
\n144 text<\/span>
\nTotal: 2846 bytes (13 files)<\/span><\/p>\n

Checked with calc and it correct. Needs formatting better - Total is wrong place etc.:<\/p>\n

BEGIN {print \"BYTES\", \"\\t\", \"FILE\" }<\/span>
\n{print $5, \"\\t\", $9<\/span>
\nsum += $5<\/span>
\n+filenum}<\/span>
\nEND {print sum,\":Bytes (\"filenum NR,\" files)\" }<\/span><\/p>\n

now put all this in another exe..? The awk scr has to always be in the same dir as the sh script..hacky..! Needs to be written as a full one-liner in a sh script sometime...but works for now.<\/p>\n

vi filesum.sh<\/span><\/p>\n

ls -l | awk -f fls.awk<\/span><\/p>\n

chmod u+x\u00a0filesum.sh<\/span><\/p>\n

.\/filesum.sh *<\/span>
\nBYTES FILE<\/span><\/p>\n

512 awk.scr<\/span>
\n157 block.awk<\/span>
\n240 blocklist.awk<\/span>
\n293 cheques.awk<\/span>
\n23 filesum.sh<\/span>
\n140 fls.awk<\/span>
\n64 grades.txt<\/span>
\n72 multiline_record.txt<\/span>
\n612 names.txt<\/span>
\n368 phonelist.awk<\/span>
\n121 presidents.txt<\/span>
\n109 spent.txt<\/span>
\n144 text<\/span>
\n2855 :Bytes (14 files)<\/span><\/p>\n

It even correlates! with<\/p>\n

du -bc *<\/span>
\n512 awk.scr<\/span>
\n157 block.awk<\/span>
\n240 blocklist.awk<\/span>
\n293 cheques.awk<\/span>
\n23 filesum.sh<\/span>
\n140 fls.awk<\/span>
\n64 grades.txt<\/span>
\n72 multiline_record.txt<\/span>
\n612 names.txt<\/span>
\n368 phonelist.awk<\/span>
\n121 presidents.txt<\/span>
\n109 spent.txt<\/span>
\n144 text<\/span>
\n2855 total<\/span><\/p>\n

Formatting print - kernel very diff from PDF output - experiment..<\/p>\n

printf '|\\n'<\/span>
\n|<\/span><\/p>\n

printf '(\"|\")\\n'<\/span>
\n(\"|\")<\/span><\/p>\n

awk sh as a grep search of $1:<\/strong><\/p>\n

vi acronyms.txt<\/span><\/p>\n

BASIC<\/strong> Beginner's All-Purpose Symbolic Instruction Code
\nCICS<\/strong> Customer Information Control System
\nCOBOL<\/strong> Common Business Oriented Language
\nDBMS<\/strong> Data Base Management System
\nGIGO<\/strong> Garbage In, Garbage Out
\nGIRL<\/strong> Generalized Information Retrieval Language
\nAWK<\/strong> Aho Weinberger Kernighan
\nPERL<\/strong> Practical Extraction and Reporting Language
\nFORTRAN\u00a0<\/strong>Formula Translation<\/p>\n

vi search.sh<\/span><\/p>\n

#! \/bin\/sh
\n# assign shell's $1 to awk search variable
\nawk '$1 ~ search' search=$1 acronyms.txt<\/strong><\/p>\n

.\/search.sh GI\u00a0<\/span>
\nGI<\/strong>GO Garbage In, Garbage Out<\/span>
\nGI<\/strong>RL Generalized Information Retrieval Language<\/span><\/p>\n

.\/search.sh a<\/span><\/p>\n

.\/search.sh A<\/span>
\nBASIC Beginner's All-Purpose Symbolic Instruction Code <\/span>
\nAWK Aho Weinberger Kernighan<\/span>
\nFORTRAN Formula Translation<\/span><\/p>\n

.\/search.sh AW<\/span>
\nAWK Aho Weinberger Kernighan<\/span><\/p>\n

for loop to read\u00a0word of each field at a time:<\/strong><\/p>\n

vi awkfor.awk<\/span><\/p>\n

{for ( i = 1; i <= NF; i++ )<\/span>
\nprint $i}<\/span><\/p>\n

awk -e '{for ( i = 1; i <= NF; i++ ) print $i }' acronyms.txt<\/span><\/p>\n

BASIC<\/span>
\nBeginner's<\/span>
\nAll-Purpose<\/span>
\nSymbolic<\/span>
\nInstruction<\/span>
\nCode<\/span>
\nCICS<\/span>
\nCustomer<\/span>
\nInformation<\/span>
\nControl<\/span>
\nSystem<\/span>
\nCOBOL<\/span>
\nCommon<\/span>
\nBusiness<\/span>
\nOriented<\/span>
\nLanguage<\/span>
\nDBMS<\/span>
\nData<\/span>
\nBase<\/span>
\nManagement<\/span>
\nSystem<\/span>
\nGIGO<\/span>
\nGarbage<\/span>
\nIn,<\/span>
\nGarbage<\/span>
\nOut<\/span>
\nGIRL<\/span>
\nGeneralized<\/span>
\nInformation<\/span>
\nRetrieval<\/span>
\nLanguage<\/span><\/p>\n

This can be used as a basis for statistics word count example:<\/p>\n

awk -e '{for ( i = 1; i <= NF; i++ ) print $i }' acronyms.txt | sort | uniq -c<\/span>
\n1 All-Purpose<\/span>
\n1 Base<\/span>
\n1 BASIC<\/span>
\n1 Beginner's<\/span>
\n1 Business<\/span>
\n1 CICS<\/span>
\n1 COBOL<\/span>
\n1 Code<\/span>
\n1 Common<\/span>
\n1 Control<\/span>
\n1 Customer<\/span>
\n1 Data<\/span>
\n1 DBMS<\/span>
\n2 Garbage<\/span><\/strong>
\n1 Generalized<\/span>
\n1 GIGO<\/span>
\n1 GIRL<\/span>
\n1 In,<\/span>
\n2 Information<\/span><\/strong>
\n1 Instruction<\/span>
\n2 Language<\/span><\/strong>
\n1 Management<\/span>
\n1 Oriented<\/span>
\n1 Out<\/span>
\n1 Retrieval<\/span>
\n1 Symbolic<\/span>
\n2 System<\/span><\/strong><\/p>\n

Now count word distribution of some random text:<\/p>\n

fortune | awk -e '{for ( i = 1; i <= NF; i++ ) print $i }' | sort | uniq -c<\/span><\/p>\n

1 A:<\/span>
\n2 an<\/span>
\n1 and<\/span>
\n1 between<\/span>
\n1 difference<\/span>
\n1 drunk.<\/span>
\n2 Irish<\/span>
\n1 less<\/span>
\n1 One<\/span>
\n1 Q:<\/span>
\n1 the<\/span>
\n1 wake?<\/span>
\n1 wedding<\/span>
\n1 What's<\/span><\/p>\n

Or letter\/char distribution in the sys dictionaries:<\/p>\n

cat \/usr\/share\/dict\/british-english | awk -e '{FS=\"\"\u00a0<\/strong>; for ( i = 1; i <= NF<\/strong>; i++ ) print $i }' | sort | uniq -c<\/span>
\n26262 '<\/strong><\/span>
\n63180 a<\/span>
\n1288 A<\/span>
\n10 \u00c3\u00a1<\/span>
\n6 \u00c3:<\/span>
\n3 \u00c3\u00a5<\/span>
\n1 \u00c3\u2026<\/span>
\n4 \u00c3\u00a4<\/span>
\n14293 b<\/span>
\n1247 B<\/span>
\n30435 c<\/span>
\n1419 C<\/span>
\n5 \u00c3\u00a7<\/span>
\n27771 d<\/span>
\n734 D<\/span>
\n88096 e<\/strong><\/span><\/p>\n

596 E....<\/strong><\/span><\/p>\n

89882 s<\/span><\/p>\n

That stat is misleading for letter s because this dic lists plurals\/ownership :\u00a01 youngsters,1 youngster's. The word count total is\u00a099156 so many extra s.<\/p>\n

You would need the total letter sample in the dictionary for stats.<\/p>\n

awk -e '{for ( i = 1; i <= NF; i++ ) print $i }' < alphabet.txt | sort | uniq -c<\/span>
\n1 a<\/span>
\n1 b<\/span>
\n1 c<\/span>
\n1 d<\/span>
\n1 e<\/span>
\n1 f<\/span>
\n1 g<\/span>
\n1 h<\/span>
\n1 i<\/span>
\n1 j<\/span>
\n1 k<\/span>
\n1 l<\/span>
\n1 m<\/span>
\n1 n<\/span>
\n1 o<\/span>
\n1 p<\/span>
\n1 q<\/span>
\n1 r<\/span>
\n1 s<\/span>
\n1 t<\/span>
\n1 u<\/span>
\n1 v<\/span>
\n1 w<\/span>
\n1 x<\/span>
\n1 y<\/span>
\n1 z<\/span><\/p>\n

Do reverse loop - count down for same result:<\/p>\n

awk -e '{for ( i = NF; i >= NF; i-- ) print $i }' < alphabet.txt<\/span>
\na<\/span>
\nb<\/span>
\nc<\/span>
\nd<\/span>
\ne<\/span>
\nf<\/span>
\ng<\/span>
\nh<\/span>
\ni<\/span>
\nj<\/span>
\nk<\/span>
\nl<\/span>
\nm<\/span>
\nn<\/span>
\no<\/span>
\np<\/span>
\nq<\/span>
\nr<\/span>
\ns<\/span>
\nt<\/span>
\nu<\/span>
\nv<\/span>
\nw<\/span>
\nx<\/span>
\ny<\/span>
\nz<\/span><\/p>\n

Count no. of loops\/lines processed:<\/p>\n

awk -e '{for ( i = 1; i <= NF; i++ ) print $i, x += 1 }' < alphabet.txt<\/span>
\na 1<\/span>
\nb 2<\/span>
\nc 3<\/span>
\nd 4<\/span>
\ne 5<\/span>
\nf 6<\/span>
\ng 7<\/span>
\nh 8<\/span>
\ni 9<\/span>
\nj 10<\/span>
\nk 11<\/span>
\nl 12<\/span>
\nm 13<\/span>
\nn 14<\/span>
\no 15<\/span>
\np 16<\/span>
\nq 17<\/span>
\nr 18<\/span>
\ns 19<\/span>
\nt 20<\/span>
\nu 21<\/span>
\nv 22<\/span>
\nw 23<\/span>
\nx 24<\/span>
\ny 25<\/span>
\nz 26<\/span><\/p>\n

cat TheRavenV1_3.txt<\/span>
\nOnce upon a midnight dreary, while I pondered, weak and weary,
\nOver many a quaint and curious volume of forgotten lore\u201d
\nWhile I nodded, nearly napping, suddenly there came a tapping,
\nAs of some one gently rapping, rapping at my chamber door.
\n,Tis some visitor, I muttered, tapping at my chamber door\u201d
\nOnly this and nothing more.<\/p>\n

Ah, distinctly I remember it was in the bleak December;
\nAnd each separate dying ember wrought its ghost upon the floor.
\nEagerly I wished the morrow;\u201dvainly I had sought to borrow
\nFrom my books surcease of sorrow\u201dsorrow for the lost Lenore\u201d
\nFor the rare and radiant maiden whom the angels name Lenore\u201d
\nNameless here for evermore.<\/p>\n

And the silken, sad, uncertain rustling of each purple curtain
\nThrilled me\u201dfilled me with fantastic terrors never felt before;
\nSo that now, to still the beating of my heart, I stood repeating
\n,Tis some visitor entreating entrance at my chamber door\u201d
\nSome late visitor entreating entrance at my chamber door;\u201d
\nThis it is and nothing more.<\/p>\n

cat TheRavenV1_3.txt | awk -e '{FS=\"\" ; for ( i = 1; i <= NF; i++ ) print $i }' | sort | uniq -c<\/span>
\n221 <\/span>
\n11 ,<\/span>
\n4 ;<\/span>
\n5 .<\/span>
\n2 ,<\/span>
\n3 <\/span>
\n3 <\/span>
\n9 \u201d<\/span>
\n55 a<\/span>
\n4 A<\/span>
\n1 and<\/span>
\n12 b<\/span>
\n16 c<\/span>
\n26 d<\/span>
\n1 D<\/span>
\n1 dreary,<\/span>
\n91 e<\/span>
\n1 E<\/span>
\n13 f<\/span>
\n2 F<\/span>
\n20 g<\/span>
\n32 h<\/span>
\n45 i<\/span>
\n7 I<\/span>
\n3 k<\/span>
\n26 l<\/span>
\n2 L<\/span>
\n33 m<\/span>
\n1 midnight<\/span>
\n54 n<\/span>
\n1 N<\/span>
\n64 o<\/span>
\n2 O<\/span>
\n1 Once<\/span>
\n15 p<\/span>
\n1 pondered,<\/span>
\n1 q<\/span>
\n67 r<\/span>
\n37 s<\/span>
\n2 S<\/span>
\n62 t<\/span>
\n4 T<\/span>
\n14 u<\/span>
\n1 upon<\/span>
\n8 v<\/span>
\n10 w<\/span>
\n1 W<\/span>
\n1 weak<\/span>
\n1 weary,<\/span>
\n1 while<\/span>
\n15 y<\/span><\/p>\n

Examining the history histograph function - how it works - can be applied elsewhere?:<\/strong><\/p>\n

history | awk '{h[$2]++}END{for(i in h){print h[i],i|\"sort -rn|head -20\"}}' |awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"o%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span>
\nawk 419 <\/span>
\nperl 216 # <\/span>
\ncat 135 ## <\/span>
\nvi 39 <\/span>
\nprintf 34 ## <\/span>
\necho 26 # <\/span>
\nfor 17 <\/span>
\nbudellmint 15 <\/span>
\n.\/search.sh 14 <\/span>
\nuptime 12 ## <\/span>
\nls 7 ## <\/span>
\ndf 5 # <\/span>
\nhistory 4 # <\/span>
\nhexdump 4 # <\/span>
\nod 3 # <\/span>
\nclear 3 # <\/span>
\n90 3 # <\/span>
\n87 3 # <\/span>
\n74 3 # <\/span>
\n70 3 #<\/span><\/p>\n

Just see what happens piping RavenV3 in:<\/p>\n

cat TheRavenV1_3.txt | awk '{h[$2]++}END{for(i in h){print h[i],i|\"sort -rn|head -20\"}}' |awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span>
\nthe 2
\nsome 2
\nI 2
\n2
\nupon 1
\nthis 1
\nthat 1
\nof 1
\nmy 1
\nme\u201dfilled 1
\nmany 1
\nlate 1
\nit 1
\nhere 1
\neach 1
\ndistinctly 1<\/p>\n

FIRST - 20 lines of 169 words in RavenV3<\/p>\n

wc -l < TheRavenV1_3.txt<\/p>\n

20
\nwc -w < TheRavenV1_3.txt
\n169<\/p>\n

The function is actually two awk commands that can be treated separately - good!<\/p>\n

history | awk '{h[$2]++}END{for(i in h){print h[i],i|\"sort -rn|head -20\"}}'
\n419 awk
\n216 perl
\n122 cat
\n39 vi...<\/p>\n

The first above is easyish - loops to find how many occurrences of each command used, sorts high to low numerically, then heads it to only 20. (The full history record varies on setup and linux dist.<\/p>\n

history | wc -l
\n1000<\/p>\n

The history awk\u00a0function\u00a0reads $2 for the command names and puts them in $1 for output:<\/p>\n

history | tail -3<\/span>
\n2208 wc -w<\/strong> < TheRavenV1_3.txt <\/span>
\n2209 history<\/strong> | head -3<\/span>
\n2210 history<\/strong> | tail -3<\/span><\/p>\n

history | awk '{h[$2]++}END{for(i in h){print h[i],i|\"sort -rn|head -20\"}}' |awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}' | head -2<\/span>
\nawk<\/strong> 419
\nperl<\/strong> 216 #<\/p>\n

I think<\/em> it puts the contents of $2 of history list into an incrementing array named h[$2], so contains the command name, later ID'd by $2,h[$2], with the increment number ID'd by h[i] and i in the for loop:\u00a0This can be shown by printing the contents of each section:<\/p>\n

Array command list contents:<\/p>\n

history | awk '{h[$2]++; print $2}' | uniq -c | head<\/span>
\n5 cat<\/span>
\n3 perl<\/span>
\n1 uptime<\/span>
\n1 vi<\/span>
\n22 cat<\/span>
\n1 sudo<\/span>
\n2 perl<\/span>
\n1 uptime<\/span>
\n10 cat<\/span>
\n10 uptime<\/span><\/p>\n

Array Counter:
\nstevee@AMDA8 ~\/Awk $ history | awk '{h[$2]++; print h[$2]}' | head -3<\/span>
\n1
\n2
\n3<\/p>\n

For loop counter contents:<\/p>\n

history | awk '{h[$2]++}END{for(i in h){print h[i],i }}'<\/span>
\n1 rm<\/span>
\n419 awk<\/span>
\n1 sudo<\/span>
\n2 cd<\/span>
\n3 clear<\/span>
\n39 vi...<\/span><\/p>\n

This also shows I have used only 45 different commands in the whole 1000 size list:<\/p>\n

history | awk '{h[$2]++}END{for(i in h){print h[i],i | \"sort -rn\"}}' | wc -l<\/span>
\n45<\/p>\n

Summary Part1<\/strong> - output of history command column $2 are stored in a numbered\u00a0incrementing array, counted for occurrence of each command and labelled using a FOR loop, then numerically sorted by quantity and headed.<\/p>\n

Part 2<\/strong><\/p>\n

awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span><\/p>\n

The next awk command can't be shown directly as it needs the prior awk\u00a0output, but knowing the format of the first part that feeds it comprises only 2 columns, a different 2 column input can be fed to learn it's behaviour. Using cat to give a number list and a name file from prior above examples, to give text, input can be fed in:<\/p>\n

cat -n acronyms.txt | awk '{print $1,$2}' | awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span>
\nBASIC 1 <\/span>
\nCICS 2 <\/span>
\nCOBOL 3 <\/span><\/p>\n

Now it is obvious that the multiplier is 60*$1. If reduced to 1, it shows:<\/p>\n

cat -n acronyms.txt | awk '{print $1,$2}' | awk '!max{max=$1;}{r=\"\";i=s=1*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span>
\nBASIC 1 # <\/span>
\nCICS 2 ## <\/span>
\nCOBOL 3 <\/span>
\nDBMS 4 # <\/span>
\nGIGO 5 ## <\/span>
\nGIRL 6 <\/span>
\nAWK 7 # <\/span>
\nPERL 8 ## <\/span>
\nFORTRAN 9 \u00a0<\/span><\/p>\n

Summary Part 2\u00a0<\/strong><\/p>\n

An experienced programmer would have seen immediately that it is part 2 that is the generic useful part for histogram usage as this contains the multiplier to suit various number ranges, and reads a NOT MAX and MAX value and uses a dividend (somehow!) to scale the output according the maximum and minimum of the range (?? not sure bout that or how).<\/p>\n

Point is, it can be used in isolation for many basic number lists - experimenting now...<\/p>\n

It works correctly\u00a0on the random.txt file, scaling automatically AND padding single digits properly in a terminal, with the 60 multiplier! Don't know how it works yet...(why 60?) This scales all below by 2\/3...very clever...<\/p>\n

cat random.txt | awk '{print $1,$2}' | awk '!max{max=$1;}{r=\"\";i=s=60*$1\/max;while(i-->0)r=r\"#\";printf \"%15s %5d %s %s\",$2,$1,r,\"\\n\";}'<\/span>
\n90 <\/span>
\n87 # <\/span>
\n74 ## <\/span>
\n40 <\/span>
\n90 <\/span>
\n70 ## <\/span>
\n46 # <\/span>
\n10 # <\/span>
\n35 <\/span>
\n\u00a0 8 <\/span>
\n\u00a0 1 # <\/span>
\n\u00a0 0<\/span><\/p>\n

Factorial.awk:<\/p>\n

-<\/p>\n

awk '<\/span><\/p>\n

# factorial: return factorial of user-supplied number<\/span>
\nBEGIN {<\/span>
\n# prompt user; use printf, not print, to avoid the\u00a0<\/span>newline<\/span>
\nprintf(\"Enter number: \")<\/span>
\n}<\/span>
\n# check that user enters a number<\/span>
\n$1 ~ \/^[0-9]+$\/ {<\/span>
\n# assign value of $1 to number & fact<\/span>
\nnumber = $1<\/span>
\nif (number == 0)<\/span>
\nfact = 1<\/span>
\nelse<\/span>
\nfact = number<\/span>
\n# loop to multiply fact*x until x = 1<\/span>
\nfor (x = number - 1; x > 1; x--)<\/span>
\nfact *= x<\/span>
\nprintf(\"The factorial of %d is %g\\n\", number, fact)<\/span>
\n# exit -- saves user from typing CRTL-D.<\/span>
\nexit<\/span>
\n}<\/span>
\n# if not a number, prompt again.<\/span>
\n{ printf(\"\\nInvalid entry. Enter a number: \")<\/span>
\n}' -<\/span><\/p>\n

chmod +x factorial.awk<\/span><\/p>\n

.\/factorial.awk<\/span>
\nEnter number: 4<\/span>
\nThe factorial of 4 is 24<\/span><\/p>\n

.\/factorial.awk<\/span>
\nEnter number: f<\/span><\/p>\n

Invalid entry. Enter a number:\u00a00
\nThe factorial of 0 is 1<\/span><\/p>\n

vi grades.txt<\/span><\/p>\n

mona 70 77 85 83 70 89
\njohn 85 92 78 94 88 91
\nandrea 89 90 85 94 90 95
\njasper 84 88 80 92 84 82
\ndunce 64 80 60 60 61 62
\nellis 90 98 89 96 96 92<\/p>\n

vi grades.awk<\/span><\/p>\n

# grades.awk -- average student grades and determine
\n# letter grade as well as class averages.
\n# $1 = student name; $2 - $NF = test scores.
\n# set output field separator to tab.
\nBEGIN { OFS = \"\\t\" }
\n# action applied to all input lines
\n{
\n# add up grades
\ntotal = 0
\nfor (i = 2; i <= NF; ++i)
\ntotal += $i
\n# calculate average
\navg = total \/ (NF - 1)
\n# assign student's average to element of array
\nstudent_avg[NR] = avg
\n# determine letter grade
\nif (avg >= 90) grade = \"A\"
\nelse if (avg >= 80) grade = \"B\"
\nelse if (avg >= 70) grade = \"C\"
\nelse if (avg >= 60) grade = \"D\"
\nelse grade = \"F\"
\n# increment counter for letter grade array
\n++class_grade[grade]
\n# print student name, average and letter gradeprint $1, avg, grade
\n}
\n# print out class statistics
\nEND {
\n# calculate class average
\nfor (x = 1; x <= NR; x++)
\nclass_avg_total += student_avg[x]
\nclass_average = class_avg_total \/ NR
\n# determine how many above\/below average
\nfor (x = 1; x <= NR; x++)
\nif (student_avg[x] >= class_average)
\n++above_average
\nelse
\n++below_average
\n# print results
\nprint \"\"
\nprint \"Class Average: \", class_average
\nprint \"At or Above Average: \", above_average
\nprint \"Below Average: \", below_average
\n# print number of students per letter grade
\nfor (letter_grade in class_grade)
\nprint letter_grade \":\", class_grade
\n[letter_grade] | \"sort\"
\n}<\/p>\n

awk -f grades.awk grades.test<\/span><\/p>\n

Class Average: 83.4167<\/span>
\nAt or Above Average: 4<\/span>
\nBelow Average: 2<\/span>
\nA: 2<\/span>
\nB: 2<\/span>
\nC: 2<\/span>
\nD: 2<\/span><\/p>\n

Doesn't add up and prog had to be \"fixed\" wrongly<\/em> by me..Needs 2 @ avg and 4 above avge for 8 students total - Proof read the oh dear!ing thing OReilly!! This OP part is missing..<\/p>\n

mona 79\u00a0C
\njohn 88B
\nandrea 90.5A<\/p>\n

jasper 85B
\ndunce\u00a064.5D
\nellis\u00a093.5A<\/p>\n

The keyword in is also an operator that can be used in a conditional expression to test that a subscript is\u00a0a member of an array. The expression:
\nitem in<\/strong> array<\/em><\/p>\n

returns 1 if array[item] exists and 0 if it does not.<\/p>\n

vi lookup.awk<\/span><\/p>\n

------<\/p>\n

awk '<\/span># lookup -- reads local glossary file and prompts user<\/span>
\nfor query<\/span>
\n#0<\/span>
\nBEGIN { FS = \"\\t\"; OFS = \"\\t\"<\/span>
\n# prompt user<\/span>
\nprintf(\"Enter a glossary term: \")<\/span>
\n}<\/span>
\n#1 read local file named glossary<\/span>
\nFILENAME == \"glossary.txt\" {<\/span>
\n# load each glossary entry into an array<\/span><\/p>\n

entry[$1] = $2<\/span>
\nnext<\/span>
\n}<\/span>
\n#2 scan for command to exit program<\/span>
\n$0 ~ \/^(quit|[qQ]|exit|[Xx])$\/ { exit }<\/span>
\n#3 process any non-empty line<\/span>
\n$0 != \"\" {<\/span>
\nif ( $0 in entry ) {<\/span>
\n# it is there, print definition<\/span>
\nprint entry[$0]<\/span>
\n} else<\/span>
\nprint $0 \" not found\"<\/span>
\n}<\/span>
\n#4 prompt user again for another term<\/span>
\n{<\/span>
\nprintf(\"Enter another glossary term (q to quit): \")<\/span>
\n}' glossary.txt -<\/span><\/p>\n

\"lookup.awk.png\"<\/a><\/p>\n

---------<\/p>\n

cp -v acronyms.txt glossary.txt<\/span><\/p>\n

chmod +x lookup.awk<\/span><\/p>\n

.\/lookup.awk<\/span>
\nEnter a glossary term: AWK<\/span>
\nAho Weinberger Kernighan<\/span>
\nEnter another glossary term (q to quit): COBOL<\/span>
\nCommon Business Oriented Language<\/span>
\nEnter another glossary term (q to quit): q<\/span><\/p>\n

cat glossary.txt<\/span>
\nACID Atomicity, Consistency, Isolation, Durability<\/span>
\nAWK Aho Weinberger Kernighan<\/span>
\nBASIC Beginner's All-Purpose Symbolic Instruction Code <\/span>
\nCICS Customer Information Control System<\/span>
\nCOBOL Common Business Oriented Language<\/span>
\nDBMS Data Base Management System<\/span>
\nGIGO Garbage In, Garbage Out<\/span>
\nGIRL Generalized Information Retrieval Language<\/span>
\nPERL Practical Extraction and Reporting Language<\/span>
\nFORTRAN Formula Translation<\/span><\/p>\n

Program Piping<\/strong><\/p>\n

Coz awk acts per line, it can alter output of a program piped thru it e.g. a prime prog prints primes per line, so awk can count the number of primes output per request:<\/p>\n

.\/bash.sh<\/span>
\nHow many prime numbers ?: 4<\/span>
\n1<\/span>
\n2<\/span>
\n3<\/span>
\n5<\/span>
\n7<\/span><\/p>\n

.\/bash.sh | awk '{print NR, $0}'<\/span>
\nHow many prime numbers ?: 4<\/span>
\n1 1<\/span>
\n2 2<\/span>
\n3 3<\/span>
\n4 5<\/span>
\n5 7<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Create yourself a copy of the awk\/sed man page - it's easier to read: man -t awk | ps2pdf - > awkman.pdf man -t sed\u00a0| ps2pdf - > sedman.pdf From sed & awk by OReilly - older kernel\/system example problems...examples don't copy in lines, but columns often, and major parts of examples are assumed knowledge\/steps, ...\u00a0 Awk\/Perl\/SED Notepad<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-4339","page","type-page","status-publish","hentry"],"a3_pvc":{"activated":true,"total_views":4,"today_views":0},"_links":{"self":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/pages\/4339","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/comments?post=4339"}],"version-history":[{"count":9,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/pages\/4339\/revisions"}],"predecessor-version":[{"id":10337,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/pages\/4339\/revisions\/10337"}],"wp:attachment":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/media?parent=4339"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}