{"id":4716,"date":"2016-09-05T14:41:01","date_gmt":"2016-09-05T13:41:01","guid":{"rendered":"https:\/\/stevepedwards.today\/DebianAdmin\/?p=4716"},"modified":"2022-09-22T20:07:59","modified_gmt":"2022-09-22T19:07:59","slug":"developing-one-liners-with-command-substitution","status":"publish","type":"post","link":"https:\/\/stevepedwards.today\/DebianAdmin\/developing-one-liners-with-command-substitution\/","title":{"rendered":"Developing One Liners with Command Substitution"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Understanding differences between the output of one command being taken as the input (the argument) for another, OR whether the output of a command is read by the shell (an unseen \"subshell\" if you like) and interpreted (or not) as a command or an argument in itself, can be very confusing when piping commands, depending what commands, where they are in the chain, and whether they accept STDIN or not.<\/p>\n

The difference between variable expansion using the $() format or whether backtics `$5`; double quotes \"$5\"; single quotes '$5' or no quotes $5; can be easily mistaken - especially if many braces {}, brackets () or slashes \/\/ etc are involved.<\/p>\n

The content of the backtics example is taken as a command for the shell to run first, whose results may\u00a0become an\u00a0argument, but the $() expansion needs to be run by a\u00a0command so it\u00a0becomes the argument. Confused? These example should summarise some\u00a0variations:<\/p>\n

stevee@AMDA8 ~\/Awk $ b=5; echo $b<\/span>
\n5<\/span>
\nstevee@AMDA8 ~\/Awk $ b=5; echo \"$b\"<\/span>
\n5<\/span>
\nstevee@AMDA8 ~\/Awk $ b=5; echo '$b'<\/span>
\n$b<\/span>
\nstevee@AMDA8 ~\/Awk $ b=5; echo `$b`<\/span>
\n5: command not found\u00a0<\/span><\/p>\n

stevee@AMDA8 ~\/Awk $\u00a0$(b)<\/span>
\nb: command not found<\/span><\/p>\n

stevee@AMDA8 ~\/Awk $ $b<\/span>
\n5: command not found<\/span><\/p>\n

stevee@AMDA8 ~\/Awk $ printf $b<\/span>
\n5<\/span><\/p>\n

stevee@AMDA8 ~\/Awk $ printf $(b)<\/span>
\nb: command not found<\/span><\/p>\n

stevee@AMDA8 ~\/Cprogs $ a=1; b=2; expr $a+$b<\/span>
\n1+2<\/span>
\nstevee@AMDA8 ~\/Cprogs $ a=1; b=2; expr a+b<\/span>
\na+b<\/span>
\nstevee@AMDA8 ~\/Cprogs $ echo `expr $a + $b`<\/span>
\n3<\/span>
\nstevee@AMDA8 ~\/Cprogs $ echo `expr $a - $b`<\/span>
\n-1<\/span>
\nstevee@AMDA8 ~\/Cprogs $ echo `expr $a \\* $b`<\/span>
\n2<\/span>
\nstevee@AMDA8 ~\/Cprogs $ echo `expr $a \/ $b`<\/span>
\n0<\/span>
\nstevee@AMDA8 ~\/Cprogs $ echo `expr $b \/ $a`<\/span>
\n2<\/span><\/p>\n

stevee@AMDA8 ~\/Awk $<\/span> $USERNAME<\/span>
\nstevee: command not found<\/span><\/p>\n

It pays to experiment and practise to get answers. For example, nmap can take a list of IP addresses to scan, if separated by whitespace, including tabs, e.g.<\/p>\n

nmap 192.168.1.2 \u00a0 \u00a0 192.168.1.3<\/span><\/p>\n

Normally, you may\u00a0feed nmap a prepared IP list from a file if there are many targets, but what if another command outputs some IPs you wanted to use, but were not in a space delimited format, like arp -a for example? This gives the MAC and IP addresses for all devices up on a network. On a class C \/24 network that's up to 254 device addresses. You could direct them into a text file and edit them, then feed the list to nmap that way:<\/p>\n

nmap -iL iplist.txt<\/span><\/p>\n

but depending where you got the list from, say a DHCP list, it may still need formatting to be suitable anyway, such as the DHCP list from my router accessed by webpage:<\/p>\n

1 192.168.1.2 00-1A-A0-5D-1D-AD FIXED IP <\/span>
\n2 192.168.1.3 B8-27-EB-E8-6B-1D FIXED IP <\/span>
\n3 192.168.1.4 B8-27-EB-71-14-27 FIXED IP <\/span>
\n4 192.168.1.5 00-23-54-3A-EB-9A FIXED IP <\/span>
\n5 192.168.1.7 B8-27-EB-73-A8-A5 FIXED IP <\/span>
\n6 192.168.1.8 B8-27-EB-B9-D1-C0 FIXED IP <\/span>
\n7 192.168.1.14 5C-93-A2-D5-83-3B 23:41:21<\/span><\/p>\n

A better solution may be to create a one-liner to use anywhere else in future, from another generally system available source\/cmd like:<\/p>\n

arp -a<\/span>
\n? (192.168.1.3) at b8:27:eb:e8:6b:1d [ether] on wlan0<\/span>
\n? (192.168.1.4) at b8:27:eb:71:14:27 [ether] on wlan0<\/span><\/p>\n

Practise inventing situations like this to get to know how to output what you want for the commands you know, then look at optimising it after it works - whether one command can act in place of itself in one instance rather than 2, or whether it has better options than you know by reading it's man page.<\/p>\n

To get the IPs in a format that nmap can accept, I need the IPs on a single line, so building in stages: first isolate the 2nd field:<\/p>\n

arp -a | awk '{print $2<\/strong>}'<\/span>
\n(192.168.1.3)<\/span>
\n(192.168.1.1)<\/span>
\n(192.168.1.4)<\/span>
\n(192.168.1.7)<\/span>
\n(192.168.1.2)<\/span>
\n(192.168.1.5)<\/span>
\n(192.168.1.8)<\/span><\/p>\n

Remove the brackets by substituting\u00a0the first and last chars between\u00a0the \"anchors\"; the first (^) and last ($) non printable line delimiters - that define a line. The dot represents ANY single char before and after them so matches each bracket ():<\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.<\/strong>\/\/g' | sed 's\/.$<\/strong>\/\/g'<\/span>
\n192.168.1.3<\/span>
\n192.168.1.1<\/span>
\n192.168.1.4<\/span>
\n192.168.1.7<\/span>
\n192.168.1.2<\/span>
\n192.168.1.5<\/span>
\n192.168.1.8<\/span><\/p>\n

Now they need to be on one line, by having the newline (\\n) stripped and replaced with whitespace or a tab (\\t), but the paste command was designed for this already:<\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | paste -s<\/span>
\n192.168.1.3 192.168.1.1 192.168.1.4 192.168.1.7 192.168.1.2 192.168.1.5 192.168.1.8<\/span><\/p>\n

Or tr and perl are better\u00a0to use when learning concepts, as sed and awk don't work in the same simple search and replace format for \"\\n\" type chars in Mint.<\/p>\n

This seems to be what paste -s uses anyway, so you could also use 1 white space \" \" instead of a tab \\t, also:<\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | tr \"\\n\" \"\\t\"<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | perl -pe 's\/\\n\/\\t\/g'<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

sed and awk fail for this simple format requiring more cryptic parameters:<\/strong><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | sed 's\/\\n\/\\t\/g'<\/strong><\/span>
\n192.168.1.2<\/span>
\n192.168.1.7...<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | sed '{:q;N;s\/\\n\/ \/g;t q}'<\/strong><\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | awk 'gsub(\"\\n\",\"\\t\")'<\/span><\/p>\n

no OP<\/span><\/p>\n

arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | awk '{printf \"%s \",$0} END {print \"\"}'<\/span>
\n192.168.1.2 192.168.1.7 192.168.1.1 192.168.1.3 192.168.1.8 192.168.1.4 192.168.1.5<\/span><\/p>\n

Now the result is in the right list form as an argument that nmap requires to read multiple IPs, but how to get all this on the command line AFTER the nmap command when nmap cannot take input from a pipe end, such as 192.xxx | nmap.<\/p>\n

This is where you see how the shell handles different output as an argument or a command, so works or not, if either any form of quotes is used (fail) or whether variable expansion or backtics is used (pass):<\/p>\n

nmap 'arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | paste -s'<\/span><\/p>\n

Starting Nmap 6.40 ( https:\/\/nmap.org ) at 2016-09-04 23:02 BST<\/span>
\nFailed to resolve \"arp -a | awk {print\".<\/span><\/strong>
\nUnable to split netmask from target expression: \"} | sed s\/^.\/\/g | sed s\/.$\/\/g | paste -s\"<\/span><\/strong>
\nWARNING: No targets were specified, so 0 hosts scanned.<\/span>
\nNmap done: 0 IP addresses (0 hosts up) scanned in 0.30 seconds<\/span><\/p>\n

nmap `arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | paste -s`<\/code><\/span><\/p>\n

Starting Nmap 6.40 ( https:\/\/nmap.org ) at 2016-09-04 23:03 BST<\/span>
\nUnable to split netmask from target expression: \"arp -a | awk '{print }' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | paste -s\"<\/span><\/strong>
\nWARNING: No targets were specified, so 0 hosts scanned.<\/span>
\nNmap done: 0 IP addresses (0 hosts up) scanned in 0.13 seconds<\/span><\/p>\n

nmap $(arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | paste -s)<\/span><\/p>\n

Starting Nmap 6.40 ( https:\/\/nmap.org ) at 2016-09-04 23:00 BST<\/span>
\nNmap scan report for 192.168.1.3<\/span>
\nHost is up (0.013s latency).<\/span>
\nNot shown: 995 closed ports<\/span>
\nPORT STATE SERVICE<\/span>
\n22\/tcp open ssh<\/span>
\n139\/tcp open netbios-ssn<\/span>
\n445\/tcp open microsoft-ds<\/span>
\n8080\/tcp open http-proxy<\/span>
\n8081\/tcp open blackice-icecap....<\/span><\/p>\n

nmap `arp -a | awk '{print $2}' | sed 's\/^.\/\/g' | sed 's\/.$\/\/g' | paste -s`<\/span><\/p>\n

Starting Nmap 6.40 ( https:\/\/nmap.org ) at 2016-09-04 23:00 BST<\/span>
\nNmap scan report for 192.168.1.3<\/span>
\nHost is up (0.013s latency).<\/span>
\nNot shown: 995 closed ports<\/span>
\nPORT STATE SERVICE<\/span>
\n22\/tcp open ssh<\/span>
\n139\/tcp open netbios-ssn<\/span>
\n445\/tcp open microsoft-ds<\/span>
\n8080\/tcp open http-proxy<\/span>
\n8081\/tcp open blackice-icecap...<\/span><\/p>\n

You can see how cryptic\u00a0looking this can get for a working example of backtics and tr, and being aware of what spaces are required or not:<\/p>\n

nmap `arp -a|awk '{print $2}'|sed 's\/^.\/\/g'|sed 's\/.$\/\/g'|tr \"\\n\" \" \" `<\/span><\/strong><\/p>\n

There are some oddities that can occur if you investigate these variations, for example, the live ASCII bar chart example has uptime embedded in a perl function inside backtics, and works as intended:<\/p>\n

perl -e 'while(1) {`uptime` =~ \/average: ([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)); sleep 1 }'<\/span><\/p>\n

3.41 #<\/span>
\n3.41 #<\/span>
\n3.45 #<\/span>
\n3.45 #<\/span>
\n3.26 ##<\/span><\/p>\n

BUT..If you replace the backtics with single quotes, an error is not thrown, and the command does not \"fail\" to work completely, but it gives no output, and\u00a0loops until stopped:<\/p>\n

perl -e 'while(1) {'uptime' =~ \/average: ([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)); sleep 1 }'<\/span><\/p>\n

no OP<\/span><\/p>\n

no OP...<\/span><\/p>\n

Yet if you run uptime alone in single quotes, it gives the same output as it would without single quotes, yet that data does not become an argument for the function as before:<\/p>\n

'uptime'<\/span>
\n23:50:05 up 6:27, 3 users, load average: 3.60, 3.81, 4.63<\/span><\/p>\n

The same occurs for double quotes.<\/p>\n

For an attempt at variable expansion, it fails noisily:<\/p>\n

perl -e 'while(1) {$(uptime)<\/strong> =~ \/average: ([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)); sleep 1 }'<\/span>
\nBareword found where operator expected at -e line 1, near \"$(uptime\"<\/span>
\n(Missing operator before uptime?)<\/span>
\nsyntax error at -e line 1, near \"$(uptime\"<\/span>
\nExecution of -e aborted due to compilation errors.<\/span><\/p>\n

The cause is shown by trying to run the expansion alone, as the\u00a0first space delimited output of the uptime command - the time of day - is read by the shell as a potential command, which of course, does not exist.<\/p>\n

$(uptime)<\/span>
\n23:54:07: command not found<\/span><\/p>\n

Yet it can be run as a variable expansion argument for the right command:<\/p>\n

echo $(uptime)<\/span>
\n23:58:39 up 6:37, 5 users, load average: 0.96, 0.85, 1.09<\/span><\/p>\n

BUT, it cannot be run in the function like that inside backtics, even though echo is present to try to expand it:<\/p>\n

perl -e 'while(1) { `echo $(uptime)`<\/strong> =~ \/average: ([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)); sleep 1 }'<\/span>
\nsh: 1: Syntax error: \")\" unexpected<\/span><\/p>\n

perl -e `while(1) { `echo $(uptime)` =~ \/average: ([\\d.]+)\/; printf(\"% 5s %s\\n\", $1, \"#\" x ($1 * 10)); sleep 1 }`<\/span>
\n><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

Understanding differences between the output of one command being taken as the input (the argument) for another, OR whether the output of a command is read by the shell (an unseen \"subshell\" if you like) and interpreted (or not) as a command or an argument in itself, can be very confusing when piping commands, depending ...\u00a0 Developing One Liners with Command Substitution<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-4716","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"a3_pvc":{"activated":true,"total_views":1,"today_views":0},"_links":{"self":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4716","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/comments?post=4716"}],"version-history":[{"count":4,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4716\/revisions"}],"predecessor-version":[{"id":8941,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4716\/revisions\/8941"}],"wp:attachment":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/media?parent=4716"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/categories?post=4716"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/tags?post=4716"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}