{"id":4145,"date":"2016-08-17T15:36:29","date_gmt":"2016-08-17T14:36:29","guid":{"rendered":"https:\/\/stevepedwards.today\/DebianAdmin\/?p=4145"},"modified":"2016-08-17T15:36:29","modified_gmt":"2016-08-17T14:36:29","slug":"gnuplot-basic-starter-example-data-file-for-system-profile-plots","status":"publish","type":"post","link":"https:\/\/stevepedwards.today\/DebianAdmin\/gnuplot-basic-starter-example-data-file-for-system-profile-plots\/","title":{"rendered":"Gnuplot – Basic Starter Example Data File for System Profile Plots"},"content":{"rendered":"
<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

This example is my starter data.dat file for building complexity if required, later. It combines the basics required for a time of day x axis against y axis numerical data, with x\/y autoscaling and title functions etc. Paste it into some file name:<\/p>\n

vi plot1.gnu<\/span><\/p>\n

set title 'user CPU time'<\/span>
\nset autoscale<\/span>
\nset terminal png medium size 640,480<\/span>
\nset xdata time<\/span>
\nset xlabel 'Time of Day' font 'Arial,14'<\/span>
\nset ylabel '%CPU util' font 'Arial,14'<\/span>
\nset timefmt '%Y-%m-%d %H:%M:%S'<\/span>
\nset output 'userCPU.png'<\/span>
\nplot 'data1.dat' using 1:3 title 'user cpu%' with linespoints<\/span><\/p>\n

This requires the data file \"data1.dat\" to be available in your home dir, if running gnuplot from there. The 1:3 reads the 3rd column data from left to right to plot it against the pre-defined x time axis. It outputs a PNG of the plot, also in the home directory. The rest is self evident - titles etc.<\/p>\n

The data can be a column list of numbers created from any source - by hand etc. For this example I'm collecting data from sar\/sadf.<\/p>\n

sudo apt-get install sysstat<\/span><\/p>\n

Turn on sadc logging in<\/p>\n

sudo vi \/etc\/default\/sysstat<\/span><\/p>\n

ENABLED=\"true\"<\/span><\/p>\n

sar<\/span>
\nCannot open \/var\/log\/sysstat\/sa17: No such file or directory<\/span>
\nPlease check if data collecting is enabled in \/etc\/default\/sysstat<\/span><\/p>\n

After a\u00a0\"linux jiffie\" or two, and the creation of a binary log folder e.g:\u00a0ls \/var\/log\/sysstat\/sa17<\/span>,\u00a0sar will be able to output data from 10 min intervals, as will sadf:<\/p>\n

\"sar1.png\"<\/a><\/p>\n

stevee@AMDA8 ~ $ sadf -p -P 0,1,2,3 | head\u00a0<\/span>
\nAMDA8 599 2016-08-17 11:55:01 UTC cpu0 %user<\/strong> 10.28<\/span>
\nAMDA8 599 2016-08-17 11:55:01 UTC cpu0 %nice 0.07<\/span>
\n.....<\/span><\/p>\n

This is the data I will strip columns from for that data.dat file\u00a0as the y axis in is already configured to read time\/date info in this format of columns 3 and\/or 4 above:\u00a0set timefmt '%Y-%m-%d %H:%M:%S'<\/span><\/p>\n

sadf -p -P 0,1,2,3 | grep user | grep cpu0 | awk '{print $3,$4,$8}'<\/span><\/p>\n

2016-08-17 11:55:01 10.28<\/span>
\n2016-08-17 12:45:01 7.67....<\/span><\/p>\n

3 columns allows you to see where the 1:3 fits in the gnuplot conf file, and will allow an extra date on the x axis to be added if you want later.<\/p>\n

Direct this into data.dat:\u00a0sadf -p -P 0,1,2,3 | grep user | grep cpu0 | awk '{print $3,$4,$8}' > data1.dat<\/span><\/p>\n

cat data1.dat<\/span>
\n2016-08-17 11:55:01 10.28<\/span>
\n2016-08-17 12:05:01 5.27<\/span>
\n2016-08-17 12:15:01 17.73<\/span>
\n2016-08-17 12:25:01 13.63<\/span>
\n2016-08-17 12:35:01 4.38<\/span>
\n2016-08-17 12:45:01 7.67<\/span>
\n2016-08-17 12:55:01 8.40<\/span>
\n2016-08-17 13:05:01 11.02<\/span>
\n2016-08-17 13:15:01 13.57<\/span>
\n2016-08-17 13:25:01 10.07<\/span>
\n2016-08-17 13:35:01 5.68<\/span>
\n2016-08-17 13:45:01 3.20<\/span>
\n2016-08-17 13:55:01 4.32<\/span>
\n2016-08-17 14:05:01 2.40<\/span>
\n2016-08-17 14:15:01 9.42<\/span>
\n2016-08-17 14:25:01 10.45<\/span>
\n2016-08-17 14:35:01 14.02<\/span>
\n2016-08-17 14:45:01 14.45<\/span>
\n2016-08-17 14:55:01 16.20<\/span><\/p>\n

Now you just run gnuplot with the conf file:<\/p>\n

gnuplot plot1.gnu<\/span><\/p>\n

An aptly named userCPU.png should appear in your home dir:<\/p>\n

ls<\/span>
\nbin Cprogs Desktop Downloads Pictures Public userCPU.png<\/strong><\/span>
\nContacts data1.dat Documents Music plot1.gnu Templates Videos<\/span><\/p>\n

Open it:<\/p>\n

xdg-open userCPU.png\u00a0<\/span><\/p>\n

\"userCPU.png\"<\/a><\/p>\n

Compare the data values from the data.dat file and check their graph points:<\/p>\n

cat data1.dat | awk '{print $3}' | sed ':a;N;$!ba;s\/\\n\/; \/g'<\/span>
\n10.28; 5.27; 17.73; 13.63; 4.38; 7.67; 8.40; 11.02; 13.57; 10.07; 5.68; 3.20; 4.32; 2.40; 9.42; 10.45; 14.02; 14.45; 16.20; 8.94\u00a0<\/span><\/p>\n

Nice! Now you can tart it up with colours and whatever you want by adding more functions to the conf file etc.<\/p>\n

https:\/\/gnuplot.sourceforge.net\/docs_4.2\/node1.html<\/a><\/span><\/p>\n

Hmmm...just noticed...the latest sar data is 1hr 08 behind real time...<\/p>\n

date<\/span>
\nWed Aug 17 16:43:37<\/strong> BST 2016<\/span><\/p>\n

cat data1.dat | tail -n1<\/span>
\n2016-08-17 15:35:01<\/strong> 6.11<\/span><\/p>\n

OK, a few hrs later, a thing to note is that you need to increase the PNG size as you add more data else the x axis time gets squished, so I changed:<\/p>\n

set terminal png large size 1280,960<\/span><\/p>\n

\"userCPUlg.png\"<\/a><\/p>\n

The x axis time fields (with seconds) are set by the data, not<\/strong> the conf file, so if you wanted to ignore the seconds you have to strip that colon separated column from the output. Then you could remove the %S from<\/p>\n

set timefmt '%Y-%m-%d %H:%M:%S'<\/span><\/p>\n

If you study the data, the time is taken every 01 secs, so that is what needs removal:<\/p>\n

2016-08-17 17:05:01<\/strong> 24.73<\/span>
\n 2016-08-17 17:15:01<\/strong> 7.24<\/span>
\n 2016-08-17 17:25:01<\/strong> 2.75<\/span><\/p>\n

Appending sed to strip these chars from the line does the job:<\/p>\n

sadf -p -P 0,1,2,3 | grep user | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g'<\/span><\/p>\n

2016-08-17 11:55<\/strong> 10.28<\/span>
\n 2016-08-17 12:05<\/strong> 5.27<\/span>
\n 2016-08-17 12:15<\/strong> 17.73<\/span><\/p>\n

Now there is data with only 10 min intervals. Amending the conf file for that also gives month and day, but very squashed so needs rotation:<\/p>\n

set format x \"%m-%d %H:%M\"<\/span><\/p>\n

\"nosecs.png\"<\/a><\/p>\n

set format x \"%H:%M\"<\/span><\/p>\n

\"nosecs2.png\"<\/a><\/p>\n

set xtics rotate<\/span><\/p>\n

\"rotatedef.png\"<\/a><\/p>\n

That defaults to 90 degs, but 45 may be better for added info but I cannot get this \"by 45\" to work:<\/p>\n

set title 'user CPU time'
\nset autoscale
\nset terminal png large size 1280,960
\nset xdata time
\nset xlabel 'Time of Day' font 'Arial,14' offset -2,0<\/strong>
\nset ylabel '%CPU util' font 'Arial,12'
\nset timefmt '%Y-%m-%d %H:%M:%S'
\nset xtics rotate<\/strong>
\nset format x \"%H:%M\"<\/strong>
\nset output 'userCPU.png'
\nplot 'data1.dat' using 1:3 title 'user cpu%' with linespoints
\n<\/span><\/p>\n

Finally, to include the full ymd\/time date:
\nset format x \"%Y-%m-%d %H:%M\"<\/span><\/p>\n

\"userCPU.png\"<\/a><\/p>\n

Each time you want an instant updated plot viewed, run the three cmds on one line e.g:<\/p>\n

sadf -p -P 0,1,2,3 | grep user | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu;\u00a0xdg-open userCPU.png<\/span><\/p>\n

\"userCPU.png\"<\/a><\/p>\n

It's easy to view the other CPU% usage options like idle and system just by changing the grepped term and\/or the png name. Swap user for idle for example - note the y axis autoscale increase:<\/p>\n

sadf -p -P 0,1,2,3 | grep idle<\/strong> | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu; mv -v userCPU.png idleCPU.png\u00a0<\/strong>; xdg-open idleCPU.png<\/strong><\/span><\/p>\n

\"idleCPU.png\"<\/a><\/p>\n

The idle is almost a mirror image of the user graph! The system must be similar to the user stats then?<\/p>\n

sadf -p -P 0,1,2,3 | grep system | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu; mv -v userCPU.png system.png; xdg-open system.png<\/span><\/p>\n

\"system.png\"<\/a><\/p>\n

Very similar:<\/p>\n

\"usersystem.png\"<\/a><\/p>\n

If xdg-open complains with:<\/p>\n

(eog:18950): EOG-WARNING **: Couldn't load icon: Error writing<\/span><\/p>\n

stick it in the bit bucket to get the cmd line back:<\/p>\n

sadf -p -P 0,1,2,3 | grep system | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu; mv -v userCPU.png system.png; xdg-open system.png &2> \/dev\/null <\/strong><\/span>
\n\u02dcuserCPU.png, -> \u02dcsystem.png,<\/span>
\n[1] 19094<\/span>
\nstevee@AMDA8 ~ $<\/span><\/p>\n

With a change of log folder at 2400hrs, you may want to run data from a prior day with the same format for these commands:<\/p>\n

sadf \/var\/log\/sysstat\/sa17<\/strong>\u00a0-P 0,1,2,3 | grep nice | grep cpu0 | awk '{print $3,$4,$8}' | sed 's\/:01\/\/g' > data1.dat; gnuplot plot1.gnu; mv -v userCPU.png nice.png; xdg-open nice.png &2> \/dev\/null<\/span><\/p>\n

Running for niceness also shows the autoscaling smaller y axis:<\/p>\n

\"nice.png\"<\/a><\/p>\n

Finally, just for completeness of terminology, the iowait<\/strong> field (steal is all 0%):<\/p>\n

\"iowait.png\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

<\/div>\n

<\/path><\/svg><\/i> \"Loading\"<\/p>\n

<\/div>\n

This example is my starter data.dat file for building complexity if required, later. It combines the basics required for a time of day x axis against y axis numerical data, with x\/y autoscaling and title functions etc. Paste it into some file name: vi plot1.gnu set title 'user CPU time' set autoscale set terminal png ...\u00a0 Gnuplot – Basic Starter Example Data File for System Profile Plots<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-4145","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"a3_pvc":{"activated":true,"total_views":1,"today_views":0},"_links":{"self":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4145","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/comments?post=4145"}],"version-history":[{"count":0,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/posts\/4145\/revisions"}],"wp:attachment":[{"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/media?parent=4145"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/categories?post=4145"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stevepedwards.today\/DebianAdmin\/wp-json\/wp\/v2\/tags?post=4145"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}