Author Topic: Daily Log file - Is it possible to just record Min, Average and Max Values?  (Read 6503 times)

Stumpy

  • Newbie
  • *
  • Posts: 19
    • View Profile
I have been logging data to a text file using AHMI v3.99y Beta28 using the Basicdatalogger2. It works perfectly! But I now want summarize the daily logs.

In the Basicdatalogger or ChartWithLogging or Datasubscriber would there be an easy way to record just that days Min, Average & Max Values to a daily Log file for Each PLCAddressItem. I have about 10 monitored items.

I am currently testing and recording using BasicDataLogger2 writing a daily log file but then having to post process to summarize the data for min, Avg & Max Values separately for each PLCAddressItem using spreadsheet.

Any Help much appreciated!

Thank You in advance!

Hugh

DavidSr

  • Full Member
  • ***
  • Posts: 170
    • View Profile
I am not the best person to answer that regarding the HMI portion of this, Archie is,  but I do what you are asking for in the PLC, pretty easy to store all the data and then use plc instructions for Averaging and finding min / max values. You don't need the chart for that but you could chart the results of the the PLC instructions .
What PLC are you using?
« Last Edit: March 12, 2019, 07:26:33 PM by DavidSr »
David

Stumpy

  • Newbie
  • *
  • Posts: 19
    • View Profile
Hi DadvidSr

Thank you for your reply, sadly in this instance I do not have access to the PLC for programming, its located remotely.
Also the internet link is very slow, so realistically I can compile and then upload an AHMI project (taking  about 3 or 4Hrs to upload). Note this not a mission critical app, just something to take the chore out of post processing data! Although the daily log details give great insight of workings, I now really only want the Min, Avg & Max readings cutting the current daily log points down from 2400 to 30. This over the years should save disk space!

I am using opcda driver and its an ABB 800xA. I can read the OPC data no problem and my current append to daily log file gives some robustness regarding history for previous values that day should a power outage occur. A ups is in place to cover brown outs but some times longer unplanned outages occur (ie storms affecting power grid) and a loss of log data until grid power is restored.

Cheers

Hugh

Godra

  • Hero Member
  • *****
  • Posts: 1447
    • View Profile
Hugh,

read carefully this part of DavidSr's post: "... store all the data and then use plc instructions for Averaging and finding min / max values ..."

How would a BasicDataLogger2 go about doing all the post processing without storing all the required data first?

You might be able to create a code to do this for you by reading the values from the log file and then processing them, similar to this:

https://docs.microsoft.com/en-us/dotnet/visual-basic/programming-guide/concepts/linq/aggregation-operations

or this:

https://www.dotnetperls.com/math-max-min-vbnet

This is just my view of this.
« Last Edit: March 13, 2019, 08:07:12 PM by Godra »

Phrog30

  • Guest
Use a database.  There are plenty of free ones, with tutorials.  Search MySQL, or SQLite, my two favorites.

The apps I have referenced in the past have examples.  If you want more info let me know.

Stumpy

  • Newbie
  • *
  • Posts: 19
    • View Profile
Thank you one and all for your time and the feedback... It opens ones eyes to options!

Godra: Great links to coded examples I will try these out on my saved data file(s)...

Phrog30: I do also like the option going forward using a database. So Yes More Info PLEASE! ....


I will attempt both options

Thanks Again

Regards

Hugh

DavidSr

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Always glad to help Hugh .

While the other solutions / options suggested will probably do what you want, and were offered because you requested them, and both of those guys are offering you good options but you are headed down a path that is far more complicated then just using the PLC to store and process the data you want. It's already there, all you have to do is copy each change in what ever interval you want to other word addresses and let the PLC do the processing. Then it is easy to display it on the HMI and chart the processed results if you want.


MySql server  is a great server, I have several  running but setting that up to do what you want seems like an overkill.

I understand you have a slow internet speed but once it is programmed I think it will be far more reliable then the SQL or any other PC based solution.

If you want to try to do it in the PLC, Perhaps you can get someone local to the PLC to set it up for you. If you need PLC programming help check out https://plctalk.net

Whatever you decide in the end there are people willing to help you especially since you are trying to help yourself.  Let us know how it goes. I wish you all the best.
David

Stumpy

  • Newbie
  • *
  • Posts: 19
    • View Profile
Actually David,
 You have a very valid point. I had not thought of your solution because the system is remotely located. However your last comment sparked a light bulb moment. I do have the option of running a "Soft Controller" (simulation on PC/Laptop) that I could load up the existing PLC Program and then develop to cater for my data recording requirements. Then carry out the necessary proving tests before loading up during a future site visit. I think that would work :)

So I guess I should update my previous statement...  I will attempt ALL Three options. It should keep me busy for a while.


Thanks Again for the input!

Hugh

Phrog30

  • Guest
Here is a sample app with database stuff,
https://www.advancedhmi.com/forum/index.php?topic=2323.msg13814#msg13814

I don't think it's overkill to use a database. I'm no expert, but I've dabbled enough that's it's the quickest and easiest way to store and retrieve data. If you are new, then it might be a challenge. The biggest benefit to me is the easy way to query data.

DavidSr

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Here is a sample app with database stuff,
https://www.advancedhmi.com/forum/index.php?topic=2323.msg13814#msg13814

I don't think it's overkill to use a database. I'm no expert, but I've dabbled enough that's it's the quickest and easiest way to store and retrieve data. If you are new, then it might be a challenge. The biggest benefit to me is the easy way to query data.
James I would agree if he wanted to store and retrieve a lot of data, I have done a lot of MySql, including running several websites running on it with PHP front ends,  it is really fast and if it's run on Linux very stable  but he only wants 3 items. That is why I said it seems to be an overkill. Doing math with Sql is more complicated then doing it in the PLC also.

 You cannot out-run a PLC with Sql, you have to retrieve, process,  store and then retrieve the processed data then interface it with the HMI with some code. The PLC can do it in milliseconds with every scan and display it on the HMI as fast as the comm link is, No Queries required.  I am currently reading, several thousand data points,  sorting, averaging and getting min / max  and displaying it on the AVHMI, longest scan is only 40 ms when I let it all run. I am charting them and could put the results on one of my  MySQL servers but I don't have a need for that. Another advantage of doing it in the PLC is once you have that information you can use it for process control if you need it.

If you really want to put it in a SQL database, do it after the PLC has processed it. Anyway that is how I see it, Many ways to skin it none are wrong,  some are more efficient, really depends on what he wants to do with the results.
« Last Edit: March 15, 2019, 12:42:48 AM by DavidSr »
David

Stumpy

  • Newbie
  • *
  • Posts: 19
    • View Profile
Forgive my lack of clarity moving forward just had to pause for thought. For me all three options present valid solutions with slightly different learning opportunities. Initially I did want the easiest/quickest solution to summarize my data but as with most things once you get a glimpse at the possibilities:- ideas form, requirements evolve/change, and so I now also want to use this to learn and gain experience allowing me to better understand and implement future system applications as they are needed.
 
I do find great comfort in the fact that as you have all demonstrated here that help is at hand when needed... So I hope its OK to pester you for more info, as I would like to learn/explore more regarding solutions using Databases and PHP with website front ends, (kind of an ultimate longer term goal in the back of my mind). Perhaps first I need to digest and solve the 3 solutions to the original query, helping bring me up to speed and use it as a stepping stone moving forward ie eventual goal of database solutions with web based front end.... of which I can only dream about from my starting point now!  As the saying goes on this side of the pond "Rome was not built in one day", but with continued help/support from this excellent forum success seems to be within my grasp which I see as a great starting point! :)

Kind Regards

Hugh

DavidSr

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Hugh I won't say what you "Should Do", that would be presumptuous of me, and There are guys that know a whole lot more than I do but I will say how "I" would do it based on what I know and understand if I wanted to go down the path you laid out.

Start with storing the data in words in the PLC

Process the data in the PLC
 You said you wanted Avg and Min Max so the processed data is all you need, 3 values at what ever interval you want to record them.
Now you can use the Advanced HMI to Chart/Log just those 3 values.  Since you only want the min max and avg you don't need to grab all the values you stored in the PLC  to get those. If you post process in a DB you will need to grab all of it.

Now you can display the 3 different words ( For each of the 10 items you want)  on the HMI, if you want, You can chart with out displaying it but it sounds like you wanted to see it in real time also. The word  that has the MIN., The word  that has the Max and The word  that has the Average value. You can use a digital or analog Panel meter for that .

One project I am using this method for is storing wind speed every second for 15 minutes using a first in first out then averaging the 900 words to get the Avg Wind Speed for the previous 15 minuets. It runs continuously so it is updating the avg every second.  I show real time wind speed on the HMI and the Avg. I also Grab the Temp from a RTD connected to an IR module. I continually read the Temp on the HMI.
I chart the values  of both avg wind speed and temp with a time interval in the HMI of 900,000  ( That's 15 minutes.) (15 x 60 ) / .001

Now I could put that data in a db but I am not interested it doing that.  I am running 12 different set ups like that on one obsolete PLC 5 with varying file lengths. Several of them storing, averaging, and totaling data over a 6 month period but running continuously. That's a lot of Data and the PLC does it in less than a Second every scan.


So That is the simplest way to get and see the data you want using the same method.

For your project, you can set up a chart in the HMI, for the 3 values you want, 1 chart for each item with the 3 values.  Depends on what you want.
Now you have the data you want - and you can use Open Office, Libre Office or Excell to view the spread sheets and create other charts .
You can also export the sheets to A database in one of those office programs. No SQL needed, Open Office and Libre Office both have database programs and they are open source. see https://openoffice.org


This way you have a feel for it and altogether it would not take much time at  to get all that with the database in your office program.That is not the best Database solution and I wouldn't use it if I wanted a long term solution but it will get you started and  It is very easy to set up a simple db in those office programs but  it is not a good long term solution for a DB doing what you want.


Now if you want to continue, with a robust DB then do as James suggested and set up A SQL, DB for it. You can set it up on the same machine you are developing AHMI on, And depending on where you are running the run-time of ADVHMI you can run it on that as well.

The great thing about MySql is it is super fast and you can run a lot of separate DBs on it on the same box and set up relational DBs so you can get data from more then one DB to analyze together, That also saves on the number of records needed because you can share a set of records across databases. You need to know a lot more about setting up a SQL DB then you do in an office Program DB though.

Have Fun!
« Last Edit: March 16, 2019, 11:53:39 AM by DavidSr »
David

DavidSr

  • Full Member
  • ***
  • Posts: 170
    • View Profile
Wind speed capture, store and average. Max is maximum ever and is not computed over a time period, You can do the same with minimum but that is always going to be a zero in this case so no need. If you want to get MIN/MAX for a period you have to  write logic for that and you can read it from the same file and place the results in another word.
There maybe better, more compact ways to do this but this is what I am doing. The real time value is placed in the last word that is not seen in this graphic.  All the data is moving from the bottom towards the top. The oldest data is to the right of where I show the average in the graphic.  You can use a FLL (File fill instruction) instead of copy for integers but I started doing this with Floating point values for other things so I just use the same method for clarity. You cannot use the FLL instruction for floating point values in PLC 5.

All the other numbers you see are the captured wind speed values each second. 


It is usually better to use a higher resolution for the timer time base such as .01 or .001 but I did not need it and to get 15 minutes the data file would have had to be much larger.
« Last Edit: March 16, 2019, 12:12:07 PM by DavidSr »
David

Stumpy

  • Newbie
  • *
  • Posts: 19
    • View Profile
Can I just say "WOW" A lot of hard graft (work) and effort has gone in to that project, especially with attention to detail. Its fantastic! I think it would take me a few years to code something similar! :(  After seeing that I am just jealous ;) but inspired at what is possible.
I can see how you have collated the 900 words of data, smart move for the hi resolution over the 15mins. Also I am very impressed with the Weather Station Graphics! Not to digress too far from the main topic but what I am interested in what kit is used in particular the wind direction and its granularity/resolution? plus is it analogue input?
It looks like the PLC I will be using has a built in "resource/overhead" graph as part of its download so I can hopefully avoid over burdening the processor or memory which is nice since I know my coding initially wont be efficient!

Hugh

DavidSr

  • Full Member
  • ***
  • Posts: 170
    • View Profile
You are giving me way too much credit.

  It would take you all of 30 minutes to do all of it. That is why I said use a PLC for the data processing, not a database. The graphics are just stock Advanced HMI stuff.
It's late for me, I will answer more fully how I get the data next week. It's simple if you have the right stuff.  The weather station is really a very small part of my HMI screen. I have it on a 25 inch monitor so there  is a lot more I did not show.


Can I just say "WOW" A lot of hard graft (work) and effort has gone in to that project, especially with attention to detail. Its fantastic! I think it would take me a few years to code something similar! :(  After seeing that I am just jealous ;) but inspired at what is possible.
I can see how you have collated the 900 words of data, smart move for the hi resolution over the 15mins. Also I am very impressed with the Weather Station Graphics! Not to digress too far from the main topic but what I am interested in what kit is used in particular the wind direction and its granularity/resolution? plus is it analogue input?
It looks like the PLC I will be using has a built in "resource/overhead" graph as part of its download so I can hopefully avoid over burdening the processor or memory which is nice since I know my coding initially wont be efficient!

Hugh
« Last Edit: March 16, 2019, 09:33:01 PM by DavidSr »
David