Page 1 of 1

Inconsistencies in the data !!HELP!!

Posted: Thu Sep 30, 2021 4:54 pm
by amonphi
Hi, can anyone explain to me or noticed inconsistencies in the data? I explain to me .....

I have a weather station Ecowitt GW1003, if I compare any reading at a certain date and time, between Ecowitt.net server and stored data in my database on Altervista, in my template they are completely different; I attach a summary photo where you can see better; I inserted a different day for the rain (for obvious reasons).

My Cronjob for Update takes every 5 minutes, in fact I noticed that the data I see on the Ecowitt server, I'll see it 5 minutes later in the Altervista database. Do I need to reduce the minutes for the update from 5 to 1 or 2? What does it mean? Slowdowns or Blocks?

Did I mean a bad value in the Units database, and / or limits in Meteotemplate?

Thank you all
Riepilogo confronto dati.jpg
Riepilogo confronto dati.jpg (159.77 KiB) Viewed 5257 times
04.jpg
04.jpg (301.79 KiB) Viewed 5257 times

Re: Inconsistencies in the data !!HELP!!

Posted: Sat Oct 02, 2021 3:12 pm
by davidefa
You have to remove the cron job that calls update.php every 5 minutes.
It is not necessary as you are using the ecowitt plugin ( this will influence only slightly the mean value calculated and saved every 5 minutes in the db ).
If you take a look at your cache/apiCache.txt ( where api.php saves the values before writing to the database )
you can see there is an 'intruder' ( a value out of sequence ), in the field 'date' the value "2021-10-02 16:10:17"

Code: Select all

{"timestamp":["1633183809",1633183817,"1633183870","1633183933"],"date":["2021-10-02 16:10:09","2021-10-02 16:10:17","2021-10-02 16:11:10","2021-10-02 16:12:13"],"T":["22.11","22.1","22.11","22.11"],"H":["57","57","57","57"],"P":["1018.47","1018.59","1018.47","1018.47"],"W":["1.8","0","0","1.8"],"G":["8.64","8.6","8.64","2.16"],"B":["105","99","99","254"],"R":["0","0","0","0"],"RR":["0","0","0","0"],"S":["171.27","1 72.5","172.45","166.38"],"D":[13.2,13.2,13.2,13.2],"A":[22.8,23.1,23.1,22.8],"UV":["1.0","1.0","1.0","1.0"]}
When meteotemplate saves a new record in the db calculates the last 5 minutes mean values but uses the actual time as 'timestamp'
Using the previously reported data meteotemplate will use the values received at: "2021-10-02 16:10:09", "2021-10-02 16:10:17", "2021-10-02 16:11:10", "2021-10-02 16:12:13", "2021-10-02 16:13:14", "2021-10-02 16:14:15" and save the calculated mean value with a timestamp of "2021-10-02 16:15:00".
Not sure if ecowitt.net does the same or uses the timestamp of the first sample ( this could account for the 5 minutes shift )

If you enable the csv data saving ( in the ecowitt plugin ) you could use a spreadsheet to check meteotemplate and ecowitt.net calculations.

P.S.
If you want an update every exactly every 60 seconds you should set ( in WsView ) the custom update interval to 59 seconds ( see wxforum )

Re: Inconsistencies in the data !!HELP!!

Posted: Sat Oct 02, 2021 6:46 pm
by amonphi
davidefa wrote: Sat Oct 02, 2021 3:12 pm If you enable the csv data saving ( in the ecowitt plugin ) you could use a spreadsheet to check meteotemplate and ecowitt.net calculations.
Hi Davidefa,
First of all, thanks as always for the explanations and very useful info, I think I understand, both how the average calculation of the values is made, both the explanation in wxforum (change that I have made as described and that you anticipated me).

I disabled from cron-job.org the cronjob that made the update every 5 minutes, and now in cache / apica.txt the 'intruder' in the field 'date' disappeared ("2021-10-02 20:00: 47 "," 2021-10-02 20:01:47 "," 2021-10-02 20:02:47 ").

As for CSV Data Saving I had enabled it in the plugin but, once opened with Excel (distributed the text of a single cell in separate columns to facilitate reading) It is not clear to me as a check Meteotemplate and ecowitt.net Calculations; The data in .csv with those in the altervista database that were busted?

Thanks again

Re: Inconsistencies in the data !!HELP!!

Posted: Sun Oct 03, 2021 12:37 pm
by amonphi
davidefa wrote: Sat Oct 02, 2021 3:12 pm You have to remove the cron job that calls update.php every 5 minutes.
It is not necessary as you are using the ecowitt plugin ( this will influence only slightly the mean value calculated and saved every 5 minutes in the db ).
If you take a look at your cache/apiCache.txt ( where api.php saves the values before writing to the database )
you can see there is an 'intruder' ( a value out of sequence ), in the field 'date' the value "2021-10-02 16:10:17"
I made a comparison and there are still discrepancies unfortunately at this point I think I have some settings to be reviewed surely (this also because the template first was with the PCE FWS-20 with Raspberry, only this year I switched to Ecowitt with GW1000 for the Loading data).

For example, in addition to disabling the CronJob, should I also change Database Update Setup? I realized now that I selected "Wu". If yes, what should I set up with GW1000?
Database update.jpg
Database update.jpg (205.56 KiB) Viewed 5229 times

Re: Inconsistencies in the data !!HELP!!

Posted: Sat Oct 09, 2021 9:25 pm
by davidefa
For example, in addition to disabling the CronJob, should I also change Database Update Setup? I realized now that I selected "Wu". If yes, what should I set up with GW1000?
No that does not matter ( if you don't use update.php ).
As for CSV Data Saving I had enabled it in the plugin but, once opened with Excel (distributed the text of a single cell in separate columns to facilitate reading) It is not clear to me as a check Meteotemplate and ecowitt.net Calculations; The data in .csv with those in the altervista database that were busted?
What do you mean by saying: ' those in the altervista database that were busted?'
Please show an example

Re: Inconsistencies in the data !!HELP!!

Posted: Sun Oct 10, 2021 12:02 pm
by amonphi
davidefa wrote: Sat Oct 09, 2021 9:25 pm
What do you mean by saying: ' those in the altervista database that were busted?'
Please show an example
Hi, writing with the translator unfortunately I explained myself wrongly.

I meant that, same data (from .csv plugin file, and those of the allata or alldataExtra database in the Altervista) on the same date and time, they are different.

The data in real time in ecowitt.net and those at http://meteopaparano.altervista.org/tem ... apiLog.txt, let's say that they match ... what made me think of some mistake, it's time In which I go to see those data, considering the fact that the template is the average of the last 5 minutes (as you explained to me) and in the last line of ecowitt.net, it is as if I remained back of 5 minutes. I attach the images (captured at the same moment naturally) so it is clearer.
Data_Weather_station.jpg
Data_Weather_station.jpg (212.01 KiB) Viewed 5176 times
Data_template_apiLog.jpg
Data_template_apiLog.jpg (43.67 KiB) Viewed 5176 times