Database update / WU / failed cron-job

[dtwrlewis]

Having intermittent problems here http://www.dtwrlewis.com/radlett/indexDesktop.php. Database updates via WU IENGLAND417 - updated the WU APIKey - updated Meteotemplate to 18. It worked for 12 hrs then failed today. Strangely, even tho cron-job history says it failed at each attempt today - the home page seems to be occasionally updating with current data. Any ideas

Screen Shot 2021-11-07 at 11.07.34 AM.png (342.9 KiB) Viewed 11516 times

[dtwrlewis]

JUst an update on this - my template stopped updating at 8.30 local time. All subsequent cron-job report fail. However the current conditions block is somehow still working. Wondering if I need to do a fresh instal? This is for my dad's weather station. Mine is on the same server ..../rothesay and updates via weathercat api - no problems

[davidefa]

From your cache/updateLog.txt e cache/updateLog.txt I see that your cron job is sometime working ( 17:40, 21:30 and 21:46 ).
On my test pc I have no problem retrieving your data from wu using your station id and api key.
So I think the problem lies ( somehow ) in the cron job.
To verify this try calling your update.php 'manually':
- from your browser go to the page: http://www.dtwrlewis.com/radlett/update ... sword=xxxx ( substitute xxxx with your update password ), this should not timeout
- in cache/apiLog.txt you should see a long log starting with something similar to this

Handling data from wu
update 'SW' = wu
update 'U' = 1636407011 (2021-11-08 21:30:11)
update 'T' = 10.5
update 'H' = 97
update 'P' = 1020.32
update 'W' = 0
update 'G' = 8
update 'R' = 0.25
update 'RR' = 0
update 'B' = 173
update 'S' =

- wait 5 minutes repeat the above points a couple times
If everything works as expected try disabling the original cron job and set a new one ( you could use also a free cron job from https://cron-job.org/en/ just for testing )

Even if wu is now working ok, we don't know if this morning wu had 'a few hiccups' ( not very difficult to believe ) and the cron job gave up after repeated errors (check if you cron job has a detailed log of the failures )

[dtwrlewis]

Thanks davidefa. When I call the update.php manually it takes forever (2-4mins) to result in "success' page.

It seem to be a problem with the update.php

Handling data from wu

update 'SW' = wu

update 'U' = 1636668612 (2021-11-11 22:10:12)

update 'T' = 8

update 'H' = 96

update 'P' = 1017.27

update 'W' = 0

update 'G' = 0

update 'R' = 0

update 'RR' = 0

update 'B' = 147

update 'S' =

Begin of common part of api script

Start handling live data

Checking if extra sensors should be logged

Extra sensor settings file not found.

Extra sensors data to save in db:

add live data: S:

add live data: D: 7.5

add live data: DTime: 1636668568

add live data: A: 7.5

add live data: ATime: 1636668568

calculated D: 7.4

calculated A: 7.4

Save meteotemplateLive.txt: {"SW":"wu","U":1636668612,"T":8,"H":96,"P":1017.26999999999998181010596454143524169921875,"W":0,"G":0,"R":0,"RR":0,"B":147,"S":null,"SWTime":1636668961,"UTime":1636668961,"TTime":1636668961,"HTime":1636668961,"PTime":1636668961,"WTime":1636668961,"GTime":1636668961,"RTime":1636668961,"RRTime":1636668961,"BTime":1636668961,"STime":1636668961,"D":7.4000000000000003552713678800500929355621337890625,"DTime":1636668961,"A":7.4000000000000003552713678800500929355621337890625,"ATime":1636668961}

[davidefa]

To verify what is taking so long yo can substitute your update/update.php script with the attached one ( make a backup of the original script ), only added a few extra print to show the timing.
The script tries to retrieve data from wu in two different ways, maybe the call to file_get_contents on line 246 times out.
Call the script 'manually' as you already did, at the end you can find an extended log in cache/updateLog.txt.

[dtwrlewis]

Thank you again davidefa.

Current date/time: 2021-11-13 14:46:33
Loading data from https://api.weather.com/v2/pws/observat ... on=decimal
Calling file_get_contents: 2021-11-13 14:46:33
Exiting file_get_contents:2021-11-13 14:47:33
Calling curlMain: 2021-11-13 14:47:33
Exiting curlMain: 2021-11-13 14:47:33
Loaded data items from WU = {"observations":[{"stationID":"IENGLAND417","obsTimeUtc":"2021-11-13T14:45:34Z","obsTimeLocal":"2021-11-13 14:45:34","neighborhood":"Radlett","softwareType":"weatherlink.com 1.10","country":"GB","solarRadiation":null,"lon":-0.318778,"realtimeFrequency":null,"epoch":1636814734,"lat":51.677654,"uv":null,"winddir":329,"humidity":82.0,"qcStatus":1,"metric":{"temp":11.8,"heatIndex":11.8,"dewpt":8.9,"windChill":11.8,"windSpeed":1.6,"windGust":12.9,"pressure":1014.56,"precipRate":0.00,"precipTotal":0.00,"elev":100.0}}]}
Calling api.php date/time: 2021-11-13 14:47:33
Success
Exiting api.php date/time: 2021-11-13 14:47:33

updateLog.txt.zip

(1.18 KiB) Downloaded 602 times

[davidefa]

It seems that the data are updated regularly now.
I removed the file_get_contents that times out after 1 min ( data are retrieved with curl ).
You can now revert your cron job to the original one every minute.

I uploaded a test file ( www.dtwrlewis.com/radlett/update/test0.php ) which uses file_get_content to retrieve the content of 2 sites:
- the first call result in a incomplete page
- the second call times out

For some reason this command file_get_contents is not working properly in your site ( you may contact your provider for a clue )
This means that blocks and plugins that use this command to access external sites do not ( or may not ) work properly.
For example the forecast block does not retrieves yr.no forecasts.
( but these blocks/plugins can be modified to work in your site )

[dtwrlewis]

Thanks davidefa

My main concern is the station data. I can live without the external block for now. I really appreciate your help with solving the data update issue.

[dtwrlewis]

PS - Noticed your site location. We have a small vacation property in a village called Farnocchia. We used to have a Vantage Vue running remotely over a few years. https://www.wunderground.com/dashboard/ ... 1-16/daily But the power supply and internet connection was so unreliable that we shut down the feed 2 years ago. Looking forward visiting Italy next year (Pandemic permitting!)

[davidefa]

I can live without the external block for now.

Sorry, what do you mean by 'external block'?

[dtwrlewis]

Blocks that access external sites e.g the yr.no forecast