E. Updates Setup

In this part of the installation process you will set up a way of regular updates of the database. There are several options and the one that is the best for you will depend mostly on the way you get your data from your station.

You will need to specify how you want to do the updates - which file to use, format of it etc. If you are using one of the most common SW types like Weather Display or Cumulus, it will be very easy. If you want to read a bit more about how the data is actually handeled by the scripts click the below link.

Data caching and logging

The recommended interval of data in the database is 5 minutes. Some of you might be logging data at 1 minute interval. The problem with using 1 minute interval is not technical and theoretically there is nothing that would prevent you from doing this (and it can easily be configured in the scripts), however, it is not recommended because it will generate quite a lot of data and slow down the overall behavior of your site and template.

On the other hand, it is nice to have data at shorter intervals and make sure you don't miss any extremes. To solve this problem, the template uses a method which is a compromise between database size and interval length. This method is based on the fact that it only makes sense to log some parameters often. For example, it is obviously useful to log wind speed and temperature often, however, loging for example pressure or humidity at 1-minute intervals is useless and would only generate mostly duplicate values. So how does the script work?

The script uses the following method. In an ideal situation, you set your weather software to upload your current conditions as a textfile (in case of Cumulus it is the realtime.txt, in case of WD the clientraw.txt etc.) at some shorter interval, for example 1 minute. Then, when setting up the uploads, you run the upload script every minute (or as often as the txt file changes). However, in the script configuration you set you want your database interval to be 5 minutes.

What then happens is this:

The script is executed every one minute, it reads the txt file with current conditions and saves this information. It then checks the last time your database was updated. If it is less than your specified interval (ie. in this example 5 minutes), it will not perform an update, but leave the values saved in temporary cache file. After another minute, the script is executed again and it will again look at the values of the current conditions text file. It will then add these values to the temporary file again and check the database. This procedure continues until you reach a time, when the last database record was created at a time in the past, longer than your set interval of database update - in this example, the process will continue, data will be added to the temporary cache file every minute, until the script detects that the interval between last update and now is longer than 5 mintues. At that moment, it will perform database update. This update is then done by taking into account all those temporarly saved values.

In particular, the following method is used:

temperature - the database actually contains 3 columns for temperature values - max, min and average. So the script takes all the measured temperature values, finds minimum, maximum and average of them and saves them into particular columns.
humidity - average value is saved
pressure - average value is saved
wind speed - average value is saved
wind gust - maximum value is saved
precipitation - this parameter is a daily sum so it always takes the last value, which in fact is also the maximum
solar radiation - max value is saved
rain rate - average value is saved

This way, for example with regards to temperature, even though you are saving data to database only every 5 minutes, you get the max and min from the shorter 1-minute intervals. At the same time, for other parameters such as humidity or pressure, you get the average of all those temporary values, which however shouldn´t make much of a difference because these do not change very often. And for some parameters logging in shorter intervals would be useless - in this case precipitation, because it is a daily sum.

In version 12, a new method of updates was introduced - an API. The major advantage of this is that this API is now built into several popular SW, such as Weather Display, Meteobridge, WeatherCat or WeeWx. If you are using one of these, you should see a setup dialog for Meteotemplate directly in your SW. You will only have to specify the URL of the api script and your update password you specified in the Main settings. The URL of the API script is simply the URL of your template folder and api.php. The api.php is in the template root folder. You then set the interval, the database will be updated every 5 minutes, but you can set the interval of calling the API to something shorter. The API script will cache the data and update the database every 5 minutes using aggregated data from the cache file. This is also advantageous because upon each trigger of the API script, a file of current conditions is saved. In some blocks/plugins you can then use real-time data, which will not use the 5 minute data from database (used for general statistics etc.), but rather take data directly from this real time file. In case you use the API, you do not need to do anything else and you can ignore the update setup and CRON setup explained below.

Here is a list of SW currently supporting the API, make sure you are using the latest version of that SW, which has Meteotemplate built-in. If you do this, you can then ignore the rest of this page and you don't need any CRON jobs etc. The API is by far the most reliable, fastest, most accurate, easiest to use method and it will allow you to use all the possible data your station sends.
List of SW and its support of the API:

Weather Display - full support of the API
Meteobridge - full support of the API
WeeWx - full support of the API with the installed WeeWX-Meteotemplate uploader service found here
WeatherCat - full support of the API
Cumulus - no support

If your SW is not on the list above, you can email me and the developer as well, in the menu on the left you have a link to the API, which the developer can use to integrate Meteotemplate into their software.

To use the API you will only need to provide the API URL - the api.php script is in the root folder of your template, the same directory as the index files. So the URL is simply ..../api.php. Last thing you will need is the update password, which is what you specified in the Main settings.

In case for some reason you still want to use the previous way of getting updates vie Weather Display, Weather Cat, Meteobridge (using the clientraw.txt etc.), follow the instructions below and use a CRON job, however it is highly recommended to use the API if this option is available in the SW you use.

Go to your Control Panel and click the Database Update setup link in the Database section. Then follow the instructions on the page. The possible options for database update are:

API - use with Weather Display, WeatherCat, Meteobridge, WeeWx or if you create a script yourself following the syntax of the API (availbale in this wiki as well). In this case you only tell the template to use the API and in the corresponding software you set up Meteotemplate updates. The URL is the url of the api.php script located in your template root folder (.../api.php) and update password is your update password provided in the Main setup.
Cumulus - using realtime.txt, you will need a CRON job as well
Meteobridge - using Meteobridge built in HTTP request function to the API
Weather Underground
Custom text file - this pretty much works for files from any software, you can specify exactly the format of it and do the updates based on that
NetAtmo - this will use the NetAtmo API. For this you will need an additional Plugin, go to Plugins section at meteotemplate.com and download the NetAtmo plugin and follow the instructions in the plugin documentation.
WeatherLink - this will use data directly from wl.com. For this you will need an additional Plugin, go to Plugins section at meteotemplate.com and download the WL-IP plugin and follow the instructions in the plugin documentation.
WView - let WView create a text file with the current conditions - you can decide your own format - then use the Custom text file option on the Database Updates setup page
WANET - this will use data directly from WANET API. For this you will need an additional Plugin, go to Plugins section at meteotemplate.com and download the WANET plugin and follow the instructions in the plugin documentation.

The updates are performed by PHP scripts that are located in the update folder of your template. IF YOU ARE USING THE API, YOU SET THE API URL FOR THE api.php script on your server AND YOU DO NOT NEED ANY CRON JOB. Other update options (custom file, Cumulus, WU, NetAtmo, WeatherLink etc.) will require a CRON job for the "update/update.php" page. Probably the easiest method is using a so-called CRON JOB. If you are familiar with what a CRON job is and have the possibility to create one, then this option will be the easiest. If however, you do not know what a CRON job is or cannot set it up on your server, then you can choose some of the alternative methods below. CRON jobs are also available at some free online sites (see CRON jobs below).

Setting up CRON

Please note: If you are using the built-in Meteotemplate update feature in Weather Display, Meteobridge, WeeWx uploaded or WeatherCat, you do NOT need a CRON job, the software will handle this for you and you can skip this section.

A CRON job allows execution of a script at defined time intervals (or time of the day etc.). If you are using your own server then you can usually set it up yourself. If you are using external webhosting, then there usually is an option to add a CRON job too, but the number of these and the time intervals will vary.

For the database, the interval of database updates is 5 minutes, therefore you need to make sure that this is possible with your CRON and in fact if you use a shorter interval you will get even more accurate results. All you then have to do is specify the URL of the update script (which will depend on which software you use) and the interval.

The best and most reliable way is setting up CRON job on your own server or via your webhosting provider (note: some servers do not accept the question mark symbol in the url which specifies your password, in such case replace the question mark with a space - update.php password=...). If however, you do not have your own server, your webhosting does not provide this or the interval is too long, then you can try using external webhosting service, I have found a great site, which offers free CRON jobs at 1 minute intervals! Just go to https://cron-job.org/en/, register and then follow the instructions below to set this up. You might find other services offering CRON jobs as well and it does not matter which one you use as long as it works and supports as short intervals as possible. Another possibility is MyWebCron, which is also free and offers minutely CRON jobs.

Extra Sensors

In version 15, support for logging data from extra sensors was made possible. To do this, you first need to make sure your page is being updated. Whenever an update is triggered, the API file named "meteotemplateLive.txt" is created in the root of your template. This is always the best way to check your template is being updated.

If you are using one of the softwares with built-in support (WD, WC, WeeWX, Meteobridge), they should automatically detect which sensors are available and based on that send data to the API. If you are using other update types (Cumulus, WeatherLink, etc.), the corresponding plugins will also automatically detect what is available. If you have some extra sensors available, you can set up their logging to an extra table called "alldataExtra". To do this you first need to go to your Meteotemplate Control panel (the page which you see after you login as admin, easiest way to get there is to click the admin icon in the footer on the right or if you are already logged in as admin, use the Admin tab in the menu (the one on the very right) and click Control panel). On the left, second from the top, you will see a link "Extra sensors". When you click this link you should see a list of all the possible parameters the API/template supports. Some of them are already being logged by the default alldata table. For the rest, if the template detects an extra sensor data available in your meteotemplateLive.txt, it will give you the option to enable its logging in the extra database table. By default all options are off. When you hit Save, the extra table will be created with the corresponding columns and the API update script will start automatically logging data to this extra table as well. You can of course change this in the future if for example you get a new sensor or a previously available sensor no longer is available. Whatever is enabled here is logged in the extra table.