Page 1 of 1

Too much data in the DB ?

Posted: Fri Jun 18, 2021 9:48 pm
by linux150
Hi!
I found a way to export my history data from Meteohub via WD-Export. Then i cumulated the rain with an formular in Excel and then i import all data. My problem now is, that my site is very slow.
On the serverside i did contact the helpdesk about the slow response, and he switched me to a faster one.
Now i think my database is the problem, because the export of WD create an entry for every minute!
So my datas are from 2009 and now i have about 6.000.000 !! Data entrys in my DB.
And so i think the slow response from my site is because the big sql DB ?!
And why do i post this in programming? Well i want to ask if theres any way in SQL to get less DB Entrys in calculating a average of, for example, 5 Minutes.
Means for me, select DB Entry from 00:00 to 00:05 (5 entrys) calculate an average and write on entry (maybe in another table).
I hope you know what i mean. I have the RAW WD CSV Files (for each month) also but not with the cumulated rain. This was also a lot of work, so i ask for a solution in SQL. Maybe calculating once more in Excel would be easier ...?

Sorry for my english, i hope u understand my problem ....

If you want test my site you find in my profil.

LG
Martin

Re: Too much data in the DB ?

Posted: Sat Jun 19, 2021 2:26 am
by John B
Hello, Martin,

This might help:
http://www.meteotemplate.com/web/wiki/wiki4GB.php

On the left side select Data Import (if it is not already). You might find what you are seeking there.

Good luck,
John

Re: Too much data in the DB ?

Posted: Sat Jun 19, 2021 8:08 pm
by linux150
Thanks, exactly what i need :-)
I did read most of the wiki but i think i was blind a little bit.
I will try and hope it will boost my site a little bit, because in the moment it is lame.

LG
Martin

Re: Too much data in the DB ?

Posted: Sun Jun 20, 2021 2:07 pm
by linux150
Automatic didn’t work, I did get an error from my host. Something about my .htaccess
But didn’t matter, I backup the alldata and renamed new to alldata.
All work fine now
Tnx agn
Martin

Re: Too much data in the DB ?

Posted: Sun Jun 20, 2021 7:12 pm
by John B
Yes, Martin, it seems a bit complicated, even though it is not. I found that reading through that part of the Wiki until I understood which parts needed were important. I was fortunate as it worked first time. :)

As I said last time, good luck.

John

Re: Too much data in the DB ?

Posted: Wed Aug 04, 2021 2:15 pm
by PNCATTE
Hello,

Following a problem with my host I lost the history of my database
As I had saved a data file in .CSV I wanted to reload it from the control panel mode in Data Import but it gives me an error code:
E 406 The browser does not accept the MIME type of the requested page.

do you have any idea how to load the data

Thanks in advance
erreur E  406.png
erreur E 406.png (140.35 KiB) Viewed 31059 times

Re: Too much data in the DB ?

Posted: Thu Aug 05, 2021 2:12 pm
by davidefa
I've tried your setup and works for me ( with firefox )

e406.png
e406.png (84.8 KiB) Viewed 31041 times
( both with and without leading 'www.' )


I've searched about error 406 and found that even if the error is 'client side' generated the source may be more difficult to pinpoint ( should be some 'incomprehension' between server and browser ).
I suggest the following:
- try using a different browser; even starting the browser in 'clean mode' ( without extension enabled ) may be worth trying
- try using a different computer/operating system
( if you don't have access to a different computer you may download a virtualizing software such as vmware, a virtual machine with linux already installed and repeat the import from within the virtual ambient )

Re: Too much data in the DB ?

Posted: Thu Aug 05, 2021 5:25 pm
by PNCATTE
Hello,

Thanks for this information, I tried with three other browsers and a MAC ditto
I can't find my problem if I launch the link:
http://fleurie-meteolive.fr/MTE/install/alldata2013.csv

I see downloading the data file

Can you try by accessing my site with the password: 12345 to see if it is not from a corrupted file on my server?

http://fleurie-meteolive.fr/MTE/indexDesktop.php


Thank you for your help
Pascal

Re: Too much data in the DB ?

Posted: Thu Aug 05, 2021 7:03 pm
by davidefa
The 12345 password does not work for me

Re: Too much data in the DB ?

Posted: Fri Aug 06, 2021 6:32 am
by PNCATTE
Hello,

For the URL you just have to note:
../install/alldata2013.csv

Thank you again for the information

Kind regards

Re: Too much data in the DB ?

Posted: Tue Aug 24, 2021 11:23 am
by johnathan
The app development process involves creating an app from scratch. The original idea for the idea may be to create a social networking application, but app development company https://mlsdev.com/blog/53-how-much-doe ... mobile-app can help you with a variety of different ideas. Some experts can help you design a game or a tool for social interaction, while others can help you build shopping carts or virtual walls for a real brick-and-mortar shop. Others can help you create the back end of your site or the front end, like web applications and email functionality. They can also help you design the interface and develop the look and feel of your web pages.