My project needs to save a large amount of historical data from sensors, and the number of them is large.
Although we have eliminated part of the data by the anti-jitter and anti-duplication, but becaues of my project needs to keeping running for a long time, so it still will generates a lot of records.
I used tried SQL Server and MySQL to save these records, and divide the data into tables by sensor(source) or time (Daily, Monthly).
(by sensor) his_data_Dev1 his_data_Dev2 his_data_Dev3 (by time(daily)) his_Data_20210301 his_Data_20210302 his_Data_20210303 (by time(monthly)) his_Data_202101 his_Data_202102
example data:(no device column in divide by device)
But they are not easy to query and still have performance bottlenecks.
Currently looking for a better solution, which can better save large amounts of data (records for several years), and faster query speed (especially aggregate computing, we need to aggregate and calculate the historical values of a each device in units of years.).
The solution accepts to change to another database – but at least it should support .NET Core access.