Member-only story
Windowed/delta grouping of time series data in MySQL
I ran into an odd problem in MySQL recently that I didn’t have a good solution for. I have time series data where a recommendation is posted each day, but I want the “minimum date” when a change occurred. Imagine a log that tells you the state of the system each day, but it doesn’t track what changed. You need to only show deltas in your output, or do something different when something changes (or notify a user to review something). This way, if multiple events happened in a row, I just want that first date/time.
In my production system, if I see a change happened and I marked it “reviewed”, I didn’t want a new state record with the same value to force me to review it again. I just want that notification if it’s changed since my last review. So, it would seem like “any new recommendation after my last review” is all I need to check — compare two dates. But the problem is bigger than that, because I couldn’t reliably figure out if the state AFTER my review was still the same as it was BEFORE. So I need some kind of query on the state that shows me the minimum date of the state after it changed.
Here’s a dataset that captures the problem. My real dataset involves a number of fields, but they can be effectively lumped together like this test. We have a value that can change, and a datetime:
create database db_test_window_grouping;create table test_windowed_grouping (
recordentry varchar(40) not null,
dated datetime,
primary key…