Silly timestamp precision problem

[tom lawtonParticipant]

Hi-

I’ve got some data that has been saved with one less digit of precision on its timestamp than it needed… So there are two consecutive rows with the same value. Easy enough, fixed with a function column “time+mod(#-1,2)*0.0005”.

Except. The data has been pre-cleaned to throw out bad rows, so there are sometimes jumps in the timestamp. So it’s not always the even rows that need the 0.0005 adding… 😀

Soo… what’s the smartest way to fix this?! I was thinking to check the value of the previous row, and if the same, add the required increment; but I’m never quite sure how- or if it’s even possible- in DataGraph…?

Thanks for any tips!

Tom

[dgteamModerator]

Hi Tom,

Here’s a suggestion: Use the .delta column property to identify when the time in two consecutive rows did not change. When the value doesn’t change, you get a zero, and if the value does change, you get a one. Then, you can have a column that uses the delta column to determine when to add the additional time increment.

Here, we made up some data to mimic your situation, where some columns have repeated times values, and two locations don’t have this issue.

Plotting the original data on the X-axis versus the row number looks like this.

Plotting the corrected column now gives you a unique point at every time value.

 

If you wanted to see where you had missing data, you could do a check for that too.

The ‘skip’ column looks a little messy, as it is returning “invalid date” in some rows, but you can select this column to plot with a Lines command.  Here’s what everything looks like plotted together.

If you had access to the pre-cleaned data, perhaps another approach would be to use the raw data without removing rows before importing. Then you can mask and filter the data within DataGraph 🙂

In any case hope this helps and gives you some ideas!

 

[tom lawtonParticipant]

Use the .delta column property to identify when the time in two consecutive rows did not change.

Perfect! That is exactly what was needed- Now, to try to remember it exists for next time this happens! I guess I didn’t find it as it doesn’t appear in the Function Reference (I don’t think?)- and I didn’t look for (think of?| know about?!) Column Properties…

Thanks so much,

Tom

[Author]

Posts