Combine Sets

This combines a time series of sets into a single set. Best explained with a slightly contrived example. Consider a set that has 31 values, just one time value.

Use the filter action to extract a portion that changes in time. I just want extract the entries that have the same number after the decimal. One way to do that is using the expression abs(mod(v,1)-T)<0.001.

The logical expression puts the current value into the variable ‘v’ and then evaluates the statement to see if it is true or false. It is true if the value is ≠0, false if it is 0. One important trick here is that it is not going to work to do mod(v,1)==T as you might assume. The reason is that 0.1 is actually an infinite fraction in binary, just like 1/3 is an infinite fraction in decimals. That means that you will run into round-off. To get around that we make sure that the difference is small in absolute value. The value 0.001 just just small enough.

If you look at the lower right corner I’m looking at the valute at 0.2. At t=0.3 the values will be 0.3,1.3,3.3 etc.

Now combine the time values of the ‘Filter’ variable. The result is

The Set variable is disk based, except for the parameter table. When you combine the time values this continuous to be the case. In this particular example all of the time values share the same data file, since they came from the same object originally. In more realistic examples each time value of the input Set uses a different file to store the data. When you combine the time values, the output will reference all of the files.

A key of the design for the Set variable is to make it really efficient to send a large data set to an external program. If you look at the DTSet class template in the C++ code you can see how the set type is structured and how you can read the data from the underlying file. This references the same files as ImageTank uses, so no copying is needed. The external program just reads in what it needs, so it can easily handle many gigabytes.