Insertion-filter for close samples

[tezc]

See https://redislabs.atlassian.net/browse/PM-1996

Insertion filter support

Many sensors report data periodically. Often, the difference between the measured value and the previous measured value is negligible and related to random noise or to measurement accuracy limitations. In such situations it may be preferable not to add the new measurement to the time series. This PR introduces new configuration parameters to the keys to ignore the current sample if the above scenario occurs.

Implementation:

• New module configuration arguments:

  IGNORE_MAX_TIME_DIFF - a non-negative integer [milliseconds]; default (hard coded) = 0
  IGNORE_MAX_VAL_DIFF - a non-negative floating-point; default (hard coded) = 0
  

• Defined two new per-key parameters:

  ignoreMaxTimeDiff - a non-negative integer [milliseconds]; default = IGNORE_MAX_TIME_DIFF
  ignoreMaxValDiff - a non-negative floating-point; default = IGNORE_MAX_VAL_DIFF
  

  These are arguments to key configuration parameters:

   TS.CREATE key … [IGNORE ignoreMaxTimediff ignoreMaxValDiff]
   TS.ALTER key … [IGNORE ignoreMaxTimediff ignoreMaxValDiff]
  

• Optional argument can be passed to

  TS.ADD key timestamp value … [IGNORE ignoreMaxTimeDiff ignoreMaxValDiff]
  TS.INCRBY key addend ... [IGNORE ignoreMaxTimeDiff ignoreMaxValDiff]
  TS.DECRBY key addend ... [IGNORE ignoreMaxTimeDiff ignoreMaxValDiff]
  

  IGNORE … is used when creating a new time series to set the per-key parameters and is ignored otherwise(the existing per-key configuration parameters is used).

• For each call to TS.ADD, if the following condition is met:

  if ((series is not a compaction) &&
      (series[DUPLICATE_POLICY] == LAST) &&
      (timestamp ≥ max_timestamp) &&
      (timestamp - max_timestamp ≤ ignoreMaxTimeDiff) &&
      abs(value - value_at_max_timestamp) ≤ ignoreMaxValDiff)   )

  then sample is ignored. When a sample is ignored, the return value for TS.ADD and for the applicable array element in TS.MADD is max_timestamp of the key.

• Additional changes

  • Added ignoreMaxTimeDiff and ignoreMaxValDiff parameters to TS.INFO output.
  • Added ignoreMaxTimeDiff and ignoreMaxValDiff parameters to RDB.

• Minor unrelated change

  • Fix to skip a test if it is an address sanitizer run

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.15%. Comparing base (ec58c02) to head (673e79e).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1543      +/-   ##
==========================================
+ Coverage   88.99%   89.15%   +0.15%     
==========================================
  Files          35       35              
  Lines        5836     5899      +63     
==========================================
+ Hits         5194     5259      +65     
+ Misses        642      640       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.