Package com.illumon.iris.downsampling

Class Downsampler

java.lang.Object
com.illumon.iris.downsampling.Downsampler
public class Downsampler
extends Object
A simple utility for downsampling time-series data. This uses a "builder" style as opposed to a collection of overloads due to the large number of options. By default, this runs in "online" mode and uses a ticking time table to update as new sample data arrives in the underlying table. The default this uses upperBin for sampling. This can be overridden by setting timeBinMode. The output is grouped by the specified timestamp and "key columns" and aggregates are specified with the last/first/min/max etc methods. Example of sampling trade data: downsampledTrades = downsample(trades, "Timestamp", "00:05:00", "Sym") .last("Price", "Size") .sum("Volume=Size") .execute() Example of sampling quote data: downsampledQuotes = downsample(quotes, "Timestamp", "00:05:00", "Sym") .last("Bid", "BidSize", "Ask", "AskSize") .maintainState("Bid", "BidSize", "Ask", "AskSize") .execute()

  • Method Details

    • downsample

      public static Downsampler.Builder downsample​(Database db, Table table, MatchPair timestampColumn, long periodValue, String... keyColumns)
      Initialize a downsampler builder that will sample the given table at the given time period. The sampling will use time bins based on the given timestamp column and grouped by the given keys.
      Parameters:
      db - Database that will be used to generate time table
      table - Raw data to table
      timestampColumn - Column from which to generate time bins
      periodValue - Sampling period/frequency in nanoseconds
      keyColumns - Columns by which to group the result
      Returns:

    • downsample

      public static Downsampler.Builder downsample​(Database db, Table table, MatchPair timestampColumn, String period, String... keyColumns)
      Initialize a downsampler builder that will sample the given table at the given time period. The sampling will use time bins based on the given timestamp column and grouped by the given keys.
      Parameters:
      db - Database that will be used to generate time table
      table - Raw data to table
      timestampColumn - Column from which to generate time bins
      period - Sampling period/frequency in HH:mm:ss format
      keyColumns - Columns by which to group the result
      Returns:

    • downsample

      public static Downsampler.Builder downsample​(Database db, Table table, MatchPair timestampColumn, String period, Collection<String> keyColumns)
      Initialize a downsampler builder that will sample the given table at the given time period. The sampling will use time bins based on the given timestamp column and grouped by the given keys.
      Parameters:
      db - Database that will be used to generate time table
      table - Raw data to table
      timestampColumn - Column from which to generate time bins
      period - Sampling period/frequency in HH:mm:ss format
      keyColumns - Columns by which to group the result
      Returns:

    • downsample

      public static Downsampler.Builder downsample​(Database db, Table table, String timestampColumn, long periodValue, String... keyColumns)
      Initialize a downsampler builder that will sample the given table at the given time period. The sampling will use time bins based on the given timestamp column and grouped by the given keys.
      Parameters:
      db - Database that will be used to generate time table
      table - Raw data to table
      timestampColumn - Column from which to generate time bins
      periodValue - Sampling period/frequency in nanoseconds
      keyColumns - Columns by which to group the result
      Returns:

    • downsample

      public static Downsampler.Builder downsample​(Database db, Table table, String timestampColumn, String period, String... keyColumns)
      Initialize a downsampler builder that will sample the given table at the given time period. The sampling will use time bins based on the given timestamp column and grouped by the given keys.
      Parameters:
      db - Database that will be used to generate time table
      table - Raw data to table
      timestampColumn - Column from which to generate time bins
      period - Sampling period/frequency in HH:mm:ss format
      keyColumns - Columns by which to group the result
      Returns:

    • downsample

      public static Downsampler.Builder downsample​(Database db, Table table, String timestampColumn, String period, Collection<String> keyColumns)
      Initialize a downsampler builder that will sample the given table at the given time period. The sampling will use time bins based on the given timestamp column and grouped by the given keys.
      Parameters:
      db - Database that will be used to generate time table
      table - Raw data to table
      timestampColumn - Column from which to generate time bins
      period - Sampling period/frequency in HH:mm:ss format
      keyColumns - Columns by which to group the result
      Returns: