Package io.deephaven.csv

Class CsvTools

java.lang.Object
io.deephaven.csv.CsvTools
public class CsvTools extends Object
Utilities for reading and writing CSV files to and from Tables

  • Field Details

  • Constructor Details

    • CsvTools

      public CsvTools()

  • Method Details

    • builder

      public static io.deephaven.csv.CsvSpecs.Builder builder()
      Creates a CsvSpecs.Builder with CsvTools-specific values. Sets ColumnNameLegalizer.INSTANCE as CsvSpecs.Builder.headerLegalizer(Function) and CsvSpecs.Builder.headerValidator(Predicate); sets a new instance of DeephavenTimeZoneParser as CsvSpecs.Builder.customTimeZoneParser(CustomTimeZoneParser).
      Returns:
      the builder

    • readCsv

      @ScriptApi public static Table readCsv(String path) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from path by importing CSV data.

      If path is a non-file URL, the CSV will be parsed via readCsv(URL, CsvSpecs).

      Otherwise, the path will be parsed via readCsv(Path, CsvSpecs), which will apply decompression based on the path.

      Parameters:
      path - the path
      Returns:
      the table
      Throws:
      io.deephaven.csv.util.CsvReaderException
      See Also:

    • readCsv

      @ScriptApi public static Table readCsv(InputStream stream) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from stream by importing CSV data. The stream will be closed upon return.
      Parameters:
      stream - an InputStream providing access to the CSV data.
      Returns:
      a Deephaven Table object
      Throws:
      io.deephaven.csv.util.CsvReaderException
      See Also:

    • readCsv

      @ScriptApi public static Table readCsv(URL url) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from url by importing CSV data.
      Parameters:
      url - the url
      Returns:
      the table
      Throws:
      io.deephaven.csv.util.CsvReaderException
      See Also:

    • readCsv

      @ScriptApi public static Table readCsv(Path path) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from path by importing CSV data.

      Paths that end in ".tar.zip", ".tar.bz2", ".tar.gz", ".tar.7z", ".tar.zst", ".zip", ".bz2", ".gz", ".7z", ".zst", or ".tar" will be decompressed.

      Parameters:
      path - the file path
      Returns:
      the table
      Throws:
      io.deephaven.csv.util.CsvReaderException
      See Also:

    • readCsv

      @ScriptApi public static Table readCsv(String path, io.deephaven.csv.CsvSpecs specs) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from path by importing CSV data according to the specs.

      If path is a non-file URL, the CSV will be parsed via readCsv(URL, CsvSpecs).

      Otherwise, the path will be parsed via readCsv(Path, CsvSpecs), which will apply decompression based on the path.

      Parameters:
      path - the path
      specs - the csv specs
      Returns:
      the table
      Throws:
      io.deephaven.csv.util.CsvReaderException
      See Also:

    • readCsv

      @ScriptApi public static Table readCsv(InputStream stream, io.deephaven.csv.CsvSpecs specs) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from stream by importing CSV data according to the specs. The stream will be closed upon return.
      Parameters:
      stream - The stream
      specs - The CSV specs.
      Returns:
      The table.
      Throws:
      io.deephaven.csv.util.CsvReaderException - If some error occurs.

    • readCsv

      @ScriptApi public static Table readCsv(URL url, io.deephaven.csv.CsvSpecs specs) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from url by importing CSV data according to the specs.
      Parameters:
      url - the url
      specs - the csv specs
      Returns:
      the table
      Throws:
      io.deephaven.csv.util.CsvReaderException - If some CSV reading error occurs.

    • readCsv

      @ScriptApi public static Table readCsv(Path path, io.deephaven.csv.CsvSpecs specs) throws io.deephaven.csv.util.CsvReaderException
      Creates an in-memory table from path by importing CSV data according to the specs.

      A path that ends in ".tar.zip", ".tar.bz2", ".tar.gz", ".tar.7z", ".tar.zst", ".zip", ".bz2", ".gz", ".7z", ".zst", or ".tar" will be decompressed.

      Parameters:
      path - the path
      specs - the csv specs
      Returns:
      the table
      Throws:
      io.deephaven.csv.util.CsvReaderException - If some CSV reading error occurs.
      See Also:

    • renamesForHeaderless

      public static Collection<Pair> renamesForHeaderless(Collection<String> columnNames)
      Convert an ordered collection of column names to use for a result table into a series of rename pairs to pass to Table.renameColumns(Collection).
      Parameters:
      columnNames - The column names
      Returns:
      A collection of rename columns

    • renamesForHeaderless

      public static Collection<Pair> renamesForHeaderless(String... columnNames)
      Convert an array of column names to use for a result table into a series of rename pairs to pass to Table.renameColumns(Collection).
      Parameters:
      columnNames - The column names
      Returns:
      A collection of rename columns

    • readHeaderlessCsv

      @ScriptApi public static Table readHeaderlessCsv(String filePath, Collection<String> columnNames) throws io.deephaven.csv.util.CsvReaderException
      Equivalent to CsvTools.readCsv(filePath, CsvTools.builder().hasHeaderRow(false).build()).renameColumns(renamesForHeaderless(columnNames));
      Throws:
      io.deephaven.csv.util.CsvReaderException

    • readHeaderlessCsv

      @ScriptApi public static Table readHeaderlessCsv(String filePath, String... columnNames) throws io.deephaven.csv.util.CsvReaderException
      Equivalent to CsvTools.readCsv(filePath, CsvTools.builder().hasHeaderRow(false).build()).renameColumns(renamesForHeaderless(columnNames));
      Throws:
      io.deephaven.csv.util.CsvReaderException

    • readCsv

      @ScriptApi @Deprecated public static Table readCsv(InputStream is, String format) throws io.deephaven.csv.util.CsvReaderException
      Deprecated.
      See readCsv(InputStream, CsvSpecs)
      Creates an in-memory table by importing CSV data. The first row must be column names. Column data types are inferred from the data.
      Parameters:
      is - an InputStream providing access to the CSV data.
      format - an Apache Commons CSV format name to be used to parse the CSV, or a single non-newline character to use as a delimiter.
      Returns:
      a Deephaven Table object
      Throws:
      io.deephaven.csv.util.CsvReaderException

    • readCsv

      @ScriptApi @Deprecated public static Table readCsv(InputStream is, char separator) throws io.deephaven.csv.util.CsvReaderException
      Deprecated.
      See readCsv(InputStream, CsvSpecs)
      Creates an in-memory table by importing CSV data. The first row must be column names. Column data types are inferred from the data.
      Parameters:
      is - an InputStream providing access to the CSV data.
      separator - a char to use as the delimiter value when parsing the file.
      Returns:
      a Deephaven Table object
      Throws:
      io.deephaven.csv.util.CsvReaderException

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, boolean compressed, String destPath, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to compress (bz2) the file being written
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, boolean compressed, String destPath, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to compress (bz2) the file being written
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, PrintStream out, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      out - the stream to write to
      columns - a list of columns to include in the export
      Throws:
      IOException - if there is a problem writing to the stream

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, PrintStream out, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      out - the stream to write to
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if there is a problem writing to the stream

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean compressed, ZoneId timeZone, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to zip the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean compressed, ZoneId timeZone, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to zip the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean compressed, ZoneId timeZone, boolean nullsAsEmpty, char separator, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to zip the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      separator - the delimiter for the CSV
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table[] sources, String destPath, boolean compressed, ZoneId timeZone, String tableSeparator, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      sources - an array of Deephaven table objects to be exported
      destPath - path to the CSV file to be written
      compressed - whether to compress (bz2) the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      tableSeparator - a String (normally a single character) to be used as the table delimiter
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table[] sources, String destPath, boolean compressed, ZoneId timeZone, String tableSeparator, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      sources - an array of Deephaven table objects to be exported
      destPath - path to the CSV file to be written
      compressed - whether to compress (bz2) the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      tableSeparator - a String (normally a single character) to be used as the table delimiter
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table[] sources, String destPath, boolean compressed, ZoneId timeZone, String tableSeparator, char fieldSeparator, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV.
      Parameters:
      sources - an array of Deephaven table objects to be exported
      destPath - path to the CSV file to be written
      compressed - whether to compress (bz2) the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      tableSeparator - a String (normally a single character) to be used as the table delimiter
      fieldSeparator - the delimiter for the CSV files
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean compressed, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, String... columns) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to zip the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean compressed, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to zip the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, String destPath, boolean compressed, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, boolean nullsAsEmpty, char separator, String... columns) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      destPath - path to the CSV file to be written
      compressed - whether to zip the file being written
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      separator - the delimiter for the CSV
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, Writer out, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, boolean nullsAsEmpty, String... columns) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - Writer used to write the CSV
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsv

      @ScriptApi public static void writeCsv(Table source, Writer out, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, boolean nullsAsEmpty, char separator, String... columns) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - Writer used to write the CSV
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      separator - the delimiter for the CSV
      columns - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsvHeader

      @ScriptApi public static void writeCsvHeader(Writer out, String... columns) throws IOException
      Writes the column name header row to a CSV file.
      Parameters:
      out - the Writer to which the header should be written
      columns - a list of column names to be written
      Throws:
      IOException - if the Writer cannot be written to

    • writeCsvHeader

      @ScriptApi public static void writeCsvHeader(Writer out, char separator, String... columns) throws IOException
      Writes the column name header row to a CSV file.
      Parameters:
      out - the Writer to which the header should be written
      separator - a char to use as the delimiter value when writing out the header
      columns - a list of column names to be written
      Throws:
      IOException - if the Writer cannot be written to

    • writeCsvPaginate

      @ScriptApi public static void writeCsvPaginate(Table source, String destPath, String filename) throws IOException
      Writes a Deephaven table to one or more files, splitting it based on the MAX_CSV_LINE_COUNT setting.
      Parameters:
      source - a Deephaven table to be exported
      destPath - the path in which the CSV file(s) should be written
      filename - the base file name to use for the files. A dash and starting line number will be concatenated to each file.
      Throws:
      IOException - if the destination files cannot be written

    • writeCsvPaginate

      @ScriptApi public static void writeCsvPaginate(Table source, String destPath, String filename, boolean nullsAsEmpty) throws IOException
      Writes a Deephaven table to one or more files, splitting it based on the MAX_CSV_LINE_COUNT setting.
      Parameters:
      source - a Deephaven table to be exported
      destPath - the path in which the CSV file(s) should be written
      filename - the base file name to use for the files. A dash and starting line number will be concatenated to each file.
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      Throws:
      IOException - if the destination files cannot be written

    • writeToMultipleFiles

      @ScriptApi public static void writeToMultipleFiles(Table table, String path, String filename, long startLine) throws IOException
      Writes a subset of rows from a Deephaven table to a CSV file.
      Parameters:
      table - a Deephaven table from which rows should be exported
      path - the destination path in which the output CSV file should be created
      filename - the base file name to which a dash and starting line number will be concatenated for the file
      startLine - the starting line number from the table to export; the ending line number will be startLine + MAX_CSV_LINE_COUNT-1, or the end of the table
      Throws:
      IOException - if the destination file cannot be written

    • writeToMultipleFiles

      @ScriptApi public static void writeToMultipleFiles(Table table, String path, String filename, long startLine, boolean nullsAsEmpty) throws IOException
      Writes a subset of rows from a Deephaven table to a CSV file.
      Parameters:
      table - a Deephaven table from which rows should be exported
      path - the destination path in which the output CSV file should be created
      filename - the base file name to which a dash and starting line number will be concatenated for the file
      startLine - the starting line number from the table to export; the ending line number will be startLine + MAX_CSV_LINE_COUNT-1, or the end of the table
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      Throws:
      IOException - if the destination file cannot be written

    • writeCsvContents

      @ScriptApi public static void writeCsvContents(Table source, Writer out, ZoneId timeZone, String... colNames) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - a Writer to which the header should be written
      timeZone - a time zone constant relative to which date time data should be adjusted
      colNames - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsvContents

      @ScriptApi public static void writeCsvContents(Table source, Writer out, ZoneId timeZone, boolean nullsAsEmpty, String... colNames) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - a Writer to which the header should be written
      timeZone - a time zone constant relative to which date time data should be adjusted
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      colNames - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsvContents

      @ScriptApi public static void writeCsvContents(Table source, Writer out, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, String... colNames) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - a Writer to which the header should be written
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      colNames - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsvContents

      @ScriptApi public static void writeCsvContents(Table source, Writer out, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, boolean nullsAsEmpty, String... colNames) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - a Writer to which the header should be written
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      colNames - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • writeCsvContents

      @ScriptApi public static void writeCsvContents(Table source, Writer out, ZoneId timeZone, @Nullable @Nullable BiConsumer<Long,Long> progress, boolean nullsAsEmpty, char separator, String... colNames) throws IOException
      Writes a table out as a CSV file.
      Parameters:
      source - a Deephaven table object to be exported
      out - a Writer to which the header should be written
      timeZone - a time zone constant relative to which date time data should be adjusted
      progress - a procedure that implements BiConsumer, and takes a progress Integer and a total size Integer to update progress
      nullsAsEmpty - if nulls should be written as blank instead of '(null)'
      separator - the delimiter for the CSV
      colNames - a list of columns to include in the export
      Throws:
      IOException - if the target file cannot be written

    • separatorCsvEscape

      protected static String separatorCsvEscape(String str, String separator)
      Returns a String value for a CSV column's value. This String will be enclosed in double quotes if the value includes a double quote, a newline, or the separator.
      Parameters:
      str - the String to be escaped
      separator - the delimiter for the CSV
      Returns:
      the input String, enclosed in double quotes if the value contains a comma, newline or double quote

    • fromLegacyFormat

      public static io.deephaven.csv.CsvSpecs fromLegacyFormat(String format)