Package com.illumon.iris.importers

Class SchemaFromQuandl

java.lang.Object
com.illumon.iris.importers.SchemaFromQuandl
public class SchemaFromQuandl
extends Object
Downloads Quandl metadata and creates matching schema files.

  • Method Summary

    Modifier and Type Method Description
    static String getTableImportSchema​(File xmlMetaData, String namespace, String tableName, String groupingColumn, String partitionColumn)
    Parses a provided XML file with Quandl datatable metadata and returns an Iris schema file with an ImportSource block to match the metadata.
    static String getTableImportSchema​(String xmlMetaDataUri, String namespace, String tableName, String groupingColumn, String partitionColumn)
    Downloads an XML file with Quandl datatable metadata from the specified URI, and returns an Iris schema file with an ImportSource block to match the metadata.
    static String getTableImportSchema​(Document doc, String namespace, String tableName, String groupingColumn, String partitionColumn)
    Takes a DOM Document with Quandl XML datatable metadata and returns an Iris schema including an ImportSource block to match the structure of the metadata.
    static String getTimeSeriesImportSchema​(File xmlMetaData, String namespace, String tableName, String groupingColumn, String datasetColumn, String partitionColumn)
    Parses a provided XML file with Quandl timeseries metadata and returns an Iris schema file with an ImportSource block to match the metadata.
    static String getTimeSeriesImportSchema​(String xmlMetaDataUri, String namespace, String tableName, String groupingColumn, String datasetColumn, String partitionColumn)
    Downloads an XML file with Quandl time series metadata from the specified URI, and returns an Iris schema file with an ImportSource block to match the metadata.
    static String getTimeSeriesImportSchema​(Document doc, String namespace, String tableName, String groupingColumn, String datasetColumn, String partitionColumn)
    Takes a DOM Document with Quandl XML time series metadata and returns an Iris schema including an ImportSource block to match the structure of the metadata.
    static void internalMain​(com.fishlib.io.logger.Logger log, String... args)
    Used when called from other Iris processes
    static void main​(String... args)
    Regular main entry point, used when this module is called from a java command line, or from an IntelliJ run configuration.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • getTableImportSchema

      public static String getTableImportSchema​(File xmlMetaData, String namespace, String tableName, String groupingColumn, String partitionColumn)
      Parses a provided XML file with Quandl datatable metadata and returns an Iris schema file with an ImportSource block to match the metadata.
      Parameters:
      xmlMetaData - File pointing to XML document of Quandl metadata
      namespace - Iris namespace
      tableName - Iris table name
      groupingColumn - optional (null if not used) column to group on
      partitionColumn - column for data partitioning (defaults to Date)
      Returns:
      string of <ImportSource> block for schema file

    • getTableImportSchema

      public static String getTableImportSchema​(String xmlMetaDataUri, String namespace, String tableName, String groupingColumn, String partitionColumn)
      Downloads an XML file with Quandl datatable metadata from the specified URI, and returns an Iris schema file with an ImportSource block to match the metadata.
      Parameters:
      xmlMetaDataUri - URI from which to download XML document of Quandl metadata
      namespace - Iris namespace
      tableName - Iris table name
      groupingColumn - optional (null if not used) column to group on
      partitionColumn - column for data partitioning (defaults to Date)
      Returns:
      string of <ImportSource> block for schema file, or null if an IOException occurs while attempting to download the metadata.

    • getTableImportSchema

      public static String getTableImportSchema​(Document doc, String namespace, String tableName, String groupingColumn, String partitionColumn)
      Takes a DOM Document with Quandl XML datatable metadata and returns an Iris schema including an ImportSource block to match the structure of the metadata.
      Parameters:
      doc - XML document of Quandl metadata
      namespace - Iris namespace
      tableName - Iris table name
      groupingColumn - optional (null if not used) column to group on
      partitionColumn - column for data partitioning (defaults to Date)
      Returns:
      string of <ImportSource> block for schema file

    • getTimeSeriesImportSchema

      public static String getTimeSeriesImportSchema​(File xmlMetaData, String namespace, String tableName, String groupingColumn, String datasetColumn, String partitionColumn)
      Parses a provided XML file with Quandl timeseries metadata and returns an Iris schema file with an ImportSource block to match the metadata.
      Parameters:
      xmlMetaData - File pointing to XML document of Quandl metadata
      namespace - Iris namespace
      tableName - Iris table name
      groupingColumn - optional (null if not used) column to group on
      datasetColumn - name to use for generated column that will store data set names
      partitionColumn - column for data partitioning (defaults to Date)
      Returns:
      string of <ImportSource> block for schema file

    • getTimeSeriesImportSchema

      public static String getTimeSeriesImportSchema​(String xmlMetaDataUri, String namespace, String tableName, String groupingColumn, String datasetColumn, String partitionColumn)
      Downloads an XML file with Quandl time series metadata from the specified URI, and returns an Iris schema file with an ImportSource block to match the metadata.
      Parameters:
      xmlMetaDataUri - URI from which to download XML document of Quandl metadata
      namespace - Iris namespace
      tableName - Iris table name
      groupingColumn - optional (null if not used) column to group on
      datasetColumn - name to use for generated column that will store data set names
      partitionColumn - column for data partitioning (defaults to Date)
      Returns:
      string of <ImportSource> block for schema file

    • getTimeSeriesImportSchema

      public static String getTimeSeriesImportSchema​(Document doc, String namespace, String tableName, String groupingColumn, String datasetColumn, String partitionColumn)
      Takes a DOM Document with Quandl XML time series metadata and returns an Iris schema including an ImportSource block to match the structure of the metadata.
      Parameters:
      doc - XML document of Quandl metadata
      namespace - Iris namespace
      tableName - Iris table name
      groupingColumn - optional (null if not used) column to group on
      datasetColumn - name to use for generated column that will store data set names
      partitionColumn - column for data partitioning (defaults to Date)
      Returns:
      string of <ImportSource> block for schema file

    • internalMain

      public static void internalMain​(com.fishlib.io.logger.Logger log, String... args)
      Used when called from other Iris processes
      Parameters:
      log - Logger created upstream
      args - Varargs list of arguments in Apache CLI format

    • main

      public static void main​(String... args)
      Regular main entry point, used when this module is called from a java command line, or from an IntelliJ run configuration.
      Parameters:
      args - Varargs list of arguments in Apache CLI format