Package com.illumon.iris.db.tables.appendable.local

Class LocalAppendableTable

java.lang.Object

com.illumon.iris.db.tables.appendable.local.LocalAppendableTable

All Implemented Interfaces:

AppendableTable<LocalAppendableColumn>, BufferedAppendable, BufferedAppendableTable<LocalAppendableColumn>, TableTransactionProcessor

public class LocalAppendableTable
extends Object
implements BufferedAppendableTable<LocalAppendableColumn>, TableTransactionProcessor

AppendableTable for output to a local table location. In general only one thread at a time should call any of the methods that change the underlying table. Acceptable patterns include:

1. A single thread appends all rows, occasionally flushing, eventually terminating with close and/or release. Other threads may call call force asynchronously.
2. Many threads append to the columns individually, with a central thread eventually closing after all writes are done.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	LocalAppendableTable.TableDefinitionMismatchException

Field Summary

Fields

Modifier and Type	Field	Description
static int	OPTION_APPEND	Option for tables that should append to existing data.
static int	OPTION_SKIP_PREPARE_LOCATION	Option for utilities that take care of preparing the writable location before constructing an appendable table.
static int	OPTION_WRITE_THROUGH	Option for tables that should write through to concurrently-readable data stores.
static int	OPTIONS_DEFAULT	Option for tables that should truncate existing data and not write through to concurrently-readable data stores.

Constructor Summary

Constructors

Constructor	Description
LocalAppendableTable(TableDefinition inputDefinition, WritableLocalTableLocation location, int options, StringCacheProvider stringCacheProvider)	Construct a LocalAppendableTable for the supplied definition, location, options, and string caching.

Method Summary

Modifier and Type	Method	Description
void	abortTransaction()	Abort the current pending transaction.
void	close()	Flush all appended data to the underlying store, release any resources held, and force all appended data to be persisted to permanent storage.
void	finishTransaction()	Finish the current pending transaction.
void	flush()	Write all appended data to the underlying store and prepare for subsequent writes.
void	force()	Force all flushed data to be persisted to permanent storage.
LocalAppendableColumn	getColumn(String columnName)	Get a column for this table by name.
Map<String,LocalAppendableColumn>	getColumnMap()	Get a map of all of the AppendableTables by name.
LocalAppendableColumn[]	getColumns()	Get an array of all of the AppendableTables for this table.
TableDefinition	getDefinition()
WritableLocalTableLocation	getTableLocation()	Get the TableLocation for this appendable table.
boolean	inTransaction()
boolean	maybeForce()	Perform a BufferedAppendable.force() if there is data pending persistence.
void	prepareCheckpoint()	Optional operation.
static long	prepareLocation(WritableLocalTableLocation location, TableDefinition definition, boolean append, boolean writeThrough)	Prepare the location and its checkpoint record for writing.
void	release()	Release any resources held.
void	setDataIndexes(List<String[]> dataIndexes)	Set the list of data indices for this location.
void	setImportDetails(Properties importDetails)	Set a trivial import state object wrapping the supplied import details to be part of checkpoints for the output table location.
void	setImportState(ImportState importState)	Set an import state object to be part of checkpoints for the output table location, and invoke the appropriate ImportState.onResume(FullTableLocationKey).
void	setSortedColumns(List<SortPair> sortedColumns)	Set the ordered list of sorted columns for this location.
long	size()
void	startTransaction()	Begin a new transaction.
void	updateImportState(UnaryOperator<ImportState> updater)	Set the import state object (as in setImportState(ImportState)) after applying the supplied updater function to the existing import state object.
void	updateSize(long size)	Set the size of this appendable table in rows.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.tables.appendable.AppendableTable

addRecords

Field Details

OPTIONS_DEFAULT

public static final int OPTIONS_DEFAULT

Option for tables that should truncate existing data and not write through to concurrently-readable data stores.

See Also:

Constant Field Values

OPTION_APPEND

public static final int OPTION_APPEND

Option for tables that should append to existing data.

See Also:

Constant Field Values

OPTION_WRITE_THROUGH

public static final int OPTION_WRITE_THROUGH

Option for tables that should write through to concurrently-readable data stores.

See Also:

Constant Field Values

OPTION_SKIP_PREPARE_LOCATION

public static final int OPTION_SKIP_PREPARE_LOCATION

Option for utilities that take care of preparing the writable location before constructing an appendable table. Such utilities must call prepareLocation(WritableLocalTableLocation, TableDefinition, boolean, boolean) appropriately.

See Also:

Constant Field Values

Constructor Details

LocalAppendableTable

public LocalAppendableTable(@NotNull TableDefinition inputDefinition, @NotNull WritableLocalTableLocation location, int options, @NotNull StringCacheProvider stringCacheProvider)

Construct a LocalAppendableTable for the supplied definition, location, options, and string caching.

Note that the table name used for persisting a per-location writable table definition will be derived from inputDefinition and location according to findTableName(TableDefinition, LocalTableLocation).

Parameters:

inputDefinition - The definition

location - The destination

options - Options bitmap

stringCacheProvider - String cache provider to use

Method Details

prepareLocation

public static long prepareLocation(@NotNull WritableLocalTableLocation location, @NotNull TableDefinition definition, boolean append, boolean writeThrough)

Prepare the location and its checkpoint record for writing.

Parameters:

location - The WritableLocalTableLocation

definition - The TableDefinition

append - Whether this utility will specify OPTION_APPEND

writeThrough - Whether this utility will specify OPTION_WRITE_THROUGH

Returns:

The visible size of the location after preparation.

getDefinition

public TableDefinition getDefinition()

Specified by:

getDefinition in interface AppendableTable<LocalAppendableColumn>

getColumns

public LocalAppendableColumn[] getColumns()

Description copied from interface: AppendableTable

Get an array of all of the AppendableTables for this table.

Specified by:

getColumns in interface AppendableTable<LocalAppendableColumn>

Returns:

an array of AppendableTable

getColumn

public LocalAppendableColumn getColumn(@NotNull String columnName)

Description copied from interface: AppendableTable

Get a column for this table by name.

Specified by:

getColumn in interface AppendableTable<LocalAppendableColumn>

Parameters:

columnName - The column name

Returns:

The desired column

getColumnMap

public Map<String,LocalAppendableColumn> getColumnMap()

Description copied from interface: AppendableTable

Get a map of all of the AppendableTables by name.

Specified by:

getColumnMap in interface AppendableTable<LocalAppendableColumn>

Returns:

a map of column name to column

size

public long size()

Specified by:

size in interface AppendableTable<LocalAppendableColumn>

Returns:

The size of this appendable table in rows

updateSize

public void updateSize(long size)

Description copied from interface: AppendableTable

Set the size of this appendable table in rows. This is for use by utilities that write directly to an AppendableTable's columns, in order to record that a set of rows has been added to all columns and should be available for checkpointing or reading. All columns should be flushed before this call, either externally or via the AppendableTable.flush() method.

Specified by:

updateSize in interface AppendableTable<LocalAppendableColumn>

Parameters:

size - The new size

setSortedColumns

public void setSortedColumns(@NotNull List<SortPair> sortedColumns)

Description copied from interface: AppendableTable

Set the ordered list of sorted columns for this location.

Specified by:

setSortedColumns in interface AppendableTable<LocalAppendableColumn>

Parameters:

sortedColumns - the ordered list of sorted columns

setDataIndexes

public void setDataIndexes(@NotNull List<String[]> dataIndexes)

Description copied from interface: AppendableTable

Set the list of data indices for this location.

Specified by:

setDataIndexes in interface AppendableTable<LocalAppendableColumn>

Parameters:

dataIndexes - the data indices.

updateImportState

public void updateImportState(@NotNull UnaryOperator<ImportState> updater)

Set the import state object (as in setImportState(ImportState)) after applying the supplied updater function to the existing import state object.

Parameters:

updater - The import state updater

setImportState

public void setImportState(@Nullable ImportState importState)

Set an import state object to be part of checkpoints for the output table location, and invoke the appropriate ImportState.onResume(FullTableLocationKey).

Parameters:

importState - The import state

setImportDetails

public void setImportDetails(@NotNull Properties importDetails)

Set a trivial import state object wrapping the supplied import details to be part of checkpoints for the output table location.

Parameters:

importDetails - The import details

inTransaction

public boolean inTransaction()

Specified by:

inTransaction in interface TableTransactionProcessor

Returns:

True if this processor is currently in a transaction, else false

startTransaction

public void startTransaction()

Description copied from interface: TableTransactionProcessor

Begin a new transaction. Causes any existing pending transaction to be aborted.

Specified by:

startTransaction in interface TableTransactionProcessor

abortTransaction

public void abortTransaction()

Description copied from interface: TableTransactionProcessor

Abort the current pending transaction.

This should not be called if there is no current transaction.

Specified by:

abortTransaction in interface TableTransactionProcessor

finishTransaction

public void finishTransaction()

Description copied from interface: TableTransactionProcessor

Finish the current pending transaction.

This makes the rows that were part of the transaction eligible to be made visible and durable, but does not in itself make them visible or durable.

This should not be called if there is no current transaction.

Specified by:

finishTransaction in interface TableTransactionProcessor

flush

public void flush()

Description copied from interface: BufferedAppendable

Write all appended data to the underlying store and prepare for subsequent writes. Does not imply BufferedAppendable.force().

Specified by:

flush in interface AppendableTable<LocalAppendableColumn>

Specified by:

flush in interface BufferedAppendable

prepareCheckpoint

public void prepareCheckpoint()

Optional operation. Prepare a checkpoint snapshot for the next call to BufferedAppendable.force(). Should be called directly after BufferedAppendable.flush(), with no intervening modification operations. Callers should use release() in the event that this method throws, rather than close().

Specified by:

prepareCheckpoint in interface BufferedAppendable

force

public void force()

Description copied from interface: BufferedAppendable

Force all flushed data to be persisted to permanent storage. Will consistently and atomically persist the last prepared checkpoint in implementations that implement BufferedAppendable.prepareCheckpoint(). Does not imply BufferedAppendable.flush(), as this is a distinct operation in all implementations. May be called concurrently by threads other than the "writing thread".

Specified by:

force in interface BufferedAppendable

maybeForce

public boolean maybeForce()

Description copied from interface: BufferedAppendable

Perform a BufferedAppendable.force() if there is data pending persistence. Implementations that don't keep track of pending data explicitly will always invoke BufferedAppendable.force() pessimistically.

Specified by:

maybeForce in interface BufferedAppendable

Returns:

Whether this call may have done any actual work (i.e. there was data pending or assumed to be pending)

close

public void close()

Description copied from interface: BufferedAppendable

Flush all appended data to the underlying store, release any resources held, and force all appended data to be persisted to permanent storage. This should be the final access to this appendable (if successful) with the exception of any asynchronous BufferedAppendable.force() operations. A BufferedAppendable.release() is appropriate afterwards, especially on error.

Specified by:

close in interface AppendableTable<LocalAppendableColumn>

Specified by:

close in interface BufferedAppendable

release

public void release()

Description copied from interface: BufferedAppendable

Release any resources held. Called after an error in preparation for termination. For external callers, this should be done as the final access to this column with the exception of any asynchronous BufferedAppendable.force() operations.

Specified by:

release in interface BufferedAppendable

getTableLocation

public WritableLocalTableLocation getTableLocation()

Get the TableLocation for this appendable table.

Returns: