A C D E F G H I J K P R S T V

A

appendType(StringBuilder, Schema) - Static method in class co.cask.hydrator.common.HiveSchemaConverter

asRecord(SparseVector) - Static method in class co.cask.hydrator.common.spark.VectorUtils

C

cloneRecord(StructuredRecord, Schema, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Creates a builder based off the given record.

co.cask.hydrator.common - package co.cask.hydrator.common

co.cask.hydrator.common.batch - package co.cask.hydrator.common.batch

co.cask.hydrator.common.batch.action - package co.cask.hydrator.common.batch.action

co.cask.hydrator.common.batch.sink - package co.cask.hydrator.common.batch.sink

co.cask.hydrator.common.http - package co.cask.hydrator.common.http

co.cask.hydrator.common.spark - package co.cask.hydrator.common.spark

Condition - Enum in co.cask.hydrator.common.batch.action

Conditions for when an action should run.

ConditionConfig - Class in co.cask.hydrator.common.batch.action

Base plugin config for post actions that contain a setting for when the action should run.

ConditionConfig() - Constructor for class co.cask.hydrator.common.batch.action.ConditionConfig

ConditionConfig(String) - Constructor for class co.cask.hydrator.common.batch.action.ConditionConfig

ConfigurationUtils - Class in co.cask.hydrator.common.batch

Utility class for providing operations on the Configuration.

configurePipeline(PipelineConfigurer) - Method in class co.cask.hydrator.common.ReferenceBatchSink

configurePipeline(PipelineConfigurer) - Method in class co.cask.hydrator.common.ReferenceBatchSource

configurePipeline(PipelineConfigurer) - Method in class co.cask.hydrator.common.ReferenceRealtimeSink

configurePipeline(PipelineConfigurer) - Method in class co.cask.hydrator.common.ReferenceRealtimeSource

Constants - Class in co.cask.hydrator.common

Class that contains common names and description used across plugins.

Constants.Reference - Class in co.cask.hydrator.common

Common Reference Name property name and description

Constants.Reference() - Constructor for class co.cask.hydrator.common.Constants.Reference

convertBytes(Object) - Method in class co.cask.hydrator.common.RecordConverter

convertField(Object, Schema) - Method in class co.cask.hydrator.common.RecordConverter

createInstance() - Static method in class co.cask.hydrator.common.batch.JobUtils

Creates a new instance of Job.

D

decrypt(byte[], Schema) - Method in class co.cask.hydrator.common.FieldEncryptor

DEFAULT - Static variable in class co.cask.hydrator.common.KeyValueListParser

E

encrypt(Object, Schema) - Method in class co.cask.hydrator.common.FieldEncryptor

EXTERNAL_DATASET_TYPE - Static variable in class co.cask.hydrator.common.Constants

F

FieldEncryptor - Class in co.cask.hydrator.common

Encrypts and decrypts fields based on their schema.

FieldEncryptor(KeystoreConf, int) - Constructor for class co.cask.hydrator.common.FieldEncryptor

fromRecord(StructuredRecord) - Static method in class co.cask.hydrator.common.spark.VectorUtils

G

get() - Method in class co.cask.hydrator.common.http.HTTPRequestor

getCategoricalFeatureInfo(Schema, String, String, String, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Get the feature to cardinality mapping provided by the user.

getCharset() - Method in class co.cask.hydrator.common.http.HTTPPollConfig

getConnectTimeout() - Method in class co.cask.hydrator.common.http.HTTPConfig

getFeatureList(Schema, String, String, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Get the feature list of the features that have to be used for training/prediction depending on the featuresToInclude or featuresToInclude list.

getInputFieldValue(StructuredRecord, String, Splitter) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Gets the input field for feature generation.

getInputFormatClassName() - Method in class co.cask.hydrator.common.SourceInputFormatProvider

getInputFormatConfiguration() - Method in class co.cask.hydrator.common.SourceInputFormatProvider

getInterval() - Method in class co.cask.hydrator.common.http.HTTPPollConfig

getIvHex() - Method in class co.cask.hydrator.common.KeystoreConf

getKeyAlias() - Method in class co.cask.hydrator.common.KeystoreConf

getKeyPassword() - Method in class co.cask.hydrator.common.KeystoreConf

getKeystoreInputStream(String) - Method in class co.cask.hydrator.common.FieldEncryptor

getKeystorePassword() - Method in class co.cask.hydrator.common.KeystoreConf

getKeystorePath() - Method in class co.cask.hydrator.common.KeystoreConf

getKeystoreType() - Method in class co.cask.hydrator.common.KeystoreConf

getNonDefaultConfigurations(Configuration) - Static method in class co.cask.hydrator.common.batch.ConfigurationUtils

Get the Map of properties which are newly added or have updated in the configuration instance passed to this method as a parameter.

getOutputFormatClassName() - Method in class co.cask.hydrator.common.batch.sink.SinkOutputFormatProvider

getOutputFormatConfiguration() - Method in class co.cask.hydrator.common.batch.sink.SinkOutputFormatProvider

getOutputSchema(Schema, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

getReadTimeout() - Method in class co.cask.hydrator.common.http.HTTPPollConfig

getRequestHeadersMap() - Method in class co.cask.hydrator.common.http.HTTPConfig

getTransformation() - Method in class co.cask.hydrator.common.KeystoreConf

getUrl() - Method in class co.cask.hydrator.common.http.HTTPConfig

H

HiveSchemaConverter - Class in co.cask.hydrator.common

Helper class for converting a Schema into a hive schema.

HTTPConfig - Class in co.cask.hydrator.common.http

Base config class for HTTP plugins.

HTTPConfig() - Constructor for class co.cask.hydrator.common.http.HTTPConfig

HTTPConfig(String, String) - Constructor for class co.cask.hydrator.common.http.HTTPConfig

HTTPPollConfig - Class in co.cask.hydrator.common.http

Config class for HTTP Poll plugin.

HTTPPollConfig() - Constructor for class co.cask.hydrator.common.http.HTTPPollConfig

HTTPPollConfig(String, String, long) - Constructor for class co.cask.hydrator.common.http.HTTPPollConfig

HTTPPollConfig(String, String, long, String) - Constructor for class co.cask.hydrator.common.http.HTTPPollConfig

HTTPRequestor - Class in co.cask.hydrator.common.http

Utility code for performing a get request and formatting it as a StructuredRecord.

HTTPRequestor(HTTPPollConfig) - Constructor for class co.cask.hydrator.common.http.HTTPRequestor

I

IdUtils - Class in co.cask.hydrator.common

Utility class for Id related operations.

initialize() - Method in class co.cask.hydrator.common.FieldEncryptor

initialize(RealtimeContext) - Method in class co.cask.hydrator.common.ReferenceRealtimeSink

initialize(RealtimeContext) - Method in class co.cask.hydrator.common.ReferenceRealtimeSource

J

JobUtils - Class in co.cask.hydrator.common.batch

A utility for providing operations on Job.

K

KeystoreConf - Class in co.cask.hydrator.common

Config for keystore properties.

KeystoreConf() - Constructor for class co.cask.hydrator.common.KeystoreConf

KeystoreConf(String, String, String, String, String, String, String) - Constructor for class co.cask.hydrator.common.KeystoreConf

KeyValueListParser - Class in co.cask.hydrator.common

Parses a list of key value pairs.

KeyValueListParser(String, String) - Constructor for class co.cask.hydrator.common.KeyValueListParser

Create a parser that uses the given regexes to parse a list of key value pairs.

P

parse(String) - Method in class co.cask.hydrator.common.KeyValueListParser

Parses the given list of key value pairs.

parseDuration(String) - Static method in class co.cask.hydrator.common.TimeParser

Parses a duration String to its long value.

parseRuntime(String) - Method in class co.cask.hydrator.common.TimeParser

Parses time strings into timestamps, with support for some basic math.

R

RecordConverter<INPUT,OUTPUT> - Class in co.cask.hydrator.common

Converts an object with a schema into another type of object with the same schema.

RecordConverter() - Constructor for class co.cask.hydrator.common.RecordConverter

REFERENCE_NAME - Static variable in class co.cask.hydrator.common.Constants.Reference

REFERENCE_NAME_DESCRIPTION - Static variable in class co.cask.hydrator.common.Constants.Reference

ReferenceBatchSink<IN,KEY_OUT,VAL_OUT> - Class in co.cask.hydrator.common

A BatchSink that verifies referenceName property

ReferenceBatchSink(ReferencePluginConfig) - Constructor for class co.cask.hydrator.common.ReferenceBatchSink

ReferenceBatchSource<KEY_IN,VAL_IN,OUT> - Class in co.cask.hydrator.common

A BatchSource that verifies referenceName property

ReferenceBatchSource(ReferencePluginConfig) - Constructor for class co.cask.hydrator.common.ReferenceBatchSource

referenceName - Variable in class co.cask.hydrator.common.http.HTTPPollConfig

referenceName - Variable in class co.cask.hydrator.common.ReferencePluginConfig

ReferencePluginConfig - Class in co.cask.hydrator.common

PluginConfig that contains a referenceName property that is used to create an external Dataset for metadata, lineage purposes.

ReferencePluginConfig(String) - Constructor for class co.cask.hydrator.common.ReferencePluginConfig

ReferenceRealtimeSink<I> - Class in co.cask.hydrator.common

A RealtimeSink that verifies referenceName property and creates an externalDataset.

ReferenceRealtimeSink(ReferencePluginConfig) - Constructor for class co.cask.hydrator.common.ReferenceRealtimeSink

ReferenceRealtimeSource<T> - Class in co.cask.hydrator.common

A RealtimeSource that verifies referenceName property and creates an externalDataset.

ReferenceRealtimeSource(ReferencePluginConfig) - Constructor for class co.cask.hydrator.common.ReferenceRealtimeSource

RowRecordTransformer - Class in co.cask.hydrator.common

Transforms Rows into Records.

RowRecordTransformer(Schema, String) - Constructor for class co.cask.hydrator.common.RowRecordTransformer

runCondition - Variable in class co.cask.hydrator.common.batch.action.ConditionConfig

S

SCHEMA - Static variable in class co.cask.hydrator.common.http.HTTPRequestor

SchemaValidator - Class in co.cask.hydrator.common

Utility class for with methods for validating schema fields and its types, validating fields are simple, checking if schema is subset of another, checking if fields are present in schema, etc

shouldFollowRedirects() - Method in class co.cask.hydrator.common.http.HTTPConfig

shouldRun(BatchActionContext) - Method in class co.cask.hydrator.common.batch.action.ConditionConfig

SinkOutputFormatProvider - Class in co.cask.hydrator.common.batch.sink

An implementation of OutputFormatProvider based on Configuration and output format class.

SinkOutputFormatProvider(Class<? extends OutputFormat>, Configuration) - Constructor for class co.cask.hydrator.common.batch.sink.SinkOutputFormatProvider

SinkOutputFormatProvider(String, Configuration) - Constructor for class co.cask.hydrator.common.batch.sink.SinkOutputFormatProvider

SinkOutputFormatProvider(String, Map<String, String>) - Constructor for class co.cask.hydrator.common.batch.sink.SinkOutputFormatProvider

SourceInputFormatProvider - Class in co.cask.hydrator.common

An implementation of InputFormatProvider based on Configuration and input format class.

SourceInputFormatProvider(Class<? extends InputFormat>, Configuration) - Constructor for class co.cask.hydrator.common.SourceInputFormatProvider

SourceInputFormatProvider(String, Configuration) - Constructor for class co.cask.hydrator.common.SourceInputFormatProvider

SparkUtils - Class in co.cask.hydrator.common.spark

Spark plugin Utility class.

SPARSE_SCHEMA - Static variable in class co.cask.hydrator.common.spark.VectorUtils

T

TimeParser - Class in co.cask.hydrator.common

Parses time expressions and does simple time math.

TimeParser(long) - Constructor for class co.cask.hydrator.common.TimeParser

toHiveSchema(Schema) - Static method in class co.cask.hydrator.common.HiveSchemaConverter

Translate the given schema into a hive schema.

toRecord(Row) - Method in class co.cask.hydrator.common.RowRecordTransformer

transform(INPUT, Schema) - Method in class co.cask.hydrator.common.RecordConverter

V

validate() - Method in class co.cask.hydrator.common.batch.action.ConditionConfig

validate() - Method in class co.cask.hydrator.common.http.HTTPConfig

validate() - Method in class co.cask.hydrator.common.http.HTTPPollConfig

validateConfigParameters(Schema, String, String, String, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Validate the config parameters for the spark sink and spark compute classes.

validateFeatureGeneratorConfig(Schema, Map<String, String>, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Validate config parameters for feature generator classes.

validateFieldsArePresentInSchema(Schema, String...) - Static method in class co.cask.hydrator.common.SchemaValidator

Iterates through the required fields and checks if they are present in the schema

validateId(String) - Static method in class co.cask.hydrator.common.IdUtils

validateLabelFieldForTrainer(Schema, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Validate label field for trainer.

validateOutputSchemaAndInputSchemaIfPresent(String, String, PipelineConfigurer) - Static method in class co.cask.hydrator.common.SchemaValidator

Validate output schema fields and if input schema is present, check if output schema is a subset of the input schema and return output schema.

validateOutputSchemaIsSubsetOfInputSchema(Schema, Schema) - Static method in class co.cask.hydrator.common.SchemaValidator

Checks that all the fields in output schema is part of input schema and the fields schema type matches.

validateSchemaFieldsAreSimple(Schema) - Static method in class co.cask.hydrator.common.SchemaValidator

Iterates through the schema fields and checks their type is simple

validateTextField(Schema, String) - Static method in class co.cask.hydrator.common.spark.SparkUtils

Validate the input field to be used for text based feature generation.

valueOf(String) - Static method in enum co.cask.hydrator.common.batch.action.Condition

Returns the enum constant of this type with the specified name.

values() - Static method in enum co.cask.hydrator.common.batch.action.Condition

Returns an array containing the constants of this enum type, in the order they are declared.

VectorUtils - Class in co.cask.hydrator.common.spark

Vector utility class to handle sparse vectors.

VectorUtils() - Constructor for class co.cask.hydrator.common.spark.VectorUtils

A C D E F G H I J K P R S T V