Index (gobblin-utility 0.11.0 API)

A B C D E F G H I J K L M N O P Q R S T U V W Z _

A

accept(Path) - Method in class gobblin.util.AvroUtils.AvroPathFilter
accept(Path) - Method in class gobblin.util.filters.AndPathFilter
accept(Path) - Method in class gobblin.util.filters.HiddenFilter
accept(Path) - Method in class gobblin.util.filters.RegexPathFilter
accept(Path) - Method in class gobblin.util.filters.TarGpgPathFilter
accept(Class<?>) - Method in class gobblin.util.WritableShimSerialization
acceptDb(String) - Method in class gobblin.data.management.copy.hive.WhitelistBlacklist
acceptsLogFileExtensions(Set<String>) - Method in class gobblin.util.logs.LogCopier.Builder: Set the set of acceptable log file extensions.
acceptTable(String, String) - Method in class gobblin.data.management.copy.hive.WhitelistBlacklist
acquirePermits(long) - Method in class gobblin.util.limiter.CountBasedLimiter
acquirePermits(long) - Method in interface gobblin.util.limiter.Limiter: Acquire a given number of permits.
acquirePermits(long) - Method in class gobblin.util.limiter.MultiLimiter
acquirePermits(long) - Method in class gobblin.util.limiter.NonRefillableLimiter
acquirePermits(long) - Method in class gobblin.util.limiter.NoopLimiter
acquirePermits(long) - Method in class gobblin.util.limiter.PoolBasedLimiter
acquirePermits(long) - Method in class gobblin.util.limiter.RateBasedLimiter
acquirePermits(long) - Method in class gobblin.util.limiter.TimeBasedLimiter
add(T) - Method in class gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable: Offer an element to the container.
add(ResourceRequirement) - Method in class gobblin.util.request_allocation.ResourceRequirement: Vector addition of this and other ResourceRequirement.
add(ResourceRequirement, ResourceRequirement, ResourceRequirement) - Static method in class gobblin.util.request_allocation.ResourceRequirement
addBrokerKeys(Map<String, String>, Iterable<Map.Entry<S, T>>) - Static method in class gobblin.broker.SharedResourcesBrokerFactory
addClusterMapping(String, String) - Method in class gobblin.util.ClustersNames
addClusterMapping(URL, String) - Method in class gobblin.util.ClustersNames
addExtension(Path, String...) - Static method in class gobblin.util.PathUtils: Suffix all extensions to path.
addGobblinSite() - Static method in class gobblin.util.HadoopUtils: Add "gobblin-site.xml" as a Configuration resource.
AdditionalCloseableInputStream - Class in gobblin.util.io: An extension of FSDataInputStream that takes additional Closeable objects, which will all be closed when InputStream is closed.
AdditionalCloseableInputStream(InputStream, Closeable...) - Constructor for class gobblin.util.io.AdditionalCloseableInputStream
addListener(L) - Method in class gobblin.util.callbacks.CallbacksDispatcher
addListener(PathAlterationListener) - Method in class gobblin.util.filesystem.PathAlterationObserver: Add a file system listener.
addObserver(PathAlterationObserver) - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Add a file system observer to this monitor.
addPathAlterationObserver(PathAlterationListener, Optional<PathAlterationObserver>, Path) - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Create and attach PathAlterationObserverSchedulers for the given root directory and any nested subdirectories under the root directory to the given PathAlterationObserverScheduler.
addToHadoopConfiguration(Configuration) - Static method in class gobblin.util.WritableShimSerialization: Helper method to add this serializer to an existing Hadoop config.
addWeakListener(L) - Method in class gobblin.util.callbacks.CallbacksDispatcher: Only weak references are stored for weak listeners.
ALL - Static variable in enum gobblin.util.Id.Parts
AllEqualPrioritizer() - Constructor for class gobblin.util.request_allocation.RequestAllocatorConfig.AllEqualPrioritizer
AllocatedRequestsIterator<T extends Request<T>> - Interface in gobblin.util.request_allocation: An Iterator over Request that also provides with the total resources used by all consumed entries.
AllocatedRequestsIteratorBase<T extends Request<T>> - Class in gobblin.util.request_allocation: A basic implementation of AllocatedRequestsIterator.
AllocatedRequestsIteratorBase(Iterator<AllocatedRequestsIteratorBase.RequestWithResourceRequirement<T>>, ResourcePool) - Constructor for class gobblin.util.request_allocation.AllocatedRequestsIteratorBase
AllocatedRequestsIteratorBase.RequestWithResourceRequirement<T> - Class in gobblin.util.request_allocation: Stores and element and its ResourceRequirement.
AllocatedRequestsIteratorBase.TExtractor<T> - Class in gobblin.util.request_allocation: A Function used to extract the actual Request from a AllocatedRequestsIteratorBase.RequestWithResourceRequirement.
allocateRequests(Iterator<? extends Requestor<T>>, ResourcePool) - Method in class gobblin.util.request_allocation.HierarchicalAllocator
allocateRequests(Iterator<? extends Requestor<T>>, ResourcePool) - Method in class gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator
allocateRequests(Iterator<? extends Requestor<T>>, ResourcePool) - Method in interface gobblin.util.request_allocation.RequestAllocator: Compute the subset of accepted Requests from the input Requestors which fit withing the ResourcePool.
ALLOCATOR_ALIAS_KEY - Static variable in class gobblin.util.request_allocation.RequestAllocatorUtils
allowParallelization() - Method in class gobblin.util.request_allocation.RequestAllocatorConfig.Builder
allowParallelization(int) - Method in class gobblin.util.request_allocation.RequestAllocatorConfig.Builder
AndPathFilter - Class in gobblin.util.filters: Combines multiple PathFilters.
AndPathFilter(PathFilter...) - Constructor for class gobblin.util.filters.AndPathFilter
append(Path, int, Progressable) - Method in class gobblin.util.filesystem.ThrottledFileSystem
append(byte[]...) - Method in class gobblin.util.guid.Guid: Creates a new Guid which is a unique, replicable representation of the pair (this, byteArrays).
append(Guid...) - Method in class gobblin.util.guid.Guid: Creates a new Guid which is a unique, replicable representation of the pair (this, guids).
append(HasGuid...) - Method in class gobblin.util.guid.Guid: Creates a new Guid which is a unique, replicable representation of the pair (this, objs).
ApplicationLauncherUtils - Class in gobblin.util: A utility class for the application launcher.
ApplicationLauncherUtils() - Constructor for class gobblin.util.ApplicationLauncherUtils
apply(WorkUnitState) - Method in class gobblin.util.PublisherUtils.WorkUnitStateSuccess
apply(AllocatedRequestsIteratorBase.RequestWithResourceRequirement<T>) - Method in class gobblin.util.request_allocation.AllocatedRequestsIteratorBase.TExtractor
applyCommandLineOptions(CommandLine, T) - Method in class gobblin.runtime.cli.PublicMethodsCliObjectFactory: For each method for which the helper created an Option and for which the input CommandLine contains that option, this method will automatically call the method on the input object with the correct arguments.
AUTH_TOKEN_PATH - Static variable in class gobblin.util.ProxiedFileSystemUtils
AUTH_TYPE_KEY - Static variable in class gobblin.util.ProxiedFileSystemUtils
AutoCloseableLock - Class in gobblin.util: An auto-closeable Lock that can be used with try-with-resources.
AutoCloseableLock(Lock) - Constructor for class gobblin.util.AutoCloseableLock
AutoResetEvent - Class in gobblin.util.concurrent
AutoResetEvent() - Constructor for class gobblin.util.concurrent.AutoResetEvent
AutoResetEvent(boolean) - Constructor for class gobblin.util.concurrent.AutoResetEvent
AutoReturnableObject<T> - Class in gobblin.util: Borrow an object from a GenericObjectPool and returns it automatically on close.
AutoReturnableObject(GenericObjectPool<T>) - Constructor for class gobblin.util.AutoReturnableObject
AvroFlattener - Class in gobblin.util: This class provides methods to flatten an Avro Schema to make it more optimal for ORC (Hive does not support predicate pushdown for ORC with nested fields: ETL-7214) The behavior of Avro Schema un-nesting is listed below: 1.
AvroFlattener() - Constructor for class gobblin.util.AvroFlattener
AvroPathFilter() - Constructor for class gobblin.util.AvroUtils.AvroPathFilter
AvroUtils - Class in gobblin.util: A Utils class for dealing with Avro objects
AvroUtils() - Constructor for class gobblin.util.AvroUtils
AvroUtils.AvroPathFilter - Class in gobblin.util
awaitNextRetry() - Method in class gobblin.util.ExponentialBackoff: Block until next retry can be executed.
awaitNextRetryIfAvailable() - Method in class gobblin.util.ExponentialBackoff: Block until next retry can be executed unless max retries has been reached.
AzkabanTags - Class in gobblin.util: Utility class for collecting metadata specific to a Azkaban runtime environment.
AzkabanTags() - Constructor for class gobblin.util.AzkabanTags

B

BaseLimiterType - Enum in gobblin.util.limiter: An enumeration of types of Limiters supported out-of-the-box.
basicDataSource - Variable in class gobblin.util.jdbc.DataSourceProvider
BatchedMeterDecorator - Class in gobblin.util.io: A decorator to a Meter that batches updates for performance.
BatchedMeterDecorator(Meter, int) - Constructor for class gobblin.util.io.BatchedMeterDecorator
bindSharedResourceAtScope(SharedResourceFactory<T, K, S>, K, S, T) - Method in class gobblin.broker.SharedResourcesBrokerImpl
BLACKLIST - Static variable in class gobblin.data.management.copy.hive.WhitelistBlacklist
BROKER_CONF_FILE_KEY - Static variable in class gobblin.broker.SharedResourcesBrokerFactory
BrokerConfigurationKeyGenerator - Class in gobblin.broker: Generates key strings that the default SharedResourcesBroker can understand.
BrokerConfigurationKeyGenerator() - Constructor for class gobblin.broker.BrokerConfigurationKeyGenerator
BruteForceAllocator<T extends Request<T>> - Class in gobblin.util.request_allocation: A RequestAllocator that simply adds every work unit to a ConcurrentBoundedPriorityIterable, then returns the iterator.
BruteForceAllocator(RequestAllocatorConfig<T>) - Constructor for class gobblin.util.request_allocation.BruteForceAllocator
BruteForceAllocator.Factory - Class in gobblin.util.request_allocation
build() - Method in class gobblin.broker.SharedResourcesBrokerImpl.SubscopedBrokerBuilder
build() - Method in class gobblin.util.jdbc.DataSourceBuilder
build() - Method in class gobblin.util.logs.LogCopier.Builder: Build a new LogCopier instance.
build() - Method in class gobblin.util.request_allocation.RequestAllocatorConfig.Builder
build() - Method in class gobblin.util.request_allocation.ResourceRequirement.Builder
build() - Method in class gobblin.util.test.StdoutWriter.Builder
builder() - Static method in class gobblin.util.jdbc.DataSourceBuilder
Builder() - Constructor for class gobblin.util.logs.LogCopier.Builder
builder(ResourceEstimator<T>) - Static method in class gobblin.util.request_allocation.RequestAllocatorConfig
Builder(ResourceEstimator<T>) - Constructor for class gobblin.util.request_allocation.RequestAllocatorConfig.Builder
Builder(ResourcePool) - Constructor for class gobblin.util.request_allocation.ResourceRequirement.Builder
Builder() - Constructor for class gobblin.util.test.StdoutWriter.Builder
buildLimiter(Config) - Method in class gobblin.util.limiter.CountBasedLimiter.Factory
buildLimiter(Config) - Method in interface gobblin.util.limiter.LimiterFactory
buildLimiter(Config) - Method in class gobblin.util.limiter.PoolBasedLimiter.Factory
buildLimiter(Config) - Method in class gobblin.util.limiter.RateBasedLimiter.Factory
buildLimiter(Config) - Method in class gobblin.util.limiter.TimeBasedLimiter.Factory
buildObject(CommandLine) - Method in interface gobblin.runtime.cli.CliObjectFactory: Build an instance of T from the parsed CommandLine.
buildObject(String[], int, boolean, String) - Method in interface gobblin.runtime.cli.CliObjectFactory: Build an instance of T from the input arguments.
buildObject(CommandLine) - Method in class gobblin.runtime.cli.PublicMethodsCliObjectFactory
buildObject(String[], int, boolean, String) - Method in class gobblin.runtime.cli.PublicMethodsCliObjectFactory
byteBufferToOutputStream(ByteBuffer, OutputStream) - Static method in class gobblin.util.io.StreamUtils: Reads the full contents of a ByteBuffer and writes them to an OutputStream.
bytesWritten() - Method in class gobblin.util.test.StdoutWriter

C

call() - Method in class gobblin.util.callbacks.CallbacksDispatcher.CallbackCallable
call() - Method in class gobblin.util.executors.MDCPropagatingCallable
Callback<L,R> - Class in gobblin.util.callbacks: A helper class to wrap a function and provide a name for logging purposes
Callback() - Constructor for class gobblin.util.callbacks.Callback
CallbackCallable() - Constructor for class gobblin.util.callbacks.CallbacksDispatcher.CallbackCallable
CallbackResult<R> - Class in gobblin.util.callbacks: A helper class to simplify the access the result of a callback (if any)
CallbackResult() - Constructor for class gobblin.util.callbacks.CallbackResult
CallbackResults() - Constructor for class gobblin.util.callbacks.CallbacksDispatcher.CallbackResults
CallbacksDispatcher<L> - Class in gobblin.util.callbacks: A helper to dispatch callbacks to a set of listeners.
CallbacksDispatcher(Optional<ExecutorService>, Optional<Logger>) - Constructor for class gobblin.util.callbacks.CallbacksDispatcher: Constructor
CallbacksDispatcher() - Constructor for class gobblin.util.callbacks.CallbacksDispatcher
CallbacksDispatcher(Logger) - Constructor for class gobblin.util.callbacks.CallbacksDispatcher
CallbacksDispatcher(ExecutorService, Logger) - Constructor for class gobblin.util.callbacks.CallbacksDispatcher
CallbacksDispatcher.CallbackCallable<R> - Class in gobblin.util.callbacks
CallbacksDispatcher.CallbackResults<L,R> - Class in gobblin.util.callbacks
cancel(T) - Method in class gobblin.util.concurrent.TaskScheduler: Attempts to cancel the specified ScheduledTask.
canProxyAs(String, String, Path) - Static method in class gobblin.util.ProxiedFileSystemUtils: Returns true if superUserName can proxy as userNameToProxyAs using the specified superUserKeytabLocation, false otherwise.
checkAndNotify() - Method in class gobblin.util.filesystem.PathAlterationObserver: Check whether the file and its chlidren have been created, modified or deleted.
cleanJobStagingData(State, Logger) - Static method in class gobblin.util.JobLauncherUtils: Cleanup staging data of all tasks of a job.
cleanStagingData(List<? extends State>, Logger) - Static method in class gobblin.util.JobLauncherUtils: Cleanup the staging data for a list of Gobblin tasks.
cleanTaskStagingData(State, Logger) - Static method in class gobblin.util.JobLauncherUtils: Cleanup staging data of a Gobblin task.
cleanTaskStagingData(State, Logger, Closer, Map<String, ParallelRunner>) - Static method in class gobblin.util.JobLauncherUtils: Cleanup staging data of a Gobblin task using a ParallelRunner.
cleanup() - Method in class gobblin.util.test.StdoutWriter
CliApplication - Interface in gobblin.runtime.cli: An application that can be called by GobblinCli.
CliObjectFactory<T> - Interface in gobblin.runtime.cli: A factory used to create an object from CLI arguments backed by commons-cli library.
CliObjectFactory.HelpArgumentFound - Exception in gobblin.runtime.cli: Thrown if help argument (-h) was found in the app arguments.
CliObjectOption - Annotation Type in gobblin.runtime.cli: Specify additional information to use when building a CLI option from a method.
CliObjectSupport - Annotation Type in gobblin.runtime.cli: Specifies that the annotated constructor should be used to instantiate the object for running from CLI.
CLIPasswordEncryptor - Class in gobblin.util: A command line tool for encrypting password.
CLIPasswordEncryptor() - Constructor for class gobblin.util.CLIPasswordEncryptor
close() - Method in class gobblin.broker.SharedResourcesBrokerImpl
close() - Method in class gobblin.util.AutoCloseableLock
close() - Method in class gobblin.util.AutoReturnableObject: Return the borrowed object to the pool.
close() - Method in class gobblin.util.callbacks.CallbacksDispatcher
close() - Method in class gobblin.util.concurrent.TaskScheduler: Closes this TaskScheduler, ensuring that new tasks cannot be created and cancelling existing tasks.
close() - Method in class gobblin.util.filesystem.FileSystemInstrumentation
close() - Method in class gobblin.util.filesystem.ThrottledFileSystem
close() - Method in class gobblin.util.HiveJdbcConnector
close() - Method in class gobblin.util.io.AdditionalCloseableInputStream
close() - Method in class gobblin.util.io.CloseableHttpConn
close() - Method in class gobblin.util.io.SeekableFSInputStream
close() - Method in class gobblin.util.NoopCloseable
close() - Method in class gobblin.util.ParallelRunner
close() - Method in class gobblin.util.test.FastSequentialSource.FastSequentialExtractor
close() - Method in class gobblin.util.test.HelloWorldSource.ExtractorImpl
close() - Method in class gobblin.util.test.StdoutWriter
close() - Method in class gobblin.util.test.TestingSource.Extract
CloseableHttpConn - Class in gobblin.util.io: Wraps a HttpURLConnection into a Closeable object.
CloseableHttpConn() - Constructor for class gobblin.util.io.CloseableHttpConn
closeChannelsOnComplete() - Method in class gobblin.util.io.StreamCopier: Close the input and output Channels after copy, whether the copy was successful or not.
closed - Variable in class gobblin.util.filesystem.FileSystemInstrumentation
CLUSTER_IDENTIFIER_TAG_NAME - Static variable in class gobblin.util.ClusterNameTags
ClusterNameTags - Class in gobblin.util: Utility class for collecting metadata specific to the current Hadoop cluster.
ClusterNameTags() - Constructor for class gobblin.util.ClusterNameTags
ClustersNames - Class in gobblin.util: Allows conversion of URLs identifying a Hadoop cluster (e.g.
ClustersNames() - Constructor for class gobblin.util.ClustersNames
combine(Guid...) - Static method in class gobblin.util.guid.Guid: Combine multiple Guids into a single Guid.
combinePaths(String...) - Static method in class gobblin.util.PathUtils
combineProperties(Properties...) - Static method in class gobblin.util.PropertiesUtils: Combine a variable number of Properties into a single Properties.
combineSysAndJobProperties(Properties, Properties) - Static method in class gobblin.util.JobConfigurationUtils: Get a new Properties instance by combining a given system configuration Properties object (first) and a job configuration Properties object (second).
commit() - Method in class gobblin.util.test.StdoutWriter
CompactionRecordCountProvider - Class in gobblin.util.recordcount: Implementation of RecordCountProvider, which provides record count from file path.
CompactionRecordCountProvider() - Constructor for class gobblin.util.recordcount.CompactionRecordCountProvider
compare(T, T) - Method in class gobblin.util.request_allocation.RequestAllocatorConfig.AllEqualPrioritizer
compare(T, T) - Method in class gobblin.util.request_allocation.SimpleHierarchicalPrioritizer
compareRequestors(Requestor<T>, Requestor<T>) - Method in interface gobblin.util.request_allocation.HierarchicalPrioritizer
compareRequestors(Requestor<T>, Requestor<T>) - Method in class gobblin.util.request_allocation.SimpleHierarchicalPrioritizer
compareWithoutSchemeAndAuthority(Path, Path) - Static method in class gobblin.util.PathUtils: Compare two path without shedme and authority (the prefix)
ConcurrentBoundedPriorityIterable<T> - Class in gobblin.util.request_allocation: A concurrent bounded priority Iterable.
ConcurrentBoundedPriorityIterable(Comparator<? super T>, ResourceEstimator<T>, ResourcePool) - Constructor for class gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable
CONFIG_NAMESPACE - Static variable in class gobblin.util.test.HelloWorldSource
configToProperties(Config) - Static method in class gobblin.util.ConfigUtils: Convert a given Config instance to a Properties instance.
configToProperties(Config, Optional<String>) - Static method in class gobblin.util.ConfigUtils: Convert a given Config instance to a Properties instance.
configToProperties(Config, String) - Static method in class gobblin.util.ConfigUtils: Convert a given Config instance to a Properties instance.
configToState(Config) - Static method in class gobblin.util.ConfigUtils: Convert a given Config to a State instance.
configure() - Method in class gobblin.util.jdbc.DataSourceModule
ConfigUtils - Class in gobblin.util: Utility class for dealing with Config objects.
ConfigUtils() - Constructor for class gobblin.util.ConfigUtils
CONN_DRIVER - Static variable in class gobblin.util.jdbc.DataSourceProvider
CONN_URL - Static variable in class gobblin.util.jdbc.DataSourceProvider
constructFileName(String, long) - Static method in class gobblin.util.recordcount.CompactionRecordCountProvider: Construct the file name as {filenamePrefix}{recordCount}.{SystemCurrentTimeInMills}.{RandomInteger}{SUFFIX}.
constructFilePath(String, long) - Static method in class gobblin.util.recordcount.IngestionRecordCountProvider: Construct a new file path by appending record count to the filename of the given file path, separated by SEPARATOR.
constructLateFilePath(String, FileSystem, Path) - Method in class gobblin.util.recordcount.LateFileRecordCountProvider: Construct filename for a late file.
constructObject(CommandLine) - Method in class gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory
constructObject(CommandLine) - Method in class gobblin.runtime.cli.PublicMethodsCliObjectFactory
ConstructorAndPublicMethodsCliObjectFactory<T> - Class in gobblin.runtime.cli: A helper class for automatically inferring Options from the constructor and public methods in a class.
ConstructorAndPublicMethodsCliObjectFactory(Class<? extends T>) - Constructor for class gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory
contractPool(ResourceRequirement) - Method in class gobblin.util.request_allocation.ResourcePool
convertPath(Path, RecordCountProvider) - Method in class gobblin.util.recordcount.CompactionRecordCountProvider: This method currently supports converting the given Path from IngestionRecordCountProvider.
convertRecordSchema(GenericRecord, Schema) - Static method in class gobblin.util.AvroUtils: Change the schema of an Avro record.
convertStream(InputStream) - Static method in class gobblin.util.io.StreamUtils: Convert an instance of InputStream to a FSDataInputStream that is Seekable and PositionedReadable.
copy() - Method in class gobblin.util.io.StreamCopier: Execute the copy of bytes from the input to the output stream.
copy(InputStream, OutputStream) - Static method in class gobblin.util.io.StreamUtils: Copies an InputStream to and OutputStream using Channels.
copy(ReadableByteChannel, WritableByteChannel) - Static method in class gobblin.util.io.StreamUtils: Copies a ReadableByteChannel to a WritableByteChannel.
copyFile(FileSystem, Path, FileSystem, Path, Path, boolean, Configuration) - Static method in class gobblin.util.HadoopUtils: Copies a src Path from a srcFs FileSystem to a dst Path on a dstFs FileSystem.
copyFile(FileSystem, Path, FileSystem, Path, boolean, Configuration) - Static method in class gobblin.util.HadoopUtils: Copy a file from a srcFs FileSystem to a dstFs FileSystem.
copyPath(FileSystem, Path, FileSystem, Path, Configuration) - Static method in class gobblin.util.HadoopUtils: Copies data from a src Path to a dst Path.
copyPath(FileSystem, Path, FileSystem, Path, boolean, Configuration) - Static method in class gobblin.util.HadoopUtils: Copies data from a src Path to a dst Path.
COUNT_KEY - Static variable in class gobblin.util.limiter.CountBasedLimiter.Factory
CountBasedLimiter - Class in gobblin.util.limiter: An implementation of Limiter that limits the number of permits allowed to be issued.
CountBasedLimiter(long) - Constructor for class gobblin.util.limiter.CountBasedLimiter
CountBasedLimiter.Factory - Class in gobblin.util.limiter
create(Path, FsPermission, boolean, int, short, long, Progressable) - Method in class gobblin.util.filesystem.ThrottledFileSystem
create(String) - Static method in class gobblin.util.Id.Job
create(String, long) - Static method in class gobblin.util.Id.Job
create(String) - Static method in class gobblin.util.Id.MultiTask
create(String, int) - Static method in class gobblin.util.Id.MultiTask
create(String) - Static method in class gobblin.util.Id.Task
create(String, int) - Static method in class gobblin.util.Id.Task
create(Gson, TypeToken<R>) - Method in class gobblin.util.io.GsonInterfaceAdapter
create(Config) - Method in interface gobblin.util.request_allocation.ResourceEstimator.Factory
createCancelled() - Static method in class gobblin.util.callbacks.CallbackResult
createDefaultTopLevelBroker(Config, ScopeInstance<S>) - Static method in class gobblin.broker.SharedResourcesBrokerFactory: Create a root SharedResourcesBroker.
createExtractToWorkUnitStateMap(Collection<? extends WorkUnitState>) - Static method in class gobblin.util.PublisherUtils: Creates a Multimap that maps Extract to their corresponds WorkUnitStates.
createFailed(Throwable) - Static method in class gobblin.util.callbacks.CallbackResult
createFromFuture(Future<R>) - Static method in class gobblin.util.callbacks.CallbackResult
createRequestAllocator(RequestAllocatorConfig<T>) - Method in class gobblin.util.request_allocation.BruteForceAllocator.Factory
createRequestAllocator(RequestAllocatorConfig<T>) - Method in class gobblin.util.request_allocation.GreedyAllocator.Factory
createRequestAllocator(RequestAllocatorConfig<T>) - Method in class gobblin.util.request_allocation.HierarchicalAllocator.Factory
createRequestAllocator(RequestAllocatorConfig<T>) - Method in class gobblin.util.request_allocation.PreOrderAllocator.Factory
createRequestAllocator(RequestAllocatorConfig<T>) - Method in interface gobblin.util.request_allocation.RequestAllocator.Factory: Create a RequestAllocator with the input prioritizer and ResourceEstimator.
createResource(SharedResourcesBroker<S>, ScopedConfigView<S, FileSystemKey>) - Method in class gobblin.util.filesystem.FileSystemFactory
createResource(SharedResourcesBroker<S>, ScopedConfigView<S, EmptyKey>) - Method in class gobblin.util.io.StreamThrottler.Factory
createResource(SharedResourcesBroker<S>, ScopedConfigView<S, SharedLimiterKey>) - Method in class gobblin.util.limiter.broker.SharedLimiterFactory
createSuccessful(R) - Static method in class gobblin.util.callbacks.CallbackResult
cryptoKeyLocation(String) - Method in class gobblin.util.jdbc.DataSourceBuilder

D

DATASET - Static variable in class gobblin.util.dataset.DatasetUtils
DATASET_SPECIFIC_PROPS - Static variable in class gobblin.util.dataset.DatasetUtils: A configuration key that allows a user to specify config parameters on a dataset specific level.
DatasetFilterUtils - Class in gobblin.util: A utility class for filtering datasets through blacklist and whitelist.
DatasetFilterUtils() - Constructor for class gobblin.util.DatasetFilterUtils
DatasetUtils - Class in gobblin.util.dataset
DataSourceBuilder - Class in gobblin.util.jdbc
DataSourceBuilder() - Constructor for class gobblin.util.jdbc.DataSourceBuilder
DataSourceModule - Class in gobblin.util.jdbc: A Guice module defining the dependencies used by the jdbc data source.
DataSourceModule(Properties) - Constructor for class gobblin.util.jdbc.DataSourceModule
DataSourceProvider - Class in gobblin.util.jdbc: A provider class for DataSources.
DataSourceProvider(Properties) - Constructor for class gobblin.util.jdbc.DataSourceProvider
DataSourceProvider() - Constructor for class gobblin.util.jdbc.DataSourceProvider
DatePartitionType - Enum in gobblin.util: Temporal granularity types for writing (partitioner.TimeBasedWriterPartitioner) and reading (DatePartitionedAvroFileSource) date partitioned data.
deepestNonGlobPath(Path) - Static method in class gobblin.util.PathUtils: Finds the deepest ancestor of input that is not a glob.
DEFAULT_BROKER_CONF_FILE - Static variable in class gobblin.broker.SharedResourcesBrokerFactory
DEFAULT_BUFFER_SIZE - Static variable in class gobblin.util.io.StreamCopier
DEFAULT_CFG - Static variable in class gobblin.util.service.StandardServiceConfig
DEFAULT_CONN_DRIVER - Static variable in class gobblin.util.jdbc.DataSourceProvider
DEFAULT_DIMENSION_TOLERANCE - Static variable in class gobblin.util.request_allocation.ResourcePool
DEFAULT_HIVESERVER_VERSION - Static variable in class gobblin.util.HiveJdbcConnector
DEFAULT_HOCON_PULL_FILE_EXTENSIONS - Static variable in class gobblin.util.PullFileLoader
DEFAULT_JAVA_PROPS_PULL_FILE_EXTENSIONS - Static variable in class gobblin.util.PullFileLoader
DEFAULT_NUM_HELLOS - Static variable in class gobblin.util.test.HelloWorldSource
DEFAULT_PARALLEL_RUNNER_THREADS - Static variable in class gobblin.util.ParallelRunner
DefaultLimiterFactory - Class in gobblin.util.limiter: A default factory class for Limiters.
DefaultLimiterFactory() - Constructor for class gobblin.util.limiter.DefaultLimiterFactory
defaultThreadFactory() - Static method in class gobblin.util.ExecutorsUtils: Get a default ThreadFactory.
delegate() - Method in class gobblin.util.executors.MDCPropagatingExecutorService
delegate() - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
delete(Path) - Method in class gobblin.util.filesystem.ThrottledFileSystem
delete(Path, boolean) - Method in class gobblin.util.filesystem.ThrottledFileSystem
deleteEmptyParentDirectories(FileSystem, Path, Path) - Static method in class gobblin.util.PathUtils: Deletes empty directories starting with startPath and all ancestors up to but not including limitPath.
deleteIfExists(FileSystem, Path, boolean) - Static method in class gobblin.util.HadoopUtils: A wrapper around FileSystem.delete(Path, boolean) that only deletes a given Path if it is present on the given FileSystem.
deletePath(FileSystem, Path, boolean) - Static method in class gobblin.util.HadoopUtils: A wrapper around FileSystem.delete(Path, boolean) which throws IOException if the given Path exists, and FileSystem.delete(Path, boolean) returns False.
deletePath(Path, boolean) - Method in class gobblin.util.ParallelRunner: Delete a Path.
deletePathAndEmptyAncestors(FileSystem, Path, boolean) - Static method in class gobblin.util.HadoopUtils
DeprecationUtils - Class in gobblin.util.deprecation: Utilities to handle deprecations in Gobblin.
DeprecationUtils() - Constructor for class gobblin.util.deprecation.DeprecationUtils
desanitizeKey(String) - Static method in class gobblin.util.ConfigUtils
deserialize(String) - Static method in class gobblin.util.guid.Guid: Reverse of Guid.toString().
deserialize(String, Class<T>) - Static method in class gobblin.util.SerializationUtils: Deserialize a String obtained via SerializationUtils.serialize(Serializable) into an object, using BaseEncoding.base64().
deserialize(String, Class<T>, BaseEncoding) - Static method in class gobblin.util.SerializationUtils: Deserialize a String obtained via SerializationUtils.serialize(Serializable) into an object, using the given BaseEncoding, which must be the same BaseEncoding used to serialize the object.
deserializeFromBytes(byte[], Class<T>) - Static method in class gobblin.util.SerializationUtils: Deserialize a String obtained via SerializationUtils.serializeIntoBytes(Serializable) into an object.
deserializeFromFile(T, Path) - Method in class gobblin.util.ParallelRunner: Deserialize a State object from a file.
deserializeFromSequenceFile(Class<? extends Writable>, Class<T>, Path, Collection<T>, boolean) - Method in class gobblin.util.ParallelRunner: Deserialize a list of State objects from a Hadoop SequenceFile.
deserializeFromString(Class<? extends Writable>, String) - Static method in class gobblin.util.HadoopUtils: Deserialize a Writable object from a string.
deserializeFromString(Class<? extends Writable>, String, Configuration) - Static method in class gobblin.util.HadoopUtils: Deserialize a Writable object from a string.
deserializeFsPermission(State, String, FsPermission) - Static method in class gobblin.util.HadoopUtils: Get FsPermission from a State object.
deserializeState(FileSystem, Path, T) - Static method in class gobblin.util.SerializationUtils: Deserialize/read a State instance from a file.
deserializeStateFromInputStream(InputStream, T) - Static method in class gobblin.util.SerializationUtils: Deserialize/read a State instance from a file.
deserializeWriterDirPermissions(State, int, int) - Static method in class gobblin.util.HadoopUtils: Deserializes a FsPermissions object that should be used when a DataWriter is creating directories.
deserializeWriterFilePermissions(State, int, int) - Static method in class gobblin.util.HadoopUtils: Deserializes a FsPermissions object that should be used when a DataWriter is writing a file.
destroy() - Method in class gobblin.util.filesystem.PathAlterationObserver: Final processing.
driver(String) - Method in class gobblin.util.jdbc.DataSourceBuilder

E

Either<S,T> - Class in gobblin.util: Represents a value of one of two possible types.
Either() - Constructor for class gobblin.util.Either
Either.Left<S,T> - Class in gobblin.util: An instance of Either with value of type S.
Either.Right<S,T> - Class in gobblin.util: An instance of Either with value of type T.
EmailUtils - Class in gobblin.util: A utility class for sending emails.
EmailUtils() - Constructor for class gobblin.util.EmailUtils
emptyIfNotPresent(Config, String) - Static method in class gobblin.util.ConfigUtils: Return string value at path if config has path.
EmptyIterable<T> - Class in gobblin.util: A type of Iterables corresponding to empty Collections.
EmptyIterable() - Constructor for class gobblin.util.EmptyIterable
EmptyKey - Class in gobblin.broker: A dummy SharedResourceKey.
EmptyKey() - Constructor for class gobblin.broker.EmptyKey
equals(Object) - Method in class gobblin.broker.SharedResourcesBrokerImpl
equals(Object) - Method in class gobblin.util.filesystem.FileStatusEntry: Compare if this object is equal to another object
equals(Object) - Method in class gobblin.util.filesystem.FileSystemKey
equals(Object) - Method in class gobblin.util.filesystem.FileSystemLimiterKey
equals(Object) - Method in class gobblin.util.http.HttpLimiterKey
equals(Object) - Method in class gobblin.util.ImmutableProperties
equals(Object) - Method in class gobblin.util.io.StreamCopierSharedLimiterKey
estimateRequirement(T, ResourcePool) - Method in interface gobblin.util.request_allocation.ResourceEstimator
exceedsHardBound(ResourceRequirement, boolean) - Method in class gobblin.util.request_allocation.ResourcePool
exceedsSoftBound(ResourceRequirement, boolean) - Method in class gobblin.util.request_allocation.ResourcePool
ExceptionCatchingPathAlterationListenerDecorator - Class in gobblin.util.filesystem: A decorator for PathAlterationListener that catches and logs any exception thrown by the underlying listener, preventing it from failing the application.
ExceptionCatchingPathAlterationListenerDecorator() - Constructor for class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
execCallbacks(Function<? super L, R>, L) - Method in class gobblin.util.callbacks.CallbacksDispatcher
execCallbacks(Function<? super L, R>) - Method in class gobblin.util.callbacks.CallbacksDispatcher
execute() - Method in class gobblin.util.executors.IteratorExecutor: Execute the tasks in the task Iterator.
execute(Runnable) - Method in class gobblin.util.executors.MDCPropagatingExecutorService
execute(Runnable) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
executeAndGetResults() - Method in class gobblin.util.executors.IteratorExecutor: Execute the tasks in the task Iterator.
executeStatements(String...) - Method in class gobblin.util.HiveJdbcConnector: Executes the given SQL statements.
EXECUTOR_SERVICE_SHUTDOWN_TIMEOUT - Static variable in class gobblin.util.ExecutorsUtils
EXECUTOR_SERVICE_SHUTDOWN_TIMEOUT_TIMEUNIT - Static variable in class gobblin.util.ExecutorsUtils
ExecutorsUtils - Class in gobblin.util: A utility class to use with Executors in cases such as when creating new thread pools.
ExecutorsUtils() - Constructor for class gobblin.util.ExecutorsUtils
exists(Path) - Method in class gobblin.util.filesystem.ThrottledFileSystem
ExponentialBackoff - Class in gobblin.util: Utility class for exponential backoff.
ExponentialBackoff.NoMoreRetriesException - Exception in gobblin.util: Thrown if no more retries are available for ExponentialBackoff.
Extract() - Constructor for class gobblin.util.test.TestingSource.Extract
EXTRACT_LIMIT_COUNT_LIMIT_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
EXTRACT_LIMIT_POOL_SIZE_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
EXTRACT_LIMIT_RATE_LIMIT_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
EXTRACT_LIMIT_RATE_LIMIT_KEY_DEP - Static variable in class gobblin.util.limiter.DefaultLimiterFactory: Deprecated.
EXTRACT_LIMIT_RATE_LIMIT_TIMEUNIT_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
EXTRACT_LIMIT_RATE_LIMIT_TIMEUNIT_KEY_DEP - Static variable in class gobblin.util.limiter.DefaultLimiterFactory: Deprecated.
EXTRACT_LIMIT_TIME_LIMIT_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
EXTRACT_LIMIT_TIME_LIMIT_KEY_DEP - Static variable in class gobblin.util.limiter.DefaultLimiterFactory: Deprecated.
EXTRACT_LIMIT_TIME_LIMIT_TIMEUNIT_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
EXTRACT_LIMIT_TIME_LIMIT_TIMEUNIT_KEY_DEP - Static variable in class gobblin.util.limiter.DefaultLimiterFactory: Deprecated.
EXTRACT_LIMIT_TYPE_KEY - Static variable in class gobblin.util.limiter.DefaultLimiterFactory
ExtractorImpl(int) - Constructor for class gobblin.util.test.HelloWorldSource.ExtractorImpl
extractPropertiesWithPrefix(Properties, Optional<String>) - Static method in class gobblin.util.PropertiesUtils: Extract all the keys that start with a prefix in Properties to a new Properties instance.

F

Factory() - Constructor for class gobblin.util.filesystem.ThrottledFileSystem.Factory
Factory() - Constructor for class gobblin.util.io.StreamThrottler.Factory
Factory() - Constructor for class gobblin.util.limiter.CountBasedLimiter.Factory
Factory() - Constructor for class gobblin.util.limiter.PoolBasedLimiter.Factory
Factory() - Constructor for class gobblin.util.limiter.RateBasedLimiter.Factory
Factory() - Constructor for class gobblin.util.limiter.TimeBasedLimiter.Factory
Factory() - Constructor for class gobblin.util.request_allocation.BruteForceAllocator.Factory
Factory() - Constructor for class gobblin.util.request_allocation.GreedyAllocator.Factory
Factory() - Constructor for class gobblin.util.request_allocation.HierarchicalAllocator.Factory
Factory() - Constructor for class gobblin.util.request_allocation.PreOrderAllocator.Factory
FACTORY_ALIAS - Static variable in class gobblin.util.limiter.CountBasedLimiter
FACTORY_NAME - Static variable in class gobblin.util.filesystem.FileSystemFactory
FAIL_IF_NO_GLOBAL_LIMITER_KEY - Static variable in class gobblin.util.limiter.broker.SharedLimiterFactory
FAIL_ON_UNKNOWN_RESOURCE_ID - Static variable in class gobblin.util.limiter.broker.SharedLimiterFactory
FastSequentialExtractor() - Constructor for class gobblin.util.test.FastSequentialSource.FastSequentialExtractor
FastSequentialSource - Class in gobblin.util.test: A very simple Source that just emits long values counting up from 1 until either it reaches a user specified value or a user specified time elapses.
FastSequentialSource() - Constructor for class gobblin.util.test.FastSequentialSource
FastSequentialSource.FastSequentialExtractor - Class in gobblin.util.test
FIELD_LOCATION_DELIMITER - Static variable in class gobblin.util.AvroUtils
FieldWeighter - Class in gobblin.util.binpacking: A WorkUnitWeighter implementation that parses the weight from a field in the work unit.
FieldWeighter() - Constructor for class gobblin.util.binpacking.FieldWeighter
FileListUtils - Class in gobblin.util: Utility class for listing files on a FileSystem.
FileListUtils() - Constructor for class gobblin.util.FileListUtils
FileStatusEntry - Class in gobblin.util.filesystem
FileStatusEntry(Path) - Constructor for class gobblin.util.filesystem.FileStatusEntry
FileSystemFactory<S extends ScopeType<S>> - Class in gobblin.util.filesystem: A SharedResourceFactory for creating FileSystems.
FileSystemFactory() - Constructor for class gobblin.util.filesystem.FileSystemFactory
FileSystemInstrumentation - Class in gobblin.util.filesystem: A base class for FileSystem decorators used for FileSystemInstrumentationFactorys.
FileSystemInstrumentation(FileSystem) - Constructor for class gobblin.util.filesystem.FileSystemInstrumentation
FileSystemInstrumentationFactory<S extends ScopeType<S>> - Class in gobblin.util.filesystem: A factory that instruments FileSystem.
FileSystemInstrumentationFactory() - Constructor for class gobblin.util.filesystem.FileSystemInstrumentationFactory
FileSystemKey - Class in gobblin.util.filesystem
FileSystemKey(URI, Configuration) - Constructor for class gobblin.util.filesystem.FileSystemKey
FileSystemLimiterKey - Class in gobblin.util.filesystem: SharedLimiterKey used for NameNode throttling.
FileSystemLimiterKey(URI) - Constructor for class gobblin.util.filesystem.FileSystemLimiterKey
fileToProperties(String, Configuration) - Static method in class gobblin.util.JobConfigurationUtils: Load the properties from the specified file into a Properties object.
fileToProperties(String) - Static method in class gobblin.util.JobConfigurationUtils: Load the properties from the specified file into a Properties object.
filter(List<String>, List<Pattern>, List<Pattern>) - Static method in class gobblin.util.DatasetFilterUtils
filter(Set<String>, List<Pattern>, List<Pattern>) - Static method in class gobblin.util.DatasetFilterUtils
FilterStreamUnpacker - Class in gobblin.util.io: Contains utilities for getting uderlying streams to a filter stream.
FilterStreamUnpacker() - Constructor for class gobblin.util.io.FilterStreamUnpacker
findFullPrefixKeys(Properties, Optional<String>) - Static method in class gobblin.util.ConfigUtils: Finds a list of properties whose keys are complete prefix of other keys.
findWrappedMeteredInputStream(InputStream) - Static method in class gobblin.util.io.MeteredInputStream: Find the lowest MeteredInputStream in a chain of FilterInputStreams.
findWrappedMeteredOutputStream(OutputStream) - Static method in class gobblin.util.io.MeteredOutputStream: Find the lowest MeteredOutputStream in a chain of FilterOutputStreams.
flatten(Schema, boolean) - Method in class gobblin.util.AvroFlattener: Flatten the Schema to un-nest recursive Records (to make it optimal for ORC)
flattenWorkUnits(Collection<WorkUnit>) - Static method in class gobblin.util.JobLauncherUtils: Utility method that takes in a List of WorkUnits, and flattens them.
ForceQueuePolicy - Class in gobblin.util.executors: RejectedExecutionHandler that just re-attempts to put into the queue.
ForceQueuePolicy() - Constructor for class gobblin.util.executors.ForceQueuePolicy
ForkOperatorUtils - Class in gobblin.util: Utility class for use with the ForkOperator class.
ForkOperatorUtils() - Constructor for class gobblin.util.ForkOperatorUtils
FormatException(String) - Constructor for exception gobblin.util.StringParsingUtils.FormatException
FormatException(String, Throwable) - Constructor for exception gobblin.util.StringParsingUtils.FormatException
FormatException(Throwable) - Constructor for exception gobblin.util.StringParsingUtils.FormatException
formatJvmArguments(Optional<String>) - Static method in class gobblin.util.JvmUtils: Formats the specified jvm arguments such that any tokens are replaced with concrete values;
forName(String) - Static method in enum gobblin.util.limiter.BaseLimiterType: Get a BaseLimiterType for the given name.
fromByteArrays(byte[]...) - Static method in class gobblin.util.guid.Guid: Generate a Guid for an array of byte arrays.
fromHasGuid(HasGuid...) - Static method in class gobblin.util.guid.Guid: Generate a Guid for an array of HasGuid.
fromStrings(String...) - Static method in class gobblin.util.guid.Guid: Generate a Guid for an array of Strings.
FS_SCHEMES_NON_ATOMIC - Static variable in class gobblin.util.HadoopUtils: A Collection of all known FileSystem schemes that do not support atomic renames or copies.

G

generateDumpScript(Path, FileSystem, String, String) - Static method in class gobblin.util.HeapDumpForTaskUtils: Generate the dumpScript, which is used when OOM error is thrown during task execution.
generateKey(SharedResourceFactory, SharedResourceKey, ScopeType, String) - Static method in class gobblin.broker.BrokerConfigurationKeyGenerator: Generate a SharedResourcesBroker configuration key for a particular SharedResourceFactory, SharedResourceKey and ScopeType.
get() - Method in class gobblin.util.AutoReturnableObject
get(String, Optional<String>) - Static method in class gobblin.util.concurrent.TaskSchedulerFactory: Gets an instance of the TaskScheduler with the specified type and ensures that it is started.
get(TaskSchedulerType, Optional<String>) - Static method in class gobblin.util.concurrent.TaskSchedulerFactory: Gets an instance of the TaskScheduler with the specified type and ensures that it is started.
get() - Method in class gobblin.util.Either: Get the value of this Either instance, returned as class Object.
get(Configuration, SharedResourcesBroker<S>) - Static method in class gobblin.util.filesystem.FileSystemFactory: Equivalent to FileSystem.get(Configuration), but uses the input SharedResourcesBroker to configure add-ons to the FileSystem (e.g.
get(URI, Configuration, SharedResourcesBroker<S>) - Static method in class gobblin.util.filesystem.FileSystemFactory: Equivalent to FileSystem.get(URI, Configuration), but uses the input SharedResourcesBroker to configure add-ons to the FileSystem (e.g.
get(EnumSet<Id.Parts>) - Method in class gobblin.util.Id
get() - Method in class gobblin.util.jdbc.DataSourceProvider
getAutoScope(SharedResourcesBroker<S>, ConfigView<S, FileSystemKey>) - Method in class gobblin.util.filesystem.FileSystemFactory
getAutoScope(SharedResourcesBroker<S>, ConfigView<S, EmptyKey>) - Method in class gobblin.util.io.StreamThrottler.Factory
getAutoScope(SharedResourcesBroker<S>, ConfigView<S, SharedLimiterKey>) - Method in class gobblin.util.limiter.broker.SharedLimiterFactory
getAzkabanTags() - Static method in class gobblin.util.AzkabanTags: Uses AzkabanTags.getAzkabanTags(Configuration) with default Hadoop Configuration
getAzkabanTags(Configuration) - Static method in class gobblin.util.AzkabanTags: Gets all useful Azkaban runtime properties required by metrics as a Map.
getBoolean(Config, String, boolean) - Static method in class gobblin.util.ConfigUtils: Return boolean value at path if config has path.
getBytesProcessedMeter() - Method in class gobblin.util.io.MeteredInputStream
getBytesProcessedMeter() - Method in class gobblin.util.io.MeteredOutputStream
getBytesProcessedMeter() - Method in interface gobblin.util.io.MeteredStream
getChildren() - Method in class gobblin.util.filesystem.FileStatusEntry: Return the directory's files.
getClusterName(String) - Method in class gobblin.util.ClustersNames
getClusterName() - Method in class gobblin.util.ClustersNames: Returns the cluster name on which the application is running.
getClusterName(Configuration) - Method in class gobblin.util.ClustersNames: Returns the cluster name on which the application is running.
getClusterNameTags() - Static method in class gobblin.util.ClusterNameTags: Uses ClusterNameTags.getClusterNameTags(Configuration) with default Hadoop Configuration.
getClusterNameTags(Configuration) - Static method in class gobblin.util.ClusterNameTags: Gets all useful Hadoop cluster metrics.
getCodecFactory(Optional<String>, Optional<String>) - Static method in class gobblin.util.WriterUtils: Creates a CodecFactory based on the specified codec name and deflate level.
getConfFromProperties(Properties) - Static method in class gobblin.util.HadoopUtils
getConfFromState(State) - Static method in class gobblin.util.HadoopUtils
getConfFromState(State, Optional<String>) - Static method in class gobblin.util.HadoopUtils: Provides Hadoop configuration given state.
getConfig() - Method in class gobblin.broker.KeyedScopedConfigViewImpl
getConfig(Config, String, Config) - Static method in class gobblin.util.ConfigUtils: Return Config value at path if config has path.
getConfigOrEmpty(Config, String) - Static method in class gobblin.util.ConfigUtils
getConfigView(S, K, String) - Method in class gobblin.broker.SharedResourcesBrokerImpl: Get a ConfigView for the input scope, key, and factory.
getConnection() - Method in class gobblin.util.HiveJdbcConnector
getDataPublisherFinalDir(State, int, int) - Static method in class gobblin.util.WriterUtils: Get the Path corresponding the to the directory a given BaseDataPublisher should commits its output data.
getDatasetSpecificProps(Iterable<String>, State) - Static method in class gobblin.util.dataset.DatasetUtils: Given a Iterable of dataset identifiers (e.g., name, URN, etc.), return a Map that links each dataset with the extra configuration information specified in the state via DatasetUtils.DATASET_SPECIFIC_PROPS.
getDateTimeFieldType() - Method in enum gobblin.util.DatePartitionType
getDateTimePattern() - Method in enum gobblin.util.DatePartitionType
getDecoratedObject() - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
getDecoratedObject() - Method in class gobblin.util.io.BatchedMeterDecorator
getDefaultExecutor(Logger) - Static method in class gobblin.util.callbacks.CallbacksDispatcher
getDefaultJacksonFactory() - Static method in class gobblin.util.json.JsonUtils: Get a Jackson JsonFactory configured with standard settings.
getDefaultObjectMapper() - Static method in class gobblin.util.json.JsonUtils: Get a Jackson ObjectMapper configured with standard settings.
getDefaultWriterFilePath(State, int, int) - Static method in class gobblin.util.WriterUtils: Creates the default Path for the ConfigurationKeys.WRITER_FILE_PATH key.
getDeserializer(Class<WritableShim>) - Method in class gobblin.util.WritableShimSerialization
getDirectorySchema(Path, FileSystem, boolean) - Static method in class gobblin.util.AvroUtils: Get the latest avro schema for a directory
getDirectorySchema(Path, Configuration, boolean) - Static method in class gobblin.util.AvroUtils: Get the latest avro schema for a directory
getDouble(Config, String, double) - Static method in class gobblin.util.ConfigUtils: Return double value at path if config has path.
getExpectedRecordCount() - Method in class gobblin.util.test.FastSequentialSource.FastSequentialExtractor
getExpectedRecordCount() - Method in class gobblin.util.test.HelloWorldSource.ExtractorImpl
getExpectedRecordCount() - Method in class gobblin.util.test.TestingSource.Extract
getExtractor(WorkUnitState) - Method in class gobblin.util.test.FastSequentialSource
getExtractor(WorkUnitState) - Method in class gobblin.util.test.HelloWorldSource
getExtractor(WorkUnitState) - Method in class gobblin.util.test.TestingSource
getExtractsForPredicate(Multimap<Extract, WorkUnitState>, Predicate<WorkUnitState>) - Static method in class gobblin.util.PublisherUtils: Given a Multimap of Extracts to WorkUnitStates, filter out any Extracts where all of the corresponding WorkUnitStates do not meet the given Predicate.
getFactorySpecificConfig() - Method in class gobblin.broker.KeyedScopedConfigViewImpl
getField(Schema, String) - Static method in class gobblin.util.AvroUtils: Given a GenericRecord, this method will return the field specified by the path parameter.
getField(DateTime) - Method in enum gobblin.util.DatePartitionType
getFieldSchema(Schema, String) - Static method in class gobblin.util.AvroUtils: Given a GenericRecord, this method will return the schema of the field specified by the path parameter.
getFieldValue(GenericRecord, String) - Static method in class gobblin.util.AvroUtils: Given a GenericRecord, this method will return the field specified by the path parameter.
getFileStatus(Path) - Method in class gobblin.util.filesystem.ThrottledFileSystem
getForkId(String, int) - Static method in class gobblin.util.ForkOperatorUtils: Get the fork branch ID of a branch of a given task.
getFsConfiguration(State) - Static method in class gobblin.util.WriterUtils
getGson(Class<T>) - Static method in class gobblin.util.io.GsonInterfaceAdapter
getHadoopTokens(State) - Static method in class gobblin.util.hadoop.TokenUtils: Get Hadoop tokens (tokens for job history server, job tracker and HDFS) using Kerberos keytab.
getHighWatermark() - Method in class gobblin.util.test.FastSequentialSource.FastSequentialExtractor
getHighWatermark() - Method in class gobblin.util.test.HelloWorldSource.ExtractorImpl
getHighWatermark() - Method in class gobblin.util.test.TestingSource.Extract
getHostName() - Static method in class gobblin.util.HostUtils
getImplicitBroker() - Static method in class gobblin.broker.SharedResourcesBrokerFactory: Get the implicit SharedResourcesBroker in the callers thread.
getInstance() - Static method in class gobblin.util.ClustersNames
getInt(Config, String, Integer) - Static method in class gobblin.util.ConfigUtils: Return Integer value at path if config has path.
getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class gobblin.util.request_allocation.BruteForceAllocator
getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class gobblin.util.request_allocation.GreedyAllocator
getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class gobblin.util.request_allocation.PreOrderAllocator
getJoinIterator(Iterator<? extends Requestor<T>>, ConcurrentBoundedPriorityIterable<T>) - Method in class gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator
getJvmInputArguments() - Static method in class gobblin.util.JvmUtils: Gets the input arguments passed to the JVM.
getKey() - Method in interface gobblin.util.concurrent.ScheduledTask: The unique key used to identify this task.
getKeyedConfig() - Method in class gobblin.broker.KeyedScopedConfigViewImpl
getKeyedScopedConfig() - Method in class gobblin.broker.KeyedScopedConfigViewImpl
getLeft() - Method in class gobblin.util.Either.Left
getLevel() - Method in class gobblin.util.filesystem.FileStatusEntry: Return the level
getListeners() - Method in class gobblin.util.callbacks.CallbacksDispatcher
getListeners() - Method in class gobblin.util.filesystem.PathAlterationObserver: Returns the set of registered file system listeners.
getLog() - Method in class gobblin.util.callbacks.CallbacksDispatcher
getLong(Config, String, Long) - Static method in class gobblin.util.ConfigUtils: Return Long value at path if config has path.
getLowestIntervalUnit(String) - Static method in enum gobblin.util.DatePartitionType
getMultiFieldValue(GenericRecord, String) - Static method in class gobblin.util.AvroUtils
getName() - Method in class gobblin.util.filesystem.FileSystemFactory
getName() - Method in class gobblin.util.io.StreamThrottler.Factory
getName() - Method in class gobblin.util.limiter.broker.SharedLimiterFactory
getObservers() - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Returns the set of PathAlterationObserver registered with this monitor.
getOptionallyThrottledFileSystem(FileSystem, State) - Static method in class gobblin.util.HadoopUtils: Calls HadoopUtils.getOptionallyThrottledFileSystem(FileSystem, int) parsing the qps from the input State at key HadoopUtils.MAX_FILESYSTEM_QPS.
getOptionallyThrottledFileSystem(FileSystem, int) - Static method in class gobblin.util.HadoopUtils: Get a throttled FileSystem that limits the number of queries per second to a FileSystem.
getOptions() - Method in interface gobblin.runtime.cli.CliObjectFactory
getOptions() - Method in class gobblin.runtime.cli.ConstructorAndPublicMethodsCliObjectFactory
getParent() - Method in class gobblin.util.filesystem.FileStatusEntry: Return the parent entry.
getPath() - Method in class gobblin.util.filesystem.FileStatusEntry: Return the path from the instance FileStatus variable
getPathForBranch(State, String, int, int) - Static method in class gobblin.util.ForkOperatorUtils: Get a new path with the given branch name as a sub directory.
getPathWithoutSchemeAndAuthority(Path) - Static method in class gobblin.util.PathUtils: Removes the Scheme and Authority from a Path.
getPatternList(State, String) - Static method in class gobblin.util.DatasetFilterUtils
getPatternList(State, String, String) - Static method in class gobblin.util.DatasetFilterUtils
getPatternsFromStrings(List<String>) - Static method in class gobblin.util.DatasetFilterUtils: Convert a list of Strings to a list of Patterns.
getPos() - Method in class gobblin.util.io.SeekableFSInputStream
getPrefix() - Method in class gobblin.util.Id
getPrefix() - Method in class gobblin.util.Id.Job
getPrefix() - Method in class gobblin.util.Id.MultiTask
getPrefix() - Method in class gobblin.util.Id.Task
getPrincipalUsingHostname(String, String) - Static method in class gobblin.util.HostUtils
getPropAsBoolean(Properties, String, String) - Static method in class gobblin.util.PropertiesUtils
getProperty(String) - Method in class gobblin.util.ImmutableProperties: Need to override this method otherwise it will call super.get(key).
getPropertyNameForBranch(String, int, int) - Static method in class gobblin.util.ForkOperatorUtils: Get a new property key from an original one with branch index (if applicable).
getPropertyNameForBranch(String, int) - Static method in class gobblin.util.ForkOperatorUtils: Get a new property key from an original one with branch index (if applicable).
getPropertyNameForBranch(WorkUnitState, String) - Static method in class gobblin.util.ForkOperatorUtils: Get a new property key from an original one based on the branch id.
getProxiedFileSystem(String, Properties, URI) - Static method in class gobblin.util.ProxiedFileSystemCache: Deprecated.
use #fromProperties
getProxiedFileSystem(String, Properties, Configuration) - Static method in class gobblin.util.ProxiedFileSystemCache: Deprecated.
use #fromProperties
getProxiedFileSystem(String, Properties, URI, Configuration) - Static method in class gobblin.util.ProxiedFileSystemCache: Deprecated.
Use #fromProperties
getProxiedFileSystem(State, ProxiedFileSystemWrapper.AuthType, String, String) - Method in class gobblin.util.ProxiedFileSystemWrapper: Deprecated.

Same as @see #getProxiedFileSystem(State, AuthType, String, String, Configuration) where state properties will be copied into Configuration.
getProxiedFileSystem(State, ProxiedFileSystemWrapper.AuthType, String, String, Configuration) - Method in class gobblin.util.ProxiedFileSystemWrapper: Deprecated.

Getter for proxiedFs, using the passed parameters to create an instance of a proxiedFs.
getProxiedFileSystemUsingKeytab(State, URI, Configuration) - Static method in class gobblin.util.ProxiedFileSystemCache: Deprecated.
use #fromKeytab.
getProxiedFileSystemUsingKeytab(String, String, Path, URI, Configuration) - Static method in class gobblin.util.ProxiedFileSystemCache: Deprecated.
use #fromKeytab.
getProxiedFileSystemUsingToken(String, Token<?>, URI, Configuration) - Static method in class gobblin.util.ProxiedFileSystemCache: Deprecated.
use #fromToken.
getRateIfRateControlled(FileSystem) - Static method in class gobblin.util.RateControlledFileSystem: Deprecated.

Determines whether the file system is rate controlled, and if so, returns the allowed rate in operations per second.
getRateLimiter() - Method in class gobblin.util.filesystem.ThrottledFileSystem
getRateLimiter() - Method in class gobblin.util.RateControlledFileSystem: Deprecated.
getRecordCount(Path) - Method in class gobblin.util.recordcount.CompactionRecordCountProvider: Get the record count through filename.
getRecordCount(Path) - Method in class gobblin.util.recordcount.IngestionRecordCountProvider: The record count should be the last component before the filename extension.
getRecordCount(Path) - Method in class gobblin.util.recordcount.LateFileRecordCountProvider: Get record count from a given filename (possibly having LATE_COMPONENT{RandomInteger}), using the original FileNameWithRecordCountFormat.
getRequestor() - Method in interface gobblin.util.request_allocation.Request
getRequests(Comparator<T>) - Method in interface gobblin.util.request_allocation.PushDownRequestor: Return an Iterator of Requests already sorted by the input prioritizer.
getResourceRequirementBuilder() - Method in class gobblin.util.request_allocation.ResourcePool: Use to create a ResourceRequirement compatible with this ResourcePool.
getRight() - Method in class gobblin.util.Either.Right
getScheduledTask(K) - Method in class gobblin.util.concurrent.TaskScheduler: Gets the ScheduledTask with the specified key.
getScheduledTasks() - Method in class gobblin.util.concurrent.TaskScheduler: Gets all ScheduledTasks.
getSchema() - Method in class gobblin.util.test.FastSequentialSource.FastSequentialExtractor
getSchema() - Method in class gobblin.util.test.HelloWorldSource.ExtractorImpl
getSchema() - Method in class gobblin.util.test.TestingSource.Extract
getSchemaFromDataFile(Path, FileSystem) - Static method in class gobblin.util.AvroUtils: Get Avro schema from an Avro data file.
getScope(S) - Method in class gobblin.broker.SharedResourcesBrokerImpl
getScopedConfig() - Method in class gobblin.broker.KeyedScopedConfigViewImpl
getScopedView(S) - Method in class gobblin.broker.KeyedScopedConfigViewImpl
getSerializer(Class<WritableShim>) - Method in class gobblin.util.WritableShimSerialization
getSharedResource(SharedResourceFactory<T, K, S>, K) - Method in class gobblin.broker.SharedResourcesBrokerImpl
getSharedResourceAtScope(SharedResourceFactory<T, K, S>, K, S) - Method in class gobblin.broker.SharedResourcesBrokerImpl
getStateFromConf(Configuration) - Static method in class gobblin.util.HadoopUtils
getString(Config, String, String) - Static method in class gobblin.util.ConfigUtils: Return string value at path if config has path.
getStringList(Config, String) - Static method in class gobblin.util.ConfigUtils: An extension to Config.getStringList(String).
getTableNameWriterFilePath(State) - Static method in class gobblin.util.WriterUtils: Creates Path for the ConfigurationKeys.WRITER_FILE_PATH key according to ConfigurationKeys.EXTRACT_TABLE_NAME_KEY.
getTokenFromSeqFile(String, Path) - Static method in class gobblin.util.ProxiedFileSystemUtils: Retrives a Token from a given sequence file for a specified user.
getUnderlyingMeter() - Method in class gobblin.util.io.BatchedMeterDecorator
getUsageString() - Method in interface gobblin.runtime.cli.CliObjectFactory: Get a usage string for display on the command line.
getUsageString() - Method in class gobblin.runtime.cli.PublicMethodsCliObjectFactory
getWorkunits(SourceState) - Method in class gobblin.util.test.FastSequentialSource
getWorkunits(SourceState) - Method in class gobblin.util.test.HelloWorldSource
getWorkunits(SourceState) - Method in class gobblin.util.test.TestingSource
getWriterFileName(State, int, int, String, String) - Static method in class gobblin.util.WriterUtils: Get the value of ConfigurationKeys.WRITER_FILE_NAME for the a given DataWriter.
getWriterFilePath(State, int, int) - Static method in class gobblin.util.WriterUtils: Get the Path corresponding the the relative file path for a given DataWriter.
getWriterFS(State, int, int) - Static method in class gobblin.util.WriterUtils
getWriterFs(State) - Static method in class gobblin.util.WriterUtils
getWriterOutputDir(State, int, int) - Static method in class gobblin.util.WriterUtils: Get the Path corresponding the to the directory a given DataWriter should be writing its output data.
getWriterStagingDir(State, int, int) - Static method in class gobblin.util.WriterUtils: Get the Path corresponding the to the directory a given DataWriter should be writing its staging data.
getWriterStagingDir(State, int, int, String) - Static method in class gobblin.util.WriterUtils: Get the staging Path for DataWriter that has attemptId in the path.
GLOB_TOKENS - Static variable in class gobblin.util.PathUtils
GLOBAL_PROPS_EXTENSION - Static variable in class gobblin.util.PullFileLoader
GLOBAL_PROPS_PATH_FILTER - Static variable in class gobblin.util.PullFileLoader
globStatus(Path) - Method in class gobblin.util.filesystem.ThrottledFileSystem
gobblin.broker - package gobblin.broker: A SharedResourcesBroker is an object that provides and manages objects that are accessed from multiple points in the application, allowing disconnected components to use shared objects, as well as easy user configuration for those objects.
gobblin.data.management.copy.hive - package gobblin.data.management.copy.hive
gobblin.runtime.cli - package gobblin.runtime.cli
gobblin.util - package gobblin.util
gobblin.util.binpacking - package gobblin.util.binpacking
gobblin.util.callbacks - package gobblin.util.callbacks: This package provides some simple instrumentation to handling callback execution.
gobblin.util.concurrent - package gobblin.util.concurrent
gobblin.util.dataset - package gobblin.util.dataset
gobblin.util.deprecation - package gobblin.util.deprecation
gobblin.util.executors - package gobblin.util.executors
gobblin.util.filesystem - package gobblin.util.filesystem
gobblin.util.filters - package gobblin.util.filters
gobblin.util.guid - package gobblin.util.guid
gobblin.util.hadoop - package gobblin.util.hadoop
gobblin.util.http - package gobblin.util.http
gobblin.util.io - package gobblin.util.io
gobblin.util.iterators - package gobblin.util.iterators
gobblin.util.jdbc - package gobblin.util.jdbc
gobblin.util.json - package gobblin.util.json
gobblin.util.limiter - package gobblin.util.limiter
gobblin.util.limiter.broker - package gobblin.util.limiter.broker
gobblin.util.logs - package gobblin.util.logs
gobblin.util.recordcount - package gobblin.util.recordcount
gobblin.util.reflection - package gobblin.util.reflection
gobblin.util.request_allocation - package gobblin.util.request_allocation
gobblin.util.service - package gobblin.util.service
gobblin.util.test - package gobblin.util.test
GOBBLIN_UTIL_JDBC_PREFIX - Static variable in class gobblin.util.jdbc.DataSourceProvider
GobblinCli - Class in gobblin.runtime.cli: Instantiates a CliApplication and runs it.
GobblinCli() - Constructor for class gobblin.runtime.cli.GobblinCli
GobblinConstructorUtils - Class in gobblin.util.reflection: Helper methods to instantiate classes
GobblinConstructorUtils() - Constructor for class gobblin.util.reflection.GobblinConstructorUtils
GreedyAllocator<T extends Request<T>> - Class in gobblin.util.request_allocation: A RequestAllocator that selects Requests without any order guarantees until the ResourcePool is full, then stops.
GreedyAllocator(RequestAllocatorConfig<T>) - Constructor for class gobblin.util.request_allocation.GreedyAllocator
GreedyAllocator.Factory - Class in gobblin.util.request_allocation
GsonInterfaceAdapter - Class in gobblin.util.io: A Gson interface adapter that makes it possible to serialize and deserialize polymorphic objects.
GsonInterfaceAdapter() - Constructor for class gobblin.util.io.GsonInterfaceAdapter
Guid - Class in gobblin.util.guid: Class wrapping a byte array representing a guid.
Guid(byte[]) - Constructor for class gobblin.util.guid.Guid: Creates a Guid by computing the sha of the input bytes.
guid() - Method in interface gobblin.util.guid.HasGuid
GUID_LENGTH - Static variable in class gobblin.util.guid.Guid

H

HadoopUtils - Class in gobblin.util: A utility class for working with Hadoop.
HadoopUtils() - Constructor for class gobblin.util.HadoopUtils
hasContent(FileSystem, Path) - Static method in class gobblin.util.HadoopUtils
hasFailed() - Method in class gobblin.util.callbacks.CallbackResult
HasGuid - Interface in gobblin.util.guid: Represents an object for which we can compute a unique, replicable Guid.
hashCode() - Method in class gobblin.broker.SharedResourcesBrokerImpl
hashCode() - Method in class gobblin.util.filesystem.FileStatusEntry: Returns a hash code value for the object, which is defined as the hash code of the path name.
hashCode() - Method in class gobblin.util.filesystem.FileSystemKey
hashCode() - Method in class gobblin.util.filesystem.FileSystemLimiterKey
hashCode() - Method in class gobblin.util.http.HttpLimiterKey
hashCode() - Method in class gobblin.util.ImmutableProperties
hashCode() - Method in class gobblin.util.io.StreamCopierSharedLimiterKey
hasNext() - Method in class gobblin.util.iterators.InterruptibleIterator
hasNext() - Method in class gobblin.util.request_allocation.AllocatedRequestsIteratorBase
hasNonEmptyPath(Config, String) - Static method in class gobblin.util.ConfigUtils: Check if the given key exists in config and it is not null or empty Uses StringUtils.isNotBlank(CharSequence)
hasRejectedElement() - Method in class gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable
HDFS_ILLEGAL_TOKEN_REGEX - Static variable in class gobblin.util.HadoopUtils
HeapDumpForTaskUtils - Class in gobblin.util: A utility class for generating script to move the heap dump .prof files to HDFS for hadoop tasks, when Java heap out of memory error is thrown.
HeapDumpForTaskUtils() - Constructor for class gobblin.util.HeapDumpForTaskUtils
HELLO_ID_FULL_KEY - Static variable in class gobblin.util.test.HelloWorldSource
HELLO_ID_KEY - Static variable in class gobblin.util.test.HelloWorldSource
helloMessage(int) - Static method in class gobblin.util.test.HelloWorldSource.ExtractorImpl
HelloWorldSource - Class in gobblin.util.test: Hello world!
HelloWorldSource() - Constructor for class gobblin.util.test.HelloWorldSource
HelloWorldSource.ExtractorImpl - Class in gobblin.util.test
HelpArgumentFound() - Constructor for exception gobblin.runtime.cli.CliObjectFactory.HelpArgumentFound
HiddenFilter - Class in gobblin.util.filters: A PathFilter that filters out hidden files (those starting with '_' or '.').
HiddenFilter() - Constructor for class gobblin.util.filters.HiddenFilter
HierarchicalAllocator<T extends Request<T>> - Class in gobblin.util.request_allocation: A RequestAllocator optimized for HierarchicalPrioritizers.
HierarchicalAllocator() - Constructor for class gobblin.util.request_allocation.HierarchicalAllocator
HierarchicalAllocator.Factory - Class in gobblin.util.request_allocation
HierarchicalPrioritizer<T extends Request> - Interface in gobblin.util.request_allocation: A Comparator for Requests that can also compare Requestors, and which guarantees that given Requests r1, r2, then r1.getRequestor > r2.getRequestor implies r1 > r2.
HIVE_EXECUTION_SIMULATE - Static variable in class gobblin.util.HiveJdbcConnector
HiveJdbcConnector - Class in gobblin.util: A class for managing a Hive JDBC connection.
HIVESERVER_CONNECTION_STRING - Static variable in class gobblin.util.HiveJdbcConnector
HIVESERVER_PASSWORD - Static variable in class gobblin.util.HiveJdbcConnector
HIVESERVER_URL - Static variable in class gobblin.util.HiveJdbcConnector
HIVESERVER_USER - Static variable in class gobblin.util.HiveJdbcConnector
HIVESERVER_VERSION - Static variable in class gobblin.util.HiveJdbcConnector
HIVESITE_DIR - Static variable in class gobblin.util.HiveJdbcConnector
HostUtils - Class in gobblin.util
HostUtils() - Constructor for class gobblin.util.HostUtils
HttpLimiterKey - Class in gobblin.util.http: HttpLimiterKey used for http client throttling.
HttpLimiterKey(String) - Constructor for class gobblin.util.http.HttpLimiterKey
HUMAN_READABLE_SIZE_PATTERN - Static variable in class gobblin.util.StringParsingUtils
humanReadableToByteCount(String) - Static method in class gobblin.util.StringParsingUtils: Convert a human readable string (e.g.

I

Id - Class in gobblin.util
Id(String) - Constructor for class gobblin.util.Id
Id(String, long) - Constructor for class gobblin.util.Id
Id.Job - Class in gobblin.util
Id.MultiTask - Class in gobblin.util
Id.Parts - Enum in gobblin.util
Id.Task - Class in gobblin.util
ImmutableProperties - Class in gobblin.util: Immutable wrapper for Properties.
ImmutableProperties(Properties) - Constructor for class gobblin.util.ImmutableProperties
inferFromConfig(RequestAllocatorConfig<T>) - Static method in class gobblin.util.request_allocation.RequestAllocatorUtils: Infer and construct a RequestAllocator from an input Config.
IngestionRecordCountProvider - Class in gobblin.util.recordcount: Implementation of RecordCountProvider, which provides record count from file path.
IngestionRecordCountProvider() - Constructor for class gobblin.util.recordcount.IngestionRecordCountProvider
initialize(URI, Configuration) - Method in class gobblin.util.filesystem.InstrumentedFileSystem
initialize() - Method in class gobblin.util.filesystem.PathAlterationObserver: Initialize the observer.
INSTANCE - Static variable in class gobblin.util.NoopCloseable
INSTANCE_NAME - Static variable in enum gobblin.util.Id.Parts
InstrumentedFileSystem - Class in gobblin.util.filesystem: A base FileSystem that uses FileSystemFactory to create an underlying FileSystem with available instrumentations.
InstrumentedFileSystem(String, FileSystem) - Constructor for class gobblin.util.filesystem.InstrumentedFileSystem
InstrumentedFileSystemUtils - Class in gobblin.util.filesystem: Common methods for Instrumented FileSystems.
InstrumentedFileSystemUtils() - Constructor for class gobblin.util.filesystem.InstrumentedFileSystemUtils
InstrumentedHDFSFileSystem - Class in gobblin.util.filesystem: A InstrumentedFileSystem for the hdfs scheme.
InstrumentedHDFSFileSystem() - Constructor for class gobblin.util.filesystem.InstrumentedHDFSFileSystem
InstrumentedLocalFileSystem - Class in gobblin.util.filesystem: A InstrumentedFileSystem for the file scheme.
InstrumentedLocalFileSystem() - Constructor for class gobblin.util.filesystem.InstrumentedLocalFileSystem
InstrumentedWebHDFSFileSystem - Class in gobblin.util.filesystem: A InstrumentedFileSystem for the webhdfs scheme.
InstrumentedWebHDFSFileSystem() - Constructor for class gobblin.util.filesystem.InstrumentedWebHDFSFileSystem
instrumentFileSystem(FileSystem, SharedResourcesBroker<S>, ConfigView<S, FileSystemKey>) - Method in class gobblin.util.filesystem.FileSystemInstrumentationFactory: Return an instrumented version of the input FileSystem.
instrumentFileSystem(FileSystem, SharedResourcesBroker<S>, ConfigView<S, FileSystemKey>) - Method in class gobblin.util.filesystem.ThrottledFileSystem.Factory
InterruptibleIterator<T> - Class in gobblin.util.iterators: An iterator that allows ending prematurely (i.e.
InterruptibleIterator() - Constructor for class gobblin.util.iterators.InterruptibleIterator
invokeConstructor(Class<T>, String, Object...) - Static method in class gobblin.util.reflection.GobblinConstructorUtils: Utility method to create an instance of clsName using the constructor matching the arguments, args
invokeFirstConstructor(Class<T>, List<Object>...) - Static method in class gobblin.util.reflection.GobblinConstructorUtils: Convenience method on top of ConstructorUtils.invokeConstructor(Class, Object[]) that returns a new instance of the cls based on a constructor priority order.
invokeLongestConstructor(Class<T>, Object...) - Static method in class gobblin.util.reflection.GobblinConstructorUtils: Returns a new instance of the cls based on a set of arguments.
isAbsoluteAndSchemeAuthorityNull(Path) - Static method in class gobblin.util.PathUtils: Is an absolute path (ie a slash relative path part) AND a scheme is null AND authority is null.
isAncestor(Path, Path) - Static method in class gobblin.util.PathUtils: Checks whether possibleAncestor is an ancestor of fullPath.
isCompressed(byte[]) - Static method in class gobblin.util.io.StreamUtils
isDirectory() - Method in class gobblin.util.filesystem.FileStatusEntry: Return whether the path is a directory from the instance FileStatus variable.
isExists() - Method in class gobblin.util.filesystem.FileStatusEntry: Indicate whether the file existed the last time it was checked.
isFull() - Method in class gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable
isGlob(Path) - Static method in class gobblin.util.PathUtils
isScopeTypeAncestor(S, S) - Static method in class gobblin.broker.SharedResourcesBrokerUtils: Determine if a ScopeType is an ancestor of another ScopeType.
isSuccessful() - Method in class gobblin.util.callbacks.CallbackResult
isTimeInRange(List<String>, String, String, DateTime) - Static method in class gobblin.util.TimeRangeChecker: Checks if a specified time is on a day that is specified the a given List of acceptable days, and that the hours + minutes of the specified time fall into a range defined by startTimeStr and endTimeStr.
isValid() - Method in class gobblin.broker.ResourceInstance
isValid() - Method in class gobblin.broker.TTLResourceEntry
iterator() - Method in class gobblin.util.EmptyIterable
iterator() - Method in class gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable
IteratorExecutor<T> - Class in gobblin.util.executors: Executes tasks in an Iterator.
IteratorExecutor(Iterator<Callable<T>>, int, ThreadFactory) - Constructor for class gobblin.util.executors.IteratorExecutor

J

JobConfigurationUtils - Class in gobblin.util: A utility class for working with job configurations.
JobConfigurationUtils() - Constructor for class gobblin.util.JobConfigurationUtils
JobLauncherUtils - Class in gobblin.util: Utility class for the job scheduler and job launchers.
JobLauncherUtils() - Constructor for class gobblin.util.JobLauncherUtils
JsonUtils - Class in gobblin.util.json: JSON related utilities
JsonUtils() - Constructor for class gobblin.util.json.JsonUtils
JvmUtils - Class in gobblin.util

K

KeyedScopedConfigViewImpl<S extends ScopeType<S>,K extends SharedResourceKey> - Class in gobblin.broker: An implementation of ScopedConfigView that knows how to extract relevant subconfiguration from an input Config.
KeyedScopedConfigViewImpl() - Constructor for class gobblin.broker.KeyedScopedConfigViewImpl
klazz - Variable in class gobblin.runtime.cli.PublicMethodsCliObjectFactory

L

LateFileRecordCountProvider - Class in gobblin.util.recordcount
LateFileRecordCountProvider() - Constructor for class gobblin.util.recordcount.LateFileRecordCountProvider
LATEST_MOD_TIME_ORDER - Static variable in class gobblin.util.FileListUtils
left(S) - Static method in class gobblin.util.Either: Create an instance of Either with value of type S.
Left(S) - Constructor for class gobblin.util.Either.Left
Limiter - Interface in gobblin.util.limiter: An interface for classes that implement some logic limiting on the occurrences of some events, e.g., data record extraction using an Extractor.
LIMITER_CLASS_KEY - Static variable in class gobblin.util.limiter.broker.SharedLimiterFactory
LimiterFactory - Interface in gobblin.util.limiter: Interface for factories that build Limiters.
listFilesRecursively(FileSystem, Path) - Static method in class gobblin.util.FileListUtils
listFilesRecursively(FileSystem, Iterable<Path>) - Static method in class gobblin.util.FileListUtils
listFilesRecursively(FileSystem, Path, PathFilter) - Static method in class gobblin.util.FileListUtils: Helper method to list out all files under a specified path.
listFilesRecursively(FileSystem, Path, PathFilter, boolean) - Static method in class gobblin.util.FileListUtils: Helper method to list out all files under a specified path.
LISTING_FILES_PER_PERMIT - Static variable in class gobblin.util.filesystem.ThrottledFileSystem: Listing operations will use 1 permit per this many listed elements.
listMostNestedPathRecursively(FileSystem, Path) - Static method in class gobblin.util.FileListUtils: Method to list out all files, or directory if no file exists, under a specified path.
listMostNestedPathRecursively(FileSystem, Iterable<Path>) - Static method in class gobblin.util.FileListUtils
listMostNestedPathRecursively(FileSystem, Path, PathFilter) - Static method in class gobblin.util.FileListUtils: Method to list out all files, or directory if no file exists, under a specified path.
listPathsRecursively(FileSystem, Path, PathFilter) - Static method in class gobblin.util.FileListUtils: Helper method to list out all paths under a specified path.
listStatus(Path) - Method in class gobblin.util.filesystem.ThrottledFileSystem
listStatus(Path, PathFilter) - Method in class gobblin.util.filesystem.ThrottledFileSystem
listStatusRecursive(FileSystem, Path) - Static method in class gobblin.util.HadoopUtils: Deprecated.
Use FileListUtils.listFilesRecursively(FileSystem, Path).
LOAD_HADOOP_CONFIGURATION - Static variable in class gobblin.broker.SharedResourcesBrokerFactory
loadPullFile(Path, Config, boolean) - Method in class gobblin.util.PullFileLoader: Load a single pull file.
loadPullFilesRecursively(Path, Config, boolean) - Method in class gobblin.util.PullFileLoader: Find and load all pull files under a base Path recursively.
Log4jConfigurationHelper - Class in gobblin.util.logs: A helper class for programmatically configuring log4j.
Log4jConfigurationHelper() - Constructor for class gobblin.util.logs.Log4jConfigurationHelper
LogCopier - Class in gobblin.util.logs: A utility class that periodically reads log files in a source log file directory for changes since the last reads and appends the changes to destination log files with the same names as the source log files in a destination log directory.
LogCopier.Builder - Class in gobblin.util.logs: A builder class for LogCopier.
logFailures(List<Either<T, ExecutionException>>, Logger, int) - Static method in class gobblin.util.executors.IteratorExecutor: Log failures in the output of IteratorExecutor.executeAndGetResults().
loggingDecorator(ExecutorService) - Static method in class gobblin.util.ExecutorsUtils: Creates an ListeningExecutorService whose submit and execute methods propagate the MDC information across thread boundaries.
loggingDecorator(ScheduledExecutorService) - Static method in class gobblin.util.ExecutorsUtils: Creates an ListeningScheduledExecutorService whose submit, execute, schedule, scheduleAtFixedRate, and scheduleWithFixedDelay methods propagate the MDC information across thread boundaries.
loggingDecorator(Runnable) - Static method in class gobblin.util.ExecutorsUtils: Creates an Runnable which propagates the MDC information across thread boundaries.
loggingDecorator(Callable<T>) - Static method in class gobblin.util.ExecutorsUtils: Creates an Callable which propagates the MDC information across thread boundaries.
LoggingUncaughtExceptionHandler - Class in gobblin.util: A type of Thread.UncaughtExceptionHandler that logs uncaught exceptions.
LoggingUncaughtExceptionHandler(Optional<Logger>) - Constructor for class gobblin.util.LoggingUncaughtExceptionHandler
logStatistics(Optional<Logger>) - Method in class gobblin.util.request_allocation.ConcurrentBoundedPriorityIterable: Log statistics about this ConcurrentBoundedPriorityIterable.

M

M_OUTPUT_FILE_PREFIX - Static variable in class gobblin.util.recordcount.CompactionRecordCountProvider
main(String[]) - Static method in class gobblin.runtime.cli.GobblinCli
main(String[]) - Static method in class gobblin.util.CLIPasswordEncryptor
mark() - Method in class gobblin.util.io.BatchedMeterDecorator
mark(long) - Method in class gobblin.util.io.BatchedMeterDecorator
MAX_ACTIVE_CONNS - Static variable in class gobblin.util.jdbc.DataSourceProvider
MAX_FILESYSTEM_QPS - Static variable in class gobblin.util.HadoopUtils
MAX_IDLE_CONNS - Static variable in class gobblin.util.jdbc.DataSourceProvider
MAX_RECORDS_PER_WORK_UNIT - Static variable in class gobblin.util.test.FastSequentialSource
MAX_SECONDS_KEY - Static variable in class gobblin.util.limiter.TimeBasedLimiter.Factory
MAX_SECONDS_PER_WORK_UNIT - Static variable in class gobblin.util.test.FastSequentialSource
maxActiveConnections(int) - Method in class gobblin.util.jdbc.DataSourceBuilder
maxIdleConnections(int) - Method in class gobblin.util.jdbc.DataSourceBuilder
MAXIMUM_PORT - Static variable in class gobblin.util.PortUtils
MDCPropagatingCallable<T> - Class in gobblin.util.executors
MDCPropagatingCallable(Callable<T>) - Constructor for class gobblin.util.executors.MDCPropagatingCallable
MDCPropagatingExecutorService - Class in gobblin.util.executors
MDCPropagatingExecutorService(ExecutorService) - Constructor for class gobblin.util.executors.MDCPropagatingExecutorService
MDCPropagatingRunnable - Class in gobblin.util.executors
MDCPropagatingRunnable(Runnable) - Constructor for class gobblin.util.executors.MDCPropagatingRunnable
MDCPropagatingScheduledExecutorService - Class in gobblin.util.executors
MDCPropagatingScheduledExecutorService(ScheduledExecutorService) - Constructor for class gobblin.util.executors.MDCPropagatingScheduledExecutorService
mergePaths(Path, Path) - Static method in class gobblin.util.PathUtils
mergeTemplateWithUserCustomizedFile(Properties, Properties) - Static method in class gobblin.util.TemplateUtils: create a complete property file based on the given template
MeteredInputStream - Class in gobblin.util.io: A FilterInputStream that counts the bytes read from the underlying InputStream.
MeteredOutputStream - Class in gobblin.util.io: A FilterOutputStream that counts the bytes read from the underlying OutputStream.
MeteredOutputStream(OutputStream, Meter, int) - Constructor for class gobblin.util.io.MeteredOutputStream: Builds a MeteredOutputStream.
MeteredStream - Interface in gobblin.util.io: A InputStream or OutputStream that measures bytes processed.
MINIMUM_PORT - Static variable in class gobblin.util.PortUtils
mkdirs(Path, FsPermission) - Method in class gobblin.util.filesystem.ThrottledFileSystem
mkdirsWithRecursivePermission(FileSystem, Path, FsPermission) - Static method in class gobblin.util.WriterUtils: Create the given dir as well as all missing ancestor dirs.
movePath(FileSystem, Path, FileSystem, Path, Configuration) - Static method in class gobblin.util.HadoopUtils: Moves a src Path from a srcFs FileSystem to a dst Path on a dstFs FileSystem.
movePath(FileSystem, Path, FileSystem, Path, boolean, Configuration) - Static method in class gobblin.util.HadoopUtils: Moves a src Path from a srcFs FileSystem to a dst Path on a dstFs FileSystem.
movePath(Path, FileSystem, Path, Optional<String>) - Method in class gobblin.util.ParallelRunner: Move a Path.
movePath(Path, FileSystem, Path, boolean, Optional<String>) - Method in class gobblin.util.ParallelRunner: Move a Path.
MR_OUTPUT_FILE_PREFIX - Static variable in class gobblin.util.recordcount.CompactionRecordCountProvider
MultiLimiter - Class in gobblin.util.limiter: A Limiter that contains multiple underlying Limiters and requests permits from all those limiters.
MultiLimiter(Limiter...) - Constructor for class gobblin.util.limiter.MultiLimiter

N

NAME - Static variable in class gobblin.util.io.StreamThrottler.Factory
NAME - Static variable in class gobblin.util.limiter.broker.SharedLimiterFactory
NamedFuture() - Constructor for class gobblin.util.ParallelRunner.NamedFuture
newAppId(String) - Static method in class gobblin.util.ApplicationLauncherUtils: Create a new app ID.
newBuilder() - Static method in class gobblin.util.logs.LogCopier: Get a new LogCopier.Builder instance for building a LogCopier.
newChildInstance(Path) - Method in class gobblin.util.filesystem.FileStatusEntry: Create a new child instance.
newConfiguration() - Static method in class gobblin.util.HadoopUtils
newConnectorWithProps(Properties) - Static method in class gobblin.util.HiveJdbcConnector: Create a new HiveJdbcConnector based on the configs in a Properties object
newDaemonThreadFactory(Optional<Logger>, Optional<String>) - Static method in class gobblin.util.ExecutorsUtils: Get a new ThreadFactory that uses a LoggingUncaughtExceptionHandler to handle uncaught exceptions, uses the given thread name format, and produces daemon threads.
newEmbeddedConnector(int) - Static method in class gobblin.util.HiveJdbcConnector: Create a new HiveJdbcConnector using the specified Hive server version.
newJobId(String) - Static method in class gobblin.util.JobLauncherUtils: Create a new job ID.
newLimiter(State) - Static method in class gobblin.util.limiter.DefaultLimiterFactory: Create a new Limiter instance of one of the types in BaseLimiterType.
newMultiTaskId(String, int) - Static method in class gobblin.util.JobLauncherUtils: Create an ID for a new multi-task (corresponding to a MultiWorkUnit) for the job with the given job ID.
newScalingThreadPool(int, int, long) - Static method in class gobblin.util.executors.ScalingThreadPoolExecutor: Creates a ScalingThreadPoolExecutor.
newScalingThreadPool(int, int, long, ThreadFactory) - Static method in class gobblin.util.executors.ScalingThreadPoolExecutor: Creates a ScalingThreadPoolExecutor.
newSubscopedBuilder(ScopeInstance<S>) - Method in class gobblin.broker.SharedResourcesBrokerImpl: Get a builder to create a descendant SharedResourcesBrokerImpl (i.e.
newTaskId(String, int) - Static method in class gobblin.util.JobLauncherUtils: Create a new task ID for the job with the given job ID.
newThreadFactory(Optional<Logger>) - Static method in class gobblin.util.ExecutorsUtils: Get a new ThreadFactory that uses a LoggingUncaughtExceptionHandler to handle uncaught exceptions.
newThreadFactory(Optional<Logger>, Optional<String>) - Static method in class gobblin.util.ExecutorsUtils: Get a new ThreadFactory that uses a LoggingUncaughtExceptionHandler to handle uncaught exceptions and the given thread name format.
next() - Method in class gobblin.util.iterators.InterruptibleIterator
next() - Method in class gobblin.util.request_allocation.AllocatedRequestsIteratorBase
NO_OP_CLOSEABLE - Static variable in class gobblin.util.limiter.NonRefillableLimiter
NO_OP_PATH_FILTER - Static variable in class gobblin.util.FileListUtils
NoMoreRetriesException(String) - Constructor for exception gobblin.util.ExponentialBackoff.NoMoreRetriesException
NonRefillableLimiter - Class in gobblin.util.limiter: A type of Limiters that do not support permit refills by returning a no-op Closeable in NonRefillableLimiter.acquirePermits(long).
NonRefillableLimiter() - Constructor for class gobblin.util.limiter.NonRefillableLimiter
NoopCloseable - Class in gobblin.util: A Closeable that does nothing.
NoopCloseable() - Constructor for class gobblin.util.NoopCloseable
NoopLimiter - Class in gobblin.util.limiter: A Limiter that satisfies all requests.
NoopLimiter() - Constructor for class gobblin.util.limiter.NoopLimiter
NotEnoughPermitsException - Exception in gobblin.util.limiter: Indicates there were not enough permits in the Limiter to finish the copy.
NotEnoughPermitsException(String) - Constructor for exception gobblin.util.limiter.NotEnoughPermitsException
NotEnoughPermitsException(String, Throwable) - Constructor for exception gobblin.util.limiter.NotEnoughPermitsException
NotOnCli - Annotation Type in gobblin.runtime.cli: Specifies that the annotated method should not be offered as an option in the CLI.
nullifyFieldsForSchemaMerge(Schema, Schema) - Static method in class gobblin.util.AvroUtils: Merge oldSchema and newSchame.
NUM_HELLOS_FULL_KEY - Static variable in class gobblin.util.test.HelloWorldSource
NUM_HELLOS_KEY - Static variable in class gobblin.util.test.HelloWorldSource
NUM_WORK_UNITS - Static variable in class gobblin.util.test.FastSequentialSource

O

OBJECT_DATA - Static variable in class gobblin.util.io.GsonInterfaceAdapter
OBJECT_TYPE - Static variable in class gobblin.util.io.GsonInterfaceAdapter
onClose() - Method in class gobblin.util.filesystem.FileSystemInstrumentation: A method called when the FileSystem is being closed or when the JVM is shutting down.
onDirectoryChange(Path) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onDirectoryChange(Path) - Method in interface gobblin.util.filesystem.PathAlterationListener
onDirectoryChange(Path) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onDirectoryCreate(Path) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onDirectoryCreate(Path) - Method in interface gobblin.util.filesystem.PathAlterationListener
onDirectoryCreate(Path) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onDirectoryDelete(Path) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onDirectoryDelete(Path) - Method in interface gobblin.util.filesystem.PathAlterationListener
onDirectoryDelete(Path) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onFileChange(Path) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onFileChange(Path) - Method in interface gobblin.util.filesystem.PathAlterationListener
onFileChange(Path) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onFileCreate(Path) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onFileCreate(Path) - Method in interface gobblin.util.filesystem.PathAlterationListener
onFileCreate(Path) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onFileDelete(Path) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onFileDelete(Path) - Method in interface gobblin.util.filesystem.PathAlterationListener
onFileDelete(Path) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onInvalidate() - Method in class gobblin.broker.ResourceInstance
onInvalidate() - Method in class gobblin.broker.TTLResourceEntry
onStart(PathAlterationObserver) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onStart(PathAlterationObserver) - Method in interface gobblin.util.filesystem.PathAlterationListener
onStart(PathAlterationObserver) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
onStop(PathAlterationObserver) - Method in class gobblin.util.filesystem.ExceptionCatchingPathAlterationListenerDecorator
onStop(PathAlterationObserver) - Method in interface gobblin.util.filesystem.PathAlterationListener
onStop(PathAlterationObserver) - Method in class gobblin.util.filesystem.PathAlterationListenerAdaptor
open(Path, int) - Method in class gobblin.util.filesystem.ThrottledFileSystem

P

pack(List<WorkUnit>, WorkUnitWeighter) - Method in class gobblin.util.binpacking.WorstFitDecreasingBinPacking
PARALLEL_RUNNER_THREADS_KEY - Static variable in class gobblin.util.ParallelRunner
parallelize(List<F>, Function<F, T>, int, int, Optional<Logger>) - Static method in class gobblin.util.ExecutorsUtils: A utility method to parallelize loops.
ParallelRunner - Class in gobblin.util: A class that is responsible for running certain methods in parallel.
ParallelRunner(int, FileSystem) - Constructor for class gobblin.util.ParallelRunner
ParallelRunner(int, FileSystem, ParallelRunner.FailPolicy) - Constructor for class gobblin.util.ParallelRunner
ParallelRunner.FailPolicy - Enum in gobblin.util: Policies indicating how ParallelRunner should handle failure of tasks.
ParallelRunner.NamedFuture - Class in gobblin.util: A future with a name / message for reporting.
parse(String) - Static method in enum gobblin.util.concurrent.TaskSchedulerType: Return the TaskSchedulerType with the specified name.
parse(String) - Static method in class gobblin.util.Id
parseSchemaFromFile(Path, FileSystem) - Static method in class gobblin.util.AvroUtils: Parse Avro schema from a schema file.
passWord(String) - Method in class gobblin.util.jdbc.DataSourceBuilder
PASSWORD - Static variable in class gobblin.util.jdbc.DataSourceProvider
PathAlterationListener - Interface in gobblin.util.filesystem: A listener that receives events of general file system modifications.
PathAlterationListenerAdaptor - Class in gobblin.util.filesystem
PathAlterationListenerAdaptor() - Constructor for class gobblin.util.filesystem.PathAlterationListenerAdaptor
PathAlterationObserver - Class in gobblin.util.filesystem
PathAlterationObserver(String) - Constructor for class gobblin.util.filesystem.PathAlterationObserver: Construct an observer for the specified directory.
PathAlterationObserver(String, PathFilter) - Constructor for class gobblin.util.filesystem.PathAlterationObserver: Construct an observer for the specified directory and file filter.
PathAlterationObserver(Path) - Constructor for class gobblin.util.filesystem.PathAlterationObserver: Construct an observer for the specified directory.
PathAlterationObserver(Path, PathFilter) - Constructor for class gobblin.util.filesystem.PathAlterationObserver: Construct an observer for the specified directory and file filter.
PathAlterationObserver(FileStatusEntry, PathFilter) - Constructor for class gobblin.util.filesystem.PathAlterationObserver: The comparison between path is always case-sensitive in this general file system context.
PathAlterationObserverScheduler - Class in gobblin.util.filesystem: A runnable that spawns a monitoring thread triggering any registered PathAlterationObserver at a specified interval.
PathAlterationObserverScheduler() - Constructor for class gobblin.util.filesystem.PathAlterationObserverScheduler
PathAlterationObserverScheduler(long) - Constructor for class gobblin.util.filesystem.PathAlterationObserverScheduler
PathUtils - Class in gobblin.util
PathUtils() - Constructor for class gobblin.util.PathUtils
POOL_SIZE_KEY - Static variable in class gobblin.util.limiter.PoolBasedLimiter.Factory
PoolBasedLimiter - Class in gobblin.util.limiter: An implementation of Limiter that ony allows permits to be acquired from a pool.
PoolBasedLimiter(int) - Constructor for class gobblin.util.limiter.PoolBasedLimiter
PoolBasedLimiter.Factory - Class in gobblin.util.limiter
PortUtils - Class in gobblin.util
PortUtils() - Constructor for class gobblin.util.PortUtils
PREFIX - Static variable in class gobblin.util.Id.Job
PREFIX - Static variable in class gobblin.util.Id.MultiTask
PREFIX - Static variable in class gobblin.util.Id.Task
PreOrderAllocator<T extends Request<T>> - Class in gobblin.util.request_allocation: A RequestAllocator that operates over PushDownRequestors, getting a pre-ordered iterator of Requests from each Requestor and merging them, stopping as soon as the ResourcePool is exhausted.
PreOrderAllocator(RequestAllocatorConfig<T>) - Constructor for class gobblin.util.request_allocation.PreOrderAllocator
PreOrderAllocator.Factory - Class in gobblin.util.request_allocation
PriorityIterableBasedRequestAllocator<T extends Request<T>> - Class in gobblin.util.request_allocation
PriorityIterableBasedRequestAllocator() - Constructor for class gobblin.util.request_allocation.PriorityIterableBasedRequestAllocator
PROPERTIES_TO_TAGS_MAP - Static variable in class gobblin.util.AzkabanTags
propertiesToConfig(Properties) - Static method in class gobblin.util.ConfigUtils: Convert a given Properties to a Config instance.
propertiesToConfig(Properties, Optional<String>) - Static method in class gobblin.util.ConfigUtils: Convert all the keys that start with a prefix in Properties to a Config instance.
propertiesToTypedConfig(Properties, Optional<String>) - Static method in class gobblin.util.ConfigUtils: Convert all the keys that start with a prefix in Properties to a Config instance.
PropertiesUtils - Class in gobblin.util: A utility class for Properties objects.
PropertiesUtils() - Constructor for class gobblin.util.PropertiesUtils
propsToStringKeyMap(Properties) - Static method in class gobblin.util.PropertiesUtils: Converts a Properties object to a Map where each key is a String.
ProxiedFileSystemCache - Class in gobblin.util: A cache for storing a mapping between Hadoop users and user FileSystem objects.
ProxiedFileSystemCache() - Constructor for class gobblin.util.ProxiedFileSystemCache
ProxiedFileSystemUtils - Class in gobblin.util: Utility class for creating FileSystem objects while proxied as another user.
ProxiedFileSystemUtils() - Constructor for class gobblin.util.ProxiedFileSystemUtils
ProxiedFileSystemUtils.AuthType - Enum in gobblin.util
ProxiedFileSystemWrapper - Class in gobblin.util: Deprecated.
ProxiedFileSystemWrapper() - Constructor for class gobblin.util.ProxiedFileSystemWrapper: Deprecated.
ProxiedFileSystemWrapper.AuthType - Enum in gobblin.util: Deprecated.
Use ProxiedFileSystemUtils.AuthType.
PublicMethodsCliObjectFactory<T> - Class in gobblin.runtime.cli: A helper class for automatically inferring Options from the public methods in a class.
PublicMethodsCliObjectFactory(Class<? extends T>) - Constructor for class gobblin.runtime.cli.PublicMethodsCliObjectFactory
PublisherUtils - Class in gobblin.util: Utility class for DataPublisher.
PublisherUtils() - Constructor for class gobblin.util.PublisherUtils
PublisherUtils.SplitExtractsResult - Class in gobblin.util
PublisherUtils.WorkUnitStateSuccess - Class in gobblin.util: Implementation of Predicate that checks if a given WorkUnitState has a WorkUnitState.WorkingState equal to WorkUnitState.WorkingState.SUCCESSFUL.
PullFileLoader - Class in gobblin.util: Used to load pull files from the file system.
PullFileLoader(Path, FileSystem, Collection<String>, Collection<String>) - Constructor for class gobblin.util.PullFileLoader
PushDownRequestor<T extends Request> - Interface in gobblin.util.request_allocation: A Requestor that can provide an Iterator of Requests already sorted by the input prioritizer.
putConfigurationIntoProperties(Configuration, Properties) - Static method in class gobblin.util.JobConfigurationUtils: Put all configuration properties in a given Configuration object into a given Properties object.
putPropertiesIntoConfiguration(Properties, Configuration) - Static method in class gobblin.util.JobConfigurationUtils: Put all configuration properties in a given Properties object into a given Configuration object.
putStateIntoConfiguration(State, Configuration) - Static method in class gobblin.util.JobConfigurationUtils: Put all configuration properties in a given State object into a given Configuration object.

Q

QPS_KEY - Static variable in class gobblin.util.limiter.RateBasedLimiter.Factory

R

RANDOM - Static variable in class gobblin.util.request_allocation.AllocatedRequestsIteratorBase.RequestWithResourceRequirement
RateBasedLimiter - Class in gobblin.util.limiter: An implementation of Limiter that limits the rate of some events.
RateBasedLimiter(double) - Constructor for class gobblin.util.limiter.RateBasedLimiter
RateBasedLimiter(double, TimeUnit) - Constructor for class gobblin.util.limiter.RateBasedLimiter
RateBasedLimiter.Factory - Class in gobblin.util.limiter
RateControlledFileSystem - Class in gobblin.util: Deprecated.
use ThrottledFileSystem
RateControlledFileSystem(FileSystem, long) - Constructor for class gobblin.util.RateControlledFileSystem: Deprecated.
read() - Method in class gobblin.util.io.MeteredInputStream
read(byte[], int, int) - Method in class gobblin.util.io.MeteredInputStream
read(byte[], int, int) - Method in class gobblin.util.io.SeekableFSInputStream
read() - Method in class gobblin.util.io.SeekableFSInputStream
read() - Method in class gobblin.util.io.ThrottledInputStream
read(byte[]) - Method in class gobblin.util.io.ThrottledInputStream
read(byte[], int, int) - Method in class gobblin.util.io.ThrottledInputStream
readFrom(Path) - Method in class gobblin.util.logs.LogCopier.Builder: Set the path of the source log file directory to read from.
readRecord(Long) - Method in class gobblin.util.test.FastSequentialSource.FastSequentialExtractor
readRecord(String) - Method in class gobblin.util.test.HelloWorldSource.ExtractorImpl
readRecord(String) - Method in class gobblin.util.test.TestingSource.Extract
recordsWritten() - Method in class gobblin.util.test.StdoutWriter
recordToByteArray(GenericRecord) - Static method in class gobblin.util.AvroUtils: Convert a GenericRecord to a byte array.
refresh(Path) - Method in class gobblin.util.filesystem.FileStatusEntry
RegexPathFilter - Class in gobblin.util.filters: Use a regex to filter Path.
RegexPathFilter(String) - Constructor for class gobblin.util.filters.RegexPathFilter
RegexPathFilter(String, boolean) - Constructor for class gobblin.util.filters.RegexPathFilter
registerImplicitBroker(SharedResourcesBroker<?>) - Static method in class gobblin.broker.SharedResourcesBrokerFactory: Register a SharedResourcesBroker to be used as the implicit broker for this and all new children threads.
rejectedExecution(Runnable, ThreadPoolExecutor) - Method in class gobblin.util.executors.ForceQueuePolicy
relativizePath(Path, Path) - Static method in class gobblin.util.PathUtils
remove() - Method in class gobblin.util.iterators.InterruptibleIterator
remove() - Method in class gobblin.util.request_allocation.AllocatedRequestsIteratorBase
removeExtension(Path, String...) - Static method in class gobblin.util.PathUtils: Removes all extensions from path if they exist.
removeListener(L) - Method in class gobblin.util.callbacks.CallbacksDispatcher
removeListener(PathAlterationListener) - Method in class gobblin.util.filesystem.PathAlterationObserver: Remove a file system listener.
removeObserver(PathAlterationObserver) - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Remove a file system observer from this monitor.
removeUncomparableFields(Schema) - Static method in class gobblin.util.AvroUtils: Remove map, array, enum fields, as well as union fields that contain map, array or enum, from an Avro schema.
rename(Path, Path) - Method in class gobblin.util.filesystem.ThrottledFileSystem
renameDeprecatedKeys(State, String, List<String>) - Static method in class gobblin.util.deprecation.DeprecationUtils: Sets an option in a State to the first available value of a list of deprecatedKeys.
renamePath(FileSystem, Path, Path) - Static method in class gobblin.util.HadoopUtils: A wrapper around FileSystem.rename(Path, Path) which throws IOException if FileSystem.rename(Path, Path) returns False.
renamePath(FileSystem, Path, Path, boolean) - Static method in class gobblin.util.HadoopUtils: A wrapper around FileSystem.rename(Path, Path) which throws IOException if FileSystem.rename(Path, Path) returns False.
renamePath(Path, Path, Optional<String>) - Method in class gobblin.util.ParallelRunner: Rename a Path.
renamePathHandleLocalFSRace(FileSystem, Path, Path) - Static method in class gobblin.util.HadoopUtils: Renames a src Path on fs FileSystem to a dst Path.
renameRecursively(FileSystem, Path, Path) - Static method in class gobblin.util.HadoopUtils: This method is an additive implementation of the FileSystem.rename(Path, Path) method.
replacePortTokens(String) - Method in class gobblin.util.PortUtils: Replaces any port tokens in the specified string.
replaceScheme(URI, String, String) - Static method in class gobblin.util.filesystem.InstrumentedFileSystemUtils: Replace the scheme of the input URI if it matches the string to replace.
replaceScheme(Path, String, String) - Static method in class gobblin.util.filesystem.InstrumentedFileSystemUtils: Replace the scheme of the input Path if it matches the string to replace.
replaceScheme(Path[], String, String) - Static method in class gobblin.util.filesystem.InstrumentedFileSystemUtils: Replace the scheme of each Path if it matches the string to replace.
replaceScheme(FileStatus[], String, String) - Static method in class gobblin.util.filesystem.InstrumentedFileSystemUtils: Replace the scheme of each FileStatus if it matches the string to replace.
replaceScheme(FileStatus, String, String) - Static method in class gobblin.util.filesystem.InstrumentedFileSystemUtils: Replace the scheme of the input FileStatus if it matches the string to replace.
Request<T extends Request> - Interface in gobblin.util.request_allocation: Represents an expensive request in the request allocation problem.
RequestAllocator<T extends Request<T>> - Interface in gobblin.util.request_allocation: This interface is intended to solve the problem of selecting a subset of expensive requests based on priority when there are limited resources to perform such requests.
RequestAllocator.Factory - Interface in gobblin.util.request_allocation
RequestAllocatorConfig<T extends Request<T>> - Class in gobblin.util.request_allocation
RequestAllocatorConfig() - Constructor for class gobblin.util.request_allocation.RequestAllocatorConfig
RequestAllocatorConfig.AllEqualPrioritizer<T> - Class in gobblin.util.request_allocation
RequestAllocatorConfig.Builder<T extends Request<T>> - Class in gobblin.util.request_allocation
RequestAllocatorUtils - Class in gobblin.util.request_allocation
RequestAllocatorUtils() - Constructor for class gobblin.util.request_allocation.RequestAllocatorUtils
Requestor<T extends Request> - Interface in gobblin.util.request_allocation: A wrapper around a Iterator of Requests used for request allocation problem.
reset() - Method in class gobblin.util.io.ThrottledInputStream
resolveEncrypted(Config, Optional<String>) - Static method in class gobblin.util.ConfigUtils: Resolves encrypted config value(s) by considering on the path with "encConfigPath" as encrypted.
RESOURCE_LIMITED_PREFIX - Static variable in class gobblin.util.filesystem.FileSystemLimiterKey
ResourceEstimator<T> - Interface in gobblin.util.request_allocation: Computes the ResourceRequirement for Requests.
ResourceEstimator.Factory<T> - Interface in gobblin.util.request_allocation
ResourceInstance<T> - Class in gobblin.broker: A SharedResourceFactoryResponse that returns a newly created resource instance.
ResourceInstance() - Constructor for class gobblin.broker.ResourceInstance
ResourcePool - Class in gobblin.util.request_allocation: Represents a pool of available resources for a RequestAllocator.
ResourcePool(Map<String, Double>, Map<String, Double>, Map<String, Double>) - Constructor for class gobblin.util.request_allocation.ResourcePool
ResourcePool(ResourcePool) - Constructor for class gobblin.util.request_allocation.ResourcePool
ResourceRequirement - Class in gobblin.util.request_allocation: Represents a requirement of resources in a ResourcePool.
ResourceRequirement(ResourceRequirement) - Constructor for class gobblin.util.request_allocation.ResourceRequirement
ResourceRequirement.Builder - Class in gobblin.util.request_allocation
restoreFilePath(Path) - Static method in class gobblin.util.recordcount.LateFileRecordCountProvider: Remove the late components in the path added by LateFileRecordCountProvider.
right(T) - Static method in class gobblin.util.Either: Create an instance of Either with value of type T.
Right(T) - Constructor for class gobblin.util.Either.Right
run(String[]) - Method in interface gobblin.runtime.cli.CliApplication: Run this application.
run() - Method in class gobblin.util.executors.MDCPropagatingRunnable
run() - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler
runOneIteration() - Method in interface gobblin.util.concurrent.ScheduledTask: Run one iteration of the scheduled task.
runOneIteration() - Method in class gobblin.util.logs.LogCopier

S

s - Variable in class gobblin.util.Either
safeRenameIfNotExists(FileSystem, Path, Path) - Static method in class gobblin.util.HadoopUtils: Renames from to to if to doesn't exist in a thread-safe way.
safeRenameRecursively(FileSystem, Path, Path) - Static method in class gobblin.util.HadoopUtils: A thread safe variation of HadoopUtils.renamePath(FileSystem, Path, Path) which can be used in multi-threaded/multi-mapper environment.
sanitizeFullPrefixKey(String) - Static method in class gobblin.util.ConfigUtils
sanitizePath(String, String) - Static method in class gobblin.util.HadoopUtils: Remove illegal HDFS path characters from the given path.
sanitizePath(Path, String) - Static method in class gobblin.util.HadoopUtils: Remove illegal HDFS path characters from the given path.
ScalingThreadPoolExecutor - Class in gobblin.util.executors: ThreadPoolExecutor that scales from a min to a max number of threads as tasks get added.
schedule(T, long, TimeUnit) - Method in class gobblin.util.concurrent.TaskScheduler: Schedules a subclass of ScheduledTask to run periodically.
schedule(Runnable, long, TimeUnit) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
schedule(Callable<V>, long, TimeUnit) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
scheduleAtFixedRate(Runnable, long, long, TimeUnit) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
ScheduledTask<K> - Interface in gobblin.util.concurrent: A task which can be scheduled to run recurrently by an instance of TaskScheduler.
scheduler() - Method in class gobblin.util.logs.LogCopier
scheduleWithFixedDelay(Runnable, long, long, TimeUnit) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
SCHEME - Static variable in class gobblin.util.filesystem.InstrumentedHDFSFileSystem
SCHEME - Static variable in class gobblin.util.filesystem.InstrumentedLocalFileSystem
SCHEME - Static variable in class gobblin.util.filesystem.InstrumentedWebHDFSFileSystem
seek(long) - Method in class gobblin.util.io.SeekableFSInputStream
SeekableFSInputStream - Class in gobblin.util.io: Class that wraps an InputStream to support Seekable and PositionedReadable
SeekableFSInputStream(InputStream) - Constructor for class gobblin.util.io.SeekableFSInputStream
seekToNewSource(long) - Method in class gobblin.util.io.SeekableFSInputStream
selfScope() - Method in class gobblin.broker.SharedResourcesBrokerImpl
sendEmail(State, String, String) - Static method in class gobblin.util.EmailUtils: A general method for sending emails.
sendJobCancellationEmail(String, String, State) - Static method in class gobblin.util.EmailUtils: Send a job cancellation notification email.
sendJobCompletionEmail(String, String, String, State) - Static method in class gobblin.util.EmailUtils: Send a job completion notification email.
sendJobFailureAlertEmail(String, String, int, State) - Static method in class gobblin.util.EmailUtils: Send a job failure alert email.
SEPARATOR - Static variable in class gobblin.util.Id
SerializationUtils - Class in gobblin.util: A utility class for serializing and deserializing Objects to/from Strings.
SerializationUtils() - Constructor for class gobblin.util.SerializationUtils
serialize(T) - Static method in class gobblin.util.SerializationUtils: Serialize an object into a String.
serialize(T, BaseEncoding) - Static method in class gobblin.util.SerializationUtils: Serialize an object into a String.
serializeAsPath(GenericRecord, boolean, boolean) - Static method in class gobblin.util.AvroUtils: Serialize a generic record as a relative Path.
serializeIntoBytes(T) - Static method in class gobblin.util.SerializationUtils: Serialize an object into a byte array.
serializeState(FileSystem, Path, T) - Static method in class gobblin.util.SerializationUtils: Serialize a State instance to a file.
serializeState(FileSystem, Path, T, short) - Static method in class gobblin.util.SerializationUtils: Serialize a State instance to a file.
serializeToFile(T, Path) - Method in class gobblin.util.ParallelRunner: Serialize a State object into a file.
serializeToString(Writable) - Static method in class gobblin.util.HadoopUtils: Serialize a Writable object into a string.
serializeWriterDirPermissions(State, int, int, FsPermission) - Static method in class gobblin.util.HadoopUtils: Given a FsPermission objects, set a key, value pair in the given State for the writer to use when creating files.
serializeWriterFilePermissions(State, int, int, FsPermission) - Static method in class gobblin.util.HadoopUtils: Given a FsPermission objects, set a key, value pair in the given State for the writer to use when creating files.
set() - Method in class gobblin.util.concurrent.AutoResetEvent
setChildren(FileStatusEntry[]) - Method in class gobblin.util.filesystem.FileStatusEntry: Set the directory's files.
setGroup(FileSystem, Path, String) - Static method in class gobblin.util.HadoopUtils: Set the group associated with a given path.
setPermissions(Path, Optional<String>, Optional<String>, FileSystem, FsPermission) - Static method in class gobblin.util.HadoopUtils: Try to set owner and permissions for the path.
setProxiedFileSystem(FileSystem) - Method in class gobblin.util.ProxiedFileSystemWrapper: Deprecated.

Setter for proxiedFs.
setRequirement(String, double) - Method in class gobblin.util.request_allocation.ResourceRequirement.Builder: Specify the resource requirement along a dimension.
setWriterDirOctalPermissions(State, int, int, String) - Static method in class gobblin.util.HadoopUtils: Given a String in octal notation, set a key, value pair in the given State for the writer to use when creating directories.
setWriterFileOctalPermissions(State, int, int, String) - Static method in class gobblin.util.HadoopUtils: Given a String in octal notation, set a key, value pair in the given State for the writer to use when creating files.
SharedLimiterFactory<S extends ScopeType<S>> - Class in gobblin.util.limiter.broker: A SharedResourceFactory to create shared Limiters using a SharedResourcesBroker.
SharedLimiterFactory() - Constructor for class gobblin.util.limiter.broker.SharedLimiterFactory
SharedLimiterKey - Class in gobblin.util.limiter.broker: A SharedResourceKey for use with SharedLimiterFactory.
SharedLimiterKey(String) - Constructor for class gobblin.util.limiter.broker.SharedLimiterKey
SharedLimiterKey(String, SharedLimiterKey.GlobalLimiterPolicy) - Constructor for class gobblin.util.limiter.broker.SharedLimiterKey
SharedLimiterKey.GlobalLimiterPolicy - Enum in gobblin.util.limiter.broker
SharedResourcesBrokerFactory - Class in gobblin.broker: Used to create a default implementation of SharedResourcesBroker.
SharedResourcesBrokerFactory() - Constructor for class gobblin.broker.SharedResourcesBrokerFactory
SharedResourcesBrokerImpl<S extends ScopeType<S>> - Class in gobblin.broker: An implementation of SharedResourcesBroker using a DefaultBrokerCache for storing shared objects.
SharedResourcesBrokerImpl.SubscopedBrokerBuilder - Class in gobblin.broker: A builder used to create a descendant SharedResourcesBrokerImpl with the same backing DefaultBrokerCache.
SharedResourcesBrokerUtils - Class in gobblin.broker: General utilities for SharedResourcesBroker functionality.
SharedResourcesBrokerUtils() - Constructor for class gobblin.broker.SharedResourcesBrokerUtils
shutDown() - Method in class gobblin.util.logs.LogCopier
shutdown(SourceState) - Method in class gobblin.util.test.FastSequentialSource
shutdown(SourceState) - Method in class gobblin.util.test.HelloWorldSource
shutdown(SourceState) - Method in class gobblin.util.test.TestingSource
SHUTDOWN_PREFIX - Static variable in class gobblin.util.service.StandardServiceConfig
SHUTDOWN_TIMEOUT_MS_PROP - Static variable in class gobblin.util.service.StandardServiceConfig
shutdownExecutorService(ExecutorService, Optional<Logger>, long, TimeUnit) - Static method in class gobblin.util.ExecutorsUtils: Shutdown an ExecutorService gradually, first disabling new task submissions and later cancelling existing tasks.
shutdownExecutorService(ExecutorService, Optional<Logger>) - Static method in class gobblin.util.ExecutorsUtils: Shutdown an ExecutorService gradually, first disabling new task submissions and later cancelling existing tasks.
shutdownObject(Object, Logger) - Static method in class gobblin.broker.SharedResourcesBrokerUtils: Close Closeables and shutdown Services.
SimpleHierarchicalPrioritizer<T extends Request<T>> - Class in gobblin.util.request_allocation: A HierarchicalPrioritizer built from two input Comparators: one for Requestor and one for Request.
SimpleHierarchicalPrioritizer() - Constructor for class gobblin.util.request_allocation.SimpleHierarchicalPrioritizer
slowDeserializeGenericRecord(byte[], Schema) - Static method in class gobblin.util.AvroUtils: Deserialize a GenericRecord from a byte array.
splitExtractsByPredicate(Multimap<Extract, WorkUnitState>, Predicate<WorkUnitState>) - Static method in class gobblin.util.PublisherUtils: Given a Multimap of Extracts to WorkUnitStates, filter out any Extracts where all of the corresponding WorkUnitStates do not meet the given Predicate.
SplitExtractsResult() - Constructor for class gobblin.util.PublisherUtils.SplitExtractsResult
StandardServiceConfig - Class in gobblin.util.service: A wrapper around a typesafe Config object to provide standard configuration for services.
StandardServiceConfig(Config) - Constructor for class gobblin.util.service.StandardServiceConfig: Constructor from a typesafe config object
start() - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Start monitoring.
start() - Method in class gobblin.util.limiter.CountBasedLimiter
start() - Method in interface gobblin.util.limiter.Limiter: Start the Limiter.
start() - Method in class gobblin.util.limiter.MultiLimiter
start() - Method in class gobblin.util.limiter.NoopLimiter
start() - Method in class gobblin.util.limiter.PoolBasedLimiter
start() - Method in class gobblin.util.limiter.RateBasedLimiter
start() - Method in class gobblin.util.limiter.TimeBasedLimiter
startRateControl() - Method in class gobblin.util.RateControlledFileSystem: Deprecated.
STARTUP_PREFIX - Static variable in class gobblin.util.service.StandardServiceConfig
STARTUP_TIMEOUT_MS_PROP - Static variable in class gobblin.util.service.StandardServiceConfig
state(State) - Method in class gobblin.util.jdbc.DataSourceBuilder
StdoutWriter<D> - Class in gobblin.util.test: A simple writer implementation that writes the output to Stdout
StdoutWriter() - Constructor for class gobblin.util.test.StdoutWriter
StdoutWriter.Builder<D> - Class in gobblin.util.test
stop() - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Stop monitoring
stop(long) - Method in class gobblin.util.filesystem.PathAlterationObserverScheduler: Stop monitoring
stop() - Method in class gobblin.util.limiter.CountBasedLimiter
stop() - Method in interface gobblin.util.limiter.Limiter: Stop the Limiter.
stop() - Method in class gobblin.util.limiter.MultiLimiter
stop() - Method in class gobblin.util.limiter.NoopLimiter
stop() - Method in class gobblin.util.limiter.PoolBasedLimiter
stop() - Method in class gobblin.util.limiter.RateBasedLimiter
stop() - Method in class gobblin.util.limiter.TimeBasedLimiter
STREAM_COPIER_KEY_PREFIX - Static variable in class gobblin.util.io.StreamCopierSharedLimiterKey
StreamCopier - Class in gobblin.util.io: A class that copies an InputStream to an OutputStream in a configurable way.
StreamCopier(InputStream, OutputStream) - Constructor for class gobblin.util.io.StreamCopier
StreamCopier(ReadableByteChannel, WritableByteChannel) - Constructor for class gobblin.util.io.StreamCopier
StreamCopier.NotEnoughPermitsException - Exception in gobblin.util.io: Indicates there were not enough permits in the Limiter to finish the copy.
StreamCopierSharedLimiterKey - Class in gobblin.util.io: A subclass of SharedLimiterKey with more detailed information on the source and target of a throttled StreamCopier.
StreamCopierSharedLimiterKey(URI, URI) - Constructor for class gobblin.util.io.StreamCopierSharedLimiterKey
StreamThrottler<S extends ScopeType<S>> - Class in gobblin.util.io: A class used to throttle InputStreams.
StreamThrottler() - Constructor for class gobblin.util.io.StreamThrottler
StreamThrottler.Factory<S extends ScopeType<S>> - Class in gobblin.util.io: A SharedResourceFactory that creates StreamThrottler.
StreamUtils - Class in gobblin.util.io: Utility class of input/output stream helpers.
StreamUtils() - Constructor for class gobblin.util.io.StreamUtils
stringifyRequirement(ResourceRequirement) - Method in class gobblin.util.request_allocation.ResourcePool: Stringify a ResourceRequirement with the appropriate dimension labels.
stringInPatterns(String, List<Pattern>) - Static method in class gobblin.util.DatasetFilterUtils: Determines whether a string matches one of the regex patterns.
StringParsingUtils - Class in gobblin.util
StringParsingUtils() - Constructor for class gobblin.util.StringParsingUtils
StringParsingUtils.FormatException - Exception in gobblin.util
STRIP_SUFFIX - Static variable in class gobblin.util.ConfigUtils: A suffix that is automatically appended to property keys that are prefixes of other property keys.
submit(Callable<T>) - Method in class gobblin.util.executors.MDCPropagatingExecutorService
submit(Runnable) - Method in class gobblin.util.executors.MDCPropagatingExecutorService
submit(Runnable, T) - Method in class gobblin.util.executors.MDCPropagatingExecutorService
submit(Callable<T>) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
submit(Runnable) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
submit(Runnable, T) - Method in class gobblin.util.executors.MDCPropagatingScheduledExecutorService
submitCallable(Callable<Void>, String) - Method in class gobblin.util.ParallelRunner: Submit a callable to the thread pool
subtract(ResourceRequirement) - Method in class gobblin.util.request_allocation.ResourceRequirement: Vector addition of this and other ResourceRequirement.
survived(String, List<Pattern>, List<Pattern>) - Static method in class gobblin.util.DatasetFilterUtils: A topic survives if (1) it doesn't match the blacklist, and (2) either whitelist is empty, or it matches the whitelist.
switchName(Schema, String) - Static method in class gobblin.util.AvroUtils: Copies the input Schema but changes the schema name.

T

t - Variable in class gobblin.util.Either
tar(FileSystem, Path, Path) - Static method in class gobblin.util.io.StreamUtils: Creates a tar gzip file using a given Path as input and a given Path as a destination.
tar(FileSystem, FileSystem, Path, Path) - Static method in class gobblin.util.io.StreamUtils: Similiar to StreamUtils.tar(FileSystem, Path, Path) except the source and destination FileSystem can be different.
TarGpgPathFilter - Class in gobblin.util.filters: A PathFilter to accept encrypted tarball files.
TarGpgPathFilter() - Constructor for class gobblin.util.filters.TarGpgPathFilter
TaskScheduler<K,T extends ScheduledTask<K>> - Class in gobblin.util.concurrent: Base class for a task scheduler that can run ScheduledTasks periodically.
TaskScheduler() - Constructor for class gobblin.util.concurrent.TaskScheduler
TaskSchedulerFactory - Class in gobblin.util.concurrent: A factory which can be used to get an instance of TaskScheduler.
TaskSchedulerType - Enum in gobblin.util.concurrent: The types of supported TaskSchedulers.
TemplateUtils - Class in gobblin.util
TemplateUtils() - Constructor for class gobblin.util.TemplateUtils
TestingSource - Class in gobblin.util.test: A trivial implementation of Source to be used to testing.
TestingSource() - Constructor for class gobblin.util.test.TestingSource
TestingSource.Extract - Class in gobblin.util.test
TExtractor() - Constructor for class gobblin.util.request_allocation.AllocatedRequestsIteratorBase.TExtractor
ThrottledFileSystem - Class in gobblin.util.filesystem: A FileSystemInstrumentation for throttling calls to the underlying FileSystem using the input Limiter.
ThrottledFileSystem(FileSystem, Limiter) - Constructor for class gobblin.util.filesystem.ThrottledFileSystem
ThrottledFileSystem.Factory<S extends ScopeType<S>> - Class in gobblin.util.filesystem: Factory for ThrottledFileSystem.
ThrottledInputStream - Class in gobblin.util.io: A throttled InputStream.
ThrottledInputStream(InputStream, Limiter, MeteredInputStream) - Constructor for class gobblin.util.io.ThrottledInputStream: Builds a ThrottledInputStream.
TimeBasedLimiter - Class in gobblin.util.limiter: An implementation of Limiter that limits the time elapsed for some events.
TimeBasedLimiter(long) - Constructor for class gobblin.util.limiter.TimeBasedLimiter
TimeBasedLimiter(long, TimeUnit) - Constructor for class gobblin.util.limiter.TimeBasedLimiter
TimeBasedLimiter.Factory - Class in gobblin.util.limiter
TIMEOUT_MS_KEY - Static variable in class gobblin.util.service.StandardServiceConfig
TimeRangeChecker - Class in gobblin.util: Utility class to check if a given time is in a pre-defined range.
TimeRangeChecker() - Constructor for class gobblin.util.TimeRangeChecker
toConfigurationKey() - Method in class gobblin.broker.EmptyKey
toConfigurationKey() - Method in class gobblin.util.filesystem.FileSystemKey
toConfigurationKey() - Method in class gobblin.util.limiter.broker.SharedLimiterKey
TokenUtils - Class in gobblin.util.hadoop: A utility class for obtain Hadoop tokens and Hive metastore tokens for Azkaban jobs.
TokenUtils() - Constructor for class gobblin.util.hadoop.TokenUtils
toString() - Method in class gobblin.util.callbacks.Callback
toString() - Method in class gobblin.util.guid.Guid: Serializes the guid into a hex string.
toString() - Method in class gobblin.util.Id
toString() - Method in enum gobblin.util.limiter.BaseLimiterType
toString() - Method in class gobblin.util.limiter.broker.SharedLimiterKey
toString() - Method in class gobblin.util.request_allocation.ResourcePool
toStringMap(Object) - Static method in class gobblin.util.AvroUtils: Given a map: key -> value, return a map: key.toString() -> value.toString().
TOTAL_MULTI_WORK_UNIT_WEIGHT - Static variable in class gobblin.util.binpacking.WorstFitDecreasingBinPacking
totalResourcesUsed() - Method in interface gobblin.util.request_allocation.AllocatedRequestsIterator
totalResourcesUsed() - Method in class gobblin.util.request_allocation.AllocatedRequestsIteratorBase
toUriPath(Path) - Static method in class gobblin.util.HadoopUtils: Get the path as a string without schema or authority.
TTLResourceEntry<T> - Class in gobblin.broker: A ResourceEntry that automatically expires after a given number of milliseconds.
TTLResourceEntry(T, long, boolean) - Constructor for class gobblin.broker.TTLResourceEntry

U

uncaughtException(Thread, Throwable) - Method in class gobblin.util.LoggingUncaughtExceptionHandler
unpackFilterInputStream(FilterInputStream) - Static method in class gobblin.util.io.FilterStreamUnpacker: Finds the underlying InputStream to a FilterInputStream.
unpackFilterOutputStream(FilterOutputStream) - Static method in class gobblin.util.io.FilterStreamUnpacker: Finds the underlying OutputStream to a FilterOutputStream.
unsafeRenameIfNotExists(FileSystem, Path, Path) - Static method in class gobblin.util.HadoopUtils: Renames from to to if to doesn't exist in a non-thread-safe way.
updateLog4jConfiguration(Class<?>, String, String) - Static method in class gobblin.util.logs.Log4jConfigurationHelper: Update the log4j configuration.
url(String) - Method in class gobblin.util.jdbc.DataSourceBuilder
URL_TO_NAME_MAP_RESOURCE_NAME - Static variable in class gobblin.util.ClustersNames
useCopyInterval(long) - Method in class gobblin.util.logs.LogCopier.Builder: Set the copy interval between two iterations of copies.
useDestFileSystem(FileSystem) - Method in class gobblin.util.logs.LogCopier.Builder: Set the destination FileSystem for writing the destination log file.
useExcludingRegexPatterns(String) - Method in class gobblin.util.logs.LogCopier.Builder: Set the regex patterns used to filter logs that should not be copied.
useIncludingRegexPatterns(String) - Method in class gobblin.util.logs.LogCopier.Builder: Set the regex patterns used to filter logs that should be copied.
useLinesWrittenBeforeFlush(int) - Method in class gobblin.util.logs.LogCopier.Builder: Set the number of lines written before they are flushed to disk.
useLogFileNamePrefix(String) - Method in class gobblin.util.logs.LogCopier.Builder: Set the log file name prefix at the destination.
useMaxBytesPerLogFile(long) - Method in class gobblin.util.logs.LogCopier.Builder: Set the max bytes per log file.
useMaxMinutesPerLogFile(long) - Method in class gobblin.util.logs.LogCopier.Builder: Set the max minutes per log file.
userName(String) - Method in class gobblin.util.jdbc.DataSourceBuilder
USERNAME - Static variable in class gobblin.util.jdbc.DataSourceProvider
useScheduler(String) - Method in class gobblin.util.logs.LogCopier.Builder: Set the scheduler to use for scheduling copy tasks.
useSourceLogFileMonitorInterval(long) - Method in class gobblin.util.logs.LogCopier.Builder: Set the interval between two checks for the source log file monitor.
useSrcFileSystem(FileSystem) - Method in class gobblin.util.logs.LogCopier.Builder: Set the source FileSystem for reading the source log file.
useStrongEncryption(boolean) - Method in class gobblin.util.jdbc.DataSourceBuilder
useTimeUnit(TimeUnit) - Method in class gobblin.util.logs.LogCopier.Builder: Set the TimeUnit used for the source log file monitor interval, initial delay, copy interval.

V

valueOf(String) - Static method in enum gobblin.util.concurrent.TaskSchedulerType: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.DatePartitionType: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.Id.Parts: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.limiter.BaseLimiterType: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.limiter.broker.SharedLimiterKey.GlobalLimiterPolicy: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.ParallelRunner.FailPolicy: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.ProxiedFileSystemUtils.AuthType: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.ProxiedFileSystemWrapper.AuthType: Deprecated.

Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum gobblin.util.WriterUtils.WriterFilePathType: Returns the enum constant of this type with the specified name.
values() - Static method in enum gobblin.util.concurrent.TaskSchedulerType: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.DatePartitionType: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.Id.Parts: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.limiter.BaseLimiterType: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.limiter.broker.SharedLimiterKey.GlobalLimiterPolicy: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.ParallelRunner.FailPolicy: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.ProxiedFileSystemUtils.AuthType: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.ProxiedFileSystemWrapper.AuthType: Deprecated.

Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum gobblin.util.WriterUtils.WriterFilePathType: Returns an array containing the constants of this enum type, in the order they are declared.
verifyAllSuccessful(List<Either<T, ExecutionException>>) - Static method in class gobblin.util.executors.IteratorExecutor: Utility method that checks whether all tasks succeeded from the output of IteratorExecutor.executeAndGetResults().
verifySubset(Config, Config) - Static method in class gobblin.util.ConfigUtils: Check that every key-value in superConfig is in subConfig

W

waitOne() - Method in class gobblin.util.concurrent.AutoResetEvent
waitOne(int, TimeUnit) - Method in class gobblin.util.concurrent.AutoResetEvent
weight(WorkUnit) - Method in class gobblin.util.binpacking.FieldWeighter
WHITELIST - Static variable in class gobblin.data.management.copy.hive.WhitelistBlacklist
WhitelistBlacklist - Class in gobblin.data.management.copy.hive: A whitelist / blacklist implementation for filtering Hive tables.
WhitelistBlacklist(Config) - Constructor for class gobblin.data.management.copy.hive.WhitelistBlacklist
WhitelistBlacklist(String, String) - Constructor for class gobblin.data.management.copy.hive.WhitelistBlacklist
withAdditionalParentBroker(SharedResourcesBroker<S>) - Method in class gobblin.broker.SharedResourcesBrokerImpl.SubscopedBrokerBuilder: Specify additional ancestor SharedResourcesBrokerImpl.
withBufferSize(int) - Method in class gobblin.util.io.StreamCopier: Set the size in bytes of the buffer used to copy.
withCopySpeedMeter(Meter) - Method in class gobblin.util.io.StreamCopier: Set a Meter where copy speed will be reported.
withLimitedScopeConfig(Config) - Method in class gobblin.util.request_allocation.RequestAllocatorConfig.Builder
withoutLeadingSeparator(Path) - Static method in class gobblin.util.PathUtils: Removes the leading slash if present.
withOverridingConfig(Config) - Method in class gobblin.broker.SharedResourcesBrokerImpl.SubscopedBrokerBuilder: Specify Config overrides.
withPrioritizer(Comparator<T>) - Method in class gobblin.util.request_allocation.RequestAllocatorConfig.Builder
WorkUnitStateSuccess() - Constructor for class gobblin.util.PublisherUtils.WorkUnitStateSuccess
WorstFitDecreasingBinPacking - Class in gobblin.util.binpacking: Implements a bin packing algorithm similar to worst-fit decreasing bin packing.
WorstFitDecreasingBinPacking() - Constructor for class gobblin.util.binpacking.WorstFitDecreasingBinPacking
WritableShimSerialization - Class in gobblin.util: A serializer that understands how to write objects that implement WritableShim out to a Hadoop stream.
WritableShimSerialization() - Constructor for class gobblin.util.WritableShimSerialization
write(int) - Method in class gobblin.util.io.MeteredOutputStream
write(byte[], int, int) - Method in class gobblin.util.io.MeteredOutputStream
write(D) - Method in class gobblin.util.test.StdoutWriter
WRITER_ENCRYPTED_CONFIG_PATH - Static variable in class gobblin.util.WriterUtils
WriterUtils - Class in gobblin.util: Utility class for use with the DataWriter class.
WriterUtils() - Constructor for class gobblin.util.WriterUtils
WriterUtils.WriterFilePathType - Enum in gobblin.util: TABLENAME should be used for jobs that pull from multiple tables/topics and intend to write the records in each table/topic to a separate folder.
writeSchemaToFile(Schema, Path, FileSystem, boolean) - Static method in class gobblin.util.AvroUtils
writeSchemaToFile(Schema, Path, FileSystem, boolean, FsPermission) - Static method in class gobblin.util.AvroUtils
writeTo(Path) - Method in class gobblin.util.logs.LogCopier.Builder: Set the path of the destination log file directory to write to.

Z

zero() - Method in class gobblin.util.request_allocation.ResourceRequirement.Builder: Set all resource requirements to 0.

_

_fileStatus - Variable in class gobblin.util.filesystem.FileStatusEntry
_workunits - Variable in class gobblin.util.test.TestingSource

A B C D E F G H I J K L M N O P Q R S T U V W Z _