gobblin.util

Class DatasetFilterUtils

java.lang.Object

gobblin.util.DatasetFilterUtils

public class DatasetFilterUtils
extends Object

A utility class for filtering datasets through blacklist and whitelist.

Constructor Summary

Constructors

Constructor and Description
DatasetFilterUtils()

Method Summary

Modifier and Type	Method and Description
static List<String>	filter(List<String> topics, List<Pattern> blacklist, List<Pattern> whitelist)
static Set<String>	filter(Set<String> topics, List<Pattern> blacklist, List<Pattern> whitelist)
static List<Pattern>	getPatternList(State state, String propKey)
static List<Pattern>	getPatternList(State state, String propKey, String def)
static List<Pattern>	getPatternsFromStrings(List<String> strings) Convert a list of Strings to a list of Patterns.
static boolean	stringInPatterns(String s, List<Pattern> patterns) Determines whether a string matches one of the regex patterns.
static boolean	survived(String topic, List<Pattern> blacklist, List<Pattern> whitelist) A topic survives if (1) it doesn't match the blacklist, and (2) either whitelist is empty, or it matches the whitelist.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DatasetFilterUtils

public DatasetFilterUtils()

Method Detail

getPatternList

public static List<Pattern> getPatternList(State state,
                                           String propKey)

getPatternList

public static List<Pattern> getPatternList(State state,
                                           String propKey,
                                           String def)

getPatternsFromStrings

public static List<Pattern> getPatternsFromStrings(List<String> strings)

Convert a list of Strings to a list of Patterns.

filter

public static List<String> filter(List<String> topics,
                                  List<Pattern> blacklist,
                                  List<Pattern> whitelist)

filter

public static Set<String> filter(Set<String> topics,
                                 List<Pattern> blacklist,
                                 List<Pattern> whitelist)

survived

public static boolean survived(String topic,
                               List<Pattern> blacklist,
                               List<Pattern> whitelist)

A topic survives if (1) it doesn't match the blacklist, and (2) either whitelist is empty, or it matches the whitelist. Whitelist and blacklist use regex patterns (NOT glob patterns).

stringInPatterns

public static boolean stringInPatterns(String s,
                                       List<Pattern> patterns)

Determines whether a string matches one of the regex patterns.