co.cask.cdap.format

Class DelimitedStringsRecordFormat

java.lang.Object

co.cask.cdap.api.data.format.RecordFormat<StreamEvent,T>

co.cask.cdap.spi.stream.AbstractStreamEventRecordFormat<co.cask.cdap.api.data.format.StructuredRecord>

co.cask.cdap.format.DelimitedStringsRecordFormat

public class DelimitedStringsRecordFormat
extends AbstractStreamEventRecordFormat<co.cask.cdap.api.data.format.StructuredRecord>

Stream record format that interprets the body as string of delimited fields.

The delimiter can be explicitly set through the "delimiter" setting, and the character set can also be set through the "charset" setting. By default, the format will use a schema of one field, where the field is an array of strings. The schema can be set to a schema of fields, with the i'th field corresponding to the i'th value in the delimited text. Fields can also be parsed as scalar types - boolean, integer, long, double, float, bytes, and string. In addition, the very last field can be an array of strings.

If the "mapping" setting is provided, then we will use the mapping to parse the stream events rather than the order of the schema fields. "mapping" is in the format "index0:field0,index1:field1,..". For example, if "mapping" is "1:name,2:age", then a stream event like "sdf,bob,32,sdf,lkj" would be transformed into a record {"name":"bob", "age":32}.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CHARSET
static String	DELIMITER
static String	MAPPING

Fields inherited from class co.cask.cdap.api.data.format.RecordFormat

schema

Constructor Summary

Constructors

Constructor and Description
DelimitedStringsRecordFormat()

Method Summary

Methods

Modifier and Type	Method and Description
protected void	configure(Map<String,String> settings)
protected co.cask.cdap.api.data.schema.Schema	getDefaultSchema()
co.cask.cdap.api.data.format.StructuredRecord	read(StreamEvent event)
protected void	validateSchema(co.cask.cdap.api.data.schema.Schema desiredSchema)

Methods inherited from class co.cask.cdap.api.data.format.RecordFormat

getSchema, initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CHARSET

public static final String CHARSET

See Also:

Constant Field Values

DELIMITER

public static final String DELIMITER

See Also:

Constant Field Values

MAPPING

public static final String MAPPING

See Also:

Constant Field Values

Constructor Detail

DelimitedStringsRecordFormat

public DelimitedStringsRecordFormat()

Method Detail

read

public co.cask.cdap.api.data.format.StructuredRecord read(StreamEvent event)
                                                   throws co.cask.cdap.api.data.format.UnexpectedFormatException

Specified by:

read in class co.cask.cdap.api.data.format.RecordFormat<StreamEvent,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

co.cask.cdap.api.data.format.UnexpectedFormatException

getDefaultSchema

protected co.cask.cdap.api.data.schema.Schema getDefaultSchema()

Specified by:

getDefaultSchema in class co.cask.cdap.api.data.format.RecordFormat<StreamEvent,co.cask.cdap.api.data.format.StructuredRecord>

validateSchema

protected void validateSchema(co.cask.cdap.api.data.schema.Schema desiredSchema)
                       throws co.cask.cdap.api.data.schema.UnsupportedTypeException

Specified by:

validateSchema in class co.cask.cdap.api.data.format.RecordFormat<StreamEvent,co.cask.cdap.api.data.format.StructuredRecord>

Throws:

co.cask.cdap.api.data.schema.UnsupportedTypeException

configure

protected void configure(Map<String,String> settings)

Overrides:

configure in class co.cask.cdap.api.data.format.RecordFormat<StreamEvent,co.cask.cdap.api.data.format.StructuredRecord>