public class ColumnEncoderRecode extends ColumnEncoder
ColumnEncoder.EncoderType| Modifier and Type | Field and Description |
|---|---|
static boolean |
SORT_RECODE_MAP |
| Constructor and Description |
|---|
ColumnEncoderRecode() |
ColumnEncoderRecode(int colID) |
| Modifier and Type | Method and Description |
|---|---|
MatrixBlock |
apply(FrameBlock in,
MatrixBlock out,
int outputCol)
Apply the generated metadata to the FrameBlock and saved the result in out.
|
MatrixBlock |
apply(FrameBlock in,
MatrixBlock out,
int outputCol,
int rowStart,
int blk) |
MatrixBlock |
apply(MatrixBlock in,
MatrixBlock out,
int outputCol) |
MatrixBlock |
apply(MatrixBlock in,
MatrixBlock out,
int outputCol,
int rowStart,
int blk) |
void |
build(FrameBlock in)
Build the transform meta data for the given block input.
|
void |
buildPartial(FrameBlock in)
Partial build of internal data structures (e.g., in distributed spark operations).
|
static String |
constructRecodeMapEntry(String token,
Long code)
Returns the Recode map entry which consists of concatenation of code, delimiter and token.
|
boolean |
equals(Object o) |
HashMap<String,Long> |
getCPRecodeMaps() |
HashSet<Object> |
getCPRecodeMapsPartial() |
FrameBlock |
getMetaData(FrameBlock meta)
Construct a frame block out of the transform meta data.
|
int |
getNumDistinctValues() |
List<Callable<Object>> |
getPartialBuildTasks(FrameBlock in,
int blockSize) |
HashMap<String,Long> |
getRcdMap() |
int |
hashCode() |
void |
initMetaData(FrameBlock meta)
Construct the recodemaps from the given input frame for all columns registered for recode.
|
void |
mergeAt(ColumnEncoder other)
Merges another encoder, of a compatible type, in after a certain position.
|
void |
mergeBuildPartial(List<Future<Object>> futurePartials,
int start,
int end) |
void |
prepareBuildPartial()
Allocates internal data structures for partial build.
|
void |
readExternal(ObjectInput in)
Redirects the default java serialization via externalizable to our default hadoop writable serialization for
efficient broadcast/rdd deserialization.
|
void |
sortCPRecodeMaps() |
static String[] |
splitRecodeMapEntry(String value)
Splits a Recode map entry into its token and code.
|
void |
writeExternal(ObjectOutput out)
Redirects the default java serialization via externalizable to our default hadoop writable serialization for
efficient broadcast/rdd serialization.
|
compareTo, getColID, getColMapping, isApplicable, isApplicable, setColID, shiftCol, updateIndexRangespublic ColumnEncoderRecode(int colID)
public ColumnEncoderRecode()
public static String constructRecodeMapEntry(String token, Long code)
token - is part of Recode mapcode - is code for tokenpublic static String[] splitRecodeMapEntry(String value)
value - concatenation of token and code with delimiter in betweenpublic void sortCPRecodeMaps()
public void build(FrameBlock in)
Encoderin - input frame blockpublic List<Callable<Object>> getPartialBuildTasks(FrameBlock in, int blockSize)
getPartialBuildTasks in class ColumnEncoderpublic void mergeBuildPartial(List<Future<Object>> futurePartials, int start, int end) throws ExecutionException, InterruptedException
mergeBuildPartial in class ColumnEncoderExecutionExceptionInterruptedExceptionpublic void prepareBuildPartial()
ColumnEncoderprepareBuildPartial in interface EncoderprepareBuildPartial in class ColumnEncoderpublic void buildPartial(FrameBlock in)
ColumnEncoderbuildPartial in interface EncoderbuildPartial in class ColumnEncoderin - input frame blockpublic MatrixBlock apply(FrameBlock in, MatrixBlock out, int outputCol)
Encoderin - input frame blockout - output matrix blockoutputCol - is a offset in the output matrix. column in FrameBlock + outputCol = column in outpublic MatrixBlock apply(FrameBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)
apply in class ColumnEncoderpublic MatrixBlock apply(MatrixBlock in, MatrixBlock out, int outputCol, int rowStart, int blk)
apply in class ColumnEncoderpublic MatrixBlock apply(MatrixBlock in, MatrixBlock out, int outputCol)
apply in class ColumnEncoderpublic void mergeAt(ColumnEncoder other)
ColumnEncoderColumnEncoders are compatible with themselves and EncoderComposite is compatible with
every other ColumnEncoders. MultiColumnEncoders are compatible with every encodermergeAt in class ColumnEncoderother - the encoder that should be merged inpublic int getNumDistinctValues()
public FrameBlock getMetaData(FrameBlock meta)
Encodermeta - output frame blockpublic void initMetaData(FrameBlock meta)
meta - frame blockpublic void writeExternal(ObjectOutput out) throws IOException
ColumnEncoderwriteExternal in interface ExternalizablewriteExternal in class ColumnEncoderout - object outputIOException - if IOException occurspublic void readExternal(ObjectInput in) throws IOException
ColumnEncoderreadExternal in interface ExternalizablereadExternal in class ColumnEncoderin - object inputIOException - if IOException occurCopyright © 2021 The Apache Software Foundation. All rights reserved.