public class AggBinaryOp extends MultiThreadedHop
| Modifier and Type | Class and Description |
|---|---|
static class |
AggBinaryOp.MMultMethod |
static class |
AggBinaryOp.SparkAggType |
| Modifier and Type | Field and Description |
|---|---|
static AggBinaryOp.MMultMethod |
FORCED_MMULT_METHOD |
static double |
MAPMULT_MEM_MULTIPLIER |
_beginColumn, _beginLine, _endColumn, _endLine, _filename, _text, CPThreshold| Constructor and Description |
|---|
AggBinaryOp(String l,
Types.DataType dt,
Types.ValueType vt,
Types.OpOp2 innOp,
Types.AggOp outOp,
Hop in1,
Hop in2) |
| Modifier and Type | Method and Description |
|---|---|
boolean |
allowsAllExecTypes() |
void |
checkArity()
Check whether this Hop has a correct number of inputs.
|
MapMultChain.ChainType |
checkMapMultChain()
MapMultChain: Determine if XtwXv/XtXv pattern applies for this aggbinary
and if yes which type.
|
MMTSJ.MMTSJType |
checkTransposeSelf()
TSMM: Determine if XtX pattern applies for this aggbinary and if yes
which type.
|
Object |
clone() |
boolean |
compare(Hop that) |
void |
computeMemEstimate(MemoTable memo)
Computes the estimate of memory required to store the input/output of this hop in memory.
|
Lop |
constructLops()
NOTE: overestimated mem in case of transpose-identity matmult, but 3/2 at worst
and existing mem estimate advantageous in terms of consistency hops/lops,
and some special cases internally materialize the transpose for better cache locality
|
static double |
getMapmmMemEstimate(long m1_rows,
long m1_cols,
long m1_blen,
long m1_nnz,
long m2_rows,
long m2_cols,
long m2_blen,
long m2_nnz,
int cachedInputIndex,
boolean pmm)
Estimates the memory footprint of MapMult operation depending on which input is put into distributed cache.
|
AggBinaryOp.MMultMethod |
getMMultMethod() |
String |
getOpString() |
boolean |
hasLeftPMInput() |
boolean |
isGPUEnabled()
In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()),
the exectype is determined by checking this method as well as memory budget of this Hop.
|
boolean |
isMatrixMultiply() |
boolean |
isMultiThreadedOpType() |
void |
refreshSizeInformation()
Update the output size information for this hop.
|
void |
setHasLeftPMInput(boolean flag) |
getMaxNumThreads, setMaxNumThreadsaddAllInputs, addInput, checkAndSetForcedPlatform, checkAndSetInvalidCPDimsAndSize, clearMemEstimate, colsKnown, computeBoundsInformation, computeBoundsInformation, computeBoundsInformation, computeSizeInformation, computeSizeInformation, computeSizeInformation, constructAndSetLopsDataFlowProperties, createOffsetLop, dimsKnown, dimsKnown, dimsKnownAny, getBeginColumn, getBeginLine, getBlocksize, getDataCharacteristics, getDataType, getDim, getDim1, getDim2, getEndColumn, getEndLine, getExecType, getFilename, getForcedExecType, getHopID, getInput, getInputMemEstimate, getInputOutputSize, getIntermediateMemEstimate, getLength, getLops, getMemEstimate, getName, getNnz, getOutputMemEstimate, getParent, getPrivacy, getSparsity, getSpBroadcastSize, getText, getUpdateType, getValueType, hasMatrixInputWithDifferentBlocksizes, hasValidCPDimsAndSize, isMatrix, isMemEstimated, isOutputEmptyBlocks, isScalar, isTransposeSafe, isVisited, printErrorLocation, refreshColsParameterInformation, refreshColsParameterInformation, refreshMemEstimates, refreshRowsParameterInformation, refreshRowsParameterInformation, requiresCheckpoint, requiresCompression, requiresLineageCaching, requiresReblock, requiresRecompile, resetExecType, resetRecompilationFlag, resetRecompilationFlag, resetVisitStatus, resetVisitStatus, resetVisitStatus, resetVisitStatusForced, rowsKnown, setBeginColumn, setBeginLine, setBlocksize, setDataType, setDim, setDim1, setDim2, setEndColumn, setEndLine, setFilename, setForcedExecType, setLops, setMemEstimate, setName, setNnz, setOutputEmptyBlocks, setParseInfo, setPrivacy, setRequiresCheckpoint, setRequiresCompression, setRequiresLineageCaching, setRequiresReblock, setRequiresRecompile, setText, setUpdateType, setValueType, setVisited, setVisitedequals, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitctxAndFilenameToParseInfopublic static final double MAPMULT_MEM_MULTIPLIER
public static AggBinaryOp.MMultMethod FORCED_MMULT_METHOD
public AggBinaryOp(String l, Types.DataType dt, Types.ValueType vt, Types.OpOp2 innOp, Types.AggOp outOp, Hop in1, Hop in2)
public void checkArity()
HopcheckArity in class Hoppublic void setHasLeftPMInput(boolean flag)
public boolean hasLeftPMInput()
public AggBinaryOp.MMultMethod getMMultMethod()
public boolean isGPUEnabled()
HopisGPUEnabled in class Hoppublic Lop constructLops()
constructLops in class Hoppublic String getOpString()
getOpString in class Hoppublic void computeMemEstimate(MemoTable memo)
HopcomputeMemEstimate in class Hopmemo - memory tablepublic boolean isMatrixMultiply()
public boolean isMultiThreadedOpType()
isMultiThreadedOpType in class MultiThreadedHoppublic boolean allowsAllExecTypes()
allowsAllExecTypes in class Hoppublic MMTSJ.MMTSJType checkTransposeSelf()
public MapMultChain.ChainType checkMapMultChain()
public static double getMapmmMemEstimate(long m1_rows,
long m1_cols,
long m1_blen,
long m1_nnz,
long m2_rows,
long m2_cols,
long m2_blen,
long m2_nnz,
int cachedInputIndex,
boolean pmm)
optFindMMultMethod() to decide the execution strategy, as well as by
piggybacking to decide the number of Map-side instructions to put into a single GMR job.m1_rows - m1 rowsm1_cols - m1 colsm1_blen - m1 rows/cols per blockm1_nnz - m1 num non-zerosm2_rows - m2 rowsm2_cols - m2 colsm2_blen - m2 rows/cols per blockm2_nnz - m2 num non-zeroscachedInputIndex - true if cached input indexpmm - true if permutation matrix multiplypublic void refreshSizeInformation()
HoprefreshSizeInformation in class Hoppublic Object clone() throws CloneNotSupportedException
clone in class HopCloneNotSupportedExceptionCopyright © 2020 The Apache Software Foundation. All rights reserved.