@InterfaceAudience.Public @InterfaceStability.Stable public class FixedLengthInputFormat extends FileInputFormat<LongWritable,BytesWritable>
FixedLengthRecordReader| Modifier and Type | Field and Description |
|---|---|
static String |
FIXED_RECORD_LENGTH |
DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_NONRECURSIVE_IGNORE_SUBDIRS, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE| Constructor and Description |
|---|
FixedLengthInputFormat() |
| Modifier and Type | Method and Description |
|---|---|
RecordReader<LongWritable,BytesWritable> |
createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for a given split.
|
static int |
getRecordLength(Configuration conf)
Get record length value
|
protected boolean |
isSplitable(JobContext context,
Path file)
Is the given filename splittable? Usually, true, but if the file is
stream compressed, it will not be.
|
static void |
setRecordLength(Configuration conf,
int recordLength)
Set the length of each record
|
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, listStatus, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSizepublic static final String FIXED_RECORD_LENGTH
public static void setRecordLength(Configuration conf, int recordLength)
conf - configurationrecordLength - the length of a recordpublic static int getRecordLength(Configuration conf)
conf - configurationpublic RecordReader<LongWritable,BytesWritable> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException
InputFormatRecordReader.initialize(InputSplit, TaskAttemptContext) before
the split is used.createRecordReader in class InputFormat<LongWritable,BytesWritable>split - the split to be readcontext - the information about the taskIOExceptionInterruptedExceptionprotected boolean isSplitable(JobContext context, Path file)
FileInputFormatFileInputFormat always returns
true. Implementations that may deal with non-splittable files must
override this method.
FileInputFormat implementations can override this and return
false to ensure that individual input files are never split-up
so that Mappers process entire files.isSplitable in class FileInputFormat<LongWritable,BytesWritable>context - the job contextfile - the file name to checkCopyright © 2023 Apache Software Foundation. All rights reserved.