@InterfaceAudience.Public @InterfaceStability.Stable public class CombineFileRecordReader<K,V> extends RecordReader<K,V>
CombineFileSplit.
A CombineFileSplit can combine data chunks from multiple files.
This class allows using different RecordReaders for processing
these data chunks from different files.CombineFileSplit| Modifier and Type | Field and Description |
|---|---|
protected TaskAttemptContext |
context |
protected RecordReader<K,V> |
curReader |
protected int |
idx |
protected long |
progress |
protected Constructor<? extends RecordReader<K,V>> |
rrConstructor |
protected CombineFileSplit |
split |
| Constructor and Description |
|---|
CombineFileRecordReader(CombineFileSplit split,
TaskAttemptContext context,
Class<? extends RecordReader<K,V>> rrClass)
A generic RecordReader that can hand out different recordReaders
for each chunk in the CombineFileSplit.
|
| Modifier and Type | Method and Description |
|---|---|
void |
close()
Close the record reader.
|
K |
getCurrentKey()
Get the current key
|
V |
getCurrentValue()
Get the current value.
|
float |
getProgress()
return progress based on the amount of data processed so far.
|
void |
initialize(InputSplit split,
TaskAttemptContext context)
Called once at initialization.
|
protected boolean |
initNextRecordReader()
Get the record reader for the next chunk in this CombineFileSplit.
|
boolean |
nextKeyValue()
Read the next key, value pair.
|
protected CombineFileSplit split
protected Constructor<? extends RecordReader<K,V>> rrConstructor
protected TaskAttemptContext context
protected int idx
protected long progress
protected RecordReader<K,V> curReader
public CombineFileRecordReader(CombineFileSplit split, TaskAttemptContext context, Class<? extends RecordReader<K,V>> rrClass) throws IOException
IOExceptionpublic void initialize(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException
RecordReaderinitialize in class RecordReader<K,V>split - the split that defines the range of records to readcontext - the information about the taskIOExceptionInterruptedExceptionpublic boolean nextKeyValue()
throws IOException,
InterruptedException
RecordReadernextKeyValue in class RecordReader<K,V>IOExceptionInterruptedExceptionpublic K getCurrentKey() throws IOException, InterruptedException
RecordReadergetCurrentKey in class RecordReader<K,V>IOExceptionInterruptedExceptionpublic V getCurrentValue() throws IOException, InterruptedException
RecordReadergetCurrentValue in class RecordReader<K,V>IOExceptionInterruptedExceptionpublic void close()
throws IOException
RecordReaderclose in interface Closeableclose in interface AutoCloseableclose in class RecordReader<K,V>IOExceptionpublic float getProgress()
throws IOException,
InterruptedException
getProgress in class RecordReader<K,V>IOExceptionInterruptedExceptionprotected boolean initNextRecordReader()
throws IOException
IOExceptionCopyright © 2022 Apache Software Foundation. All rights reserved.