Class Lucene99ScalarQuantizedVectorsWriter
- java.lang.Object
-
- org.apache.lucene.codecs.KnnVectorsWriter
-
- org.apache.lucene.codecs.hnsw.FlatVectorsWriter
-
- org.apache.lucene.codecs.lucene99.Lucene99ScalarQuantizedVectorsWriter
-
- All Implemented Interfaces:
Closeable,AutoCloseable,Accountable
public final class Lucene99ScalarQuantizedVectorsWriter extends FlatVectorsWriter
Writes quantized vector values and metadata to index segments.- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.codecs.KnnVectorsWriter
KnnVectorsWriter.MergedVectorValues
-
-
Field Summary
-
Fields inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter
vectorsScorer
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Constructor Description Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, byte bits, boolean compress, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer)Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description FlatFieldVectorsWriter<?>addField(FieldInfo fieldInfo)Add a new field for indexingvoidclose()voidfinish()Called once at the end before closevoidflush(int maxDoc, Sorter.DocMap sortMap)Flush all buffered data on disk *static ScalarQuantizermergeAndRecalculateQuantiles(MergeState mergeState, FieldInfo fieldInfo, Float confidenceInterval, byte bits)Merges the quantiles of the segments and recalculates the quantiles if necessary.voidmergeOneField(FieldInfo fieldInfo, MergeState mergeState)Write field for mergingCloseableRandomVectorScorerSuppliermergeOneFieldToIndex(FieldInfo fieldInfo, MergeState mergeState)Write the field for merging, providing a scorer over the newly merged flat vectors.longramBytesUsed()Return the memory usage of this object in bytes.static DocsWithFieldSetwriteQuantizedVectorData(IndexOutput output, QuantizedByteVectorValues quantizedByteVectorValues, byte bits, boolean compress)Writes the vector values to the output and returns a set of documents that contains vectors.-
Methods inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter
getFlatVectorScorer
-
Methods inherited from class org.apache.lucene.codecs.KnnVectorsWriter
mapOldOrdToNewOrd, merge
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
-
-
-
Constructor Detail
-
Lucene99ScalarQuantizedVectorsWriter
public Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer) throws IOException
- Throws:
IOException
-
Lucene99ScalarQuantizedVectorsWriter
public Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, byte bits, boolean compress, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer) throws IOException
- Throws:
IOException
-
-
Method Detail
-
addField
public FlatFieldVectorsWriter<?> addField(FieldInfo fieldInfo) throws IOException
Description copied from class:FlatVectorsWriterAdd a new field for indexing- Specified by:
addFieldin classFlatVectorsWriter- Parameters:
fieldInfo- fieldInfo of the field to add- Returns:
- a writer for the field
- Throws:
IOException- if an I/O error occurs when adding the field
-
mergeOneField
public void mergeOneField(FieldInfo fieldInfo, MergeState mergeState) throws IOException
Description copied from class:KnnVectorsWriterWrite field for merging- Overrides:
mergeOneFieldin classKnnVectorsWriter- Throws:
IOException
-
mergeOneFieldToIndex
public CloseableRandomVectorScorerSupplier mergeOneFieldToIndex(FieldInfo fieldInfo, MergeState mergeState) throws IOException
Description copied from class:FlatVectorsWriterWrite the field for merging, providing a scorer over the newly merged flat vectors. This way any additional merging logic can be implemented by the user of this class.- Specified by:
mergeOneFieldToIndexin classFlatVectorsWriter- Parameters:
fieldInfo- fieldInfo of the field to mergemergeState- mergeState of the segments to merge- Returns:
- a scorer over the newly merged flat vectors, which should be closed as it holds a temporary file handle to read over the newly merged vectors
- Throws:
IOException- if an I/O error occurs when merging
-
flush
public void flush(int maxDoc, Sorter.DocMap sortMap) throws IOExceptionDescription copied from class:KnnVectorsWriterFlush all buffered data on disk *- Specified by:
flushin classKnnVectorsWriter- Throws:
IOException
-
finish
public void finish() throws IOExceptionDescription copied from class:KnnVectorsWriterCalled once at the end before close- Specified by:
finishin classKnnVectorsWriter- Throws:
IOException
-
ramBytesUsed
public long ramBytesUsed()
Description copied from interface:AccountableReturn the memory usage of this object in bytes. Negative values are illegal.
-
mergeAndRecalculateQuantiles
public static ScalarQuantizer mergeAndRecalculateQuantiles(MergeState mergeState, FieldInfo fieldInfo, Float confidenceInterval, byte bits) throws IOException
Merges the quantiles of the segments and recalculates the quantiles if necessary.- Parameters:
mergeState- The merge statefieldInfo- The field infoconfidenceInterval- The confidence intervalbits- The number of bits- Returns:
- The merged quantiles
- Throws:
IOException- If there is a low-level I/O error
-
writeQuantizedVectorData
public static DocsWithFieldSet writeQuantizedVectorData(IndexOutput output, QuantizedByteVectorValues quantizedByteVectorValues, byte bits, boolean compress) throws IOException
Writes the vector values to the output and returns a set of documents that contains vectors.- Throws:
IOException
-
close
public void close() throws IOException- Throws:
IOException
-
-