Class Lucene99ScalarQuantizedVectorsFormat
- java.lang.Object
-
- org.apache.lucene.codecs.KnnVectorsFormat
-
- org.apache.lucene.codecs.hnsw.FlatVectorsFormat
-
- org.apache.lucene.codecs.lucene99.Lucene99ScalarQuantizedVectorsFormat
-
- All Implemented Interfaces:
NamedSPILoader.NamedSPI
public class Lucene99ScalarQuantizedVectorsFormat extends FlatVectorsFormat
Format supporting vector quantization, storage, and retrieval- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Field Summary
Fields Modifier and Type Field Description static floatDYNAMIC_CONFIDENCE_INTERVALDynamic confidence intervalstatic StringNAMEstatic StringQUANTIZED_VECTOR_COMPONENT-
Fields inherited from class org.apache.lucene.codecs.KnnVectorsFormat
DEFAULT_MAX_DIMENSIONS, EMPTY
-
-
Constructor Summary
Constructors Constructor Description Lucene99ScalarQuantizedVectorsFormat()Constructs a format using default graph construction parametersLucene99ScalarQuantizedVectorsFormat(Float confidenceInterval, int bits, boolean compress)Constructs a format using the given graph construction parameters.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description static floatcalculateDefaultConfidenceInterval(int vectorDimension)FlatVectorsReaderfieldsReader(SegmentReadState state)Returns aKnnVectorsReaderto read the vectors from the index.FlatVectorsWriterfieldsWriter(SegmentWriteState state)Returns aFlatVectorsWriterto write the vectors to the index.StringtoString()-
Methods inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsFormat
getMaxDimensions
-
Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat
availableKnnVectorsFormats, forName, getName, reloadKnnVectorsFormat
-
-
-
-
Field Detail
-
QUANTIZED_VECTOR_COMPONENT
public static final String QUANTIZED_VECTOR_COMPONENT
- See Also:
- Constant Field Values
-
NAME
public static final String NAME
- See Also:
- Constant Field Values
-
DYNAMIC_CONFIDENCE_INTERVAL
public static final float DYNAMIC_CONFIDENCE_INTERVAL
Dynamic confidence interval- See Also:
- Constant Field Values
-
-
Constructor Detail
-
Lucene99ScalarQuantizedVectorsFormat
public Lucene99ScalarQuantizedVectorsFormat()
Constructs a format using default graph construction parameters
-
Lucene99ScalarQuantizedVectorsFormat
public Lucene99ScalarQuantizedVectorsFormat(Float confidenceInterval, int bits, boolean compress)
Constructs a format using the given graph construction parameters.- Parameters:
confidenceInterval- the confidenceInterval for scalar quantizing the vectors, when `null` it is calculated based on the vector dimension. When `0`, the quantiles are dynamically determined by sampling many confidence intervals and determining the most accurate pair.bits- the number of bits to use for scalar quantization (must be between 1 and 8, inclusive)compress- whether to compress the quantized vectors by another 50% when bits=4. If `true`, pairs of (4 bit quantized) dimensions are packed into a single byte. This must be `false` when bits=7. This provides a trade-off of 50% reduction in hot vector memory usage during searching, at some decode speed penalty.
-
-
Method Detail
-
calculateDefaultConfidenceInterval
public static float calculateDefaultConfidenceInterval(int vectorDimension)
-
fieldsWriter
public FlatVectorsWriter fieldsWriter(SegmentWriteState state) throws IOException
Description copied from class:FlatVectorsFormatReturns aFlatVectorsWriterto write the vectors to the index.- Specified by:
fieldsWriterin classFlatVectorsFormat- Throws:
IOException
-
fieldsReader
public FlatVectorsReader fieldsReader(SegmentReadState state) throws IOException
Description copied from class:FlatVectorsFormatReturns aKnnVectorsReaderto read the vectors from the index.- Specified by:
fieldsReaderin classFlatVectorsFormat- Throws:
IOException
-
-