Package org.apache.lucene.backward_codecs.lucene99

Class Lucene99ScalarQuantizedVectorsFormat

java.lang.Object

org.apache.lucene.codecs.KnnVectorsFormat

org.apache.lucene.codecs.hnsw.FlatVectorsFormat

org.apache.lucene.backward_codecs.lucene99.Lucene99ScalarQuantizedVectorsFormat

All Implemented Interfaces:

NamedSPILoader.NamedSPI

public class Lucene99ScalarQuantizedVectorsFormat
extends FlatVectorsFormat

Format supporting vector quantization, storage, and retrieval

WARNING: This API is experimental and might change in incompatible ways in the next release.

Field Summary

Fields

Modifier and Type	Field	Description
static final float	DYNAMIC_CONFIDENCE_INTERVAL	Dynamic confidence interval

Fields inherited from class org.apache.lucene.codecs.KnnVectorsFormat

DEFAULT_MAX_DIMENSIONS, EMPTY

Constructor Summary

Constructors

Constructor	Description
Lucene99ScalarQuantizedVectorsFormat()	Constructs a format using default graph construction parameters
Lucene99ScalarQuantizedVectorsFormat(Float confidenceInterval, int bits, boolean compress)	Constructs a format using the given graph construction parameters.

Method Summary

Modifier and Type	Method	Description
FlatVectorsReader	fieldsReader(SegmentReadState state)
FlatVectorsWriter	fieldsWriter(SegmentWriteState state)
String	toString()

Methods inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsFormat

getMaxDimensions

Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat

availableKnnVectorsFormats, forName, getName, reloadKnnVectorsFormat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

DYNAMIC_CONFIDENCE_INTERVAL

public static final float DYNAMIC_CONFIDENCE_INTERVAL

Dynamic confidence interval

See Also:

• Constant Field Values

Constructor Details

Lucene99ScalarQuantizedVectorsFormat

public Lucene99ScalarQuantizedVectorsFormat()

Constructs a format using default graph construction parameters

Lucene99ScalarQuantizedVectorsFormat

public Lucene99ScalarQuantizedVectorsFormat(Float confidenceInterval,
 int bits,
 boolean compress)

Constructs a format using the given graph construction parameters.

Parameters:

confidenceInterval - the confidenceInterval for scalar quantizing the vectors, when `null` it is calculated based on the vector dimension. When `0`, the quantiles are dynamically determined by sampling many confidence intervals and determining the most accurate pair.

bits - the number of bits to use for scalar quantization (must be between 1 and 8, inclusive)

compress - whether to compress the quantized vectors by another 50% when bits=4. If `true`, pairs of (4 bit quantized) dimensions are packed into a single byte. This must be `false` when bits=7. This provides a trade-off of 50% reduction in hot vector memory usage during searching, at some decode speed penalty.

Method Details

toString

public String toString()

Overrides:

toString in class Object

fieldsWriter

public FlatVectorsWriter fieldsWriter(SegmentWriteState state)
                               throws IOException

Specified by:

fieldsWriter in class FlatVectorsFormat

Throws:

IOException

fieldsReader

public FlatVectorsReader fieldsReader(SegmentReadState state)
                               throws IOException

Specified by:

fieldsReader in class FlatVectorsFormat

Throws:

IOException