public class DoubleMappedBigList extends AbstractDoubleBigList
Java's memory-mapping facilities have
the severe limitation of mapping at most Integer.MAX_VALUE bytes, as they
expose the content of a file using a MappedByteBuffer. This class can expose
a file of primitive types of arbitrary length as a BigList
that is actually based on an array of MappedByteBuffers, each mapping
a chunk of CHUNK_SIZE longs.
Mapping can happen with a specified byte order: saving and mapping data in native order
using methods from it.unimi.dsi.fastutil.io.BinIO will enhance performance significantly.
Instances of this class are not thread safe, but the copy() method provides a lightweight
duplicate that can be read independently by another thread.
Only chunks that are actually used will be duplicated lazily.
If you are modifiying the content of list, however, you will need to provide external synchronization.
AbstractDoubleBigList.DoubleRandomAccessSubList, AbstractDoubleBigList.DoubleSubList| Modifier and Type | Field and Description |
|---|---|
static long |
CHUNK_SIZE
The size in elements of a chunk created by
map(FileChannel, ByteOrder, FileChannel.MapMode). |
static int |
LOG2_BITS
Deprecated.
Use
LOG2_BYTES. |
static int |
LOG2_BYTES
The logarithm of the number of bytes of the primitive type of this list.
|
| Modifier | Constructor and Description |
|---|---|
protected |
DoubleMappedBigList(java.nio.DoubleBuffer[] buffer,
long size,
boolean[] readyToUse)
Creates a new mapped big list.
|
| Modifier and Type | Method and Description |
|---|---|
DoubleMappedBigList |
copy()
Returns a lightweight duplicate that can be read independently by another thread.
|
double |
getDouble(long index)
Returns the element at the specified position.
|
void |
getElements(long from,
double[] a,
int offset,
int length)
Copies (hopefully quickly) elements of this type-specific big list into the given array.
|
static DoubleMappedBigList |
map(java.nio.channels.FileChannel fileChannel)
Creates a new mapped big list by read-only mapping a given file channel using the
standard Java (i.e.,
DataOutput) byte order (ByteOrder.BIG_ENDIAN). |
static DoubleMappedBigList |
map(java.nio.channels.FileChannel fileChannel,
java.nio.ByteOrder byteOrder)
Creates a new mapped big list by read-only mapping a given file channel.
|
static DoubleMappedBigList |
map(java.nio.channels.FileChannel fileChannel,
java.nio.ByteOrder byteOrder,
java.nio.channels.FileChannel.MapMode mapMode)
Creates a new mapped big list by mapping a given file channel.
|
double |
set(long index,
double value)
Replaces the element at the specified position in this big list with the specified element (optional operation).
|
long |
size64() |
add, add, add, addAll, addAll, addAll, addAll, addElements, addElements, clear, compareTo, contains, ensureIndex, ensureRestrictedIndex, equals, forEach, get, getElements, hashCode, indexOf, indexOf, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, peek, peekDouble, pop, popDouble, push, push, rem, remove, removeDouble, removeElements, set, setElements, size, size, subList, top, topDouble, toStringadd, contains, containsAll, containsAll, forEach, remove, removeAll, removeAll, removeIf, retainAll, retainAll, toArray, toDoubleArray, toDoubleArrayclone, finalize, getClass, notify, notifyAll, wait, wait, waitaddAll, addAll, addAll, addAll, setElements, setElements, spliteratoradd, contains, containsAll, doubleIterator, doubleParallelStream, doubleSpliterator, doubleStream, parallelStream, remove, removeAll, removeIf, removeIf, removeIf, retainAll, stream, toArray, toDoubleArray, toDoubleArraypublic static int LOG2_BYTES
@Deprecated public static int LOG2_BITS
LOG2_BYTES.public static final long CHUNK_SIZE
map(FileChannel, ByteOrder, FileChannel.MapMode).protected DoubleMappedBigList(java.nio.DoubleBuffer[] buffer,
long size,
boolean[] readyToUse)
buffer - the underlying buffers.size - the overall number of elements in the underlying buffers (i.e., the
sum of the capacities of the byte buffers divided by the size of an element in bytes).readyToUse - an array parallel to buffer specifying which buffers do not need to be
duplicated before being used (the process will happen lazily); the array
will be used internally by the newly created mapped big list.public static DoubleMappedBigList map(java.nio.channels.FileChannel fileChannel) throws java.io.IOException
DataOutput) byte order (ByteOrder.BIG_ENDIAN).fileChannel - the file channel that will be mapped.fileChannel.java.io.IOException#map(FileChannel, ByteOrder, MapMode)public static DoubleMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder) throws java.io.IOException
fileChannel - the file channel that will be mapped.byteOrder - a prescribed byte order.fileChannel.java.io.IOException#map(FileChannel, ByteOrder, MapMode)public static DoubleMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder, java.nio.channels.FileChannel.MapMode mapMode) throws java.io.IOException
fileChannel - the file channel that will be mapped.byteOrder - a prescribed byte order.mapMode - the mapping mode: usually FileChannel.MapMode.READ_ONLY, but if intend to make the list
mutable, you have to pass FileChannel.MapMode.READ_WRITE.fileChannel.java.io.IOExceptionpublic DoubleMappedBigList copy()
Only chunks that are actually used will be duplicated lazily.
public double getDouble(long index)
DoubleBigListBigList.get(long)public void getElements(long from,
double[] a,
int offset,
int length)
DoubleBigListfrom - the start index (inclusive).a - the destination array.offset - the offset into the destination array where to store the first element copied.length - the number of elements to be copied.public double set(long index,
double value)
AbstractDoubleBigListset in interface DoubleBigListset in class AbstractDoubleBigListBigList.set(long,Object)public long size64()