public class LongMappedBigList
extends it.unimi.dsi.fastutil.longs.AbstractLongBigList
Java's memory-mapping facilities have
the severe limitation of mapping at most Integer.MAX_VALUE bytes, as they
expose the content of a file using a MappedByteBuffer. This class can expose
a file of primitive types of arbitrary length as a BigList
that is actually based on an array of MappedByteBuffers, each mapping
a chunk of CHUNK_SIZE longs.
Mapping can happen with a specified byte order: saving and mapping data in native order
using methods from BinIO will enhance performance significantly.
Instances of this class are not thread safe, but the copy() method provides a lightweight
duplicate that can be read independently by another thread.
Only chunks that are actually used will be duplicated lazily.
If you are modifiying the content of list, however, you will need to provide external synchronization.
| Modifier and Type | Field and Description |
|---|---|
static long |
CHUNK_SIZE
The size in elements of a chunk created by
map(FileChannel, ByteOrder, FileChannel.MapMode). |
static int |
LOG2_BITS
Deprecated.
Use
LOG2_BYTES. |
static int |
LOG2_BYTES
The logarithm of the number of bytes of the primitive type of this list.
|
| Modifier | Constructor and Description |
|---|---|
protected |
LongMappedBigList(java.nio.LongBuffer[] buffer,
long size,
boolean[] readyToUse)
Creates a new mapped big list.
|
| Modifier and Type | Method and Description |
|---|---|
LongMappedBigList |
copy()
Returns a lightweight duplicate that can be read independently by another thread.
|
void |
getElements(long from,
long[] a,
int offset,
int length) |
long |
getLong(long index) |
static LongMappedBigList |
map(java.nio.channels.FileChannel fileChannel)
Creates a new mapped big list by read-only mapping a given file channel using the
standard Java (i.e.,
DataOutput) byte order (ByteOrder.BIG_ENDIAN). |
static LongMappedBigList |
map(java.nio.channels.FileChannel fileChannel,
java.nio.ByteOrder byteOrder)
Creates a new mapped big list by read-only mapping a given file channel.
|
static LongMappedBigList |
map(java.nio.channels.FileChannel fileChannel,
java.nio.ByteOrder byteOrder,
java.nio.channels.FileChannel.MapMode mapMode)
Creates a new mapped big list by mapping a given file channel.
|
long |
set(long index,
long value) |
long |
size64() |
add, add, add, addAll, addAll, addAll, addAll, addElements, addElements, clear, compareTo, contains, ensureIndex, ensureRestrictedIndex, equals, forEach, get, getElements, hashCode, indexOf, indexOf, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, peek, peekLong, pop, popLong, push, push, rem, remove, removeElements, removeLong, set, setElements, size, size, subList, top, topLong, toStringadd, contains, containsAll, containsAll, forEach, remove, removeAll, removeAll, removeIf, retainAll, retainAll, toArray, toLongArray, toLongArrayclone, finalize, getClass, notify, notifyAll, wait, wait, waitaddAll, addAll, addAll, addAll, setElements, setElements, spliteratoradd, contains, containsAll, longIterator, longParallelStream, longSpliterator, longStream, parallelStream, remove, removeAll, removeIf, removeIf, removeIf, retainAll, stream, toArray, toLongArray, toLongArraypublic static int LOG2_BYTES
@Deprecated public static int LOG2_BITS
LOG2_BYTES.public static final long CHUNK_SIZE
map(FileChannel, ByteOrder, FileChannel.MapMode).protected LongMappedBigList(java.nio.LongBuffer[] buffer,
long size,
boolean[] readyToUse)
buffer - the underlying buffers.size - the overall number of elements in the underlying buffers (i.e., the
sum of the capacities of the byte buffers divided by the size of an element in bytes).readyToUse - an array parallel to buffer specifying which buffers do not need to be
duplicated before being used (the process will happen lazily); the array
will be used internally by the newly created mapped big list.public static LongMappedBigList map(java.nio.channels.FileChannel fileChannel) throws java.io.IOException
DataOutput) byte order (ByteOrder.BIG_ENDIAN).fileChannel - the file channel that will be mapped.fileChannel.java.io.IOException#map(FileChannel, ByteOrder, MapMode)public static LongMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder) throws java.io.IOException
fileChannel - the file channel that will be mapped.byteOrder - a prescribed byte order.fileChannel.java.io.IOException#map(FileChannel, ByteOrder, MapMode)public static LongMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder, java.nio.channels.FileChannel.MapMode mapMode) throws java.io.IOException
fileChannel - the file channel that will be mapped.byteOrder - a prescribed byte order.mapMode - the mapping mode: usually FileChannel.MapMode.READ_ONLY, but if intend to make the list
mutable, you have to pass FileChannel.MapMode.READ_WRITE.fileChannel.java.io.IOExceptionpublic LongMappedBigList copy()
Only chunks that are actually used will be duplicated lazily.
public long getLong(long index)
public void getElements(long from,
long[] a,
int offset,
int length)
public long set(long index,
long value)
set in interface it.unimi.dsi.fastutil.longs.LongBigListset in class it.unimi.dsi.fastutil.longs.AbstractLongBigListpublic long size64()