public class LongMappedBigList extends AbstractLongBigList
Java's memory-mapping facilities have
the severe limitation of mapping at most Integer.MAX_VALUE bytes, as they
expose the content of a file using a MappedByteBuffer. This class can expose
a file of primitive types of arbitrary length as a BigList
that is actually based on an array of MappedByteBuffers, each mapping
a chunk of CHUNK_SIZE longs.
Mapping can happen with a specified byte order: saving and mapping data in native order
using methods from it.unimi.dsi.fastutil.io.BinIO will enhance performance significantly.
Instances of this class are not thread safe, but the copy() method provides a lightweight
duplicate that can be read independently by another thread.
Only chunks that are actually used will be duplicated lazily.
If you are modifiying the content of list, however, you will need to provide external synchronization.
AbstractLongBigList.LongRandomAccessSubList, AbstractLongBigList.LongSubList| Modifier and Type | Field and Description |
|---|---|
static long |
CHUNK_SIZE
The size in elements of a chunk created by
map(FileChannel, ByteOrder, FileChannel.MapMode). |
static int |
LOG2_BITS
Deprecated.
Use
LOG2_BYTES. |
static int |
LOG2_BYTES
The logarithm of the number of bytes of the primitive type of this list.
|
| Modifier | Constructor and Description |
|---|---|
protected |
LongMappedBigList(java.nio.LongBuffer[] buffer,
long size,
boolean[] readyToUse)
Creates a new mapped big list.
|
| Modifier and Type | Method and Description |
|---|---|
LongMappedBigList |
copy()
Returns a lightweight duplicate that can be read independently by another thread.
|
void |
getElements(long from,
long[] a,
int offset,
int length)
Copies (hopefully quickly) elements of this type-specific big list into the given array.
|
long |
getLong(long index)
Returns the element at the specified position.
|
static LongMappedBigList |
map(java.nio.channels.FileChannel fileChannel)
Creates a new mapped big list by read-only mapping a given file channel using the
standard Java (i.e.,
DataOutput) byte order (ByteOrder.BIG_ENDIAN). |
static LongMappedBigList |
map(java.nio.channels.FileChannel fileChannel,
java.nio.ByteOrder byteOrder)
Creates a new mapped big list by read-only mapping a given file channel.
|
static LongMappedBigList |
map(java.nio.channels.FileChannel fileChannel,
java.nio.ByteOrder byteOrder,
java.nio.channels.FileChannel.MapMode mapMode)
Creates a new mapped big list by mapping a given file channel.
|
long |
set(long index,
long value)
Replaces the element at the specified position in this big list with the specified element (optional operation).
|
long |
size64() |
add, add, add, addAll, addAll, addAll, addAll, addElements, addElements, clear, compareTo, contains, ensureIndex, ensureRestrictedIndex, equals, forEach, get, getElements, hashCode, indexOf, indexOf, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, peek, peekLong, pop, popLong, push, push, rem, remove, removeElements, removeLong, set, setElements, size, size, subList, top, topLong, toStringadd, contains, containsAll, containsAll, forEach, remove, removeAll, removeAll, removeIf, retainAll, retainAll, toArray, toLongArray, toLongArrayclone, finalize, getClass, notify, notifyAll, wait, wait, waitaddAll, addAll, addAll, addAll, setElements, setElements, spliteratoradd, contains, containsAll, longIterator, longParallelStream, longSpliterator, longStream, parallelStream, remove, removeAll, removeIf, removeIf, removeIf, retainAll, stream, toArray, toLongArray, toLongArraypublic static int LOG2_BYTES
@Deprecated public static int LOG2_BITS
LOG2_BYTES.public static final long CHUNK_SIZE
map(FileChannel, ByteOrder, FileChannel.MapMode).protected LongMappedBigList(java.nio.LongBuffer[] buffer,
long size,
boolean[] readyToUse)
buffer - the underlying buffers.size - the overall number of elements in the underlying buffers (i.e., the
sum of the capacities of the byte buffers divided by the size of an element in bytes).readyToUse - an array parallel to buffer specifying which buffers do not need to be
duplicated before being used (the process will happen lazily); the array
will be used internally by the newly created mapped big list.public static LongMappedBigList map(java.nio.channels.FileChannel fileChannel) throws java.io.IOException
DataOutput) byte order (ByteOrder.BIG_ENDIAN).fileChannel - the file channel that will be mapped.fileChannel.java.io.IOException#map(FileChannel, ByteOrder, MapMode)public static LongMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder) throws java.io.IOException
fileChannel - the file channel that will be mapped.byteOrder - a prescribed byte order.fileChannel.java.io.IOException#map(FileChannel, ByteOrder, MapMode)public static LongMappedBigList map(java.nio.channels.FileChannel fileChannel, java.nio.ByteOrder byteOrder, java.nio.channels.FileChannel.MapMode mapMode) throws java.io.IOException
fileChannel - the file channel that will be mapped.byteOrder - a prescribed byte order.mapMode - the mapping mode: usually FileChannel.MapMode.READ_ONLY, but if intend to make the list
mutable, you have to pass FileChannel.MapMode.READ_WRITE.fileChannel.java.io.IOExceptionpublic LongMappedBigList copy()
Only chunks that are actually used will be duplicated lazily.
public long getLong(long index)
LongBigListBigList.get(long)public void getElements(long from,
long[] a,
int offset,
int length)
LongBigListfrom - the start index (inclusive).a - the destination array.offset - the offset into the destination array where to store the first element copied.length - the number of elements to be copied.public long set(long index,
long value)
AbstractLongBigListset in interface LongBigListset in class AbstractLongBigListBigList.set(long,Object)public long size64()