Package com.intellij.util.io

Class PersistentHashMap<Key,Value>

java.lang.Object

com.intellij.util.io.PersistentEnumeratorDelegate<Key>

com.intellij.util.io.PersistentHashMap<Key,Value>

All Implemented Interfaces:

Forceable, DataEnumerator<Key>, DataEnumeratorEx<Key>, KeyValueStore<Key,Value>, PersistentMap<Key,Value>, java.io.Closeable, java.lang.AutoCloseable

public class PersistentHashMap<Key,Value>
extends PersistentEnumeratorDelegate<Key>
implements PersistentMap<Key,Value>

Thread-safe implementation of persistent hash map (PHM). The implementation works in the following (generic) way:

• Particular key is translated via myEnumerator into an int.
• As part of enumeration process for the new key, additional space is reserved in myEnumerator.myStorage for offset in ".values" file (myValueStorage) where (serialized) value is stored.
• Once new value is written the offset storage is updated.
• When the key is removed from PHM, offset storage is set to zero.

It is important to note that offset is non-negative and can be 4 or 8 bytes, depending on the size of the ".values" file. PHM can work in appendable mode: for particular key additional calculated chunk of value can be appended to ".values" file with the offset of previously calculated chunk. For performance reasons we try hard to minimize storage occupied by keys / offsets in ".values" file: this storage is allocated as (limited) direct byte buffers so 4 bytes offset is used until it is possible. Generic record produced by enumerator used with PHM as part of new key enumeration is ? [.values file offset 4 or 8 bytes], however for unique integral keys enumerate_id isn't produced. Also for certain Value types it is possible to avoid random reads at all: e.g. in case Value is non-negative integer the value can be stored directly in storage used for offset and in case of btree enumerator directly in btree leaf.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	PersistentHashMap.ValueDataAppender

Field Summary

Fields

Modifier and Type	Field and Description
protected DataExternalizer<Value>	myValueExternalizer

Fields inherited from class com.intellij.util.io.PersistentEnumeratorDelegate

myEnumerator

Constructor Summary

Constructors

Constructor and Description
PersistentHashMap(java.io.File file, KeyDescriptor<Key> keyDescriptor, DataExternalizer<Value> valueExternalizer)
PersistentHashMap(java.nio.file.Path file, KeyDescriptor<Key> keyDescriptor, DataExternalizer<Value> valueExternalizer)
PersistentHashMap(java.nio.file.Path file, KeyDescriptor<Key> keyDescriptor, DataExternalizer<Value> valueExternalizer, int initialSize)
PersistentHashMap(java.nio.file.Path file, KeyDescriptor<Key> keyDescriptor, DataExternalizer<Value> valueExternalizer, int initialSize, int version)
PersistentHashMap(java.nio.file.Path file, KeyDescriptor<Key> keyDescriptor, DataExternalizer<Value> valueExternalizer, int initialSize, int version, PagedFileStorage.StorageLockContext lockContext)

Method Summary

Modifier and Type	Method and Description
void	appendData(Key key, PersistentHashMap.ValueDataAppender appender) Appends value chunk from specified appender to key's value.
void	close()
void	compact()
boolean	containsMapping(Key key)
static void	deleteFilesStartingWith(java.io.File prefixFile)
static void	deleteMap(PersistentHashMap<?,?> map) Deletes files and trying to close it before.
protected void	doDropMemoryCaches()
protected void	doForce()
protected Value	doGet(Key key)
protected void	doPut(Key key, Value value)
protected void	doRemove(Key key)
void	dropMemoryCaches()
int	enumerate(Key name)
void	force()
Value	get(Key key)
java.util.Collection<Key>	getAllKeysWithExistingMapping()
java.nio.file.Path	getBaseFile()
boolean	getReadOnly()
int	getSize()
boolean	isBusyReading()
protected boolean	isReadOnly()
boolean	makesSenseToCompact()
boolean	processKeys(Processor<? super Key> processor) Process all keys registered in the map.
boolean	processKeysWithExistingMapping(Processor<? super Key> processor)
void	put(Key key, Value value)
void	remove(Key key)
java.lang.String	toString()
protected boolean	wantNonNegativeIntegralValues()

Methods inherited from class com.intellij.util.io.PersistentEnumeratorDelegate

getAllDataObjects, getVersion, isClosed, isCorrupted, isDirty, markCorrupted, markDirty, traverseAllRecords, tryEnumerate, useBtree, valueOf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.intellij.util.io.PersistentMap

isClosed, isDirty, markDirty

Field Detail

myValueExternalizer

protected final DataExternalizer<Value> myValueExternalizer

Constructor Detail

PersistentHashMap

public PersistentHashMap(java.io.File file,
                         KeyDescriptor<Key> keyDescriptor,
                         DataExternalizer<Value> valueExternalizer)
                  throws java.io.IOException

Throws:

java.io.IOException

PersistentHashMap

public PersistentHashMap(java.nio.file.Path file,
                         KeyDescriptor<Key> keyDescriptor,
                         DataExternalizer<Value> valueExternalizer)
                  throws java.io.IOException

Throws:

java.io.IOException

PersistentHashMap

public PersistentHashMap(java.nio.file.Path file,
                         KeyDescriptor<Key> keyDescriptor,
                         DataExternalizer<Value> valueExternalizer,
                         int initialSize)
                  throws java.io.IOException

Throws:

java.io.IOException

PersistentHashMap

public PersistentHashMap(java.nio.file.Path file,
                         KeyDescriptor<Key> keyDescriptor,
                         DataExternalizer<Value> valueExternalizer,
                         int initialSize,
                         int version)
                  throws java.io.IOException

Throws:

java.io.IOException

PersistentHashMap

public PersistentHashMap(java.nio.file.Path file,
                         KeyDescriptor<Key> keyDescriptor,
                         DataExternalizer<Value> valueExternalizer,
                         int initialSize,
                         int version,
                         PagedFileStorage.StorageLockContext lockContext)
                  throws java.io.IOException

Throws:

java.io.IOException

Method Detail

wantNonNegativeIntegralValues

protected boolean wantNonNegativeIntegralValues()

isReadOnly

protected boolean isReadOnly()

dropMemoryCaches

public final void dropMemoryCaches()

doDropMemoryCaches

protected void doDropMemoryCaches()

getSize

public int getSize()

getBaseFile

public java.nio.file.Path getBaseFile()

makesSenseToCompact

public boolean makesSenseToCompact()

deleteFilesStartingWith

public static void deleteFilesStartingWith(java.io.File prefixFile)

deleteMap

public static void deleteMap(PersistentHashMap<?,?> map)

Deletes files and trying to close it before.

put

public final void put(Key key,
                      Value value)
               throws java.io.IOException

Specified by:

put in interface KeyValueStore<Key,Value>

Throws:

java.io.IOException

doPut

protected void doPut(Key key,
                     Value value)
              throws java.io.IOException

Throws:

java.io.IOException

enumerate

public final int enumerate(Key name)
                    throws java.io.IOException

Specified by:

enumerate in interface DataEnumerator<Key>

Overrides:

enumerate in class PersistentEnumeratorDelegate<Key>

Throws:

java.io.IOException

appendData

public final void appendData(Key key,
                             PersistentHashMap.ValueDataAppender appender)
                      throws java.io.IOException

Appends value chunk from specified appender to key's value. Important use note: value externalizer used by this map should process all bytes from DataInput during deserialization and make sure that deserialized value is consistent with value chunks appended. E.g. Value can be Set of String and individual Strings can be appended with this method for particular key, when get(Object) will be eventually called for the key, deserializer will read all bytes retrieving Strings and collecting them into Set

Throws:

java.io.IOException

processKeys

public final boolean processKeys(Processor<? super Key> processor)
                          throws java.io.IOException

Process all keys registered in the map. Note that keys which were removed after compact() call will be processed as well. Use processKeysWithExistingMapping(Processor) to process only keys with existing mappings

Specified by:

processKeys in interface PersistentMap<Key,Value>

Throws:

java.io.IOException

getAllKeysWithExistingMapping

public java.util.Collection<Key> getAllKeysWithExistingMapping()
                                                        throws java.io.IOException

Throws:

java.io.IOException

processKeysWithExistingMapping

public final boolean processKeysWithExistingMapping(Processor<? super Key> processor)
                                             throws java.io.IOException

Throws:

java.io.IOException

get

public final Value get(Key key)
                throws java.io.IOException

Specified by:

get in interface KeyValueStore<Key,Value>

Throws:

java.io.IOException

isBusyReading

public boolean isBusyReading()

doGet

protected Value doGet(Key key)
               throws java.io.IOException

Throws:

java.io.IOException

containsMapping

public final boolean containsMapping(Key key)
                              throws java.io.IOException

Throws:

java.io.IOException

remove

public final void remove(Key key)
                  throws java.io.IOException

Throws:

java.io.IOException

doRemove

protected void doRemove(Key key)
                 throws java.io.IOException

Throws:

java.io.IOException

force

public final void force()

Specified by:

force in interface Forceable

Specified by:

force in interface KeyValueStore<Key,Value>

Overrides:

force in class PersistentEnumeratorDelegate<Key>

doForce

protected void doForce()

close

public final void close()
                 throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Overrides:

close in class PersistentEnumeratorDelegate<Key>

Throws:

java.io.IOException

compact

public void compact()
             throws java.io.IOException

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

getReadOnly

public boolean getReadOnly()