The EmbeddedStorageManager is mostly created with factory methods of EmbeddedStorage, where the most common settings, like database directory or the root instance, can be configured.

Foundations

To achieve a more detailed customization, you can utilize the EmbeddedStorageFoundation factory type. It holds and creates on demand all the parts that form an EmbeddedStorageManager.

External Configuration

The artifact storage.embedded.configuration provides a convenience layer for configuration purposes, as well as facilities to read external configuration.

The Configuration type consolidates the most widely used parameters from the storage foundations in one place. It's output is an EmbeddedStorageFoundation from which a EmbeddedStorageManager can be created.

To read an external configuration use ConfigurationLoader and ConfigurationParser or the Load*() methods of Configuration. Currently XML and INI files are supported.

Channel Usage

are the IO threads used by the MicroStream storage engine. A single channel represents the unity of a thread, a storage directory and cached data. Increasing the number of channels means to run more IO threads.

The channel count is an important configuration value that impacts to IO performance.

Channel Configuration

For the channel configuration the following configuration are available:

• Channel count

  The number of channels that MicroStream will use. Must be

• Channel directory prefix

  The channel directory will be prefix+channelNumber e.g. "ch_0" if prefix is "ch_"

• Data file prefix default is "channel_"

Channel file size configuration is done by the the .

They can be set By storage.embedded.configuration API:

See also:

Or by setting a StorageFileProvider using theEmbeddedStorageFoundationfactory

These are the available properties of the Configuration type. The names are used accordingly in the external configuration files. They can be found as constants in ConfigurationPropertyNames.

Detailed Description

channelCount

Number of threads used by the storage engine. It depicts the numbers of subdirectories as well. Each thread manages one directory in which it writes to and reads from exclusively. The unity of thread, directory and the cached data therefor is called a "Channel".

houseKeepingInterval

Number of milliseconds for the house keeping interval. House keeping tasks are, among others:

• Garbage Collection

• Cache Check

• File Cleanup Check

In combination with houseKeepingNanoTimeBudget, it can be specified how many CPU time should be used for house keeping. E.g. interval=1000ms and budget=10000000ns means every second there's 0.01 seconds time for house keeping, so max 1% CPU time used for house keeping. This CPU time window is only used if house keeping work is pending. If nothing has to be done, no time is wasted.

houseKeepingNanoTimeBudget

Number of nanoseconds used for each housekeeping cycle. However, no matter how low the number is, one item of work will always be completed. But if there is nothing to clean up, no processor time will be wasted. Default is 10000000 (10 million nanoseconds = 10 milliseconds = 0.01 seconds). However, no matter how small the time is, one item is done at least. This is to avoid no-ops, if a too small time window is configured. This time budget is a "best effort" threshold, meaning when at 1ns left, a huge file has to be cleaned or the references of a huge collection have to be marked for GC, then this budget can be exceeded considerably.

dataFileDissolveRatio

The degree of the data payload of a storage file to avoid cleaning it up. There are logical "gaps" in database files, byte ranges of records which were replaced with newer versions of them in subsequent database files. These gaps can be removed to keep the used disk space at a minimum. A value of 1 means: Even at the smallest gap clean up the file. A value of 0 means: Gaps don't matter, never clean up anything. All live data in a file is copied to a current head file. Then the source file only consists of gaps and can be deleted without loss of data.

dataFileMinSize

Minimum file size in bytes of a storage file to avoid merging with other files during housekeeping. Must be greater than 1, maximum is 2GB.

dataFileMaxSize

Maximum file size in bytes of a storage file to avoid splitting in more files during housekeeping. Must be greater than 1, maximum is 2GB.

Involved Types

This list shows which property configures which type, used by the foundation types, to create the storage manager.

By default, MicroStream uses the operation system's standard file locking mechanism to prevent simultaneous access to the storage files. In the rare case that this is not sufficient to control the file access MicroStream provides a proprietary file lock implementation to ensure exclusive access to the storage files from different applications using MicroStream.

Using this file lock may only be necessary if, while a MicroStream application is running, a second MicroStream application may try to access the same storage and the default file locks are not reliable.

You don't need to activate this feature if:

• Only one MicroStream application will access the storage,

• MicroStream applications that may access the same storage run on the same system,

• other applications that may access the storage files don't use MicroStream to access them.

To activate the internal file lock you need to setup StorageLockFileSetup:

The default interval the locks are updated is 10 seconds, you can set a custom value in milliseconds with:

Storage.LockFileSetupProvider( final long updateInterval )

To specify the charset used by the lock files use:

Storage.LockFileSetupProvider( final Charset charset )

or, to customize both:

LockFileSetupProvider( final Charset charset , final long updateInterval )

Intervall and Time Budget

interval and time budget is configured by setting up a StorageHousekeepingController.

Available properties are:

• interval the housekeeping is triggered in milliseconds, default once per every second

• time budget for housekeeping in nanoseconds, default is 0.01 seconds

File Sizes and Payload

The desired file min and max sizes and payload ratio is configured by the StorageDataFileEvaluator:

available properties are:

• Data file minimum size Files smaller then minimum file size will be merged with other files if possible, default is 1 MB.

• Data file maximum size Files larger then maximum file size will be split in smaller ones, default is 8 MB.

• Data file payload min ratio to trigger file refactoring, default is 0.75 (75%).

Cache

The lifetime of objects in the internal entity cache can be configured by the StorageEntityCacheEvaluator:

Available properties are:

• Entity cache threshold

  Abstract threshold value, roughly comparable to size in bytes with a time component, at which a cache must be cleared of some entities. Default is 1000000000.

• Entity cache timeout Time in milliseconds after that an entity is considered to be old if not read meanwhile. must be greater zero, default is 86400000ms (1 day).

Using a Storage File Provider (one.microstream.storage.types.StorageFileProvider) allows to specify the location and naming rules for all storage related files.

available properties are:

• BaseDirectory The Microstream storages location base directory. Contains channel directories and type dictionary file.

• DeletionDirectory If configured, the storage will not delete files. Instead of deleting a file it will be moved to this directory.

• TruncationDirectory If configured, files that will get truncated are copied into this directory.