The EmbeddedStorageManager is mostly created with factory methods of EmbeddedStorage, where the most common settings, like database directory or the root instance, can be configured.

Foundations

To achieve a more detailed customization, you can utilize the EmbeddedStorageFoundation factory type. It holds and creates on demand all the parts that form an EmbeddedStorageManager.

External Configuration

The artifact storage.embedded.configuration provides a convenience layer for configuration purposes, as well as facilities to read external configuration.

The Configuration type consolidates the most widely used parameters from the storage foundations in one place. It's output is an EmbeddedStorageFoundation from which a EmbeddedStorageManager can be created.

To read an external configuration use ConfigurationLoader and ConfigurationParser or the Load*() methods of Configuration. Currently XML and INI files are supported.

If you just use Configuration.Load() the default configuration file is used, which is either a file in the classpath root named microstream-storage.properties, or the path configured via the system property microstream.storage.configuration.path.

To issue a full backup of the whole storage to be executed, the EmbeddedStorageManager provides two methods named issueFullBackup. Keep in mind that this could result in a very long running operation, depending on the storage size. Although the full backup may be a valid solution in some circumstances, the should be preferred, since it is by far more efficient.

By default, the continuous backup is disabled. If enabled the MicroStream instance will clone all changes to another directory. The backup is identical to the primary MicroStream storage.

To enable the continuous backup just set the backup directory:

With storage.embedded.configuration API:

With MicroStream foundation classes:

By default, MicroStream uses the operation system's standard file locking mechanism to prevent simultaneous access to the storage files. In the rare case that this is not sufficient to control the file access MicroStream provides a proprietary file lock implementation to ensure exclusive access to the storage files from different applications using MicroStream.

Intervall and Time Budget

interval and time budget is configured by setting up a StorageHousekeepingController.

Available properties are:

• interval the housekeeping is triggered in milliseconds, default once per every second

• time budget for housekeeping in nanoseconds, default is 0.01 seconds

File Sizes and Payload

The desired file min and max sizes and payload ratio is configured by the StorageDataFileEvaluator:

available properties are:

• Files smaller then minimum file size will be merged with other files if possible, default is 1 MB.

• Files larger then maximum file size will be split in smaller ones, default is 8 MB.

• Ratio of non-gap data contained in a storage file to prevent the file from being dissolved, default is 0.75 (75%).

Cache

The lifetime of objects in the internal entity cache can be configured by the StorageEntityCacheEvaluator:

Available properties are:

• Entity cache threshold

  Abstract threshold value, roughly comparable to size in bytes with a time component, at which a cache must be cleared of some entities. Default is 1000000000.

• Entity cache timeout Time in milliseconds after that an entity is considered to be old if not read meanwhile. must be greater zero, default is 86400000ms (1 day).

Using a Storage Live File Provider (one.microstream.storage.types.StorageLiveFileProvider

These are the available properties of the Configuration type. The names are used accordingly in the external configuration files. They can be found as constants in ConfigurationPropertyNames.

Detailed Description

channelCount

Number of threads used by the storage engine. It depicts the numbers of subdirectories as well. Each thread manages one directory in which it writes to and reads from exclusively. The unity of thread, directory and the cached data therefor is called a "Channel".

houseKeepingIntervalMs

Number of milliseconds for the house keeping interval. House keeping tasks are, among others:

• Garbage Collection

• Cache Check

• File Cleanup Check

In combination with houseKeepingNanoTimeBudget, it can be specified how many CPU time should be used for house keeping. E.g. interval=1000ms and budget=10000000ns means every second there's 0.01 seconds time for house keeping, so max 1% CPU time used for house keeping. This CPU time window is only used if house keeping work is pending. If nothing has to be done, no time is wasted.

housekeepingTimeBudgetNs

Number of nanoseconds used for each housekeeping cycle. However, no matter how low the number is, one item of work will always be completed. But if there is nothing to clean up, no processor time will be wasted. Default is 10000000 (10 million nanoseconds = 10 milliseconds = 0.01 seconds). However, no matter how small the time is, one item is done at least. This is to avoid no-ops, if a too small time window is configured. This time budget is a "best effort" threshold, meaning when at 1ns left, a huge file has to be cleaned or the references of a huge collection have to be marked for GC, then this budget can be exceeded considerably.

dataFileMinimumSize

Minimum file size in bytes of a storage file to avoid merging with other files during housekeeping. Must be greater than 1, maximum is 2GB.

dataFileMaximumSize

Maximum file size in bytes of a storage file to avoid splitting in more files during housekeeping. Must be greater than 1, maximum is 2GB.

dataFileMinimumUseRatio

The ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to prevent the file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data, inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its length as a negative value length header). The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the more file dissolving (data transfers to new files) is required and vice versa.

Involved Types

This list shows which property configures which type, used by the foundation types, to create the storage manager.