A generational, in-memory cache adapter for Nebulex, inspired by epocxy cache.

The adapter keeps up to two active generations, newer and older. Each generation uses one ETS table, or several tables with the :shards backend.

Fetches check the newer generation first. When an unexpired entry is found in the older generation, the adapter moves it into the newer one. When a new generation is created, the newer generation becomes the older one, and the previous older generation is retired; its tables are deleted after :gc_cleanup_delay. This evicts entries in groups, without a timer per entry.

See Nebulex.Adapters.Local.Generation for details on generation management and garbage collection.

Features

• Configurable backend (:ets or :shards).
• Expiration through the :ttl option. Fetches exclude expired entries (see The :ttl option).
• Eviction by generational garbage collection (see Nebulex.Adapters.Local.Generation).
• Sharding for intensive workloads, with the :shards backend and the :partitions option.
• Transactions through Nebulex.Locks, which keeps the locks in ETS on the local node. See Nebulex.Adapter.Transaction.
• Stats.
• Retries when the garbage collector deletes a generation table during an operation (see Concurrency and resilience).

Concurrency and resilience

The garbage collector can delete a generation table while an operation still holds a reference to it. Accessing that table raises ArgumentError on the ETS backend, or {:unknown_table, tab} on the :shards backend.

Cache operations catch these errors, reload the generation references from the metadata table, and retry up to 3 times, waiting 10 ms before each retry. If the last retry fails too, the error propagates to the caller. Streams built from match-spec queries are not retried, because they read the tables lazily while the stream is consumed; streams with {:in, keys} retry each chunk of keys.

The :gc_cleanup_delay option (10 seconds by default) gives in-flight operations time to finish before their tables are deleted. When a new generation is created, the oldest one is removed from the active generations at once, but its tables are deleted only after the delay. The delay does not keep the retired entries available to new operations.

Configuration options

The following options can be used to configure the adapter:

• :cache (atom/0) - Required. The defined cache module.

• :stats (boolean/0) - A flag to determine whether to collect cache stats. The default value is true.

• :backend (backend/0) - The backend or storage to be used for the adapter. The default value is :ets.

• :read_concurrency (boolean/0) - Sets the ETS :read_concurrency option for each generation table. See :ets.new/2. The default value is true.

• :write_concurrency (boolean/0) - Sets the ETS :write_concurrency option for each generation table. See :ets.new/2. The default value is true.

• :compressed (boolean/0) - Sets the ETS :compressed option for each generation table. See :ets.new/2. The default value is false.

• :backend_type - The ETS table type for each generation table. See :ets.new/2. The default value is :set.

• :partitions (pos_integer/0) - The number of ETS partitions when using the :shards backend. See :shards.new/2.

  The default value is System.schedulers_online().

• :purge_chunk_size (pos_integer/0) - This option is deprecated. It is ignored and will be removed in the next major release. Deprecated: This option is no longer used and will be removed in the next major release. Purging the older generation on put_all and put_new_all is now done with per-key deletes.

• :gc_interval (pos_integer/0) - The interval between scheduled generation rotations, in milliseconds. If omitted, periodic rotation is disabled. Size or memory checks can still trigger rotations, and you can create a generation explicitly with MyCache.new_generation/1.

  Usage

  Set :gc_interval unless you want to rotate generations yourself. Without it, entries that are never fetched stay in the cache until a size or memory check or an explicit call creates a new generation.

• :max_size (pos_integer/0) - The entry-count threshold that triggers a generation rotation during a size check. If omitted, entry-count checks are disabled.

• :allocated_memory (pos_integer/0) - The memory threshold, in bytes, that triggers a generation rotation during a memory check. If omitted, memory checks are disabled.

• :gc_memory_check_interval (mem_check_interval/0) - The interval time in milliseconds for garbage collection to run the size and memory checks.

  Usage

  Beware: For the :gc_memory_check_interval option to work, you must configure one of :max_size or :allocated_memory (or both).

  The default value is 10000.

• :gc_cleanup_delay (pos_integer/0) - The delay in milliseconds before the oldest generation is deleted. This grace period allows ongoing operations to complete before the generation is removed. The default value is 10000.

• :lock_opts (keyword/0) - Options to customize the locks manager used for cache transactions.

  The local adapter uses Nebulex.Locks for implementing transactions with an ETS-based locking mechanism optimized for single-node scenarios. This option allows you to customize the behavior of the locks manager, such as adjusting the cleanup interval or batch size for stale lock cleanup.

  Available Options:

  • :cleanup_interval - The interval in milliseconds for periodic cleanup of stale locks (defaults to 5 minutes).
  • :cleanup_batch_size - The number of locks to process per batch during cleanup (defaults to 100).

  Note: The :name and :init_callback options are managed internally by the adapter and should not be provided. See the "Start Options" section in Nebulex.Locks for all available options.

  Examples:

  # Use default lock options (recommended for most cases)
  MyCache.start_link()
  
  # Customize cleanup for high-throughput scenarios
  MyCache.start_link(
    lock_opts: [
      cleanup_interval: :timer.minutes(1),
      cleanup_batch_size: 500
    ]
  )
  
  # More frequent cleanup for memory-constrained environments
  MyCache.start_link(
    lock_opts: [
      cleanup_interval: :timer.seconds(30)
    ]
  )

  The default value is [].

Usage

Define a cache module:

defmodule MyApp.LocalCache do
  use Nebulex.Cache,
    otp_app: :my_app,
    adapter: Nebulex.Adapters.Local
end

Configure the cache in your application environment, usually in config/config.exs:

config :my_app, MyApp.LocalCache,
  gc_interval: :timer.hours(12),
  max_size: 1_000_000,
  allocated_memory: 2_000_000_000,
  gc_memory_check_interval: :timer.seconds(10)

For intensive workloads, use the :shards backend to partition the cache across several ETS tables. Set the number of partitions with :partitions (defaults to System.schedulers_online()):

config :my_app, MyApp.LocalCache,
  backend: :shards,
  gc_interval: :timer.hours(12),
  max_size: 1_000_000,
  allocated_memory: 2_000_000_000,
  gc_memory_check_interval: :timer.seconds(10),
  partitions: System.schedulers_online() * 2

Add the cache to your application's supervision tree, usually in lib/my_app/application.ex:

def start(_type, _args) do
  children = [
    MyApp.LocalCache
  ]

  opts = [strategy: :one_for_one, name: MyApp.Supervisor]

  Supervisor.start_link(children, opts)
end

See Nebulex.Cache for more information.

The :ttl option

The :ttl option sets an entry's expiration time. Fetches check this time and exclude expired entries. Expiration does not schedule a timer to remove an entry when its TTL ends, so the TTL is not an eviction mechanism; configure the GC options for that. See the "Eviction policy" section. Custom match-spec queries need their own expiration conditions.

Generation rotation can evict an entry before its TTL ends, so a TTL does not guarantee that an entry stays cached that long.

Eviction policy

The adapter evicts entries by retiring the oldest generation. Fetching an entry from the older generation moves it into the newer one, so it survives the next rotation. This approximates eviction by recent access; the adapter does not keep an exact LRU order or count how often entries are accessed.

The garbage collector creates a new generation, retiring the oldest one:

• Every :gc_interval. With periodic rotation alone, an entry that is written and never read stays available for one to two :gc_interval periods, depending on when it was written.

• When a size or memory check finds that :max_size or :allocated_memory is reached or exceeded. The checks run every :gc_memory_check_interval when either option is set, and can evict entries sooner than the periodic rotation.

Configuring the GC options

Set :gc_interval to control how often the cache rotates generations. Shorter intervals evict unread entries sooner; longer intervals keep them longer. For example, with gc_interval: :timer.hours(12) and periodic rotation alone, an entry written once and never fetched is evicted after roughly 12 to 24 hours. Expiration or additional rotations can make it unavailable sooner.

Set :max_size, :allocated_memory, or both to trigger extra rotations when the cache reaches that size. Choose the values from the expected number and size of entries and the memory available to your application.

These thresholds are checked periodically, so they are not hard limits: the cache can exceed them between checks, and retired tables stay allocated until they are deleted after :gc_cleanup_delay. The checks run every :gc_memory_check_interval, which defaults to 10 seconds.

Queryable API

Use ETS match specifications to query entry fields, or {:in, keys} to look up specific keys (see "{:in, keys} queries"). The adapter also accepts query: :expired to select expired entries.

Entries are stored as {:entry, key, value, touched, exp, tag} tuples, and match specifications use this structure. You can write them directly or build them with Ex2ms or Nebulex.Adapters.Local.QueryHelper.

# Using raw ETS match spec
iex> match_spec = [
...>   {
...>     {:entry, :"$1", :"$2", :_, :_, :_},
...>     [{:>, :"$2", 1}],
...>     [{{:"$1", :"$2"}}]
...>   }
...> ]
iex> MyCache.get_all(query: match_spec)
{:ok, [b: 2, c: 3]}

# Using Ex2ms for easier query building
iex> import Ex2ms
iex> match_spec = fun do
...>   {_, key, value, _, _, _} when value > 1 -> {key, value}
...> end
iex> MyCache.get_all(query: match_spec)
{:ok, [b: 2, c: 3]}

{:in, keys} queries

Queries with {:in, keys} (e.g., MyCache.get_all(in: keys)) look up the requested keys directly through the table's key index, avoiding a full-table scan. Lookup cost depends on the table type and the number of entries stored under each key. Any term is a valid key, and duplicate keys in the input are processed only once. Bear in mind:

• get_all behaves like fetch/2 on each key: if an entry is found in the older generation, it is moved into the newer one, and expired entries are lazily removed on read.
• count_all skips expired entries and leaves them in the table.
• delete_all removes the entries stored under the given keys, expired ones included, but counts only the entries that had not expired.
• stream is read-only; it does not move entries across generations nor remove expired ones.
• Keys are compared with the table's native key equality, the same as the single-key commands: =:= on :set, :bag, and :duplicate_bag tables, and == on :ordered_set tables (an entry stored under the integer 1 matches the key 1.0). On :set and :ordered_set tables an entry is counted or returned once, even when several given keys alias it or when it is transiently present in both generations while a concurrent read promotes it.

Building match specs with QueryHelper

Nebulex.Adapters.Local.QueryHelper builds ETS match specs from named fields (:key, :value, :touched, :exp, and :tag), so you do not need to write the internal entry tuple yourself. It requires the optional :ex2ms dependency.

use Nebulex.Adapters.Local.QueryHelper

# Get the values of the entries whose key is an integer greater than 10
ms = match_spec key: k, value: v, where: is_integer(k) and k > 10, select: v
MyCache.get_all!(query: ms)

Use these match specs with get_all, count_all, delete_all, and stream. See Nebulex.Adapters.Local.QueryHelper for the full syntax and more examples, and its "Tags and caching decorators" section for using it with Nebulex.Caching decorators and cache references.

Tagging entries

Use the :tag option to attach any Elixir term to an entry. You can then query or delete the entries with that tag. Entries written without a tag have a nil tag.

MyCache.put("user:123:profile", profile, tag: {:user, 123})
MyCache.put_all([a: 1, b: 2, c: 3], tag: :group_a)

use Nebulex.Adapters.Local.QueryHelper

# Get the values of the entries tagged :group_a
ms = match_spec value: v, tag: t, where: t == :group_a, select: v
MyCache.get_all!(query: ms)
#=> [1, 2, 3]

# Delete all the entries of user 123
ms = match_spec tag: t, where: t == {:user, 123}
MyCache.delete_all!(query: ms)

Tags are metadata, not part of the key. With the default :set table type, writing the same key replaces its entry regardless of the tag.

With Ex2ms, match the tag as the last element of the entry tuple:

import Ex2ms

match_spec = fun do
  {_, _, value, _, _, tag} when tag == :group_a -> value
end

MyCache.get_all!(query: match_spec)
#=> [1, 2, 3]

Transaction API

This adapter implements Nebulex.Adapter.Transaction using Nebulex.Locks, which keeps the locks in ETS on the local node.

The transaction command accepts the following options:

• :keys (list of term/0) - The list of keys to lock. If empty or not provided, a global lock is used. For better concurrency, always specify the keys involved in the transaction to enable fine-grained locking. The default value is [].

• :retries (:infinity | non_neg_integer/0) - The number of times to retry acquiring locks before giving up. If set to :infinity, the process will retry indefinitely until locks are acquired. The default value is 3.

• :retry_interval (non_neg_integer/0 | (attempt :: non_neg_integer() -> non_neg_integer())) - The time in milliseconds to wait between lock acquisition retries.

  Can be either:

  • A timeout value (non_neg_integer()) - A fixed interval. When using a non-negative integer, a small random jitter is added to prevent thundering herd issues.
  • An anonymous function - Receives the current attempt number (0-indexed) and must return a non-negative integer representing the interval in milliseconds. No jitter is added, giving you full control over the retry strategy. Returning 0 results in immediate retry with no delay.

  Examples:

  # Fixed interval with automatic jitter
  retry_interval: 10
  
  # Exponential backoff
  retry_interval: fn attempt ->
    round(min(100 * :math.pow(2, attempt), 5000))
  end
  
  # Linear backoff
  retry_interval: fn attempt -> 10 + (attempt * 50) end
  
  # Immediate retry on first attempt, then backoff
  retry_interval: fn
    0 -> 0
    attempt -> attempt * 100
  end
  
  # Custom strategy with manual jitter
  retry_interval: fn attempt ->
    base = 100 * attempt
    jitter = :rand.uniform(max(base, 1))
    base + jitter
  end

  The default value is 10.

• :lock_timeout (timeout/0) - The maximum time in milliseconds a lock can be held before being considered stale. Stale locks (from crashed processes or timeouts) are automatically cleaned up during acquisition attempts. The default value is 30000.

To customize the locks manager, set the :lock_opts option when starting the cache (see the configuration options above).

Extended API (extra functions)

The adapter adds these functions to the Nebulex.Cache API.

Creating new generations:

MyCache.new_generation()
MyCache.new_generation(gc_interval_reset: false)

Retrieving the current generations:

MyCache.generations()

Retrieving the newer generation:

MyCache.newer_generation()