Feature Request: TTL only Cache

[elbaro]

1. Unlimited sized Cache

@Memoize(Cache("unlimited"), timedelta(seconds=10 * 60))
def get():
    ...

2. Zero sized Cache (ignore returned value)

@Memoize(Cache("empty"), timedelta(seconds=10 * 60))
def update_global_dict():
    value = query(..)
    global_dict[value.field2] = value.field1
    global_dict[value.field3] = value.field1

[Yiling-J]

@elbaro Could you please elaborate on the use cases for unlimited and empty? Theine uses several queues and LRUs to improve hit ratio, all of which are size-aware. If size is set to unlimited or 0, the eviction policy will not work.

[elbaro]

1. Unlimited sized Cache

• You have a small set of key-values that fit in a memory, and they are periodically updated. You don't want to spam the mysql server.

@Memoize(Cache("unlimited"), timedelta(seconds=10))  # config rarely changes, so fetch once every 10 second
def get(config_name):
    return mysql_client.query(f"SELECT * .. WHERE name = {config_name}")[0].value

While Cache("lru", very_large_number) will work, it preallocates unnecessary memory and lru is not used at all.

This is similar to @cachetools.cached(cache={}) but with TTL.

2. Zero-sized Cache
   You are only interested in its side-effect, not the returned value. This is not actually zero-sized as we need to track TTL for each key.

The benefit of having a deciated class for this is to avoid the overhead of None, which is small. This feature request is optional.

reverse_index = {}

@Memoize(Cache("unlimited"), timedelta(seconds=10 * 60))
def update(key):
    value1, value2 = query(key)
    reverse_index[value1] = key
    reverse_index[value2] = key

    # we want to keep reverse_index update-to-date
    # but not interested in the returned value of update(key).
    return None     # Cache stores None which is a waste

[Yiling-J]

@elbaro The first case is very simple. I think a dictionary with a TTL field is sufficient. In your second example, what's the purpose of cache? If you want to cache reverse_index for 10 minutes,how about using a real cached method here: def get_reverse_index(key) -> Dict.

[elbaro]

reverse_index is a global dict that is shared by multiple keys. It can lookup with key = reverse_index[value].

def get_reverse_index(key) -> Dict

This will create a reverse index for each key. Given value, we don't know which dict to lookup.

Another example without data mapping:

@Memoize(Cache("?"), timedelta(seconds=10))
def log_error(err) -> None:  # log each type of error at most once every 10 seconds
    write_to_file(err)

[Yiling-J]

@elbaro I see. So how about adding a simple cache policy, which doesn't preallocate? Then you can set a very large cache size. About your second case, you can always use a constant key:

@Memoize(Cache("simple", 1), timedelta(seconds=100))
def foo(a:int) -> int:
    return a

@foo.key
def _(a:int) -> str:
    return "foo"

[elbaro]

simple sounds good, though I'd prefer writing Cache("simple") rather than Cache("simple", INT_MAX).