Quotas

Blueprint for Quota management in Kinto

For some use case you may want to limit the size of the data that can be added to a Kinto collection.

The first use case we have is to limit the data that a user can add to its bucket.

In the first version of the quota management feature we want to be able to limit three things:

• QUOTA_BYTES: The maximum total amount (in bytes) of data that can be stored in sync storage, as measured by the JSON stringification of every value plus every key's length. (At a bucket level)

• QUOTA_BYTES_PER_ITEM: The maximum size (in bytes) of each individual item in sync storage, as measured by the JSON stringification of its value plus its key length. (At a bucket level)

• MAX_ITEMS: The maximum number of items that can be stored in sync storage. (At a bucket level)

The implementation for this is inspired by chrome.storage sync quotas management.

In order to do that we need two things:

• A way to keep the size of data already there
• A way to reject a change if the resulting action exceed the quota.

Storing the data size

• A way to store the size taken by a bucket
• A way to get the number of records in the bucket

Each time we do an action on a collection, we want to update the size counters:

• On create, sum the bytes,
• On delete subtracts,
• On update sum the difference between old/new. (note: postgresql internal JSONB is going to be smaller but what counts is API input/output)

Rejecting Quota exceeding

For this part we want to add a plugin that can be activated to activate Quotas on a bucket.

In case the request is rejected because of a Quota Exceeded error message, the server will raise a 507 Insufficient Storage http error.

With a description of which quota was exceeded (MAX_ITEM_EXCEEDED, QUOTA_BYTES_EXCEEDED or QUOTA_BYTES_PER_ITEM_EXCEEDED)