Cache Clear: Overuse

Incident Report for NexusVC

Postmortem

[UPDATE] Added a cache management feature on Ally for authorized users.

Updated the server caching method.

At this time there will now be segmented company caches which have been identified in the most crucial application components using tagging.

When /cache/clear is used this will only flush the Dialer cache when agents have gotten stuck or disconnected from the websocket connection.

Features now available for cache management include:
cache:clear --tags=dialer
cache:clear --tags={user_id}
cache:clear --tags={company_code}
cache:clear --tags={company_id}
cache:clear --tags=products
cache:clear --tags={state}

or combination of tags such as:
cache:clear --tags={company_code},{company_id}

Posted Jul 11, 2023 - 05:13 EDT

Resolved

Update to the server caching method. At this time there will now be segmented company caches. When /cache/clear is used this will only flush the Dialer cache when agents have gotten stuck or disconnected from the websocket connection.

Features now available for cache management include:

cache:clear --tags=dialer
cache:clear --tags={user_id}
cache:clear --tags={company_code}
cache:clear --tags={company_id}
cache:clear --tags=products
cache:clear --tags={state}

or combination of tags such as:

cache:clear --tags={company_code},{company_id}
Posted Jul 11, 2023 - 05:07 EDT

Identified

/cache/clear command is designed for urgent issues. After tracking the amount of hits the request is getting within a specific set of time, it may be causing larger issues by flushing the cache to frequently. Implementing a per company cache option to force flushing of only the agency itself.
Posted Jul 10, 2023 - 15:16 EDT
This incident affected: Ally (Ally).