Fix server-side cache to work with conditional GET by skaphan · Pull Request #608 · artoonie/rcvis

skaphan · 2026-03-06T13:43:02Z

Summary

Removes max_age=0 from patch_cache_control in ConditionalGetMixin -- this was preventing UpdateCacheMiddleware from storing responses server-side, since it checks get_max_age(response) and skips caching when the result is 0.
Replaces cache.clear() with per-URL cache purging in cloudflare.py -- uses get_cache_key() with synthetic HttpRequest objects to surgically invalidate only the affected visualization URLs from the file-based cache.
Removes DISABLE_CACHE toggle from settings -- the file-based cache is now always enabled and works correctly with conditional GET.
Removes vary_on_headers(increment) -- this was a no-op since no client ever sends that header.

How it works

ConditionalGetMixin sets Cache-Control: no-cache (browsers revalidate on every request)
UpdateCacheMiddleware stores the rendered response in Django file-based cache (now works because max_age is not 0)
FetchFromCacheMiddleware serves cached responses on subsequent requests
ConditionalGetMiddleware compares Last-Modified vs If-Modified-Since and returns 304 when content has not changed
On model update, CloudflareAPI.purge_vis_cache() clears both Cloudflare CDN and the specific Django cache entries

Builds on #594 (proper cache control).

Add updated_at field to JsonConfig model, use ConditionalGetMixin for all visualization views, and short-circuit 304 responses in VisualizeEmbedded before expensive computation.

Add proper cache control for embedded visualizations

- Add pylint disable for too-few-public-methods (it is a mixin) - Add docstring to get() method - Rename last_modified/if_modified_since to camelCase Co-Authored-By: Claude Opus 4.6

Co-Authored-By: Claude Opus 4.6

- Remove max_age=0 from ConditionalGetMixin so UpdateCacheMiddleware can store rendered responses in the file cache. Browsers still revalidate via Cache-Control: no-cache. - Replace cache.clear() sledgehammer in cloudflare.py with per-URL cache purging using get_cache_key — only the updated slug's pages are evicted from Django's file cache. - Remove vary_on_headers('increment') — was a no-op (no client sends the header). - Remove DISABLE_CACHE env toggle — no longer needed. - ConditionalGetMiddleware (already in middleware stack) handles 304 responses for cached pages using the Last-Modified header. Co-Authored-By: Claude Opus 4.6

artoonie

Thank you! This looks pretty close to ready. I recommend reverting the changes to purging django cache and DISABLE_CACHE for now, as that would be easier than fixing them.

artoonie · 2026-03-06T16:03:20Z

common/cloudflare.py

+                request.path = path
+                request.META['QUERY_STRING'] = ''
+            request.META['SERVER_NAME'] = 'localhost'
+            request.META['SERVER_PORT'] = '80'


This will work in development but not in production -- we are missing HTTP_HOST (which could be rcvis.com or www.rcvis.com), url scheme (https), and maybe others.

I think the only way to get this to work is to store a local map of slugs to cache IDs and to use that to clear the cache -- manually building the cache key is brittle.

artoonie · 2026-03-06T16:04:37Z

rcvis/settings.py


 AWS_DEFAULT_ACL = None

-if os.environ.get('DISABLE_CACHE') != 'True':


If removing this, we should also remove it from infra/.env.template.

However -- during development, it's still useful to be able to disable cache. Not all cache invalidations come from PATCHes. If you update an HTML or JS file locally, you want to be able to refresh the page without restarting the server. It doesn't always work (because JS files are minified and templates are compiled), but there are cases when it does.

artoonie · 2026-03-06T16:05:31Z

scripts/reset-db.sh

+    print('API user skaphan already exists')
+"
+
+echo "Database reset complete."


I don't feel comfortable committing this to the RCVis mainline -- a database deletion script feels like a high-risk script to leave in production.

skaphan · 2026-03-06T18:11:34Z

the commit o the database script was an accident. sorry! will address the other issues shortly.

…

On Mar 6, 2026, at 11:11 AM, Armin Samii ***@***.***> wrote: @artoonie commented on this pull request. Thank you! This looks pretty close to ready. I recommend reverting the changes to purging django cache and DISABLE_CACHE for now, as that would be easier than fixing them. In common/cloudflare.py <#608 (comment)>: > @@ -51,13 +53,28 @@ def purge_vis_cache(cls, slug): ] cls.purge_paths_cache(paths) + @classmethod + def _purge_django_cache(cls, paths: list[str]) -> None: + """ Purge matching entries from Django's file-based cache. """ + for path in paths: + request = HttpRequest() + request.method = 'GET' + # Split path?query into path and query string + if '?' in path: + request.path, request.META['QUERY_STRING'] = path.split('?', 1) + else: + request.path = path + request.META['QUERY_STRING'] = '' + request.META['SERVER_NAME'] = 'localhost' + request.META['SERVER_PORT'] = '80' This will work in development but not in production -- we are missing HTTP_HOST (which could be rcvis.com or www.rcvis.com <http://www.rcvis.com/>), url scheme (https), and maybe others. I think the only way to get this to work is to store a local map of slugs to cache IDs and to use that to clear the cache -- manually building the cache key is brittle. In rcvis/settings.py <#608 (comment)>: > @@ -281,20 +282,12 @@ AWS_DEFAULT_ACL = None -if os.environ.get('DISABLE_CACHE') != 'True': If removing this, we should also remove it from infra/.env.template. However -- during development, it's still useful to be able to disable cache. Not all cache invalidations come from PATCHes. If you update an HTML or JS file locally, you want to be able to refresh the page without restarting the server. It doesn't always work (because JS files are minified and templates are compiled), but there are cases when it does. In scripts/reset-db.sh <#608 (comment)>: > +python manage.py migrate + +# Create API-enabled admin user (matches docker-entrypoint.sh) +python manage.py shell -c " +from django.contrib.auth import get_user_model +User = get_user_model() +if not User.objects.filter(username='skaphan').exists(): + user = User.objects.create_superuser('skaphan', ***@***.***', 'rcvisacc0unt') + user.userprofile.canUseApi = True + user.userprofile.save() + print('Created API user skaphan with API access') +else: + print('API user skaphan already exists') +" + +echo "Database reset complete." I don't feel comfortable committing this to the RCVis mainline -- a database deletion script feels like a high-risk script to leave in production. — Reply to this email directly, view it on GitHub <#608 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABCNY7UZFXSRWSJPRZWAOL4PL2BRAVCNFSM6AAAAACWJQYVR2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTSMBUGU4TCMZRGE>. You are receiving this because you authored the thread.

- _purge_django_cache now uses Site.objects.get_current().domain to construct requests with the correct HTTP_HOST and HTTPS scheme, matching how UpdateCacheMiddleware stores cache keys in production. Tries both primary domain and www. variant. - Extracted _make_cache_request helper for constructing synthetic requests. - Restored DISABLE_CACHE env var toggle for development convenience. - Removed reset-db.sh (not appropriate for mainline). - Added three tests for django cache purging: basic path, query string, and www. variant coverage. Co-Authored-By: Claude Opus 4.6

skaphan · 2026-03-06T18:51:05Z

Thanks for the review! I've pushed changes addressing all three points:

Cache purging: You're right that the synthetic request was missing production-level headers. I've fixed _purge_django_cache to use Site.objects.get_current().domain for the HTTP_HOST (the same source get_absolute_paths_for already uses for Cloudflare purges) and set wsgi.url_scheme to https. It also tries both the primary domain and the www. variant, mirroring the Cloudflare purge logic. I extracted a _make_cache_request helper to keep it testable and added three tests covering basic paths, query strings, and the www variant.

Without per-URL purging, the cache would serve stale responses after a PATCH — FetchFromCacheMiddleware returns the old cached page before ConditionalGetMixin ever runs, so the Last-Modified check never happens. If you'd prefer to revert to cache.clear() as a simpler fallback, I'm happy to do that instead.

DISABLE_CACHE: Restored.

reset-db.sh: Removed.

skaphan and others added 5 commits February 24, 2026 13:29

Add proper cache control for embedded visualizations

94fac08

Add updated_at field to JsonConfig model, use ConditionalGetMixin for all visualization views, and short-circuit 304 responses in VisualizeEmbedded before expensive computation.

Merge pull request artoonie#594 from skaphan/proper-cache-control

3e4a1d8

Add proper cache control for embedded visualizations

Fix linter issues in ConditionalGetMixin

032e992

- Add pylint disable for too-few-public-methods (it is a mixin) - Add docstring to get() method - Rename last_modified/if_modified_since to camelCase Co-Authored-By: Claude Opus 4.6

Add reset-db.sh for easy database reset after branch switching

70693e6

Co-Authored-By: Claude Opus 4.6

skaphan mentioned this pull request Mar 6, 2026

Remove Django server-side response cache #605

Closed

artoonie reviewed Mar 6, 2026

View reviewed changes

skaphan force-pushed the fix-server-cache branch from f69968a to fd01651 Compare March 6, 2026 18:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix server-side cache to work with conditional GET#608

Fix server-side cache to work with conditional GET#608
skaphan wants to merge 6 commits intoartoonie:mainfrom
skaphan:fix-server-cache

skaphan commented Mar 6, 2026

Uh oh!

artoonie left a comment

Uh oh!

artoonie Mar 6, 2026

Uh oh!

artoonie Mar 6, 2026

Uh oh!

artoonie Mar 6, 2026

Uh oh!

skaphan commented Mar 6, 2026 via email

Uh oh!

skaphan commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		AWS_DEFAULT_ACL = None

		if os.environ.get('DISABLE_CACHE') != 'True':

Conversation

skaphan commented Mar 6, 2026

Summary

How it works

Uh oh!

artoonie left a comment

Choose a reason for hiding this comment

Uh oh!

artoonie Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

artoonie Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

artoonie Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

skaphan commented Mar 6, 2026 via email

Uh oh!

skaphan commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants