Deduplicate files referenced by content #4

Open
opened 2026-09-02 00:33:21 +00:00 by omicron · 0 comments
Owner

Currently when two different articles reference the same image.png or other file, both these articles get a copy of that file in their output directory. This is wasteful. We want to dedup this eventually.

There's a few ways this could work:

  • If an article references an already existing file we instead add a symlink to it
  • all files get put in some directory like /files/sha2hash.extension, all articles then symlink to that file with the desired filename.

The first option probably makes sense? I'd need to investigate how this influences caching too.

Currently when two different articles reference the same image.png or other file, both these articles get a copy of that file in their output directory. This is wasteful. We want to dedup this eventually. There's a few ways this could work: - If an article references an already existing file we instead add a symlink to it - all files get put in some directory like `/files/sha2hash.extension`, all articles then symlink to that file with the desired filename. The first option probably makes sense? I'd need to investigate how this influences caching too.
omicron added the Kind/Feature
Priority
Medium
3
labels 2026-09-02 00:33:45 +00:00
omicron added this to the medium term milestone 2026-09-02 00:39:11 +00:00
omicron removed the
Priority
Medium
3
label 2026-09-02 00:39:30 +00:00
Sign in to join this conversation.