The workaround was to enable the Share Files app. The issue is still open here: Images and Videos not showing in Media Viewer when Share Files app is disabled · Issue #240 · owncloud/files_mediaviewer · GitHub with no updates since 2020.
I haven’t managed owncloud for a few years now, though, so I don’t have recent context.