We have multiple oCIS 6.x.x servers with S3 as their backend being used both for cloud storage across the Internet as well as NAS-like servers on LANs.
Performance on the LANs has been mostly acceptable when it’s just a few workstations, but as we scale to larger workgroups and render farms with 100s of nodes, pulling everything on-demand from S3 becomes a huge bottleneck.
Our near-term solution to this has been to set up a local cache of sorts via an SMB server (running on Ubuntu 24) on the LAN which synchronizes everything to/from oCIS via ownCloud Desktop. This works well enough to function, but has the following issues:
- Every single file has to be synchronized because ownCloud Desktop on Linux doesn’t support VFS. This becomes a huge bottleneck when there are large files, directory structures with hundreds of thousands of files, etc.
- We need to keep enough disk space available for the “cache” to hold the entire oCIS server, which can be hundreds of terabytes of data despite maybe only 10TB being in active use.
- Files which take time to write to the cache try and upload to oCIS in partial form, so they have to upload again and again as changes are detected by ownCloud Desktop rather than waiting until the files are 100% complete.
Ideally we’d love:
- A way to mount oCIS directly as SMB or similar without having to go through ownCloud Desktop
- A way to rsync, scp or sftp files directly to/from the oCIS server
How are people handling this sort of thing in your deployments? Particularly for enterprise I can’t imagine everyone is just relying on VFS in Windows through ownCloud Desktop for all file access.