Hello,
Thanks for your time.
tl;dr
keep it on the “old” structure
Long version:
For my situation, at least in the short term, it’s a matter of staying flexible and not “vendor locked” (that’s not quite the right term but I’m sure you get what I mean).
While my use of OCIS could certainly expand overtime, my current goal is to be able to access a large dataset (~40TiB), that is already organized and hosted on a TrueNAS SCALE system, remotely via the desktop client’s VFS feature. Locally that data is served over SMB/NFS, which is especially useful for less capable clients that won’t be able to run an ownCloud client/app (I’m not sure if OCIS supports sharing its data via WebDAV/SMB/NFS yet).
With that much data, transferring it all to decomposed FS so that it’s entirely managed by OCIS would obviously be a bit of an undertaking and a major commitment, as I’d have to go through that arduous process again to reverse things back to how I have them now. So essentially, while I get that a core point of ownCloud is to be a one-stop-shop for all of your data hosting needs, for the time being I’d prefer the underlying data storage to remain “in my control” and only use OCIS for this specific use case. This way if I decided to go in a different direction, or wanted to continue serving the data via SCALEs shares and only use OCIS for remote access I could do that without having to move/restructure massive amounts of data. So “keeping the data on the old structure” is correct, which I believe is how the ownCloud 10 External Storage app works now, granted I’ve never used it before. I know that sourcing data via SMB or local storage can get a bit messy when it comes to keeping ownCloud in sync, but my data is largely read from and written to rarely, so as long as there’s a periodic or manual scan option to keep things updated when I make external changes that’s fine. Notifications from the underlying filesystem/SMB server would of course be preferred, but I know that can be tricky to get working right.
Sure I could test out OCIS with a smaller subset of my data, but then I feel like its not a full benchmark of its scalability, and it would still be a big leap to move all of my data over.
In the long term, I could see being satisfied with OCIS enough that I decide I want to migrate all of my data to its “native” storage system, but that’s something that would need to be determined over a longer period of time than in a more typical use case where one is just using ownCloud for storing personal documents/photos (i.e. much less data).
So to summarize, external storage would allow for an ideal amount of granularity. I can get started with this particular use case and potentially only ever use OCIS for this purpose, leaving access to the data via other methods (i.e. SMB) up to SCALE’s built-in samba server or whatever else I may choose. Or, I could use OCIS for all of my data sharing, or even move other datasets I have over to it, but with the point being at no point during any of this do I have to make massive reversals if I decide I want to move in a different direction or my use case changes, as in the end my data will simply remain as plain data on a standard ZFS dataset that works with anything that can read ZFS and isn’t tied to a particular application (other than any I do decided to migrate to decomposed FS in the long run).
Essentially, I’m trying to keep my data store as abstracted from any particular product as much as possible, at least for now.
oCIS supports this by the integration of so called storage providers, such as the decomposed FS or S3ng is today. We have more of these on the roadmap that will fulfill the requirements above.
This is what I was trying to figure out in my OP. I was under the impression that the choice of storage provider was ultimately a matter of the underlying technology, with the data still ultimately being fully managed by OCIS, without the intention of it being manipulated externally.
I interpreted it to be similar to how with ownCloud 10 I can choose what database system to use (e.g. MySQL, MariaDB, postgresql, etc, but ownCloud of course still manages the database with it’s own entries. I could use decomposed FS for local storage, or S3 to host my data in the cloud instead, but with the idea being that I’m manipulating all of it through OCIS only. So it’s more of a “choose your own storage provider” situation and not a “bring your own data” situation, which I think is a good way of putting it. I want to bring my own data so to speak.
Please do correct me if I’m wrong in my understanding of this and if there is a way with current or future storage providers to give OCIS access to my pre-existing data that I can also access in other ways external to ownCloud without first migrating.
