oCIS max file size upload limit

[junglebell]

I am on v7.1.1

Is there a limit to max file size that can be uploaded via web UI? I am trying to upload some folders and not all files are getting uploaded. I believe it is cutting off at around 100MB threshold.

Could this be adjusted via config? Or is there a backend method to upload? I have some VM images that may be upto 30-50GB in size.

Thanks for your help

[kobergj]

No. You should be able to upload them.

Uploading a lot of small files might take a while because of the postprocessing. (There are ways to speed this up though). But eventually all files should be uploaded.

Do you see any error messages?

[junglebell]

Hi @kobergj - I did see some error messages, however it is likely a different issue.

Steps to reproduce:

1. Create a new space with default 1GB quota
2. Add a folder that has several files and total size > 1GB (1.3GB in my case)
3. Not all files will get copied - it reports an “unknown error” - which is misleading (see screenshot)
   
   
4. << This is where I thought files greater than certain size didn’t make it>>
5. However, you realize the default quota is only 1 GB, so you go change the quota.
6. Now drag and drop the same folder again to upload. It asked whether to merge or skip, I chose merge (see screenshot)

8. The system reports success - all files have been uploaded (See screenshot). However the reality is that it only uploaded the same files again, but did not bring the previously left out files. Notice the size reported on the folder is the same as before.

So while files >100MB can be uploaded, I see the following issues:
a. if the space doesn’t have enough free storage, it still proceeds to copy and gives incomprehensible error. Ideally storage check should be done before
b. Re-upload doesn’t truely upload all files. Ideally, it should merge existing files, and upload new

This was tested on v7.1.2. Thanks for your help.

[junglebell]

regarding postprocessing - yes I did notice the system flagging some files as being “processed” for a while. Could you help elaborate what steps could be taken to speed it up?

Also curious what happens in postprocessing, could some be deferred or disabled?

Thanks again for your help

[junglebell]

@kobergj - could you please review post #3 and #4 on this thread. thanks

[kobergj]

Hi @junglebell. Sorry your last comments slipped through There is the envvar POSTPROCESSING_WORKERS which will allow processing multiple files in parallel. See also postprocessing service docu: Postprocessing Service Configuration

There is also OCIS_ASYNC_UPLOADS which can be used to turn off postprocessing but this is not recommended.

For the issue - this seems a to be a bug - could you open a ticket in web? GitHub - owncloud/web: 🐲 Next generation frontend for ownCloud Infinite Scale At least the error message should be updated.