External storage file size limit?

Server
1

There seems to be a file size limit when uploading files to external object storage. When uploading a ~600MB file, locally it works but when doing it to the bucket, it does not. Smaller <128Mb files work in both locations.

Fresh install and VM
Debian 9
Apache 2
MariaDB
PHP 7.3
ownCloud 10.3.0 apt package
Clean status

No Errors Have Been Found.

{
“system”: {
“updatechecker”: false,
“instanceid”: “oc7etwc35oip”,
“passwordsalt”: “REMOVED SENSITIVE VALUE”,
“secret”: “REMOVED SENSITIVE VALUE”,
“trusted_domains”: [
“34.73.216.214”
],
“datadirectory”: “/var/www/owncloud/data”,
“overwrite.cli.url”: “https://34.73.216.214/owncloud”,
“dbtype”: “mysql”,
“version”: “10.3.0.4”,
“dbname”: “owncloud”,
“dbhost”: “localhost”,
“dbtableprefix”: “oc_”,
“dbuser”: “REMOVED SENSITIVE VALUE”,
“dbpassword”: “REMOVED SENSITIVE VALUE”,
“logtimezone”: “UTC”,
“apps_paths”: [
{
“path”: “/var/www/owncloud/apps”,
“url”: “/apps”,
“writable”: false
},
{
“path”: “/var/www/owncloud/apps-external”,
“url”: “/apps-external”,
“writable”: true
}
],
“installed”: true
}
}

Enabled:

  • comments: 0.3.0
  • configreport: 0.2.0
  • dav: 0.5.0
  • encryption: 1.4.0
  • federatedfilesharing: 0.5.0
  • federation: 0.1.0
  • files: 1.5.2
  • files_external: 0.7.1
  • files_external_s3: 1.0.0
  • files_mediaviewer: 1.0.0
  • files_sharing: 0.12.0
  • files_trashbin: 0.9.1
  • files_versions: 1.3.0
  • firstrunwizard: 1.2.0
  • market: 0.5.0
  • notifications: 0.5.0
  • provisioning_api: 0.5.0
  • systemtags: 0.3.0
  • updatenotification: 0.2.1
    Disabled:
  • external
  • user_external

Are you using external storage, if yes which one: local/smb/sftp/…
Yes Google Bucket/S3

Are you using encryption: yes/no
Yes

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/…
No

Client configuration

Chrome
Windows 10

ownCloud log (data/owncloud.log)

https://pastebin.kotter.net/view/0bac5e5d

2

Try to upload the file directly to the S3 storage. There could be limits in the S3 storage.

There are also some limits applied in php for uploads, but it’s usually around 512MB by default. You can check around the php.ini files being used by the web server. This limit should also apply to uploads to the primary storage, so if you can upload 600MB I don’t think this is the problem.

2 Likes
3

Direct upload is at least over my tested 5GB

4

I can’t say for sure, but I think the aws library we’re using uses multipart uploads for big files. This is fully handled by the aws library in ownCloud’s case.
According to https://www.zenko.io/blog/four-differences-google-amazon-s3-api/ the multipart process seems to be different between google and amazon. It’s possible that the library doesn’t support multipart uploads to google.

You can try to upload the file using awscli or some other official tool using the aws library.

2 Likes
5

This is what I found after a quick internet search:

@steven_meyer123

1 Like