Sync big files into google drive external storage
ownCloud saying the big file (3Gb of size) are in the blacklist.
Steps to reproduce
- Fresh install of ownCloud Server
- Update php.ini props to big files
- Integrate with google drive
- Try upload some big file
Operating system: Ubuntu 18.04
Web server: Apache2
Database: mysql 8
PHP version: 7.4
ownCloud version: 10.6
Storage backend (external storage):Google drive
Operating system: Win 10 x64
OS language: windows
- Client logfile, Web server error log, Server logfile: https://gist.github.com/reberthkss/3619b4a37ea8ac67e9b7c40118d2c8fc
Could you please paste your php.ini ?
And could you provide the message you are getting when trying to upload the big file? (screenshot)
I’ve updated the gist with php.ini data!
Follow print below:
ps.: The file is successful uploaded to google drive account but ownCloud side is not recognizing.
Opened https://github.com/owncloud/core/issues/38328 . For now, I guess the workaround is to increase the allowed php memory to fit the whole file.
Side note: code follows the example provided by the google documentation. Note that ownCloud requires a data stream. Chunked / ranged downloads (which seems to be the natural alternative according to the google documentation) doesn’t fit here and will need to be adapted, which is quite complex to do.
Thank you to reply. I’ll try increase php memory limit.
The unique parameter to change is memory_limit? I turn from 256MB to 4G
Increase memory_limit to 4G doesn’t solve the problem:
Some things to take into account:
- Make sure you have plently of memory available. Other processes running in the server could also eat memory, so the php process might hit a hard limit before reaching the established 4GB memory for php.
- Since this is a server problem, it’s way more interesting the server logs. We know the client will keep malfunction until the problem is solved in the server.
- Double-check the memory limit is correctly applied. You’ll likely need to restart the web server. The easiest way to check is to download the file and check if there is an error log in apache; you’ll see the
Allowed memory size of 536870912 bytes exhausted (tried to allocate 3410408792 bytes) error in the apache logs. There you can see the limit of ~512MB for php
Did you restart your web server after making the change?
[Fri Jan 22 16:07:12.014813 2021] [php7:error] [pid 2190] [client 126.96.36.199:56859] PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 3410408792 bytes) in /var/www/html/cloud/apps/files_external/3rdparty/guzzlehttp/streams/src/Stream.php on line 135
It seems the php value memory_limit doesn’t is the reference for this process
Even I am facing the same issue with google drive transfers
there is a memory_limit directive in the .htaccess file inside the ownCloud’s directory. You might need to change that too.
Successfully uploaded big file! Thank you man!
What about output_buffering?
Output Buffering must be turned off in
php.ini , or PHP will return memory-related errors:
Would it make a difference if they turned that off?
Doubt. The problem is in the google library, that doesn’t stream the contents