Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes)

Server
1

Expected behaviour

All directories should get synced.

Actual behaviour

One directory (which has 12000+ files in the same directory) doesn’t get synced.

Steps to reproduce

  1. I execute this command: owncloudcmd -u username -p password --trust --non-interactive /var/www/something/ https://myowncloudserver.domain.com

  2. Then I get this error:

11-29 09:25:17:163 [ info sync.csync.csync ]:   ## Starting remote discovery ##
11-29 09:25:17:163 [ info sync.accessmanager ]: 6 "PROPFIND" "https://cloud.mydomain.com/remote.php/dav/files/filesync/" has X-Request-ID "eecbc783-e450-4204-8510-117ced4e574a"
11-29 09:25:17:164 [ info sync.networkjob ]:    OCC::LsColJob created for "https://cloud.mydomain.com" + "" "OCC::DiscoverySingleDirectoryJob"
11-29 09:25:17:239 [ info sync.networkjob.lscol ]:      LSCOL of QUrl("https://cloud.mydomain.com/remote.php/dav/files/filesync/") FINISHED WITH STATUS "OK"
11-29 09:25:17:240 [ info sync.csync.updater ]: Checking for rename based on fileid 00227869ocpw2kdzfkgx
11-29 09:25:17:241 [ info sync.csync.updater ]: file: images, instruction: INSTRUCTION_NEW <<=
11-29 09:25:17:241 [ info sync.accessmanager ]: 6 "PROPFIND" "https://cloud.mydomain.com/remote.php/dav/files/filesync/images" has X-Request-ID "23660c82-4d0a-4e05-a319-5325dcab11a9"
11-29 09:25:17:241 [ info sync.networkjob ]:    OCC::LsColJob created for "https://cloud.mydomain.com" + "/images" "OCC::DiscoverySingleDirectoryJob"
11-29 09:30:17:321 [ warning sync.networkjob ]: Network job timeout QUrl("https://cloud.mydomain.com/remote.php/dav/files/filesync/images")
11-29 09:30:17:321 [ warning sync.networkjob ]: QNetworkReply::NetworkError(OperationCanceledError) "Connection timed out" QVariant(Invalid)
11-29 09:30:17:321 [ info sync.networkjob.lscol ]:      LSCOL of QUrl("https://cloud.mydomain.com/remote.php/dav/files/filesync/images") FINISHED WITH STATUS "OperationCanceledError Connection timed out"
11-29 09:30:17:321 [ warning sync.discovery ]:  LSCOL job error "Operation canceled" 0 QNetworkReply::NetworkError(OperationCanceledError)
11-29 09:30:17:321 [ warning sync.csync.updater ]:      opendir failed for images - errno 5
11-29 09:30:17:321 [ warning sync.engine ]:     ERROR during  csync_update :  "An error occurred while opening a folder images: Operation canceled"
11-29 09:30:17:326 [ info sync.database ]:      Closing DB "/var/www/path/uploads/._sync_e3c4d804bfac.db"
11-29 09:30:17:327 [ info sync.engine ]:        CSync run took  302200 ms
11-29 09:30:17:329 [ info sync.database ]:      Closing DB "/var/www/path/uploads/._sync_e3c4d804bfac.db"

Server configuration

Operating system:

Distributor ID: Debian
Description: Debian GNU/Linux 9.4 (stretch)
Release: 9.4
Codename: stretch

Web server:

Server version: Apache/2.4.25 (Debian)
Server built: 2018-03-31T08:47:16

Database:

mysql Ver 15.1 Distrib 10.1.26-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

PHP version:

PHP 7.0.27-0+deb9u1 (cli) (built: Jan 5 2018 13:51:52) ( NTS )
Copyright © 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright © 1998-2017 Zend Technologies
with Zend OPcache v7.0.27-0+deb9u1, Copyright © 1999-2017, by Zend Technologies

ownCloud version:

10.0.8

Client configuration

Client version:

owncloudcmd -v
ownCloud version 2.5.1 (build 10450)

Operating system:

Debian GNU/Linux 9 (stretch)

OS language:

English

Qt version used by client package (Linux only, see also Settings dialog):

Using Qt 5.10.1, built against Qt 5.10.1

Client package (From ownCloud or distro) (Linux only):

owncloud-client

Installation path of client:

/usr/bin/owncloudcmd

Logs

Server log:

{"reqId":"f1881c1c-c343-4d2d-bf36-7252a72a2167","level":3,"time":"2018-11-29T02:13:31+00:00","remoteAddr":"client_ip","user":"my_username","app":"PHP","method":"PROPFIND","url":"\/remote.php\/dav\/files\/filesync\/images","message":"Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) at \/var\/www\/owncloud\/lib\/composer\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/PDOStatement.php#142"}

We have increased the memory on our server from 2 GB to 12 GB and set the memory_limit to -1 which is unlimited in the php.ini file. Rebooted the server, tried again and we still get the same message. There is almost 11 GB of unused memory on the server at the moment.

My best guess: somewhere in the PHP code the application reads the whole directory at once instead of part by part? I might be wrong on this. Don’t know how exactly this works in PHP but in C# you can use the type IEnumerable for this (instead of a generic List).

2

Hey,

maybe this isn’t a valid value, you havn’t found/edited the right php.ini file or the web server wasn’t restarted in between?

3

Hey,

it seems you also have reported this in the bug report below. I don’t think that it is a good way to post this at two places in parallel, especially as this looks to me a configuration issue rather then a bug in ownCloud.

1 Like
4

I typed php --ini to check which php.ini file is being used by PHP. So it is the correct one. And there is only one version of PHP installed on this server. Also the value -1 is valid for memory_limit.

memory_limit [integer]

This sets the maximum amount of memory in bytes that a script is allowed to allocate. This helps prevent poorly written scripts for eating up all available memory on a server. Note that to have no memory limit, set this directive to -1 .

Source: http://php.net/manual/en/ini.core.php#ini.sect.resource-limits

5

Isn’t the ini path for the commandline php different from that of the php on apache?

6

I also checked using the following command:

php -r "phpinfo();" | grep php.ini

and I got the same php.ini file:

Configuration File (php.ini) Path => /etc/php/7.0/cli
Loaded Configuration File => /etc/php/7.0/cli/php.ini
7

Hey,

i still think you’re looking at the wrong php.ini / place. You’re still working on the command line (even with the phpinfo which is AFAIK using a completely different php.ini.

I think it would be also great if you could close the github issue to help the developers keeping the bugtracker clean from issues related to configuration problems.

1 Like
8

Running from the commandline:

$ php -r ‘phpinfo();’ | grep php.ini
Configuration File (php.ini) Path => /etc/php5/cli
Loaded Configuration File => /etc/php5/cli/php.ini

Invoked by browser reveals this:

Configuration File (php.ini) Path /etc/php5/apache2
Loaded Configuration File /etc/php5/apache2/php.ini

1 Like
9

I have tried to close it but after I click on “Close” nothing happens. Don’t know if a moderator has to approve it or something. I also tried to “Close and comment” also didn’t work.

Can’t even comment there anymore. Might be an github temporary issue?

10

@tom42 and @alfredb you were both right. I was looking at the wrong php.ini file. I put the phpinfo() method into index.php and run it in the browser and it outputted the following: /etc/php/7.0/apache2/php.ini and when I checked that php.ini file I saw this: memory_limit = 128M

This issue can be closed. Thank you very much for your help!

1 Like
11

Even after I increased it in the right php.ini I still get the same error. I also have restarted apache. What else could it be?