Operational cancelled when doing a large sync

Server
1

I’ve migrated from a bare-metal installation with Apache, ownCloud and MySQL to a Kubernetes deployment, based on the docker-compose.yml. The web interface seems fine but when doing a sync which includes a large number of files, the client eventually fails with Server replied with an error while reading directory '/path/goes/here' : Operation canceled

Steps to reproduce

  1. Install fresh ownCloud installation in Docker/Kubernetes, based on
  2. Add existing user data
  3. Run occ files:scan to rebuild the database

Expected behaviour

Sync should occur normally

Actual behaviour

Small syncs (approx 100GB of data) complete OK. Large syncs (approx 900GB of data) fail during Checking for changes in remote with error Server replied with an error while reading directory '/path/goes/here' : Operation canceled.

Database health appears fine, never runs out of available connections or reports an error

Server configuration

Operating system: Fedora 31 running ownCloud Docker image 10.3.2

Web server:

[jonathan@zeus ~]$ kubectl exec -it owncloud-68c4cbb6b6-b9484 -- apache2 -v
Server version: Apache/2.4.41 (Ubuntu)
Server built:   2019-08-14T14:36:32

Database:

[jonathan@zeus ~]$ kubectl exec -it mysql-869b967588-tctpq -- mysql -V
mysql  Ver 15.1 Distrib 10.4.8-MariaDB, for Linux (x86_64) using readline 5.1

PHP version:

[jonathan@zeus ~]$ kubectl exec -it owncloud-68c4cbb6b6-b9484 -- php -v
PHP 7.3.11-0ubuntu0.19.10.1 (cli) (built: Oct 24 2019 11:38:49) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.3.11, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.3.11-0ubuntu0.19.10.1, Copyright (c) 1999-2018, by Zend Technologies

ownCloud version: (see ownCloud admin page)

10.3.2

Updated from an older ownCloud or fresh install:

Fresh install with user data imported

Where did you install ownCloud from:

Docker Hub

https://hub.docker.com/layers/owncloud/server/10.3/images/sha256-b3517953e3b6a463b9b51615395711fc10ee52da84d0607f33e10989d68a41c6

Signing status (ownCloud 9.0 and above):

No errors have been found.

The content of config/config.php:

List of activated apps:

[jonathan@zeus ~]$ kubectl exec -it owncloud-68c4cbb6b6-b9484 -- occ app:list
Enabled:
  - comments: 0.3.0
  - configreport: 0.2.0
  - dav: 0.5.0
  - federatedfilesharing: 0.5.0
  - federation: 0.1.0
  - files: 1.5.2
  - files_external: 0.7.1
  - files_mediaviewer: 1.0.1
  - files_sharing: 0.12.0
  - files_trashbin: 0.9.1
  - files_versions: 1.3.0
  - firstrunwizard: 1.2.0
  - market: 0.5.0
  - notifications: 0.5.0
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - updatenotification: 0.2.1
Disabled:
  - encryption
  - external
  - user_external
  - user_ldap

Are you using external storage, if yes which one:

Kubernetes persistent volume, mounted as local storage

[jonathan@zeus ~]$ kubectl describe pvc files
Name:          files
Namespace:     owncloud
StorageClass:  microk8s-hostpath
Status:        Bound
Volume:        pvc-f7f21d80-2e7c-4539-8496-d211b92342a6
Labels:        service=files
Annotations:   control-plane.alpha.kubernetes.io/leader:
                 {"holderIdentity":"626cfbb7-4447-11ea-906b-eebe1ec7b0d9","leaseDurationSeconds":15,"acquireTime":"2020-01-31T17:28:01Z","renewTime":"2020-...
               kubectl.kubernetes.io/last-applied-configuration:
                 {"apiVersion":"v1","kind":"PersistentVolumeClaim","metadata":{"annotations":{},"labels":{"service":"files"},"name":"files","namespace":"ow...
               pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: microk8s.io/hostpath
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      2560Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Mounted By:    owncloud-68c4cbb6b6-b9484
Events:        <none>

Are you using encryption: yes/no

No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/…

No

Client configuration

Browser: ownCloud desktop client on Linux v2.6.0

Operating system:

Logs

Web server error log

[jonathan@zeus ~]$ kubectl logs owncloud-68c4cbb6b6-b9484 
Creating volume folders...
Creating hook folders...
Waiting for MySQL...
services are ready!
Waiting for Redis...
services are ready!
Removing custom folder...
Linking custom folder...
Removing config folder...
Linking config folder...
Writing config file...
Fixing base perms...
Fixing data perms...
Fixing hook perms...
Upgrading server database...
ownCloud is already latest version
ownCloud is already latest version
Writing objectstore config...
Writing php config...
Updating htaccess config...
.htaccess has been updated
Writing apache config...
Enabling cron background...
Set mode for background jobs to 'cron'
Touching cron configs...
Starting cron daemon...
Starting apache daemon...
[Mon Feb 03 11:46:12.298522 2020] [mpm_prefork:notice] [pid 192] AH00163: Apache/2.4.41 (Ubuntu) configured -- resuming normal operations
[Mon Feb 03 11:46:12.298572 2020] [core:notice] [pid 192] AH00094: Command line: '/usr/sbin/apache2 -f /etc/apache2/apache2.conf -D FOREGROUND'

ownCloud log (data/owncloud.log)

{"reqId":"723ff548293e4d23c6ed3d16c110c24f","level":3,"time":"2020-02-03T11:46:16+00:00","remoteAddr":"10.1.40.1","user":"jonathan","app":"PHP","method":"GET","url":"\/remote.php\/dav\/files\/jonathan\/Local\/Arthur%20phonecall%202018-04-20.mp3?c=163ccf703c69e922acc31476e4a6e551&x=32&y=32&forceIcon=0&preview=1","message":"imagecreatefromstring(): Empty string or invalid image at \/var\/www\/owncloud\/lib\/private\/legacy\/image.php#573"}

Browser log

 03/02/2020 12:12:05,                                                                ,                      ownCloud,Server replied with an error while reading directory 'Documents/School/Computing/Oct Halfterm 2003' : Operation canceled