Owncloud 10.3.2 file search performance issue over UI

[mustafaa]

The file search process is very slow via the owncloud UI. There are around 20000 tiff files in a folder where the search process takes more than 20 sec to populate the list. The file search process over webDAV works fine.

Server is running on following specs and configuration:

Hardware specs:
8 CPU 64 GB RAM

Application specs:
ownCloud 10.3.2 (stable)
Server version: Apache/2.4.6 (CentOS)
PHP 7.3.16 (cli) (built: Mar 17 2020 10:18:38) ( NTS
mysql Ver 15.1 Distrib 5.5.64-MariaDB, for Linux (x86_64) using readline 5.1

Owncloud configuration:
memcache.local’ => ‘\OC\Memcache\APCu’,
‘memcache.locking’ => ‘\OC\Memcache\Redis’,
‘filelocking.enabled’ => ‘true’,
‘redis’ =>
array (
‘host’ => ‘localhost’,
‘port’ => 6379,

Following additional troubleshooting has been performed but we’re unable to resolve the performance issue:
Enabled local memory caching
Enabled Redis locking
Mysql tuning
Enabled full text search with elastic search engine.

Please assist as this is causing great impact in the process.

Please advise if any specific information is required for evaluation.

Thanks a lot in advance.

[eneubauer]

For any kind of performance issues you first need to figure out if you have a bottleneck. To figure this out you need some metrics gathering system, I usually recommend collectd.

Once you have that you can figure out whether you can optimize some service configuration or if you have to throw more resources at it. Another option is to separate some of the services out to other machines, but this depends on your results.

Finally, searching through 20000 objects will just take a while if it is done in the browser.
How exactly are you searching over WebDAV?

[mustafaa]

Hi [eneubauer],

Thank you for email and writing.

On further evaluation what I noticed is the search process is very slow only for the Active directory users, the same process works really fast with local authentication.
Are there any specific tunning setting for ldap users you can suggest or tools which would assist in capturing further information to debug?

Webdav is search over davfs drive mounted on the unix system.

Thanks again

[eneubauer]

@mustafaa

On further evaluation what I noticed is the search process is very slow only for the Active directory users, the same process works really fast with local authentication.
Are there any specific tunning setting for ldap users you can suggest or tools which would assist in capturing further information to debug?

For large user directories it is usually recommended to set up a local LDAP slave, as you can see in the following documentation: https://doc.owncloud.org/server/10.4/admin_manual/installation/deployment_recommendations.html#ldap

I’m not sure if this is possible with Microsoft AD, and unfortunately I haven’t had to set this up myself so far, so I can’t really give any further recommendations.

How large is your directory?

Are the files in an external storage that requires LDAP authentication?

@mustafaa

Webdav is search over davfs drive mounted on the unix system.

I’m still not sure if the search is actually happening on the ownCloud system or locally when this is mounted like that…

@mustafaa

Thank you for email and writing.

You can change your email notification settings for your central account in your personal settings:

mustafaa

[mustafaa]

Apologise I should have provided further information.

Owncloud server is configured to authenticate against local read only version of ldap/AD server. The files reside on a high end flash storage on the same unix server as the application. These files shared across all users via the owncloud application.

Here is the simple task users are performing as part of search process:

Users login to owncloud UI as LDAP/Active directory user --> Select the shared folder --> Drill down to specific folder — Perform a document search – Search with more than 10,000 files takes more than 1min to appear and sometime it fails as “Page unresponsive” You can wait for to become responsive or exit the page"

Another scenario

Users login to owncloud UI as local database user --> Select the shared folder --> Drill down to specific folder — Perform a document search – Search with more than 10,000 files takes less than 10 sec to populate.

The requirement is to speed-up the UI search process so at this stage webDAV is an alternative approach we’re using temporarily.

Is there a way manual way to perform a search over api for ldap users to identify the issue?