Hi, I’ve just install Owncloud 9 on an Compact Presario running Ubuntu 16.04 and a client on my mac. I’ve linked my Owncloud with my Qnap trought FTP and I’ve installed the client on my mac.
Thanks
Hi,
unfortunately there is currently no solution for that except that you’re tuning your operating system and your database for best performance. oC specific info about that is available here:
For operating system and database tuning please use your favorite search engine to find common tuning guides.
1 Like
There is currently development in process to speed up the upload of many small files:
These are only the data about PUT and GET operations, I think I will ask our biggest customers about some more detailed characteristics about PUT/GET/DELETE/MOVE oeprations, how often do they do that and so one. I also included some geolocations of users, since bundling makes extreme sense for big latencies.
Some statistics also about operations at CERN, what you can find on http://cs3.ethz.ch/ : [image]
[image]
[image]
CERN: [image]
SWITCH: [image]
How many files do you have?
3 Likes
tflidd
October 6, 2016, 8:26am
4
With some mysql tuning, I was able to increase the upload speed to about 1000 files/minute:
opened 12:22AM - 06 Dec 15 UTC
closed 04:54PM - 06 Apr 17 UTC
enhancement
discussion
performance
Upload of many small files is very slow. I tested on a raspbery pi 2 which is kn… own to have a bad i/o-performance. During uploads, mysql-operation create an important io-wait on the system. It was worse with journaling of ext4, so I disabled it (it's a test system).
Uploading a folder with 1180 files (18 MB) via the oc-client takes about 20-30 minutes (pictures, a lot of text files, take any php code of a project), but the number of sql-queries is extremely high (>50 queries per uploaded file!):
| command | number |
| --- | --- |
| select | 49,681 |
| update | 11,523 |
| insert select | 2,487 |
| set option | 1,526 |
| delete | 511 |
| insert | 423 |
| begin | 17 |
| commit | 17 |
For upload, a smaller number of queries can be obtained when files are uploaded in an external folder via ftp (takes a few seconds) then this folder is included into owncloud, the scanning process takes a few minutes (it's hard to tell, 5 minutes?) and the number of queries (some queries are due to handling the web-interface):
| command | number |
| --- | --- |
| select | 11,054 |
| update | 3,040 |
| insert select | 3,004 |
| set option | 425 |
| delete | 138 |
I remember you worked on the deletion process for OC 8.0, this scales much better: 3000 files take 3500 update and 1400 select queries. Perhaps there is a similar improvement possible for upload operations ;-)
System: RPi2, Debian Jessie
Webserver: nginx/php-fpm
Database: mariadb
Owncloud 8.2.1
Apps enabled:
- activity: 2.1.3
- calendar: 0.8.1
- contacts: 0.5.0.0
- files: 1.2.0
- files_external: 0.3.0
- files_pdfviewer: 0.7
- files_sharing: 0.7.0
- files_texteditor: 2.0
- files_trashbin: 0.7.0
- files_versions: 1.1.0
- files_videoviewer: 0.1.3
- firstrunwizard: 1.1
- galleryplus: 14.3.0
- provisioning_api: 0.3.0
- templateeditor: 0.1
I haven't tested with a pure webdav-client yet.
I am happy to see that there is more ongoing work
It depends on the folders, but some can have up to 6000 files
Thats pretty good, I’ll try that !
