Local disk space

Steps to reproduce

  1. The server is confifured with external storage
  2. I’m using S3
  3. But the local disk space continues rising

Expected behaviour

Tell us what should happen
The data need be storaged only in S3

Actual behaviour

Tell us what happens instead
The local disk of the server is utilized

Server configuration

Operating system:
Debian 10
Web server:
Apache 2.4
Database:
mysql Ver 15.1 Distrib 10.3.27-MariaDB
PHP version:
7.3.19
ownCloud version: (see ownCloud admin page)
10.6.0
Updated from an older ownCloud or fresh install:
Fresh install
Where did you install ownCloud from:


Signing status (ownCloud 9.0 and above):
Login as admin user into your ownCloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results into https://gist.github.com/ and puth the link here.

The content of config/config.php:
{
“basic”: {
“license key”: “REMOVED SENSITIVE VALUE”,
“date”: “Thu, 07 Jan 2021 01:19:57 +0000”,
“ownCloud version”: “10.6.0.5”,
“ownCloud version string”: “10.6.0”,
“ownCloud edition”: “Community”,
“server OS”: “Linux”,
“server OS version”: “Linux ip-172-31-48-117 4.19.0-13-cloud-amd64 #1 SMP Debian 4.19.160-2 (2020-11-28) x86_64”,
“server SAPI”: “apache2handler”,
“webserver version”: “Apache/2.4.38 (Debian)”,
“hostname”: “drive.digivox.com.br”,
“logged-in user”: “admin”
},
“stats”: {
“users”: {
“Database”: {
“count”: 33,
“seen”: 8,
“logged in (30 days)”: 8
}
},
“groups”: {
“OC\Group\Database”: 11
}
},
“config”: {
“instanceid”: “oca2kxku88w5”,
“passwordsalt”: “REMOVED SENSITIVE VALUE”,
“secret”: “REMOVED SENSITIVE VALUE”,
“trusted_domains”: [
“3.236.113.32”,
“drive.digivox.com.br”
],
“datadirectory”: “/var/www/owncloud/data”,
“overwrite.cli.url”: “http://3.236.113.32”,
“dbtype”: “mysql”,
“version”: “10.6.0.5”,
“dbname”: “owncloud”,
“dbhost”: “localhost”,
“dbtableprefix”: “oc_”,
“mysql.utf8mb4”: true,
“enable_certificate_management”: true,
“dbuser”: “REMOVED SENSITIVE VALUE”,
“dbpassword”: “REMOVED SENSITIVE VALUE”,
“logtimezone”: “UTC”,
“apps_paths”: [
{
“path”: “/var/www/owncloud/apps”,
“url”: “/apps”,
“writable”: false
},
{
“path”: “/var/www/owncloud/apps-external”,
“url”: “/apps-external”,
“writable”: true
}
],
“installed”: true,
“memcache.local”: “\OC\Memcache\APCu”,
“memcache.locking”: “\OC\Memcache\Redis”,
“redis”: {
“host”: “127.0.0.1”,
“port”: “6379”
},
“mail_domain”: “REMOVED SENSITIVE VALUE”,
“mail_from_address”: “REMOVED SENSITIVE VALUE”,
“mail_smtpmode”: “smtp”,
“ldapIgnoreNamingRules”: false,
“singleuser”: false,
“mail_smtphost”: “REMOVED SENSITIVE VALUE”,
“mail_smtpsecure”: “ssl”,
“mail_smtpport”: “465”,
“mail_smtpauth”: 1,
“mail_smtpauthtype”: “LOGIN”,
“mail_smtpname”: “REMOVED SENSITIVE VALUE”,
“mail_smtppassword”: “REMOVED SENSITIVE VALUE”,
“loglevel”: 0
},
“integritychecker”: {
“passing”: true,
“enabled”: true,
“result”: []
},
“core”: {
“backgroundjobs_mode”: “cron”,
“enable_external_storage”: “yes”,
“first_install_version”: “10.6.0.5”,
“grace_period”: “1609253915”,
“installedat”: “1609248931.805”,
“lastcron”: “1609982102”,
“lastupdateResult”: “[]”,
“lastupdatedat”: “1609981109”,
“public_files”: “files_sharing/public.php”,
“public_webdav”: “dav/appinfo/v1/publicwebdav.php”,
“shareapi_allow_public_upload”: “no”,
“shareapi_allow_social_share”: “no”,
“shareapi_default_permissions”: “31”,
“shareapi_enabled”: “yes”,
“shareapi_enforce_links_password_read_write_delete”: “REMOVED SENSITIVE VALUE”,
“umgmt_set_password”: “REMOVED SENSITIVE VALUE”,
“umgmt_show_backend”: “false”,
“umgmt_show_email”: “true”,
“umgmt_show_is_enabled”: “true”,
“umgmt_show_last_login”: “true”,
“umgmt_show_password”: “REMOVED SENSITIVE VALUE”,
“umgmt_show_quota”: “true”,
“umgmt_show_storage_location”: “false”
},
“apps”: [
{
“id”: “notifications”,
“name”: “Notifications”,
“summary”: “Notification system for internal events”,
“description”: “This application adds an internal notification system to an ownCloud instance. Notifications are triggered by apps like the Announcement Center or Custom Groups and displayed via a bell indicator in the WebUI as well as via the ownCloud clients.”,
“category”: “tools”,
“screenshot”: “https://raw.githubusercontent.com/owncloud/promo/e1dd604d66b4c5f765579b4c160de3268169ea3c/ownCloud%20logo%20square.png”,
“licence”: “AGPL”,
“author”: “Joas Schilling, Thomas M\u00fcller”,
“version”: “0.5.2”,
“types”: [
“logging”
],
“default_enable”: “”,
“website”: “https://github.com/owncloud/notifications”,
“bugs”: “https://github.com/owncloud/notifications/issues”,
“repository”: {
@attributes”: {
“type”: “git”
},
@value”: “https://github.com/owncloud/notifications.git
},
“commands”: {
“command”: “OCA\Notifications\Command\Generate”
},
“settings”: {
“personal”: “OCA\Notifications\Panels\Personal\NotificationsPanel”
},
“dependencies”: {
“owncloud”: {
@attributes”: {
“min-version”: “10.2”,
“max-version”: “11”
}
}
},
“use-migrations”: “true”,
“info”: [],
“remote”: [],
“public”: [],
“repair-steps”: {
“install”: [],
“pre-migration”: [],
“post-migration”: [],
“live-migration”: [],
“uninstall”: []
},
“background-jobs”: [],
“two-factor-providers”: [],
“_cached”: true,
“groups”: null,
“active”: true,
“level”: 100,
“removable”: true,
“internal”: false,
“preview”: “/apps/notifications/img/notifications.svg”,
“previewAsIcon”: true,
“appconfig”: {
“enabled”: “yes”,
“installed_version”: “0.5.2”,
“types”: “logging”
}
},
{
“id”: “user_external”,
“name”: “External user support”,
“description”: “Use external user authentication methods like IMAP, SMB and FTP”,
“licence”: “AGPL”,
“author”: “Robin Appelman”,
“documentation”: {
“admin”: “https://doc.owncloud.org/server/10.3/admin_manual/configuration/user/user_auth_ftp_smb_imap.html
},

Log in to the web-UI with an administrator account and click on
'admin' -> 'Generate Config Report' -> 'Download ownCloud config report'
This report includes the config.php settings, the list of activated apps
and other details in a well sanitized form.

or 

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your ownCloud installation folder

*ATTENTION:* Do not post your config.php file in public as is. Please use one of the above
methods whenever possible. Both, the generated reports from the web-ui and from occ config:list
consistently remove sensitive data. You still may want to review the report before sending.
If done manually then it is critical for your own privacy to dilligently
remove *all* host names, passwords, usernames, salts and other credentials before posting.
You should assume that attackers find such information and will use them against your systems.

List of activated apps:
Enabled:

  • activity: 2.6.0
  • brute_force_protection: 1.1.0
  • comments: 0.3.0
  • configreport: 0.2.0
  • dav: 0.6.0
  • federatedfilesharing: 0.5.0
  • federation: 0.1.0
  • files: 1.5.2
  • files_external: 0.7.1
  • files_external_s3: 1.0.0
  • files_mediaviewer: 1.0.3
  • files_primary_s3: 1.1.2
  • files_sharing: 0.14.0
  • files_trashbin: 0.9.1
  • files_versions: 1.3.0
  • firstrunwizard: 1.2.0
  • market: 0.6.0
  • notifications: 0.5.2
  • password_policy: 2.1.2
  • provisioning_api: 0.5.0
  • systemtags: 0.3.0
  • updatenotification: 0.2.1
    Disabled:
  • encryption
  • external
  • objectstore
  • user_external
  • user_ldap
    root@ip-172-31-48-117:/var/www/owncloud#
If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your ownCloud installation folder.

Are you using external storage, if yes which one: local/smb/sftp/…
Yes, S3
Are you using encryption: yes/no
No
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/…
No

LDAP configuration (delete this part if not used)

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your ownCloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser:

Operating system:

Logs

Web server error log

Insert your webserver log here

ownCloud log (data/owncloud.log)

Insert your ownCloud log here

root@ip-172-31-48-117:/var/www/owncloud# tail -f data/owncloud.log
{“reqId”:“a0d59d3c-e19b-45c6-aa03-db3ab45d0f5d”,“level”:0,“time”:“2021-01-07T01:28:29+00:00”,“remoteAddr”:“177.75.68.194”,“user”:“admin”,“app”:“OC\Authentication\Token\DefaultTokenProvider::updateToken”,“method”:“GET”,“url”:"/ocs/v2.php/apps/notifications/api/v1/notifications?format=json",“message”:“updating token 224, last check is now 1609981409”}
{“reqId”:“a0d59d3c-e19b-45c6-aa03-db3ab45d0f5d”,“level”:0,“time”:“2021-01-07T01:28:29+00:00”,“remoteAddr”:“177.75.68.194”,“user”:“admin”,“app”:“OC\Authentication\Token\DefaultTokenProvider::updateToken”,“method”:“GET”,“url”:"/ocs/v2.php/apps/notifications/api/v1/notifications?format=json",“message”:“updating token 224, last check is now 1609981409”}
{“reqId”:“a0d59d3c-e19b-45c6-aa03-db3ab45d0f5d”,“level”:0,“time”:“2021-01-07T01:28:29+00:00”,“remoteAddr”:“177.75.68.194”,“user”:“admin”,“app”:“OC\Authentication\Token\DefaultTokenProvider::getToken”,“method”:“GET”,“url”:"/ocs/v2.php/apps/notifications/api/v1/notifications?format=json",“message”:“token 59bba530f5f60d3e2e09b7d0cb6977237aa0148543889707e54a5e15d40366155a21539c83b017f6ee4235dd95c1a4bc034a7ffd60fa3a27768004552b26a4e8 does not exist”}
{“reqId”:“010b068a-d4af-44dd-ae26-517286e609b0”,“level”:0,“time”:“2021-01-07T01:28:29+00:00”,“remoteAddr”:“177.75.68.194”,“user”:“admin”,“app”:“OC\Authentication\Token\DefaultTokenProvider::updateToken”,“method”:“GET”,“url”:"/ocs/v1.php/cloud/activity?page=0&pagesize=100&format=json",“message”:“updating token 224, last check is now 1609981409”}
{“reqId”:“010b068a-d4af-44dd-ae26-517286e609b0”,“level”:0,“time”:“2021-01-07T01:28:29+00:00”,“remoteAddr”:“177.75.68.194”,“user”:“admin”,“app”:“OC\Authentication\Token\DefaultTokenProvider::updateToken”,“method”:“GET”,“url”:"/ocs/v1.php/cloud/activity?page=0&pagesize=100&format=json",“message”:“updating token 224, last check is now 1609981409”}
{“reqId”:“010b068a-d4af-44dd-ae26-517286e609b0”,“level”:0,“time”:“2021-01-07T01:28:29+00:00”,“remoteAddr”:“177.75.68.194”,“user”:“admin”,“app”:“OC\Authentication\Token\DefaultTokenProvider::getToken”,“method”:“GET”,“url”:"/ocs/v1.php/cloud/activity?page=0&pagesize=100&format=json",“message”:“token 59bba530f5f60d3e2e09b7d0cb6977237aa0148543889707e54a5e15d40366155a21539c83b017f6ee4235dd95c1a4bc034a7ffd60fa3a27768004552b26a4e8 does not exist”}
{“reqId”:“6eI3pxNIPb78b0RHPKsr”,“level”:0,“time”:“2021-01-07T01:28:48+00:00”,“remoteAddr”:“177.37.172.114”,“user”:“admin”,“app”:“OC\User\Session::validateToken”,“method”:“GET”,“url”:"/ocs/v2.php/apps/notifications/api/v1/notifications?format=json",“message”:“token d34636d6b014fc56b5534dae97bba0373d818a192e3714013216f48f84a0beeaea3124f36c5f1e2ab4a2c71489e9960392e007c16f80e1e7019284ff6f8b9fbc with token id 226 found, validating”}
{“reqId”:“6eI3pxNIPb78b0RHPKsr”,“level”:0,“time”:“2021-01-07T01:28:48+00:00”,“remoteAddr”:“177.37.172.114”,“user”:“admin”,“app”:“OC\User\Session::validateToken”,“method”:“GET”,“url”:"/ocs/v2.php/apps/notifications/api/v1/notifications?format=json",“message”:“token d34636d6b014fc56b5534dae97bba0373d818a192e3714013216f48f84a0beeaea3124f36c5f1e2ab4a2c71489e9960392e007c16f80e1e7019284ff6f8b9fbc with token id 226 found, validating”}
{“reqId”:“09edb330-9e88-4313-962d-d7b7398b323f”,“level”:0,“time”:“2021-01-07T01:28:51+00:00”,“remoteAddr”:“177.37.172.114”,“user”:“leandro.medeiros”,“app”:“OC\Authentication\Token\DefaultTokenProvider::updateToken”,“method”:“PROPFIND”,“url”:"/remote.php/dav/files/leandro.medeiros/",“message”:“updating token 225, last check is now 1609981928”}

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log 
c) ...

I saw the used space is from theses folders

root@ip-0000-00-000-000:/var/www/owncloud/data# du -sh * |grep G
108G admin
2.0G owncloud.log
root@ip-000-00-00-00:/var/www/owncloud/data# cd admin/
root@ip-000-00-00-00:/var/www/owncloud/data/admin# du -sh * |grep G
108G uploads
root@ip-000-00-00-0000:/var/www/owncloud/data/admin# cd uploads/
root@ip-00-00-00-00:/var/www/owncloud/data/admin/uploads# du -sh * |grep G
2.9G 1062852877
1.2G 1131187971
1.3G 1377085377
1.8G 1434721540
1.8G 1577425509
1.3G 1657923443
2.1G 1854006048
2.1G 1981767420
1.8G 2022983815
1.4G 2212055400
2.9G 2260605329
1.6G 2392708985
4.3G 2474972219
2.1G 2484234777
2.3G 2872754298
1.3G 291683155
2.3G 3001872850
2.1G 3028861231
2.1G 3055884749
1.8G 3065536251
2.3G 3283050987
2.3G 3353225746
2.1G 3461521706
1.3G 3494535432
1.2G 36349992
1.8G 3655129454
1.2G 3905499599
2.1G 4115074003
2.9G 4165765144
1.8G 4222298431
4.3G 663275805
1.3G 848344068
1.7G 910861241
1.8G 927336477
root@ip-000-00-00-000:/var/www/owncloud/data/admin/uploads#

The real data are in bucket S3 correctly, but these folders in /var/www/owncloud/data/admin/uploads are consuming my disk space.

The files inside these numeric folders are encrypted, I think.

Someone have any idea?

EDITED

I’m doing upload by ownCloud client loged with admin user. So I think it is about this process. Maybe I must wait the upload be fineshed, so can I remove this?

ownCloud uses chunking for uploads by authenticated users. These chunks will be uploaded into the users uploads folder. It is also possible to change the upload storage to some temp folder in the system.
However you need to make sure that there is enough storage available for all uploads happening concurrently. Once the upload is finished the chunks are being assembled into the actual file and put into the actual storage.
If an upload is aborted due to any reasons the chunks won’t be cleaned up automatically. Therefore it is crucial to run the following occ command regularly on any ownCloud system:

  • occ dav:cleanup-chunks

I would recommend to use cron to run this command at least once a day. Configure it like the typical system cron, for example like so (paths could differ):

# /etc/cron.d/owncloud - Debian based system
37 0 * * * www-data /usr/bin/php /var/www/owncloud/occ dav:cleanup-chunks
# or alternatively directly as a www-data user cron by running `crontab -e -u www-data`
37 0 * * * /usr/bin/php /var/www/owncloud/occ dav:cleanup-chunks
# replace www-data with apache on RHEL based systems

EDIT: Links to relevant documentation:

Awesome! Thank you very much for your explanation.

Solved!

Do you know why I can’t do ti?

owncloud_erros

Perhaps you can run the command with the --help flag to see whether it offers more verbosity through another flag (probably -v).

The error itself already states it is forbidden. So I would assume a problem with the file permissions, or that you’re not running occ as the web servers user.

I can’t determine whether you calling occ in /var/www/owncloud as user root is running it correctly. Do you have a wrapper script which runs occ as the correct user?

Hello,

I tried with two users and the error continues. However, the error is only for the command cleanup-chucks as you can see in the image.![owncloud_sequence.