Old versions not being deleted - CRON job not running

Rix999 · March 31, 2022, 4:02pm

Steps to reproduce

Expected behaviour

Old versions of files should be deleted

Actual behaviour

Old versions of files are not being deleted

Server configuration

Operating system:
cPanel

Web server:

Database:

PHP version:
7.4

ownCloud version: (see ownCloud admin page)
10.9.1

Updated from an older ownCloud or fresh install:

Where did you install ownCloud from:
Softaculous

Version retention is not working.
Old versions are not getting deleted.
I suspect the CRON job is not running.

If I:

login to OC as Admin
select ‘Admin | General’ from the left navbar
Under ‘CRON’ I saw the following:

OcCron1

So I changed to AJAX
The documentation says “Each time a user visits the ownCloud page, a single background job is executed.”
I interpret this as “when I visit a webpage on my OC instance, the CRON job will run and remove old versions of files”.
I tried that and it hasn’t removed them.

So I changed it back to CRON

I am not a server admin specialist.
My OC instance is on a shared server (installed via Softaculous) using cPanel.

On cPanel there is a page for CRON jobs

After reading many pages of OC documentation I created a CRON job as follows:
/usr/local/bin/ea-php74 /data01/MYDOMAIN/public_html/owncloud/cron.php

I told cPanel to send me an email when the CRON job runs and this is what I got

Your Data directory is invalid
Please check that the data directory contains a file “.ocdata” in its root.

Cannot create “data” directory
This can usually be fixed by giving the webserver write access to the root directory.

I assume that this means I have not specified where old file versions are stored.

I think all my OC data is here
/data01/MYDOMAIN/ownclouddata/MYNAME/
and there is a file called .ocdata in this directory
/data01/MYDOMAIN/ownclouddata/

but I now have no idea how to create the command for the CRON job in cPanel.

If there is a (easy) way to create the CRON job in OC then please tell me how to do it.

I would be grateful for any suggestions. Thanks, everyone.

alfredb · March 31, 2022, 7:27pm

This does no longer works. I’m using something like this:

/usr/bin/php -f /path/to/occ system:cron

Rix999 · April 1, 2022, 7:46am

@alfredb Thank you for your reply.
I have tried this
/usr/bin/php -f /data01/MYDOMAIN/public_html/owncloud system:cron
and got the response:
No input file specified.

What am I missing from the command?
Thanks

alfredb · April 1, 2022, 8:00am

I assume MYDOMAIN is a placeholder and owncloud is your directory, so you omitted the trailing occ.

Does this work?

/usr/bin/php -f /data01/MYDOMAIN/public_html/owncloud/occ system:cron

Rix999 · April 1, 2022, 8:23am

Thank you.
I assumed “occ” was part of the “/path/to”…ie. “path to owncloud”
Yes, “MYDOMAIN” is a placeholder for the name of the organisation. As it is a shared host the name appears in the directory path.

I have tried
/usr/bin/php -f /data01/MYDOMAIN/public_html/owncloud/occ system:cron
and got an “Internal server error” followed by a long stack trace

I remembered that the path needed to be to a directory containing the file “.ocdata” so tried this
/usr/bin/php -f /data01/MYDOMAIN/ownclouddata/occ system:cron
and got
No input file specified.

I’m admit I’m puzzled why, for something so fundamental to Owncloud (the deletion of old versions of files), this function is not handled in a more user-friendly way.

alfredb · April 1, 2022, 8:31am

Well, it is. But it depends on a working cron job.

IIRC the working directory is also relevant. Now you could try:

 cd /data01/MYDOMAIN/owncloud && /usr/bin/php -f ./occ system:cron

Rix999 · April 1, 2022, 8:59am

Thanks again - I am very grateful for your patience.

I tried
cd /data01/MYDOMAIN/owncloud && /usr/bin/php -f ./occ system:cron
and got
/usr/local/cpanel/bin/jailshell: line 0: cd: /data01/MYDOMAIN/owncloud: No such file or directory
so I tried
cd /data01/MYDOMAIN/ownclouddata && /usr/bin/php -f ./occ system:cron
and got
No input file specified.
so I tried
cd /data01/MYDOMAIN/public_html/owncloud && /usr/bin/php -f ./occ system:cron
and got
Internal Server Error

In case it helps - this is what the File Manager shows in cPanel

alfredb · April 1, 2022, 9:08am

No problem, you’re welcome.

Let’s follow this one.

Please check the owncloud.log for the error this command throws.

alfredb · April 1, 2022, 9:17am

I just saw that according to your screenshot (cPanel) that maybe you should use
/usr/local/bin/php instead of /usr/bin/php. There might be different versions of php around on your system.

Rix999 · April 1, 2022, 9:32am

Thanks.
You can view the owncloud.log here
http://www.zen17229.zen.co.uk/owncloud/owncloud.log

I tried
cd /data01/MYDOMAIN/public_html/owncloud && /usr/local/bin/php -f ./occ system:cron
and got

Your Data directory  is invalid
Please check that the data directory contains a file ".ocdata" in its root.

Cannot create "data" directory
This can usually be fixed by <a href="https://doc.owncloud.com/server/10.9/go.php?to=admin-dir_permissions" target="_blank" rel="noreferrer">giving the webserver write access to the root directory</a>.

An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /data01/MYDOMAIN/public_html/owncloud/lib/private/Console/Application.php:139
Stack trace:
#0 /data01/MYDOMAIN/public_html/owncloud/console.php(115): OC\Console\Application->loadCommands(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#1 /data01/MYDOMAIN/public_html/owncloud/occ(11): require_once('/data01/MYDOMAIN/pub...')
#2 {main}

So I thought I would try this (because this directory has the .ocdata file)
cd /data01/MYDOMAIN/ownclouddata && /usr/local/bin/php -f ./occ system:cron
and got
Could not open input file: ./occ

alfredb · April 1, 2022, 10:25am

I’ve too little info about your system, cPanel and your possibilities.

Had you ever a working cron config? Usually, all occ command must be run by the user that is running the web server. This in done by prefixing te commands by something like sudo -u www-data, or whatever your web user is.
But maybe the current cron mode simply doesn’t work on your system.

You could also try:

Switch from Cron to Webcron
Invoke (via cPanel) https://MYDOMAIN/owncloud/cron.php

Rix999 · April 1, 2022, 10:29am

Here is CRON job created by Matomo (also installed via Softaculous) which works, as far as I am aware
/usr/local/bin/php -d zlib.output_compression=“Off” -d output_handler="" /data02/MYDOMAIN/public_html/matomo/console core:archive --url=https://MYDOMAIN.com/matomo/ > /dev/null

Invoke (via cPanel) https://MYDOMAIN/owncloud/cron.php

I don’t know how to do that

Switch from Cron to Webcron

I created a CRON job at webcron
htTtp://MYDOMAIN.com/owncloud/cron.php
(I also tried https)
The job runs (it is recorded as “successful” but no files get deleted

This is the output of the CRON job

Cache-Control: no-cache, no-store, must-revalidate
Content-Length: 2
Content-Type: application/json; charset=utf-8
Pragma: no-cache
Content-Security-Policy: default-src 'none';manifest-src 'self';script-src 'self' 'unsafe-eval';style-src 'self' 'unsafe-inline';img-src 'self' data: blob:;font-src 'self';connect-src 'self';media-src 'self'
X-Permitted-Cross-Domain-Policies: none
Date: Fri, 01 Apr 2022 10:52:03 GMT
Server: Apache
Set-Cookie: ocz4mzyhscmd=86d3f02811f5b0619a18de855f67f272; path=/owncloud; HttpOnly; SameSite=Strict,oc_sessionPassphrase=efAkgk4YR%2Fk0djNQ0Q13UqN7qTpx9XLlqpjNaWGFsyljW4gXq5TYRo2FFPp6%2FmQQ%2BX6b6QktCG6ySDpTUiPgsj9v%2FQJu%2BHshWpQvoTmTq07RvBcQaI%2B%2Bx3lyArvp4fBl; path=/owncloud; HttpOnly; SameSite=Strict
X-Robots-Tag: none
X-Download-Options: noopen
Expires: Thu, 19 Nov 1981 08:52:00 GMT
X-Content-Type-Options: nosniff
X-Xss-Protection: 0
X-Frame-Options: SAMEORIGIN
[]

Here are the contents of cron.php

<?php
/**
 * CREDITS etc
 */

require_once __DIR__ . '/lib/base.php';

if (!\OC::$CLI) {
	$url = \OC::$server->getURLGenerator()->linkToRoute('core.Cron.run');
	\header("Location: $url");
	return;
}

echo 'Please use ./occ system:cron' . PHP_EOL;
$return = \system('./occ system:cron');
// in case of an error while cron execution we exit with error code as well
if ($return === false) {
	exit(1);
}

Here is my config.php

<?php
$CONFIG = array (
  'instanceid' => 'ocz4mzyhscmd',
  'passwordsalt' => 'REMOVED',
  'secret' => '0vzeicsawymxozgss4znmkxneaiasb9nxmfd4zfy6sniqllji0efirnan0obrgwfxvr7shnmowucq7geik8j7f18pvych75h',
  'trusted_domains' => 
  array (
    0 => 'MYDOMAIN.com',
  ),
  'datadirectory' => '/data02/MYDOMAIN/ownclouddata',
  'overwrite.cli.url' => 'https://MYDOMAIN.com/owncloud',
  'dbtype' => 'mysql',
  'version' => '10.9.1.2',
  'dbname' => 'MYDOMAIN_ownc347',
  'dbhost' => 'localhost',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'MYDOMAIN_ownc347',
  'dbpassword' => REMOVED',
  'logtimezone' => 'UTC',
  'installed' => true,
  'theme' => '',
  'loglevel' => 0,
  'maintenance' => false,
  'integrity.check.disabled' => true,
  'filelocking.enabled' => false,
  'trashbin_retention_obligation' => '60,120',
  'versions_retention_obligation' => 'auto',
  'web.baseUrl' => 'https://MYDOMAIN.com/index.php/apps/web',
  'allow_user_to_change_mail_address' => '',
);

Thanks for your help. I’m sure you have better things to do…

eneubauer · April 1, 2022, 11:08am

Have you tried running the command directly in an SSH session or are you capturing direct cron output?

I am asking because your SSH user might be different from the web server user, which would also be the user your cron job should run as.

→ make sure to do all testing in a cron job and not via an SSH session.
→ you can redirect all command output by appending something similar to this: >> /path/to/myjob.log 2>&1

alfredb · April 1, 2022, 11:27am

Beside the cron issue:

This is IMHO far too open. I think you should reconfigure this to narrow down the range of versions to keep.

Rix999 · April 1, 2022, 12:18pm

I read this document yesterday.
This sentence seemed a bit ambiguous to me:

“You may alter the default pattern in config.php. The default setting is auto, which sets the default pattern:”

However, I have changed it to

‘versions_retention_obligation’ => ‘30’,

and run the CRON job again from webcron.
No old versions were deleted.

Incidentally, the reason why I am trying to fix this is because, yesterday, the owncloud client on one PC (2.10.0) began uploading all files to the server even though they had not been changed. The result is that oc has effectively duplicated itself. I have had to pause most uploads as I am running out of disk space and using so much bandwidth I cannot work.

Rix999 · April 1, 2022, 12:19pm

@eneubauer Thanks for your help.
I am not using SSH.
If I have understood you correctly, I have run CRON jobs using the cPanel CRON Job page (there is a screenshot at the top of this thread).

alfredb · April 1, 2022, 12:27pm

Because this is (IMHO) not not a valid configuration.

auto, D

Delete all versions that are older than D days automatically, delete other versions according to expiration rules

You could try

'versions_retention_obligation' => 'auto,30'

Rix999 · April 1, 2022, 1:57pm

Final question:
Using cPanel File Manager, can I manually delete all files in directory ‘files_versions’ without causing a problem somewhere else in oc?

alfredb · April 1, 2022, 2:15pm

I’m not sure, but usually, it’s a bad idea, managing files under ownClouds control directly. You shouldn’t do that.

Doesn’t happen the same, when you temporarily disable file versioning app?
Or from the command line with

occ versions:cleanup [userId]

BTW, is the cron issue fixed?

Rix999 · April 1, 2022, 2:21pm

You must think me really stupid…here’s why…

Doesn’t happen the same, when you temporarily disable file versioning app?

How do I do that?

Or from the command line with
occ versions:cleanup [userId]

Which command line? On one of the client PCs (I tried that and it does nothing)? Or on the server? If it’s on the server I have no access to a command line on the server

BTW, is the cron issue fixed?

No, I would have posted profound thanks to the world if anything had been fixed.