Webcron returns 400 status - cron fails to remove old files

[mart314]

As posted above, doing this results in an ownCloud error related to untrusted domains.

I’m just curious, which URL did you use? IMHO this should be:
https://www.mydomain.de/owncloud/cron.php

Yes, exactly. That is what I have been using. And as posted above, if I manually type that URL into a webbrowser, I see a success message. If I use that URL as part of the 1&1 cronjob manager, it fails.
See post 6 in this thread.

[tom42]

Hey,

could this still be valid?

2. Maybe your system has some internal hostname within the 1und1 network and thus the webcron is accessing your ownCloud installation with a different hostname?

From what i have seen so far in the owncloud.log the domain the webcron is trying to access is logged into it.

[mart314]

could this still be valid?

2. Maybe your system has some internal hostname within the 1und1 network and thus the webcron is accessing your ownCloud installation with a different hostname?

Yes, that could still be a problem - hence my attempts to find out what the true server name is, using a clumsy PHP script. Here’s a related question: the config.php allows me to configure trusted domains but what I seem to understand here is that these are full server names, i.e. not ‘mydomain.de’ but rather ‘www.mydomain.de’ - is that right?

[alfredb]

You can add this

“remoteAddr”: “XXX.XXX.XX.XXX”,

as trusted, or preferably its name, that you can find with:

nslookup XXX.XXX.XX.XXX

[mart314]

1. I would update to the latest 10.0.10 after the cleaning the logfile to see if this could fix the log entries

OK, just did that, now on 10.0.10 - can’t really tell any difference.
But now I have even more stuff on the file system…

Is it safe to delete old checkpoints (./data/updater-data/ckeckpoint/, not the latest) and old download .zip files (./data/updater-data/download/)?

[alfredb]

Is it safe to delete old checkpoints (./data/updater-data/ckeckpoint/ , not the latest) and old download .zip files (./data/updater-data/download/ )?

Yes, you can delete them to save disk space. Downgrading isn’t supported anyway.

[alfredb]

Any update on the webcron issue?

[mart314]

Any update on the webcron issue?

Well, it’s still not working… :-
I now started to manually remove old file versions by creating copies of the files and renaming them… =8-o

Edit: To be more precise:

• Old file versions still are not deleted

• Despite having ‘cron_log’ => true, in my config.php, I do not see any cron-related log entries in owncloud.log - even if I manually open www.mydomain.de/owncloud/cron.php and get {“status”:“success”} as return value

• For some reason, calling cron.php from the 1&1 cron manager seems to be working now, at least it reports {“status”:“success”}

Having some more output from the entire cron thing would really help…

[alfredb]

I do not see any cron-related log entries in owncloud.log

You have to adjust the log level.

Cron running no entry in log

Cause I seen them. I think it has too do with log level which I raised “cron”,“method”:"–",“url”:"–",“message”:“Finished background job, the job took : 0 seconds, this job is an instance of class : OCA\Files\BackgroundJob\CleanupFileLocks with arguments : “} {“reqId”:“sIg9GBSYqtVJCw5RdHdK”,“level”:0,“time”:“2018-09-09T22:00:02+00:00”,“remoteAddr”:””,“user”:"–",“app”:“cron”,“method”:"–",“url”:"–",“message”:“Started background job of class : OCA\Files_Sharing\DeleteOrphanedSharesJob with argume…

[mart314]

OK, the picture evolves.

1. In the admin > General settings, I switched to ‘Cron > Webcron’ and set log to ‘Warnings, Errors, Fatal Issues’

2. I manually ran www.mydomain.de/owncloud/cron.php by opening that URL in a web browser. This took several seconds and returned {“status”:“success”}

3. In the log file, I now find a 176 entries of “not found while trying to get owner” similar to this:

{"reqId":"W9Lwh1KlUo0AAC2BQ1wAAABJ","level":3,"time":"2018-10-26T12:46:45+02:00","remoteAddr":"XXX.XXX.XXX.XXX","user":"--","app":"no app in context","method":"GET","url":"\/owncloud\/cron.php","message":"Exception: {\"Exception\":\"OCP\\\\Files\\\\NotFoundException\",\"Message\":\"\\\/files_trashbin\\\/files\\\/file.docx.d1540232740 not found while trying to get owner\",\"Code\":0,\"Trace\":\"#0 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Filesystem.php(961): OC\\\\Files\\\\View->getOwner('\\\/files_trashbin...')\\n#1 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Preview.php(1311): OC\\\\Files\\\\Filesystem::getOwner('\\\/files_trashbin...')\\n#2 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/hook.php(105): OC\\\\Preview::post_delete(Array)\\n#3 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/apps\\\/files_trashbin\\\/lib\\\/Trashbin.php(633): OC_Hook::emit('\\\\\\\\OCP\\\\\\\\Trashbin', 'delete', Array)\\n#4 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/apps\\\/files_trashbin\\\/lib\\\/Trashbin.php(666): OCA\\\\Files_Trashbin\\\\Trashbin::emitTrashbinPostDelete('\\\/files_trashbin...')\\n#5 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/apps\\\/files_trashbin\\\/lib\\\/Trashbin.php(836): OCA\\\\Files_Trashbin\\\\Trashbin::delete('file...', 'ocuser', '1540232740')\\n#6 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/apps\\\/files_trashbin\\\/lib\\\/Trashbin.php(756): OCA\\\\Files_Trashbin\\\\Trashbin::deleteExpiredFiles(Array, 'ocuser')\\n#7 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/apps\\\/files_trashbin\\\/lib\\\/Command\\\/Expire.php(56): OCA\\\\Files_Trashbin\\\\Trashbin::expire('ocuser')\\n#8 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Command\\\/CommandJob.php(34): OCA\\\\Files_Trashbin\\\\Command\\\\Expire->handle()\\n#9 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/BackgroundJob\\\/Job.php(57): OC\\\\Command\\\\CommandJob->run('O:33:\\\"OCA\\\\\\\\Files...')\\n#10 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/BackgroundJob\\\/QueuedJob.php(42): OC\\\\BackgroundJob\\\\Job->execute(Object(OC\\\\BackgroundJob\\\\JobList), Object(OC\\\\Log))\\n#11 \\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/cron.php(142): OC\\\\BackgroundJob\\\\QueuedJob->execute(Object(OC\\\\BackgroundJob\\\\JobList), Object(OC\\\\Log))\\n#12 {main}\",\"File\":\"\\\/mainfolder\\\/group\\\/username\\\/htdocs\\\/owncloud\\\/lib\\\/private\\\/Files\\\/View.php\",\"Line\":1717}"}

Looks like something got out of sync between the main files and their versions?!

Edit: Re-running the same thing does not produce any further log output. Maybe it considers the tasks completed? Either way, the old file versions still remain on the file system in ./data/ocuser/file_versions

Edit2: Just to re-iterate, I have these settings in config.php:

  'versions_retention_obligation' => '1,2',
  'trashbin_retention_obligation' => '1,2',
  'activity_expire_days' => 10,

but I have file versions that have a last modified date of 2017

[chacho]

Do you see cron in syslog

root@vorman:/home/pi# tail /var/log/syslog
Oct 26 07:00:01 vorman CRON[25198]: (www-data) CMD (/usr/bin/php -f /var/www/html/owncloud/cron.php)
Oct 26 07:01:29 vorman systemd[1]: Starting Daily apt download activities…
Oct 26 07:01:48 vorman systemd[1]: Started Daily apt download activities.
Oct 26 07:01:48 vorman systemd[1]: apt-daily.timer: Adding 47min 26.771105s random time.
Oct 26 07:01:48 vorman systemd[1]: apt-daily.timer: Adding 1h 5min 28.710847s random time.
Oct 26 07:09:00 vorman systemd[1]: Starting Clean php session files…
Oct 26 07:09:01 vorman systemd[1]: Started Clean php session files.
Oct 26 07:09:01 vorman CRON[25457]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
Oct 26 07:17:01 vorman CRON[25516]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Oct 26 07:24:05 vorman systemd[1]: Started Session c35 of user pi.
root@vorman:/home/pi#

[alfredb]

Do you see cron in syslog

He is on a hosted owncloud, no syslog I assume.

[tom42]

Hey,

now with 29 (or 30 after mine) posts i think the thread is already hard to follow. Maybe the issue about the log entries about files should be moved to a new thread or maybe there are already existing threads discussing the same available when searching the forums?

[alfredb]

Is there any rule to stop or split discussion after 30 posts? So what?

[mart314]

now with 29 (or 30 after mine) posts i think the thread is already hard to follow. Maybe the issue about the log entries about files should be moved to a new thread or maybe there are already existing threads discussing the same available when searching the forums?

Since I am not sure how these issues are related, it’s a bit hard for me to decide when to branch into a new thread. The overarching issue I have is that my owncloud instance is filling up with unwanted files and the mechanisms for getting rid of them seem not to be working.

But you are correct that the topic has now shifted from a webcron issue to a general cron issue (and ultimately an old file versions issue).

Edit: Is is better to have many short threads that are somewhat interrelated or one long one with all aspects? I could edit the title if that helps?

[tom42]

Hey,

Is there any rule to stop or split discussion after 30 posts? So what?

i think you’re long enough around at this community that you might know posts like “i have the same issue” of new users just because they have found some words within the topic title or some posted errors.

From my experience its always better to have different issues (the initial one was about not able running webcron) split into smaller topics. After such much posts in this thread i don’t think that any readers can follow whats the current status, which things where already solved and which where the solution.

The best example is that it seems you even had to ask the status of the webcron issue a few post above:

Any update on the webcron issue?

To me this clearly shows that its not even clear which issue now gets tackled.

Just my 2 cents on this topic, even if i contributing to mess this topic even more.

[mart314]

FYI, I just started a new thread on the primary issue of old file versions and trash files not being deleted (despite webcron apparently working now).

[tom42]

Hey,

nice and thanks. I think it would be great if you could do an additional post here with a short summary how you got the webcron to work. This post could be uses then to mark the thread as “solved” so further readers are immediately seeing your solution instead of requiring to find it within the 30+ posts.

[mart314]

nice and thanks. I think it would be great if you could do an additional post here with a short summary how you got the webcron to work.

I would - if I understood what exactly the solution was. I tried a whole range of things, and for some reason it now seems to work - although the main goal of getting rid of old files is still not achieved, so maybe it only looks like cron is working now, while it really is not.

One thing I can summarize from all this: better/more verbose reporting of cron errors would be beneficial.

Also: In between, I did a very time-consuming manual cleanup process, by creating a copy of every file with a version history (so the new copy did not have a history), deleting the original file and renaming the new file to the name of the original. I had to do this in order to free up space, as all the other efforts did not work. But I did not do this for every single file (only the largest ones and those with the most versions), so I can still tell that cron does not clean up properly. Also new versions are starting to accumulate again.

[tom42]

Hey,

nice and thanks for the summary. I think if you have suggestion the ownCloud development team is probably happy about any feedback / suggestions. From what i know this team can be reached via https://github.com/owncloud/core/issues