Item is on blacklist - how to verify/see them and remove?

Initially I though this issue was file size > 512mb but after following instructions for increasing filesize (2G) in php.ini I get no change and anyhow I see small files ~kb are impacted too. After increasing server log to debug I don’t see any relevant signs in the logs pointing keywords like ‘blacklist’, the file names, timestamps etc.

I’ve tried reinstalling oc-client, removing adding folders, force resync, using server occ files repair and restarting apache, server and oc-client.

At this stage, since I don’t want to wait 24hours each time I tweak a setting, I’d like to be able to see/edit the list of blacklisted files as referred to in the oc-client logs, how can I do this or can anyone hint at root cause?

thanks
Andrew

Expected behaviour

Tell us what should happen

All files should sync normally or when forced.

Actual behaviour

Tell us what happens instead

A few files are reported (various sizes and various folders)

Steps to reproduce

  1. normal oc-client day-to-day wrk
  2. add some files
  3. see errors in oc-client UI under activity showing 4 or 5 files with issue shown as ‘error transferring… server replied : (skipped due to earlier error trying again in X hrs)’

Server configuration

Operating system: Raspbian stretch

Web server: Apache/2.4.25 (Raspbian)

Database: 10.1.38-MariaDB-0+deb9u1 - Raspbian 9.0

PHP version: PHP version:7.2.26-1+0~20191218.33+debian9~1.gbpb5a34b

ownCloud version: 10.2.1.4

Storage backend (external storage):

Client configuration

Client version: Version 2.5.1.10973+dfsg-1ubuntu2.

Operating system: Ubuntu 19.04

OS language: En

Qt version used by client package (Linux only, see also Settings dialog):

Client package (From ownCloud or distro) (Linux only):

Installation path of client:

Logs

Template for output < 10 lines

  1. Client logfile: Output of owncloud --logwindow or owncloud --logfile log.txt

extract relevant chunk from owncloud-client log

01-14 13:19:41:302 [ warning sync.propagator ]: Could not complete propagation of “Documents/jobs/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv” by OCC::PropagateIgnoreJob(0x55ec42dfe080) with status 9 and error: “Error transferring https://awsbarker.ddns.net/owncloud/remote.php/dav/uploads/Andrew/1856036348/00000000 - server replied: (skipped due to earlier error, trying again in 22 hour(s))”
01-14 13:19:47:899 [ info sync.csync.updater ]: file: ValueWorkshop/Bridges To Prosperity - Overview-OB2Lwu5MVp0.mkv, instruction: INSTRUCTION_NONE <<=
01-14 13:19:49:172 [ info sync.csync.updater ]: file: Documents/jobs/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv, instruction: INSTRUCTION_NEW <<=
01-14 13:19:49:401 [ info sync.csync.updater ]: file: vdo2g0/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv, instruction: INSTRUCTION_NEW <<=
01-14 13:19:52:164 [ info sync.csync.reconciler ]: INSTRUCTION_NEW client file: Documents/jobs/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv
01-14 13:19:52:167 [ info sync.csync.reconciler ]: INSTRUCTION_NEW client file: vdo2g0/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv
01-14 13:19:52:189 [ info sync.engine ]: Item is on blacklist: “Documents/jobs/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv” retries: 1 for another 79352 s
01-14 13:19:52:195 [ info sync.engine ]: Item is on blacklist: “vdo2g0/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv” retries: 1 for another 79353 s
01-14 13:19:52:272 [ info sync.propagator ]: Starting INSTRUCTION_IGNORE propagation of “Documents/jobs/Peaky.Blinders.S03E02.720p.BluRay.x264-GalaxyTV.mkv” by OCC::Pro
  1. Web server error log:

When I grep the server log for keywords from above no output despite level being 0 debug. If I look arounf the above error timestamp I can only see this as unusual

{“reqId”:“FgtX5kSCexlUhthqlllb”,“level”:3,“time”:“2019-08-10T13:19:15+02:00”,“remoteAddr”:"",“user”:"–",“app”:“PHP”,“method”:"–",“url”:"–",“message”:“You are using a fallback implementation of the intl extension. Installing the native one is highly recommended instead. at /var/www/html/owncloud/lib/composer/patchwork/utf8/src/Patchwork/Utf8/Bootup/intl.php#18”}
{“reqId”:“sWp8y3aavkhKlffgVPYJ”,“level”:3,“time”:“2019-08-10T13:19:52+02:00”,“remoteAddr”:"",“user”:"–",“app”:“PHP”,“method”:"–",“url”:"–",“message”:“You are using a fallback implementation of the intl extension. Installing the native one is highly recommended instead. at /var/www/html/owncloud/lib/composer/patchwork/utf8/src/Patchwork/Utf8/Bootup/intl.php#18”}
{“reqId”:“t2O6ORKnszPFdJHceIrc”,“level”:3,“time”:“2019-08-10T13:19:59+02:00”,“remoteAddr”:"",“user”:"–",“app”:“PHP”,“method”:"–",“url”:"–",“message”:“You are using a fallback implementation of the intl extension. Installing the native one is highly recommended instead. at /var/www/html/owncloud/lib/composer/patchwork/utf8/src/Patchwork/Utf8/Bootup/intl.php#18”}
{“reqId”:“eae95961-72b6-42f2-b76b-6680d2e3471d”,“level”:4,“time”:“2019-08-10T16:13:19+02:00”,“remoteAddr”:“192.168.1.105”,“user”:"–",“app”:“remote”,“method”:“GET”,“url”:"/owncloud/status.php",“message”:“Memcache \OC\Memcache\APCu not available for local cache”}
{“reqId”:“eae95961-72b6-42f2-b76b-6680d2e3471d”,“level”:3,“time”:“2019-08-10T16:13:19+02:00”,“remoteAddr”:“192.168.1.105”,“user”:"–",“app”:“PHP”,“method”:“GET”,“url”:"/owncloud/status.php",“message”:“You are using a fallback implementation of the intl extension. Installing the native one is highly recommended instead. at /var/www/html/owncloud/lib/composer/patchwork/utf8/src/Patchwork/Utf8/Bootup/intl.php#18”}

  1. Server logfile: ownCloud log (data/owncloud.log):

You should be able to get rid of this message by installing php7.2-intl on the server and restarting the apache web server.

1 Like

Starting with version 2.5.1, the “Force sync” feature should ignore or wipe the blacklist.

Related issue:
https://github.com/owncloud/client/issues/6757

Related code change:
https://github.com/owncloud/client/pull/6760/files

You could debug currently blacklisted files in the SQLite file in your sync dir:
https://doc.owncloud.com/desktop/2.6/architecture.html#the-sync-journal

1 Like

Thanks eneubauer for rapid feedback.
I’ll have a look at this intl version, I know I have some outstanding issues with setting locale on this Raspberrypi and I suppose this might be conflicting too.

1 Like

Thanks Michael,
I couldn’t understand the SQLite reference but when I force sync although a blacklist clean out may occur I still get an Activity flag showing :

14.01.20 15:41:12, CameraUpload/Picture 1.png, /media/Data2/ownCloud,Error transferring https://awsbarker.ddns.net/owncloud/remote.php/dav/files/Andrew/CameraUpload/Picture 1.png - server replied: (skipped due to earlier error, trying again in 24 hour(s))

Whilst trying to understand how I could upgrade to php7.2-intl on server as suggested by eneubauer I found in my server nginx log these error for each file failing on the oc-client

2020/01/15 16:28:18 [error] 5503#5503: *20947 client intended to send too large body: 1234056 bytes, client: 192.168.1.105, server: awsbarker.ddns.net, request: “PUT /owncloud/remote.php/dav/files/Andrew/Fan/Washer%20Fan_files/c9HGBK0bHVQ_data/base.js HTTP/2.0”, host: “awsbarker.ddns.net

I’d forgotten (!?) that I have nginx infront of apache and I guess I need to increase the nginx config client_max_body_size >512mb;

I’ll get back once done and verified as the problem.

SOLVED

Setting the nginx config client_max_body_size to 512mb; immediately cleared the remaining Activity errors.

1 Like