OCIS seems to just hang after some time

Infinite Scale
,
1

Hi, I’ve installed OCIS on my machine using this as a guide for the most part (modifying OCIS_URL as appropriate of course, and installing via package made for my distro):
doc.owncloud[.]com/ocis/next/depl-examples/bare-metal.html

After following this guide, I start the OCIS systemd service, and it seems to work. I am able to login with the admin password output by the ocis init command, but after some time it becomes unresponsive and times out. Any attempt to access the site after this results in a timeout. I have tried restarting the service, and occasionally that will allow me to get to the login page, and browse for a short period of time before it times out again, but most of the time the site will just time out forever. I’ve even tried reinstalling (stopping the service, uninstalling the package, removing all related directories, reinstalling, and re-setting up via the guide), but the behavior remains the same. Is there something missing from the guide, or might there be some other issue that needs tracking down?

(the subsequent systemd logs from restarting the service are different from the initial log, but wouldn’t format correctly, so I’m planning on pasting them in a comment below)

Server configuration

Operating system:Arch Linux with kernel v6.5.5
Web server: Nginx v1.25.2
ownCloud version: OCIS 4.0.1
Updated from an older ownCloud or fresh install: fresh install
Where did you install ownCloud from: Arch User Repository (see also: aur.archlinux[.]org/packages/ocis)

Client configuration

Browser: Brave, Chrome, Firefox, Vivaldi
Operating system: Linux, Android, Windows

Logs

Web server error log

no errors logged

ownCloud log (data/owncloud.log)

initial systemd log…

ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"Migrating spaces directory structure..."} 
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"Migrating space types indexes..."} 
ocis[3129807]: {"level":"info","root":"/var/lib/ocis/storage/metadata","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"Migrating to messagepack metadata backend..."}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"done."}
ocis[3129807]: 2023-10-02T22:25:03-05:00 WRN missing or incomplete nats configuration. Events will not be published. pkg=rhttp service=storage-system
ocis[3129807]: {"level":"warn","error":"open /var/lib/ocis/storage/metadata/indexes/by-user-id: no such file or directory","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"error listing user indexes"}
ocis[3129807]: {"level":"warn","error":"open /var/lib/ocis/storage/metadata/indexes/by-group-id: no such file or directory","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"error listing group indexes"}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"done."}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"done."}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"Migrating spaces directory structure..."} 
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"Migrating space types indexes..."} 
ocis[3129807]: {"level":"info","root":"/var/lib/ocis/storage/users","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"Migrating to messagepack metadata backend..."}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"done."}
ocis[3129807]: {"level":"warn","error":"open /var/lib/ocis/storage/users/indexes/by-user-id: no such file or directory","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"error listing user indexes"}
ocis[3129807]: {"level":"warn","error":"open /var/lib/ocis/storage/users/indexes/by-group-id: no such file or directory","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"error listing group indexes"}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"done."}
ocis[3129807]: {"level":"info","time":"2023-10-02T22:25:03-05:00","caller":"github.com/cs3org/reva/v2@v2.16.0/pkg/storage/utils/decomposedfs/decomposedfs.go:170","message":"done."}
ocis[3129807]: 2023-10-02T22:25:05-05:00 WRN CA cert file is not ready yet. Waiting 2 seconds for it to appear. LDAP CACert=/var/lib/ocis/idm/ldap.crt service=auth-basic
ocis[3129807]: 2023-10-02T22:25:05-05:00 WRN CA cert file is not ready yet. Waiting 2 seconds for it to appear. LDAP CACert=/var/lib/ocis/idm/ldap.crt service=users
ocis[3129807]: 2023-10-02T22:25:05-05:00 WRN CA cert file is not ready yet. Waiting 2 seconds for it to appear. LDAP CACert=/var/lib/ocis/idm/ldap.crt service=groups
ocis[3129807]: 2023-10-02T22:25:06-05:00 WRN skipped as signer with same kid already loaded kid=private-key path=/var/lib/ocis/idp/private-key.pem service=idp                                                                                                                                                               ```
2

subsequent systemd logs show this when I’m able to login for a brief period of time…

ocis[3224735]: 2023-10-02T23:03:24-05:00 WRN missing or incomplete nats configuration. Events will not be published. pkg=rhttp service=storage-system
ocis[3224735]: 2023-10-02T23:03:25-05:00 WRN skipped as signer with same kid already loaded kid=private-key path=/var/lib/ocis/idp/private-key.pem service=idp
ocis[3224735]: 2023-10-02T23:03:36-05:00 ERR failed to authenticate the request error="failed to verify access token: crypto/rsa: verification error" authenticator=oidc path=/api/v0/settings/values-list service=proxy
ocis[3224735]: 2023-10-02T23:03:37-05:00 ERR failed to authenticate the request error="failed to verify access token: crypto/rsa: verification error" authenticator=oidc path=/ocs/v1.php/cloud/user service=proxy
ocis[3224735]: 2023-10-02T23:03:37-05:00 WRN core access token not set pkg=rhttp service=frontend traceid=00000000000000000000000000000000
ocis[3224735]: 2023-10-02T23:03:37-05:00 ERR error getting auth provider client error="internal error: gateway: error finding an auth provider for type: bearer" pkg=rgrpc service=gateway traceid=00000000000000000000000000000000 type=bearer
ocis[3224735]: 2023-10-02T23:04:22-05:00 WRN core access token not set pkg=rhttp service=frontend traceid=00000000000000000000000000000000
ocis[3224735]: 2023-10-02T23:04:22-05:00 ERR error getting auth provider client error="internal error: gateway: error finding an auth provider for type: bearer" pkg=rgrpc service=gateway traceid=00000000000000000000000000000000 type=bearer

or this in the cases where it stays unresponsive:

ocis[3269109]: 2023-10-02T23:21:16-05:00 WRN missing or incomplete nats configuration. Events will not be published. pkg=rhttp service=storage-system
ocis[3269109]: 2023-10-02T23:21:17-05:00 WRN skipped as signer with same kid already loaded kid=private-key path=/var/lib/ocis/idp/private-key.pem service=idp
3

I think the problem is
WRN missing or incomplete nats configuration. Events will not be published. pkg=rhttp service=storage-system
Do you run ocis with OCIS_INSECURE or the you have certificates configured?
Did you change any of the default configuration values?

4

I used the same config values as those provided on the example site (with the exception of OCIS_URL of course).

I’m running with OCIS_INSECURE set to false, and have configured certs via the Let’s Encrypt certbot for my Nginx reverse proxy as the example site says to do. Immediately after one install attempt, I also verified that my new subdomain serving the reverse proxy was added to my cert via the SSL Labs link at the bottom of the example page as well.

Do I need to configure certs for this nats thing too, or is there some other configuration for it that’s necessary?

My ocis.env file contents below:

OCIS_URL=https://my.sitename.com
PROXY_HTTP_ADDR=0.0.0.0:9200
PROXY_TLS=false
OCIS_INSECURE=false

OCIS_LOG_LEVEL=warn

OCIS_CONFIG_DIR=/etc/ocis
OCIS_BASE_DATA_PATH=/var/lib/ocis
5

So, the WRN missing or incomplete nats configuration. Events will not be published. pkg=rhttp service=storage-system log message is a red herring. The storage-system service doesn’t use events. We need to get rid of that error message.

Afais it is not necessary to configure certificates for nats, unless you set OCIS_EVENTS_ENABLE_TLS.

It seems there is some other issue that we don’t see yet. Can you try running ocis with OCIS_LOG_LEVEL=debug? Maybe this tells us what is going on…

6

Any updates on this? Same setup & same problem here.

7

Hey,

i think you could read all previous postings to see the following and provide the relevant logs:

Then i think some one could provide any updates on this, otherwise i don’t think there will be updates if the requested info is not provided. :frowning:

8

seeing as how noone else wishes to continue figuring this out I will post from my server that I built using the same process (ver. 4.0.6). However, posting the amount of info from the dump is not allowed. so made a tarball and posted on my server: download.itadmins[.]net/dump.tar.gz

The only difference is that I just get the rotating circle and nothing else.

9

I also had those warning, they went away after I changed all insecure entries from true to false in ocis.yaml

Still I have warnings spammed, I do not know where they’re coming from. I even made a second ocis instance, new, from scratch, and still they are being spammed.

WRN **http** end="21/Mar/2024:12:07:33 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="21/Mar/2024:12:07:33 +0000" status=404 time_ns=398045 traceid=6644c1b9038622e45560a6baff284d1d uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/users/9767076c-00b0-4124-ba33-1bcaa1a38ec4/received.json url=/users/9767076c-00b0-4124-ba33-1bcaa1a38ec4/received.json

WRN **core access token not set** pkg=rhttp service=frontend traceid=448ebcd1bc39b6877bca700bfd9d02b8

I’m using ocis+authelia+caddy

My compose:

  owncloud:
    container_name: owncloud
    hostname: owncloud
    image: owncloud/ocis:5
    profiles:
      - private
    networks:
      - ownmedia
    entrypoint:
      - /bin/sh
    command: ["-c", "ocis init || true; ocis server"]
    environment:
      OCIS_URL: ${OCIS_URL}
      OCIS_EXCLUDE_RUN_SERVICES: "idp"
      OCIS_LOG_LEVEL: warn
      OCIS_LOG_COLOR: "true"
      OCIS_LOG_PRETTY: "true"
      OCIS_INSECURE: "false"
      DEMO_USERS: "false"
      PROXY_TLS: "false"
      OCIS_LOG_TIMEZONE: ${TZ}
      OCIS_SERVICE_ACCOUNT_ID: ${OCIS_SERVICE_ACCOUNT_ID}
      OCIS_SERVICE_ACCOUNT_SECRET: ${OCIS_SERVICE_ACCOUNT_SECRET}
      OCIS_OIDC_ISSUER: ${OCIS_OIDC_ISSUER}
      WEB_OIDC_CLIENT_ID: ${OCIS_WEB_OIDC_CLIENT_ID}

      PROXY_OIDC_REWRITE_WELLKNOWN: "true"
      PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD: none
      PROXY_AUTOPROVISION_ACCOUNTS: "true"
      PROXY_HTTP_ADDR: ${OCIS_PROXY_HTTP_ADDR}
      PROXY_ENABLE_BASIC_AUTH: "false"
      PROXY_USER_OIDC_CLAIM: "preferred_username"
      PROXY_USER_CS3_CLAIM: "username"
      # email server 
      NOTIFICATIONS_SMTP_HOST: ${NOTIFICATIONS_SMTP_HOST}
      NOTIFICATIONS_SMTP_PORT: ${NOTIFICATIONS_SMTP_PORT} 
      NOTIFICATIONS_SMTP_SENDER: ${NOTIFICATIONS_SMTP_SENDER} 
      NOTIFICATIONS_SMTP_USERNAME: ${NOTIFICATIONS_SMTP_USERNAME}
      NOTIFICATIONS_SMTP_PASSWORD: ${NOTIFICATIONS_SMTP_PASSWORD}
      NOTIFICATIONS_SMTP_INSECURE: "false"
      NOTIFICATIONS_SMTP_AUTHENTICATION: login
      NOTIFICATIONS_SMTP_ENCRYPTION: starttls
      NOTIFICATIONS_EMAIL_TEMPLATE_PATH: ${NOTIFICATIONS_EMAIL_TEMPLATE_PATH}
    expose:
      - 9200
    volumes:
      - ${CONFIGDIR}/ownCloud/config:/etc/ocis
      - ${CLOUDDIR}/owncloud:/var/lib/ocis
    restart: unless-stopped

x
x2128×1274 446 KB

10 
WRN **http** end="21/Mar/2024:12:07:33 +0000" host=127.0.0.1 method=GET pkg=rhttp proto=HTTP/1.1 service=storage-system size=0 start="21/Mar/2024:12:07:33 +0000" status=404 time_ns=398045 traceid=6644c1b9038622e45560a6baff284d1d uri=/data/spaces/jsoncs3-share-manager-metadata%21jsoncs3-share-manager-metadata/users/9767076c-00b0-4124-ba33-1bcaa1a38ec4/received.json url=/users/9767076c-00b0-4124-ba33-1bcaa1a38ec4/received.json

This indicates that your metadata storage on disk was removed during your reset, but ocis still tries to load the shares.

11

This log was not related to any error. Fixed in the current release.