tl;dr: This turned longer than I expected, because I added some history. In a single picture Nexus is your gateway to this:
In the past seven years I have mostly been working on consulting projects or in support. Debugging different flavors of the same underlying problems again and again. Performance bottlenecks because of suboptimal SQL queries leading to database inconsistencies in the file cache. We internally started inventing code names like “Horracle” (feel free to blame me … I introduced oracle support) or “clusterfck” for Galera cluster replication problems. I guess you could say we learned a lot about database indexes, row and table locking and performance in general. And while ownCloud has come a long way in the past seven years, the root problem has not gone away: we still keep metadata in sync in a request driven LAMP stack.
“The definition of insanity is doing the same thing over and over again, but expecting different results” - Albert Einstein
We need to change. But how?
On the one hand, we should be able to embrace redis more and build a proper work queue or use something like laravel horizon to decouple potentially long running queries from the frontend. On the other hand, there have been projects to use the filesystem directly to store all metadata: including shares, tags and comments:
- we did our own experiments
- CERN had built cernbox on top of eos. They constantly pushed us to introduce APIs that would allow replacing the storage and sharing mechanism in a cleaner way in 2015.
There was however a mantra that had been instilled in us for a long time: everything must be compatible with LAMP first, performance optimizations could leave that realm.
So, sadly, we never really followed up on the CERN approach of using golang to replace the storage layer. We would have to make tiny steps in other directions first, to see the bigger picture. Some core server developers leaving, actually allowed us to think outside the box and take these first steps.
On the server side, I needed a different language to allow ownCloud to use the pkcs11 stack to efficiently access an HSM. PHP has no bindings for it. Stackoverflow is clueless. In theory, it should be possible to use the openssl bindings to use the pkcs11 engine … however that has been a known issue since 2013. A good excuse to try this golang thing and implement a small persistent deamon for it.
On the client side, we wanted to have a web fronted that used the same APIs as the desktop and mobile clients and in March 2018 had built Phoenix, a complete rewrite of the web front end in vue.js.
Furthermore, ownCloud had gotten more people on board that came with extensive golang knowledge.
Our mindset was shifting.
Probably half a year ago, the next opportunity to reevaluate the storage layer presented itself. While I had a look at the different async php implementations, as well as, swoole I realized that the best bet would be a real systems language for the storage layer. There was simply no denying it. In early 2019 CERN open sourced reva, a golang implementation of the storage layer used for cernbox. That was a huge piece of the puzzle and the bigger picture became a lot clearer.
Granted, we still have to iterate on some of the CS3 APIs, implement missing components and really make it fit the ownCloud requirements. But it is a great start.
No, let me rephrase that: this new freedom is intoxicating! Endless possibilities!
Finally, we can think about keeping open connections to all desktop clients, allowing real push notifications: instant sync. By storing metadata directly in the file system we completely eliminate the support cases for database inconsistencies. The underlying EOS storage is open source as well, is designed for High-Capacity and Low-Latency, even supports geo-replication and erasure encoding. ownCloud, the platform, will be able to embrace apps in all languages by integrating them in the web frontend.
To get this started I needed a place where we can integrate different services and components. Initially, I just wanted it to be the starting point for related development efforts, but I think it makes sense to also help admins set up and maintain an ownCloud instance in production. Something like a gateway to the ownCloud universe. Nexus quickly came to mind:
Etymology: From Latin nexus (“the act of binding together; bond”)
Have a look at the top picture, again. What do we actually see?
-
Caddy is used as a reverse proxy that automatically generates a certificate and makes tying the different web endpoints together really easy.
-
Phoenix is the new ownCloud web ui.
-
reva is the new storage layer. It might make sense to think of it as a runtime for go services that communicate vie CS3 apis. Nexus will start three instances:
-
an ocdavsvc that serves the webdav API as well as sharing and other OCP APIs
-
an authsvc that uses OpenID Connect to authenticate users backed by an LDAP server
-
a storageprovider the encapsualtes EOS and presents a CS3 api for it. A local posix storage is also implemented but needs more love. S3 and windows network drive are also planned. If you want ownCloud to access a storage you wrap it in a storage provider.
-
we are currently working on integrating a storage registry that knows the mount points of different storages in the tree of a user
-
-
EOS as the default storage solution, because it natively implements some of the properties we need for ownCloud, eg. mtime propagation.
-
Kopano Konnect as the OpenID Connect provider which is backed by an ldap server.
-
OpenLDAP as a central user database.
Some of the services can be replaced to integrate better with your environment, eg. the OIDC provider, the LDAP server or the actual storage. For now this is what we focus on.
You may have noticed that half of the services are written in golang. So if you are interested in that language, now is probably the time to get involved and learn about the different components. The Nexus is supposed to help everyone get started with this new architecture. Just do a
$ git clone git@github.com:owncloud/nexus.git
$ cd nexus && make future
That will use docker compose to bring up all the above services. Then point your browser to https://owncloud.localhost:8443/ and log in with aaliyah_abernathy:secret.
Welcome to the future!
I should probably do a deep dive into some of the topics and elaborate further, eg. why golang or a systems language makes more sense than PHP, but that will have to wait for another time. Let me know what you think and post any questions below.
I’ll try to keep it shorter next time. Unless you want me to tell you some war stories from the support battlefield about Horracle and clusterfck …
Cheers!