Need to migrate OC installations (export and import)


#1

I have a few OC servers I need to upgrade to 9.1. For various reasons, these will not be in-place upgrades, but migrations to new servers, so ideally, I just need to migrate the user accounts and data. The new OC server (9.1.5 running on CentOS 7) is built, and I'm happy with the setup. We're only using OC as a web based file transfer server, so no clients and no 3rd party plugins.

If I just want to migrate users and data, what's the process? I'd be happy to use occ or mysqldump to do the export/import if possible. The versions of OC I've got to migrate to 9.1 are 7.0.4, 8.0.15, 8.0.16 and 8.2.11. Note, these will not be merged - each will have it's own installation.

Thanks


A couple of questions
#2

I think the needed migration steps depends on if you don't want to re-create your users and don't want to loose all shares (not the data itself) or not.


#3

You could migrate all setups to your new server and then run regular upgrades, this way you keep data, user settings, shares, ....
7.0.4 -> 7.0.x (latest: https://owncloud.org/changelog/#latest7) -> 8.0.x -> 8.1.x -> 8.2.x -> 9.0.x -> 9.1.x

You could create new setups, creating users won't be such a problem. If you put back data, the clients will have to resync all data and meta information will be lost (shares, app-data, ...).


#4

Thanks for the suggestions. I was hoping that, as I'm migrating to a new installation, I could just do a mysqldump of the Users from the existing installation, and import them into the new setup, then I could just rsync the data files across to the new server. Unless there are substantial differences between the mysql format, or the data repository, would this not work?


#5

It's a bit messy, but I have a partial solution, but some things aren't working still.

If you do a mysqldump of the existing owncloud database, you can then run sed against the output file to export the following tables;
oc_users
oc_group_user
oc_group_admin
oc_groups

Example;
sed -n -e '/DROP TABLE.*'oc_groups'/,/UNLOCK TABLES/p' owncloud-sqlbkp_20170630.bak > owncloud-oc_groups.sql

If you've already created an empty (new) OC database on a new server, you can them import these tables into the new installation. A quick refresh, and you'll see all the user accounts and groups intact. Things I haven't worked out yet are;
Passwords: The passwords are in the oc_users sql table, but they don't seem to import properly. Could just be the difference between the older OC versions and 9.1.5. These can be reset through the UI, or using occ.
Quotas: No quotas are carried over, so need to be re-set

Once the user accounts are across, you can then scp the files from old to new server, then run the occ command against them to re-scan/re-fresh the OC file ownership information (make sure your chown all the data tree to the owner of your web server first - in my case 'apache')
sudo -u apache php /var/www/html/owncloud/occ files:scan --all

This sort of works, but it could do with the fine tuning. It would be great if passwords could be carried over, as on our main OC server we have over 100 accounts.

Work in progress, but it would be good if there was an official way that didn't involve 15 intermediate upgrades!


#6

I wouldn't start with manual changes of the database. Between versions, they do change some things and if you are not absolutely sure about what you are doing you are risking to break something.

You can create new users with the occ-command, this way you can set up all users automatically via a small script:
https://doc.owncloud.org/server/latest/admin_manual/configuration_server/occ_command.html?highlight=occ#user-commands-label

Regarding the passwords, you could ask the users to set a new password which you should do anyway from time to time (good passwords: https://uit.stanford.edu/service/accounts/passwords/quickguide).
If you absolutely want to keep all the passwords, you could just update the password hashes from the old database. I don't know exactly what is part of the hash, the owncloud-id and some salts in your config.php might be considered. Also keep in mind that this could have changed between OC 7.0 and OC 9.1.

Also keep in mind that you can play back all data to each user, but when you recreate the oc_filecache-table, the clients will have to sync again all data!

And if a responsible admin didn't wait for 2.5 years (release date of 7.0.4) before thinking about important security updates (the 7.0.x-branch is end of life for over 1 year!), he would run OC 9.0.11 (lowest supported version) and an upgrade to OC 9.1 would be easy :wink:


#7

Just a note: With 10.0 we released a skip upgrade migration for the first time. It works from 9.0.x to 10.0.x and with 10.0.3 a direct migration from 8.2.x is also in the pipeline ...


#8

I appreciate that OC has a rapid development cycle, but it seems that there are major version upgrades every few months, and keeping up with that in a production system is difficult. I do the smaller minor version upgrades periodically, but on busy systems, even that's difficult. If it works, we generally don't mess with it. Our main concern is security, and that's more likely to force upgrades than anything else.

These upgrades are being forced on us for security reasons now, and an in place upgrade isn't an option because we have to move from CentOS 6 to 7, so it's a brand new VM, which we then need to migrate all the user accounts, passwords and data to. If there's an easy way to do that, then I'm listening.


#9

Nextcloud offers for enterprise users an option to support a specific version up to 10 years. Perhaps you can have this with ownCloud as well. Currently you need to make a major upgrade at least once a year.