Replication documentation error

[speacock]

There are problems using the replication instructions as they are

The part about creating the authorized keys appears wrong. For me, the method suggested did not work.

The text suggests copying the contents of /data/ssh/replication.pub on LOCAL to a file, then using ssh-keyscan to copy the key of REMOTE to another file, then pasting the keys into the required authorized_key file.

However, when I used this technique, replication did not work and the testing method described for testing correct ssh set-up also failed, prompting for a password every time. The method I found to work was as follows, do not try to shortcut these steps, you must do them in this sequence for set-up to succeed, you cannot, for instance, do all the LOCAL steps in one go and then do all the REMOTE steps in one go, you have to keep switching back and forward, so make sure you keep a note of where you're up to in the process:

Instructions for setting up ssh key-based authentication

Follow the steps in the existing instructions to mount the root filesystem (/) as RW rather than RO

on LOCAL:

1. ssh into LOCAL using your newly created ssh user
2. su (otherwise you wont have permission to create the file)
3. cat /data/ssh/replication.pub>authorized_keys.LOCAL

on REMOTE:

1. ssh into REMOTE using your newly created ssh user
2. su (otherwise you wont have permission to create the file)
3. cat /data/ssh/replication.pub>authorized_keys.REMOTE

on LOCAL:
create the file /root/.ssh/authorized_keys and paste the contents of authorized_keys.REMOTE into it,
or alternatively, simply copy authorized_keys.REMOTE as authorized_keys into /root/.ssh/

on REMOTE:
create the file /root/.ssh/authorized_keys and paste the contents of authorized_keys.LOCAL into it,
or alternatively, simply copy authorized_keys.LOCAL as authorized_keys into /root/.ssh/

on LOCAL:

1. ssh to REMOTE

NOTE: you will be prompted to answer a question about authentication - answer yes and note that the system reports that it has created the known_hosts file, if it cannot create it, then you have not done things in the right sequence as you have not used su

on REMOTE:

1. ssh to LOCAL

NOTE: you will be prompted to answer a question about authentication - answer yes and note that the system reports that it has created the known_hosts file, if it cannot create it, then you have not done things in the right sequence as you have not used su

When creating your replication task in the GUI, the key that you should paste in is the one from authorized_keys.REMOTE

[speacock]

I'm happy to make the documentation changes to the Wiki, but wanted to confirm that my problem was to do with the documentation being incorrect rather than a specific problem with my configuration

[jgreco]

The whole thing seemed a bit screwed up. If you've set up ssh key auth before, it's not hard to do, the sticking point seems to be that you _must_ get that known_hosts file created while the root filesystem is mounted rw. The keys do not need to be bidirectional as far as I can tell, which seemed to be where the instructions started getting a bit odd.

I've already got another bug open about SSH port numbering from the replication config dialog.

The other point that seems bad is that things are dependent on login as root; it'd be nice to have a different userid used for SSH. In the FreeBSD world this is typically done with a wrapper to provide suid access to the desired executable. Absent that, at a minimum the sshd configuration should include a restriction that only allows you to run the one executable needed, otherwise someone who breaks into your FreeNAS server also gets free root access on your other FreeNAS server.

[dlavigne]

I don't see where this set of instructions differs from that in the documentation (other than using cat instead of cut/paste).

[jgreco]

What's needed is for someone unfamiliar with the process to walk through the instructions and point out the deficiencies.

For example, this bit got me:

"1. If you haven't done so already, create on LOCAL a user account which will be used to ssh into LOCAL."

Well, that's fine, but what's the INTENT here? It doesn't say that this has nothing at all to do with the replication system, and is actually an administrator's account. First, you don't *need* to do this if you're configuring things from the console (hi, virtualization console). Second, it says a "user" account, which contributes further to that initial misunderstanding. It then says "make a similar account on remote." Which makes it sound kind of like these accounts are actually needed for the replication system, when in fact the replication system is just using root and these can be any old user accounts that you might have already created... which, yes, it DOES say, but only in hindsight after you've gone through the process.

The instructions, as they exist, are generally usable but a bit terse. Is there anything that we can do to help correct this? I don't mind submitting some revisions myself, I know where *my* questions were - and I have done this stuff semi-routinely for many years, so I can just imagine how opaque this must be to a newbie.

[jgreco]

By the way, what's the attitude towards adding executables to the system? I'd be happy to contribute a very short little suid wrapper designed to prevent some badguy from doing

% ssh backuphost rm -fr /

or even

% ssh backup host zfs destroy pool

[gcooper]

This really isn't required with later versions of FreeNAS as one can plug in the SSH key for replication via the GUI in the Storage -> Replication Tasks -> Add Replication Task pane, and one can grab the replication key from Storage -> Replication Tasks -> View All Replication Tasks -> View Public Key. I would argue that this is a non-intuitive location for one to get the pubkey from, but it works.

[jgreco]

Well, that sounds like a heck of a better way to get this set up. Any chance that we can get a suid wrapper included? Offer still stands, I've got one we use that would be trivially modified for the purpose.

[gcooper]

Replying to jgreco:

Well, that sounds like a heck of a better way to get this set up. Any chance that we can get a suid wrapper included? Offer still stands, I've got one we use that would be trivially modified for the purpose.

Eh, not really because FreeBSD doesn't support setuid scripts.. unless you had something else in mind...

[jgreco]

FreeBSD supports suid wrappers just fine. You cannot make a script suid, but taking a small purpose-built C app to front-end it is done easily enough. Those of us in the service provider biz do stuff like this all the time; we do not like the idea of ssh'ing around as root, or unrestricted shell access, so for example our configuration pusher sits on a centralized server, and then executes commands like

ssh remoteserver -l configuser confmgr update file

where "configuser" is a non-root user on the remote host, with an ssh key, and usually a "command=foo" stanza in authorized_keys, which forces a particular binary to be run -- our wrapper. The wrapper then executes, and being a suid root binary, is then uid 0, and has an environment variable containing the user's requested command. This then gets carefully verified and sanitized. If it passes the sniff test, then it can be safely taken care of, or passed on to a script, or whatever you can imagine.

The point of this is that as it stands, replication leaves the target replication system open to remote root logins from the source replication system, so an attacker who gains control over one can trivially gain control over the other. A properly implemented suid wrapper will take care of that problem, and it isn't even that much code. Again, we have one that we use here in many variants, and I'm willing to take some time to hack together a version that should be pretty easy to drop in. This assumes that there's some easy way to add new C executables to the system as part of the build, which I haven't tried since I don't have a build env for FreeNAS set up.

[dlavigne]

Things appear to be working in 8.2beta4 and the docs have been updated accordingly. If you get a chance to test before 8.2-RELEASE, please indicate whether or not the documented procedure works for you.

[dlavigne]

Closing ticket as keys are now available in the GUI and docs are up-to-date.