Mailing List Archive

dm-crypt on top of DRBD for live migration
We want to use LVM, dm-crypt and DRBD in a 2-machine setup for KVM.

We think, a proper setup could be something like this (dm-crypt below DRBD):


   Machine 1               Machine 2

      KVM  -> -> -> -> -> ->  KVM
       |   (live migration)    .
       |                       .
      DRBD - - - - - - - - - DRBD
       |                       |
      LVM                     LVM
       |                       |
    dm-crypt                dm-crypt
       |                       |
 Disk/Partition          Disk/Partition

The KVM guest machines should run on machine 1. Live migration to
machine 2 should be supported.

Using this setup, every write to DRBD would be (independently) crypted
on both machines,
leading to additional (unnecessary?) cpu load on machine 2 before live
migrating, and additional
cpu load on machine 1 after live migration.

Could these additional cpu loads be avoided using a setup like this
(dm-crypt in top of DRBD):


   Machine 1               Machine 2

      KVM  -> -> -> -> -> ->  KVM
       |   (live migration)    .
       |                       .(b)
    dm-crypt                dm-crypt
       |                       |(a)
      DRBD - - - - - - - - - DRBD
       |                       |
      LVM                     LVM
       |                       |
 Disk/Partition          Disk/Partition

In this setup, dm-crypt runs on both machines, too, but is not used on
machine 2 until KVM
guests send write-requests after the live migration. So crypting is
done only by one machine
at every time point.

Is such a setup safe and stable?

What about caching at points (a) or (b) on machine 2?
Can KVM read cached, outdated data from dm-crypt after live migration?

Is there a workaround?

Thank You
B. Lehr & M. Müller

--
Mate ist gesunder Schlaf in Halbliterflaschen
_______________________________________________
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
Re: dm-crypt on top of DRBD for live migration [ In reply to ]
Hello Berengar,

On 12/07/2011 01:30 PM, Berengar Lehr wrote:
> We want to use LVM, dm-crypt and DRBD in a 2-machine setup for KVM.
>
> We think, a proper setup could be something like this (dm-crypt below DRBD):
>
>
> Machine 1 Machine 2
>
> KVM -> -> -> -> -> -> KVM
> | (live migration) .
> | .
> DRBD - - - - - - - - - DRBD
> | |
> LVM LVM
> | |
> dm-crypt dm-crypt
> | |
> Disk/Partition Disk/Partition
>
> The KVM guest machines should run on machine 1. Live migration to
> machine 2 should be supported.
>
> Using this setup, every write to DRBD would be (independently) crypted
> on both machines,
> leading to additional (unnecessary?) cpu load on machine 2 before live
> migrating, and additional
> cpu load on machine 1 after live migration.

We are successfully using exactly this setup ... I describe it in
another post: http://www.gossamer-threads.com/lists/drbd/users/22383#22383

>
> Could these additional cpu loads be avoided using a setup like this
> (dm-crypt in top of DRBD):
>
>
> Machine 1 Machine 2
>
> KVM -> -> -> -> -> -> KVM
> | (live migration) .
> | .(b)
> dm-crypt dm-crypt
> | |(a)
> DRBD - - - - - - - - - DRBD
> | |
> LVM LVM
> | |
> Disk/Partition Disk/Partition
>
> In this setup, dm-crypt runs on both machines, too, but is not used on
> machine 2 until KVM
> guests send write-requests after the live migration. So crypting is
> done only by one machine
> at every time point.
>
> Is such a setup safe and stable?

Looks sane, but never tried for practical reasons: you need to run
cryptsetup for every volume after promoting its underlying DRBD device.
Might be tedious work if you use one device per VM.

To automate this -- e.g. to include this in your Pacemaker HA cluster
setup -- you could use cryptsetup with a keyfile ... the question is for
what reason you want to encrypt your data, as the key needs to be
available on the server or at least on an attached device .... Maybe
someone has a better idea here.

In the first setup you only ever need to run cryptsetup to activate the
PV for your data VG after every reboot. So this is hopefully a rare case.

>
> What about caching at points (a) or (b) on machine 2?
> Can KVM read cached, outdated data from dm-crypt after live migration?

Flushing all virtual-disk caches during live-migrating is the
responsibility of qemu AFAIK, so I don't expect problems here with
another dm-crypt layer ... please someone correct me who knows more
details on that.

Regards,
Andreas

--
Need help with DRBD?
http://www.hastexo.com/now

>
> Is there a workaround?
>
> Thank You
> B. Lehr & M. Müller
>
> --
> Mate ist gesunder Schlaf in Halbliterflaschen
> _______________________________________________
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user