fr33.info

User Tools

Site Tools

linux:virtualization:lxc

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Last revision Both sides next revision
linux:virtualization:lxc [2016/10/02 00:44]
tkilla [Unprivileged containers] 		linux:virtualization:lxc [2022/01/13 23:06]
tkilla [Create new container]
Line 86: Line 86:
 **reload:** **reload:**
   sysctl -p /etc/sysctl.conf     sysctl -p /etc/sysctl.conf  
 +
 +
 +
 +
 +
 +\\
 +==== Simple Nat Bridge ====
 +
 +Easy version without libvirt - work well at OVH and hetzner. 
 +
 +**Add an additional bridge (keep eth0 as is) in /etc/network/interfaces**
 +
 +  auto lxc-bridge
 +  iface lxc-bridge inet static
 +        bridge_ports none
 +        bridge_fd 0
 +        bridge_maxwait 0
 +        address 192.168.10.1
 +        netmask 255.255.255.0
 +        up iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
 +
 +
 +**activate forwarding temporary:**
 +  echo 1 > /proc/sys/net/ipv4/ip_forward
 +
 +**activate forwarding permanent:**
 +
 +Uncomment in /etc/sysctl.conf
 +  net.ipv4.ip_forward=1
 +
 +Activate new settings:
 +  sysctl -p
 +
 +
 +**firewall rules**
 +
 +  # intern -> extern
 +  iptables -t nat -A POSTROUTING -s 192.168.10.10/24 -j SNAT --to-source 1.2.3.4
 +
 +  # ports extern -> intern - 1 rule for each $PORT
 +  iptables -t nat -A PREROUTING  -d 1.2.3.4 -p tcp --dport $PORT -j DNAT --to 192.168.10.10:${PORT}
 +
 +
 +**Container config:**
 +  lxc.network.type = veth
 +  lxc.network.flags = up
 +  lxc.network.link = lxc-bridge
 +  lxc.network.ipv4.gateway = 192.168.10.1
 +  lxc.network.ipv4 = 192.168.10.10/24
 +
 +
 +
 +
 +  * https://wiki.debian.org/LXC/SimpleBridge
 +
  
  
 \\ \\
 ==== Nat Bridge via libvirt and IPV6 ==== ==== Nat Bridge via libvirt and IPV6 ====
 +
 +**Deprecated! - if you not need DHCP, use the simple bridge method without libvirt/virsh!**
  
   apt-get install libvirt-bin   apt-get install libvirt-bin
Line 119: Line 176:
     </ip>     </ip>
     <!-- IPV6 :: -->     <!-- IPV6 :: -->
-   <ip family='ipv6' address='2001:41d0:2:bb10::2' prefix='64'>+   <ip family='ipv6' address='1234:1234:1234:1234::2' prefix='64'>
    </ip>         </ip>     
   </network>   </network>
Line 128: Line 185:
   lxc.network.flags = up   lxc.network.flags = up
   lxc.network.link = virbr0   lxc.network.link = virbr0
-  lxc.network.hwaddr = CC:AA:FF:EE:00:01   # not required+  # lxc.network.hwaddr = CC:AA:FF:EE:00:01   # not required and not useful: inter-vs-connection breaks whene set
   lxc.network.ipv4 = 192.168.122.100/24   lxc.network.ipv4 = 192.168.122.100/24
   lxc.network.ipv4.gateway = auto   # auto usually works, otherwise set main IP gateway (.254 at OVH)   lxc.network.ipv4.gateway = auto   # auto usually works, otherwise set main IP gateway (.254 at OVH)
  
   # ipv6:   # ipv6:
-  lxc.network.ipv6 = 2001:41d0:0002:bb10:0000:0000:0000:0100/64+  lxc.network.ipv6 = 1234:1234:1234:1234:0100/64
   lxc.network.ipv6.gateway = auto   lxc.network.ipv6.gateway = auto
  
Line 151: Line 208:
   ## IPv6:   ## IPv6:
   iface eth0 inet6 static   iface eth0 inet6 static
-    address 2001:41d0:2:bb10::131+    address 1234:1234:1234:1234::131
     netmask 64     netmask 64
  
Line 167: Line 224:
   virsh net-destroy default   virsh net-destroy default
   virsh net-start default   virsh net-start default
 +
 +**if you remove this net, disable autostart**
 +  virsh net-autostart default --disable
 +
  
 **activate forwarding temporary:** **activate forwarding temporary:**
Line 175: Line 236:
 Uncomment in /etc/sysctl.conf Uncomment in /etc/sysctl.conf
   net.ipv4.ip_forward=1   net.ipv4.ip_forward=1
 +
 +Activate new settings:
 +  sysctl -p
  
 **iptables config ( 1.2.3.4 is pubilc ip in root):** **iptables config ( 1.2.3.4 is pubilc ip in root):**
Line 200: Line 264:
  
   echo "IPv6 setup FOREACH vserver.."   echo "IPv6 setup FOREACH vserver.."
-  ip -6 route add 2001:41d0:2:bb10::100 dev virbr0 +  ip -6 route add 1234:1234:1234:1234::100 dev virbr0 
-  ip -6 neigh add proxy 2001:41d0:2:bb10::100 dev eth0 +  ip -6 neigh add proxy 1234:1234:1234:1234::100 dev eth0 
-  ping6 -I virbr0 -c 5 2001:41d0:2:bb10::100+  ping6 -I virbr0 -c 5 1234:1234:1234:1234::100
  
 \\ \\
Line 280: Line 344:
 http://wiki.fr33.info/doku.php/linux/virtualization/lxc?&#unprivileged_containers http://wiki.fr33.info/doku.php/linux/virtualization/lxc?&#unprivileged_containers
  
-  lxc-create -n debian8  -B btrfs -t debian -- -r jessie+FIX:  original keyserver is broken! add: --keyserver hkp://keyserver.ubuntu.com 
 + 
 + 
 +  lxc-create -n debian8  -B btrfs -t debian -- -r jessie --keyserver hkp://keyserver.ubuntu.com
  
 or or
  
-  lxc-create -n websrv -t debian-wheezy  -B btrfs+  lxc-create -n websrv -t debian-wheezy  -B btrfs --keyserver hkp://keyserver.ubuntu.com
  
 Start / Stop VS: Start / Stop VS:
Line 292: Line 359:
 Enter VS: Enter VS:
   lxc-console -n websrv   lxc-console -n websrv
 +
 +
 +In Buster, use the lxc-download script:
 +
 +  /usr/share/lxc/templates/lxc-download --list --no-validate| grep debian | grep amd64
 +  lxc-create -t /usr/share/lxc/templates/lxc-download -n <NAME> --  --no-validate -d debian -r buster -a amd64
  
  
Line 302: Line 375:
   lxc-clone --backingstore btrfs --orig vs1 --new vs2 --snapshot   lxc-clone --backingstore btrfs --orig vs1 --new vs2 --snapshot
  
 +\\
 +===== Mount external Dirs in Container =====
 +
 +The recommended way is to add the mountpoint with a relative path in the VS config:
 +
 +  lxc.mount.entry=/home/mountme home none bind,optional,relative,create=dir
 +
 +
 +Under some cicumstances it does not work (in unprivileged containers), but this works:
 +
 +  lxc.mount.entry = /home/test /home/vservers/stretch/rootfs/home/test none bind 0 0
 +
 +Also check Permissions and Ownership. chown to the root ID inside the container.
  
 \\ \\
 ===== brtfs snapshots ===== ===== brtfs snapshots =====
 +
 +the container must be stopped for a lxc-snapshot. use btrfs snapshot to backup running containers (mysql may get inconsitent)
  
 you need to create container with option  -B btrfs!! you need to create container with option  -B btrfs!!
  
   lxc-create -B btrfs -n mycontainer -t ubuntu   lxc-create -B btrfs -n mycontainer -t ubuntu
 +
 +
  
  
Line 314: Line 404:
  
   mv /home/vservers/my-lxc-container/rootfs /home/vservers/my-lxc-container/rootfs.saved   mv /home/vservers/my-lxc-container/rootfs /home/vservers/my-lxc-container/rootfs.saved
-  btrfs subvolume create /home/vservers/my-lxc-container/rootfs+  btrfs subvolume create /home/vservers/my-lxc-container/rootfs  
   btrfs subvolume list /home/vservers   btrfs subvolume list /home/vservers
-  lxc-snapshot -n webdev 
      
   # for unprivileged root container, check UID and GID of rootfs dir (here it is 100000):   # for unprivileged root container, check UID and GID of rootfs dir (here it is 100000):
   chown 100000:100000 /home/vservers/webdev/rootfs/   chown 100000:100000 /home/vservers/webdev/rootfs/
  
 +  mv /home/vservers/my-lxc-container/rootfs.saved/* /home/vservers/my-lxc-container/rootfs/
 +  lxc-snapshot -n webdev
 +
 +snapshot with comment
 +
 +  echo "working my-lxc-container before ..." > snap-comment
 +  lxc-stop -n my-lxc-container
 +  lxc-snapshot -n my-lxc-container -c snap-comment
 +  rm snap-comment
  
   * https://uli-heller.github.io/blog/2013/06/09/lxc-snapshots/   * https://uli-heller.github.io/blog/2013/06/09/lxc-snapshots/
Line 363: Line 461:
   lxc.id_map = g 0 100000 65536   lxc.id_map = g 0 100000 65536
  
 +in buster it's called idmap:
 +  lxc.idmap = u 0 100000 65536
 +  lxc.idmap = g 0 100000 65536
  
 **shift uuids to another span:** **shift uuids to another span:**
  
-  for i in `seq 0 65535`; do +Use this script: https://github.com/exaexa/chownmap 
-    find ${LXC_BASEDIR}${1}/rootfs -uid $i -exec chown $(($OFFSET+i)) \{\} \; + 
-    find ${LXC_BASEDIR}${1}/rootfs -gid $i -exec chgrp $(($OFFSET+i)) \{\} \; +  /root/bin/chownmap 0 200000 65536 /home/vservers/<containername>/rootfs/
-  done+
  
-complete script: {{:linux:virtualization:lxc-convert-unprivileged.txt|}} 
  
  
Line 461: Line 560:
  
   * https://github.com/debops/ansible-lxc/issues/15   * https://github.com/debops/ansible-lxc/issues/15
 +
 +** systemd cgroups fuckup**
 +
 +Could not find writable mount point for cgroup hierarchy 12 while trying to create cgroup
 +
 +12 is a systemd hierarchy - if you remove systemd and switch to sysvinit-core, this might be leftover.
 +
 +FIXME:
 +
 +check all of systemd is gone (uninstall ii):
 +  dpkg -l *systemd*
 +  apt remove --purge *systemd*    # without systemd apt/preferences.d/ must not be set
 +  
 +/etc/pam.d/common-session - unset this line:
 +
 +  session     optional    pam_cgfs.so -c freezer,memory,name=systemd
 +
 +Check, if 12 is still active:
 +
 +  cat /proc/self/cgroup
 +
 +WORKAROUND:
 +mcedit /etc/lxc/lxc.conf and remove 
 +  lxc.cgroup.use = @all
 +
 +* this is helpful: https://github.com/lxc/lxc/issues/1279
 +* this is not: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=769494
 +
 +\\
  
 **SSH Config** **SSH Config**
Line 477: Line 605:
  
   * http://gaijin-nippon.blogspot.de/2013/07/audit-on-lxc-host.html   * http://gaijin-nippon.blogspot.de/2013/07/audit-on-lxc-host.html
 +
 +or run this inside the container:
 +
 +  sed '/pam_loginuid.so/s/^/#/g' -i /etc/pam.d/* 
  
 FIXME maybe insecure FIXME maybe insecure
Line 489: Line 621:
  
  
 +**rsyslog error**
 +
 +TESTME
 +
 +rsyslog doesnt start on boot and errors in syslog:
 +  .. rsyslogd: imklog: cannot open kernel log(/proc/kmsg): Permission denied.
 +  .. rsyslogd-2145: activation of module imklog failed [try http://www.rsyslog.com/e/2145 ]
 +
 +Disable kernel logging in container /etc/rsyslog.conf:
 +
 +  # $ModLoad imklog   # provides kernel logging support
 +
 +
 +
 +\\
 +===== Move a root system into container =====
 +
 +you can disable many services:
 +  * udev: udev service (which is a hard dependency of systemd in Jessie) won't run in a container, but systemd recognized it
 +  * apparmor: mounts security-fs, would need to disable drop caps. bad idea
 +  * kmod
 +  * lm-sensors
 +  * dbus
 +  * kbd
 +  * hdparm
 +  * ...
 +
 +configuration changes
 +
 +  * various sysctl.conf options do not work inside container
 +  * /etc/modules -loading do not work
 + 
 +If you get errors like:
 +  INIT: Id "6" respawning too fast: disabled for 5 minutes 
 +
 +disable the matching line in /etc/inittab:
 +  # 5:23:respawn:/sbin/getty 38400 tty5
  
linux/virtualization/lxc.txt · Last modified: 2022/01/13 23:08 by tkilla

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki