LXC Containers with separate IPs

Posted on Dec 17, 2016

LXC (LinuX Containers) are kind of Chroot Enviroments on Steroids. They allow OS-level virtualization by using cgroups (control groups) provided by the Linux kernel. OS-level virtualization is not as resource-hungry as normal virtualization (there is almost zero overhead) but offers less isolation. For more details check the Wikipedia Page of LXC and OS-level virtualization.

For me personally, LXC is the perfect middle ground between heavy, full-blown VMs and microservices architectures like Docker.

In this post, I want to demonstrate how to run a LXC container with its own, separate IP address. By default, LXC containers run in their own subnet (VLAN). I’m assuming you already have LXC up and running, otherwise check the Debian Wiki and Ubuntu Documentation.

All commands are based off of a Debian Jessie (8) system.

# Network setup on Host

First, we need to modify the network setup of the host. To achieve the goal of having multiple IP addresses on one physical connection, we need to create a bridge. Open and configure /etc/network/interfaces:

# automatically starts the bridge
auto br0
iface br0 inet static
    # address of the host:
    address 10.0.0.200/24
    netmask 255.255.255.0
    # default gateway:
    gateway 10.0.0.1
    # address of the host:
    broadcast	10.0.0.200
    # link back to physical adapter
    bridge_ports eth0
    # some useful settings for bridges:
    bridge_stp off
    bridge_fd 2
    bridge_maxwait 20

Note: This is not supposed to be appended. You need to replace the iface eth0 inet ... part with the above. Defining both eth0 and br0 at the same time will NOT work.

We just create a bridge called br0 which links back to the physical hardware adapter eth0. The Host is still using same IP like with the normal configuration, but now multiple IP addresses can be used over this connection. You could also specify multiple bridge_ports (i.e. physical connections) to be used by the bridge.

Next up, restart the networking (systemctl restart networking) and check for any errors (systemctl status networking).

# Configure LXC

We’ll create a new config file for each guest.

$ sensible-editor lxc1.conf
lxc.network.type = veth # virtual ethernet
lxc.network.flags = up
lxc.network.link = br0	# adapter to use
lxc.network.ipv4 = 10.0.0.201/24 # IPv4 Address, this is the first client
lxc.network.ipv4.gateway = 10.0.0.1 # Gateway of the Host

For more settings have a look into /usr/share/doc/lxc/examples.

# Create the Container

We now have everything set. We just need to create the container. I usually do the following for a Debian container:

$ export MIRROR='http://httpredir.debian.org/debian'
$ lxc-create -t debian -f lxc1.conf -n LXC1 -- -r jessie

Options before the double-dash (--) belong to lxc-create, everything after it gets forwarded to the template (in this case /usr/share/lxc/templates/lxc-debian).

lxc-create: -t for template, -f to specify the config file and -n to name the container
lxc-debian: -r for release and environment variable MIRROR for apt download mirror

This will bootstrap a minimal Debian system in our container named ‘LXC1’.

Depending on which template you used for creating the container (-t), you might still need to adapt network config, because some templates use DHCP on the first interface by default.

# Starting the Container

After the container has been installed, we can start it.

$ lxc-start -n LXC1

In case you still need to modify the networking inside the container (due to the issue described above), you should configure it now.

For Debian based systems, open up /etc/networking/interfaces/ (inside the container) and change iface eth0 inet dhcp to iface eth0 inet manual. To apply the modifications, again restart networking and check for any errors. Unfortunately, Debian minimal neither features ping nor telnet, but we can use apt-get update to check the internet connection.

Now you have your new LXC container up and running with a separate IP!

# Controlling the Container

This is just a quick overview, for more details refer to man lxc.

By default, the root (/) of the container we created is located at /var/lib/lxc/LXC1/rootfs/. It is possible to make changes in the container from the host (e.g. by editing /var/lib/lxc/LXC1/rootfs/etc/fstab).

Execute the following commands on the Host to:

Start the Container: lxc-start -n LXC1
Stop the Container: lxc-stop -n LXC1 (or shutdown from inside the LXC Guest)
List all Containers: lxc-ls (append -f for ‘fancy’ output)
Show Information about one Container: lxc-info -n LXC1
Destroy (Remove) a Container: lxc-destroy -n LXC1
Connect to a (already running) Container: lxc-attach -n LXC1

Modern versions of LXC also have the possibility to automatically start containers after boot. To do so, append the following line to /var/lib/lxc/LXC1/config:

lxc.start.auto = 1

# Conclusion

If you have successfully configured the first LXC Container, you can set up another one (but don’t forget to change the IP in lxc.conf). Of course, to only access the machine over the local network, the container does not need to have its own IP address (this could also be done with e.g. Port Forwarding). However, there are some use cases where this is very useful.

Jack Henschel's Blog