Container Linux with systemd
This is an adaptation of a page previously found in the Platforms section of the documentation which described etcd deployments on various platform services. The original page was authored by Gyuho Lee.
The following blog post shows how to run etcd with systemd under Container Linux.
Provisioning an etcd cluster
Cluster bootstrapping in Container Linux is simplest with Ignition; coreos-metadata.service dynamically fetches the machine’s IP for discovery. Note that etcd’s discovery service protocol is only meant for bootstrapping, and cannot be used with runtime reconfiguration or cluster monitoring.
The Container Linux Config Transpiler compiles etcd configuration files into Ignition configuration files:
etcd:
version: 3.2.0
name: s1
data_dir: /var/lib/etcd
advertise_client_urls: http://{PUBLIC_IPV4}:2379
initial_advertise_peer_urls: http://{PRIVATE_IPV4}:2380
listen_client_urls: http://0.0.0.0:2379
listen_peer_urls: http://{PRIVATE_IPV4}:2380
discovery: https://discovery.etcd.io/<token>
ct would produce the following Ignition Config:
$ ct --platform=gce --in-file /tmp/ct-etcd.cnf
{"ignition":{"version":"2.0.0","config"...
{
"ignition":{"version":"2.0.0","config":{}},
"storage":{},
"systemd":{
"units":[{
"name":"etcd-member.service",
"enable":true,
"dropins":[{
"name":"20-clct-etcd-member.conf",
"contents":"[Unit]\nRequires=coreos-metadata.service\nAfter=coreos-metadata.service\n\n[Service]\nEnvironmentFile=/run/metadata/coreos\nEnvironment=\"ETCD_IMAGE_TAG=v3.1.8\"\nExecStart=\nExecStart=/usr/lib/coreos/etcd-wrapper $ETCD_OPTS \\\n --name=\"s1\" \\\n --data-dir=\"/var/lib/etcd\" \\\n --listen-peer-urls=\"http://${COREOS_GCE_IP_LOCAL_0}:2380\" \\\n --listen-client-urls=\"http://0.0.0.0:2379\" \\\n --initial-advertise-peer-urls=\"http://${COREOS_GCE_IP_LOCAL_0}:2380\" \\\n --advertise-client-urls=\"http://${COREOS_GCE_IP_EXTERNAL_0}:2379\" \\\n --discovery=\"https://discovery.etcd.io/\u003ctoken\u003e\""}]}]},
"networkd":{},
"passwd":{}}
To avoid accidental misconfiguration, the transpiler helpfully verifies etcd configurations when generating Ignition files:
etcd:
version: 3.2.0
name: s1
data_dir_x: /var/lib/etcd
advertise_client_urls: http://{PUBLIC_IPV4}:2379
initial_advertise_peer_urls: http://{PRIVATE_IPV4}:2380
listen_client_urls: http://0.0.0.0:2379
listen_peer_urls: http://{PRIVATE_IPV4}:2380
discovery: https://discovery.etcd.io/<token>
$ ct --platform=gce --in-file /tmp/ct-etcd.cnf
warning at line 3, column 2
Config has unrecognized key: data_dir_x
See Container Linux Provisioning for more details.
etcd 3.x service
Container Linux does not include etcd 3.x binaries by default. Different versions of etcd 3.x can be fetched via etcd-member.service.
Confirm unit file exists:
systemctl cat etcd-member.service
Check if the etcd service is running:
systemctl status etcd-member.service
Example systemd drop-in unit to override the default service settings:
cat > /tmp/20-cl-etcd-member.conf <<EOF
[Service]
Environment="ETCD_IMAGE_TAG=v3.2.0"
Environment="ETCD_DATA_DIR=/var/lib/etcd"
Environment="ETCD_SSL_DIR=/etc/ssl/certs"
Environment="ETCD_OPTS=--name s1 \
--listen-client-urls https://10.240.0.1:2379 \
--advertise-client-urls https://10.240.0.1:2379 \
--listen-peer-urls https://10.240.0.1:2380 \
--initial-advertise-peer-urls https://10.240.0.1:2380 \
--initial-cluster s1=https://10.240.0.1:2380,s2=https://10.240.0.2:2380,s3=https://10.240.0.3:2380 \
--initial-cluster-token mytoken \
--initial-cluster-state new \
--client-cert-auth \
--trusted-ca-file /etc/ssl/certs/etcd-root-ca.pem \
--cert-file /etc/ssl/certs/s1.pem \
--key-file /etc/ssl/certs/s1-key.pem \
--peer-client-cert-auth \
--peer-trusted-ca-file /etc/ssl/certs/etcd-root-ca.pem \
--peer-cert-file /etc/ssl/certs/s1.pem \
--peer-key-file /etc/ssl/certs/s1-key.pem \
--auto-compaction-retention 1"
EOF
mv /tmp/20-cl-etcd-member.conf /etc/systemd/system/etcd-member.service.d/20-cl-etcd-member.conf
Or use a Container Linux Config:
systemd:
units:
- name: etcd-member.service
dropins:
- name: conf1.conf
contents: |
[Service]
Environment="ETCD_SSL_DIR=/etc/ssl/certs"
etcd:
version: 3.2.0
name: s1
data_dir: /var/lib/etcd
listen_client_urls: https://0.0.0.0:2379
advertise_client_urls: https://{PUBLIC_IPV4}:2379
listen_peer_urls: https://{PRIVATE_IPV4}:2380
initial_advertise_peer_urls: https://{PRIVATE_IPV4}:2380
initial_cluster: s1=https://{PRIVATE_IPV4}:2380,s2=https://10.240.0.2:2380,s3=https://10.240.0.3:2380
initial_cluster_token: mytoken
initial_cluster_state: new
client_cert_auth: true
trusted_ca_file: /etc/ssl/certs/etcd-root-ca.pem
cert_file: /etc/ssl/certs/s1.pem
key_file: /etc/ssl/certs/s1-key.pem
peer_client_cert_auth: true
peer_trusted_ca_file: /etc/ssl/certs/etcd-root-ca.pem
peer_cert_file: /etc/ssl/certs/s1.pem
peer_key_file: /etc/ssl/certs/s1-key.pem
auto_compaction_retention: 1
$ ct --platform=gce --in-file /tmp/ct-etcd.cnf
{"ignition":{"version":"2.0.0","config"...
To see all runtime drop-in changes for system units:
systemd-delta --type=extended
To enable and start:
systemctl daemon-reload
systemctl enable --now etcd-member.service
To see the logs:
journalctl --unit etcd-member.service --lines 10
To stop and disable the service:
systemctl disable --now etcd-member.service
etcd 2.x service
Container Linux includes a unit file etcd2.service for etcd 2.x, which will be removed in the near future. See Container Linux FAQ for more details.
Confirm unit file is installed:
systemctl cat etcd2.service
Check if the etcd service is running:
systemctl status etcd2.service
To stop and disable:
systemctl disable --now etcd2.service