gRPC naming and discovery

etcd provides a gRPC resolver to support an alternative name system that fetches endpoints from etcd for discovering gRPC services. The underlying mechanism is based on watching updates to keys prefixed with the service name.

Using etcd discovery with go-grpc

The etcd client provides a gRPC resolver for resolving gRPC endpoints with an etcd backend. The resolver is initialized with an etcd client:

import (
	clientv3 "go.etcd.io/etcd/client/v3"
	etcdnaming "go.etcd.io/etcd/client/v3/naming/resolver"

	"google.golang.org/grpc"
)

...

cli, err := clientv3.NewFromURL("http://localhost:2379")
if err != nil {
    // ...
}
r, err := etcdnaming.NewBuilder(cli)
if err != nil {
    // ...
}
conn, gerr := grpc.Dial("my-service", grpc.WithResolvers(r), grpc.WithBlock(), ...)

Managing service endpoints

The etcd resolver treats all keys under the prefix of the resolution target following a “/” (e.g., “foo/bar/my-service/”) with JSON-encoded (historically go-grpc naming.Update) values as potential service endpoints. Endpoints are added to the service by creating new keys and removed from the service by deleting keys.

Adding an endpoint

New endpoints can be added to the service through etcdctl:

ETCDCTL_API=3 etcdctl put foo/bar/my-service/1.2.3.4 '{"Addr":"1.2.3.4","Metadata":"..."}'

The etcd client’s endpoints.Manager method can also register new endpoints with a key matching the Addr:

em := endpoints.NewManager(client, "foo/bar/my-service")
err := em.AddEndpoint(context.TODO(),"foo/bar/my-service/e1", endpoints.Endpoint{Addr:"1.2.3.4"});

To enable round-robin load balancing when dialing service with multiple endpoints, you can set up you connection with grpc internal round-robin load balancer:

	conn, gerr := grpc.Dial("etcd:///foo", grpc.WithResolvers(etcdResolver),
		grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy":"round_robin"}`))

Deleting an endpoint

Hosts can be deleted from the service through etcdctl:

ETCDCTL_API=3 etcdctl del foo/bar/my-service/1.2.3.4

The etcd client’s endpoints.Manager method also supports deleting endpoints:

em := endpoints.NewManager(client, "foo/bar/my-service")
err := em.DeleteEndpoint(context.TODO(), "foo/bar/my-service/e1");

Registering an endpoint with a lease

Registering an endpoint with a lease ensures that if the host can’t maintain a keepalive heartbeat (e.g., its machine fails), it will be removed from the service:

lease=`ETCDCTL_API=3 etcdctl lease grant 5 | cut -f2 -d' '`
ETCDCTL_API=3 etcdctl put --lease=$lease my-service/1.2.3.4 '{"Addr":"1.2.3.4","Metadata":"..."}'
ETCDCTL_API=3 etcdctl lease keep-alive $lease

In the golang:

lease, _ := client.Grant(context.TODO(), ttl)
em := endpoints.NewManager(client, "foo/bar/my-service")
err := em.AddEndpoint(context.TODO(), "foo/bar/my-service/e1", endpoints.Endpoint{Addr:"1.2.3.4"}, clientv3.WithLease(lease.ID));

Atomically updating endpoints

If it’s desired to modify multiple endpoints in a single transaction, endpoints.Manager can be used directly:

em := endpoints.NewManager(c, "foo")

err := em.Update(context.TODO(), []*endpoints.UpdateWithOpts{
    endpoints.NewDeleteUpdateOpts("foo/bar/my-service/e1", endpoints.Endpoint{Addr: "1.2.3.4"}),
	endpoints.NewAddUpdateOpts("foo/bar/my-service/e1", endpoints.Endpoint{Addr: "1.2.3.14"})})