Announcing Istio client-go

We are pleased to announce the initial release of the Istio client go repository which enables developers to gain programmatic access to Istio APIs in a Kubernetes environment. The generated Kubernetes informers and client set in this repository makes it easy for developers to create controllers and perform Create, Read, Update and Delete (CRUD) operations for all Istio Custom Resource Definitions (CRDs).

This was a highly requested functionality by many Istio users, as is evident from the feature requests on the clients generated by Aspen Mesh and the Knative project. If you’re currently using one of the above mentioned clients, you can easily switch to using Istio client go like this:

import (
  ...
  - versionedclient "github.com/aspenmesh/istio-client-go/pkg/client/clientset/versioned"
  + versionedclient "istio.io/client-go/pkg/clientset/versioned"
)

As the generated client sets are functionally equivalent, switching the imported client libraries should be sufficient in order to consume the newly generated library.

How to use client-go

The Istio client go repository follows the same branching strategy as the Istio API repository, as the client repository depends on the API definitions. If you want to use a stable client set, you can use the release branches or tagged versions in the client go repository. Using the client set is very similar to using the Kubernetes client go, here’s a quick example of using the client to list all Istio virtual services in the passed namespace:

package main

import (
  "log"
  "os"

  metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
  "k8s.io/client-go/tools/clientcmd"

  versionedclient "istio.io/client-go/pkg/clientset/versioned"
)

func main() {
  kubeconfig := os.Getenv("KUBECONFIG")
  namespace := os.Getenv("NAMESPACE")
  if len(kubeconfig) == 0 || len(namespace) == 0 {
    log.Fatalf("Environment variables KUBECONFIG and NAMESPACE need to be set")
  }
  restConfig, err := clientcmd.BuildConfigFromFlags("", kubeconfig)
  if err != nil {
    log.Fatalf("Failed to create k8s rest client: %s", err)
  }

  ic, err := versionedclient.NewForConfig(restConfig)
  if err != nil {
    log.Fatalf("Failed to create istio client: %s", err)
  }
  // Print all VirtualServices
  vsList, err := ic.NetworkingV1alpha3().VirtualServices(namespace).List(metav1.ListOptions{})
  if err != nil {
    log.Fatalf("Failed to get VirtualService in %s namespace: %s", namespace, err)
  }
  for i := range vsList.Items {
    vs := vsList.Items[i]
    log.Printf("Index: %d VirtualService Hosts: %+v\n", i, vs.Spec.GetHosts())
  }
}

You can find a more in-depth example here.

Useful tools created for generating Istio client-go

If you’re wondering why it took so long or why was it difficult to generate this client set, this section is for you. In Istio, we use protobuf specifications to write APIs which are then converted to Go definitions using the protobuf tool chain. There are three major challenges which you might face if you’re trying to generate Kubernetes client set from a protobuf-generated API:

• Creating Kubernetes Wrapper Types - Kubernetes client generation library only works for Go objects which follow the Kubernetes object specification for e.g. Authentication Policy Kubernetes Wrappers. This means for every API which needs programmatic access, you need to create these wrappers. Additionally, there is a fair amount of boilerplate needed for every CRD group, version and kind that needs client code generation. To automate this process, we created a Kubernetes type generator tool which can automatically create the Kubernetes types based on annotations. The annotations parsed by this tool and the various available options are explained in the README. Note that if you’re using protobuf tools to generate Go types, you would need to add these annotations as comments in the proto files, so that the comments are present in the generated Go files which are then used by this tool.

• Generating deep copy methods - In Kubernetes client machinery, if you want to mutate any object returned from the client set, you are required to make a copy of the object to prevent modifying the object in-place in the cache store. The canonical way to do this is to create a deepcopy method on all nested types. We created a tool protoc deep copy generator which is a protoc plugin and can automatically create deepcopy method based on annotations using the Proto library utility Proto Clone. Here’s an example of the generated deepcopy method.

• Marshaling and Unmarshaling types to/from JSON - For the types generated from proto definitions, it is often problematic to use the default Go JSON encoder/decoder as there are various fields like protobuf’s oneof which requires special handling. Additionally, any Proto fields with underscores in their name might serialize/deserialize to different field names depending on the encoder/decoder as the Go struct tag are generated differently. It is always recommended to use protobuf primitives for serializing/deserializing to JSON instead of relying on default Go library. We created a tool protoc JSON shim which is a protoc plugin and can automatically create Marshalers/Unmarshalers for all Go type generated from Proto definitions. Here’s an example of the code generated by this tool.

I’m hoping that the newly released client library enables users to create more integrations and controllers for the Istio APIs, and the tools mentioned above can be used by developers to generate Kubernetes client set from Proto APIs.