Install Istio with an External Control Plane
This guide walks you through the process of installing an external control plane and then connecting one or more
remote clusters to it. The external control plane deployment model allows a mesh operator to install and manage a control plane on an external cluster, separate from the data plane cluster (or multiple clusters) comprising the mesh. This deployment model allows a clear separation between mesh operators and mesh administrators. Mesh operators install and manage Istio control planes while mesh admins only need to configure the mesh.Envoy proxies (sidecars and gateways) running in the remote cluster access the external istiod via an ingress gateway which exposes the endpoints needed for discovery, CA, injection, and validation.
While configuration and management of the external control plane is done by the mesh operator in the external cluster, the first remote cluster connected to an external control plane serves as the config cluster for the mesh itself. The mesh administrator will use the config cluster to configure the mesh resources (gateways, virtual services, etc.) in addition to the mesh services themselves. The external control plane will remotely access this configuration from the Kubernetes API server, as shown in the above diagram.
Before you begin
Clusters
This guide requires that you have two Kubernetes clusters with any of the supported Kubernetes versions:5 1.25, 1.26, 1.27, 1.28.
The first cluster will host the external control plane installed in the
external-istiod
namespace. An ingress gateway is also installed in the istio-system
namespace to provide
cross-cluster access to the external control plane.The second cluster is a remote cluster that will run the mesh application workloads. Its Kubernetes API server also provides the mesh configuration used by the external control plane (istiod) to configure the workload proxies.
API server access
The Kubernetes API server in the remote cluster must be accessible to the external control plane cluster. Many cloud providers make API servers publicly accessible via network load balancers (NLBs). If the API server is not directly accessible, you will need to modify the installation procedure to enable access. For example, the east-west6 gateway used in a multicluster configuration could also be used to enable access to the API server.
Environment Variables
The following environment variables will be used throughout to simplify the instructions:
Variable | Description |
---|---|
CTX_EXTERNAL_CLUSTER | The context name in the default Kubernetes configuration file7 used for accessing the external control plane cluster. |
CTX_REMOTE_CLUSTER | The context name in the default Kubernetes configuration file7 used for accessing the remote cluster. |
REMOTE_CLUSTER_NAME | The name of the remote cluster. |
EXTERNAL_ISTIOD_ADDR | The hostname for the ingress gateway on the external control plane cluster. This is used by the remote cluster to access the external control plane. |
SSL_SECRET_NAME | The name of the secret that holds the TLS certs for the ingress gateway on the external control plane cluster. |
Set the CTX_EXTERNAL_CLUSTER
, CTX_REMOTE_CLUSTER
, and REMOTE_CLUSTER_NAME
now. You will set the others later.
Cluster configuration
Mesh operator steps
A mesh operator is responsible for installing and managing the external Istio control plane on the external cluster. This includes configuring an ingress gateway on the external cluster, which allows the remote cluster to access the control plane, and installing the sidecar injector webhook configuration on the remote cluster so that it will use the external control plane.
Set up a gateway in the external cluster
Create the Istio install configuration for the ingress gateway that will expose the external control plane ports to other clusters:
Then, install the gateway in the
istio-system
namespace of the external cluster:Run the following command to confirm that the ingress gateway is up and running:
You will notice an istiod deployment is also created in the
istio-system
namespace. This is used to configure the ingress gateway and is NOT the control plane used by remote clusters.Configure your environment to expose the Istio ingress gateway service using a public hostname with TLS.
Set the
EXTERNAL_ISTIOD_ADDR
environment variable to the hostname andSSL_SECRET_NAME
environment variable to the secret that holds the TLS certs:These instructions assume that you are exposing the external cluster’s gateway using a hostname with properly signed DNS certs as this is the recommended approach in a production environment. Refer to the secure ingress task for more information on exposing a secure gateway.
Your environment variables should look something like this:
Set up the remote config cluster
Use the
remote
profile to configure the remote cluster’s Istio installation. This installs an injection webhook that uses the external control plane’s injector, instead of a locally deployed one. Because this cluster will also serve as the config cluster, the Istio CRDs and other resources that will be needed on the remote cluster are also installed by settingglobal.configCluster
andpilot.configMap
totrue
:If you are using an IP address for the
EXTERNAL_ISTIOD_ADDR
, instead of a proper DNS hostname, modify the configuration to specify the discovery address and paths, instead of URLs:Install the configuration on the remote cluster:
Confirm that the remote cluster’s injection webhook configuration has been installed:
Confirm that the remote cluster’s validation webhook configurations have been installed:
Set up the control plane in the external cluster
Create the
external-istiod
namespace, which will be used to host the external control plane:The control plane in the external cluster needs access to the remote cluster to discover services, endpoints, and pod attributes. Create a secret with credentials to access the remote cluster’s
kube-apiserver
and install it in the external cluster:Create the Istio configuration to install the control plane in the
external-istiod
namespace of the external cluster. Notice that istiod is configured to use the locally mountedistio
configmap and theSHARED_MESH_CONFIG
environment variable is set toistio
. This instructs istiod to merge the values set by the mesh admin in the config cluster’s configmap with the values in the local configmap set by the mesh operator, here, which will take precedence if there are any conflicts:If you are using an IP address for the
EXTERNAL_ISTIOD_ADDR
, instead of a proper DNS hostname, delete the proxy metadata and update the webhook config environment variables in the configuration:Apply the Istio configuration on the external cluster:
Confirm that the external istiod has been successfully deployed:
Create the Istio
Gateway
,VirtualService
, andDestinationRule
configuration to route traffic from the ingress gateway to the external control plane:If you are using an IP address for the
EXTERNAL_ISTIOD_ADDR
, instead of a proper DNS hostname, modify the configuration. Delete theDestinationRule
, don’t terminate TLS in theGateway
, and use TLS routing in theVirtualService
:Apply the configuration on the external cluster:
Mesh admin steps
Now that Istio is up and running, a mesh administrator only needs to deploy and configure services in the mesh, including gateways, if needed.
Deploy a sample application
Create, and label for injection, the
sample
namespace on the remote cluster:Deploy the
helloworld
(v1
) andsleep
samples:Wait a few seconds for the
helloworld
andsleep
pods to be running with sidecars injected:Send a request from the
sleep
pod to thehelloworld
service:
Enable gateways
Enable an ingress gateway on the remote cluster:
See Installing Gateways13 for in-depth documentation on gateway installation.
You can optionally enable other gateways as well. For example, an egress gateway:
See Installing Gateways13 for in-depth documentation on gateway installation.
Configure and test an ingress gateway
- Make sure that the cluster is ready to configure the gateway:
Confirm that the Istio ingress gateway is running:
The Kubernetes Gateway API CRDs do not come installed by default on most Kubernetes clusters, so make sure they are installed before using the Gateway API:
- Expose the
helloworld
application on an ingress gateway:
- Set the
GATEWAY_URL
environment variable (see determining the ingress IP and ports for details):
Confirm you can access the
helloworld
application through the ingress gateway:
Adding clusters to the mesh (optional)
This section shows you how to expand an existing external control plane mesh to multicluster by adding another remote cluster. This allows you to easily distribute services and use location-aware routing and fail over16 to support high availability of your application.
Unlike the first remote cluster, the second and subsequent clusters added to the same external control plane do not provide mesh config, but instead are only sources of endpoint configuration, just like remote clusters in a primary-remote17 Istio multicluster configuration.
To proceed, you’ll need another Kubernetes cluster for the second remote cluster of the mesh. Set the following environment variables to the context name and cluster name of the cluster:
Register the new cluster
Create the remote Istio install configuration, which installs the injection webhook that uses the external control plane’s injector, instead of a locally deployed one:
If you are using an IP address for the
EXTERNAL_ISTIOD_ADDR
, instead of a proper DNS hostname, modify the configuration to specify the discovery address and path, instead of an injection URL:Create and annotate the system namespace on the remote cluster:
The
topology.istio.io/controlPlaneClusters
annotation specifies the cluster ID of the external control plane that should manage this remote cluster. Notice that this is the name of the first remote (config) cluster, which was used to set the cluster ID of the external control plane when it was installed in the external cluster earlier.Install the configuration on the remote cluster:
Confirm that the remote cluster’s injection webhook configuration has been installed:
Create a secret with credentials to allow the control plane to access the endpoints on the second remote cluster and install it:
Note that unlike the first remote cluster of the mesh, which also serves as the config cluster, the
--type
argument is set toremote
this time, instead ofconfig
.
Setup east-west gateways
Deploy east-west gateways on both remote clusters:
Wait for the east-west gateways to be assigned external IP addresses:
Expose services via the east-west gateways:
Validate the installation
Create, and label for injection, the
sample
namespace on the remote cluster:Deploy the
helloworld
(v2
) andsleep
samples:Wait a few seconds for the
helloworld
andsleep
pods to be running with sidecars injected:Send a request from the
sleep
pod to thehelloworld
service:Confirm that when accessing the
helloworld
application several times through the ingress gateway, both versionv1
andv2
are now being called:
Cleanup
Clean up the external control plane cluster:
Clean up the remote config cluster:
Clean up the optional second remote cluster if you installed it: