index.md 30,8 КБ
Newer Older
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
1
# Connecting GitLab with a Kubernetes cluster
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
2

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
3
4
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/35954) in GitLab 10.1.

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
5
6
Connect your project to Google Kubernetes Engine (GKE) or an existing Kubernetes
cluster in a few steps.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
7

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
8
9
## Overview

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
10
With one or more Kubernetes clusters associated to your project, you can use
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
11
[Review Apps](../../../ci/review_apps/index.md), deploy your applications, run
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
12
13
your pipelines, use it with [Auto DevOps](../../../topics/autodevops/index.md),
and much more, all from within GitLab.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
14

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
15
16
17
18
There are two options when adding a new cluster to your project; either associate
your account with Google Kubernetes Engine (GKE) so that you can [create new
clusters](#adding-and-creating-a-new-gke-cluster-via-gitlab) from within GitLab,
or provide the credentials to an [existing Kubernetes cluster](#adding-an-existing-kubernetes-cluster).
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
19

Thong Kuah's avatar
Thong Kuah включено в состав коммита
20
21
22
23
24
NOTE: **Note:**
From [GitLab 11.6](https://gitlab.com/gitlab-org/gitlab-ce/issues/34758) you
can also associate a Kubernetes cluster to your groups. Learn more about
[group Kubernetes clusters](../../group/clusters/index.md).

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
25
## Adding and creating a new GKE cluster via GitLab
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
26

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
27
28
29
30
31
TIP: **Tip:**
Every new Google Cloud Platform (GCP) account receives [$300 in credit upon sign up](https://console.cloud.google.com/freetrial),
and in partnership with Google, GitLab is able to offer an additional $200 for new GCP accounts to get started with GitLab's
Google Kubernetes Engine Integration. All you have to do is [follow this link](https://goo.gl/AaJzRW) and apply for credit.

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
32
NOTE: **Note:**
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
The [Google authentication integration](../../../integration/google.md) must
be enabled in GitLab at the instance level. If that's not the case, ask your
GitLab administrator to enable it. On GitLab.com, this is enabled.

### Requirements

Before creating your first cluster on Google Kubernetes Engine with GitLab's
integration, make sure the following requirements are met:

- A [billing account](https://cloud.google.com/billing/docs/how-to/manage-billing-account)
  is set up and you have permissions to access it.
- The Kubernetes Engine API is enabled. Follow the steps as outlined in the
  ["Before you begin" section of the Kubernetes Engine docs](https://cloud.google.com/kubernetes-engine/docs/quickstart#before-you-begin).

### Creating the cluster
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
48

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
49
If all of the above requirements are met, you can proceed to create and add a
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
50
new Kubernetes cluster to your project:
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
51

Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
52
1. Navigate to your project's **Operations > Kubernetes** page.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
53
54
55
56

    NOTE: **Note:**
    You need Maintainer [permissions] and above to access the Kubernetes page.

Mike Lewis's avatar
Mike Lewis включено в состав коммита
57
58
1. Click **Add Kubernetes cluster**.
1. Click **Create with Google Kubernetes Engine**.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
59
60
1. Connect your Google account if you haven't done already by clicking the
   **Sign in with Google** button.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
61
1. From there on, choose your cluster's settings:
Brett Walker's avatar
Brett Walker включено в состав коммита
62
   - **Kubernetes cluster name** - The name you wish to give the cluster.
Gareth Davies's avatar
Gareth Davies включено в состав коммита
63
   - **Environment scope** - The [associated environment](#setting-the-environment-scope-premium) to this cluster.
Brett Walker's avatar
Brett Walker включено в состав коммита
64
65
66
67
68
69
70
71
   - **Google Cloud Platform project** - Choose the project you created in your GCP
     console that will host the Kubernetes cluster. Learn more about
     [Google Cloud Platform projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects).
   - **Zone** - Choose the [region zone](https://cloud.google.com/compute/docs/regions-zones/)
     under which the cluster will be created.
   - **Number of nodes** - Enter the number of nodes you wish the cluster to have.
   - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types)
     of the Virtual Machine instance that the cluster will be based on.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
72
1. Finally, click the **Create Kubernetes cluster** button.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
73

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
74
75
After a couple of minutes, your cluster will be ready to go. You can now proceed
to install some [pre-defined applications](#installing-applications).
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
76
77
78
79

## Adding an existing Kubernetes cluster

To add an existing Kubernetes cluster to your project:
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
80

Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
81
1. Navigate to your project's **Operations > Kubernetes** page.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
82
83
84
85

    NOTE: **Note:**
    You need Maintainer [permissions] and above to access the Kubernetes page.

Mike Lewis's avatar
Mike Lewis включено в состав коммита
86
87
1. Click **Add Kubernetes cluster**.
1. Click **Add an existing Kubernetes cluster** and fill in the details:
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
88
    - **Kubernetes cluster name** (required) - The name you wish to give the cluster.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
89
90
91
92
93
94
95
96
97
98
99
    - **Environment scope** (required)- The
      [associated environment](#setting-the-environment-scope) to this cluster.
    - **API URL** (required) -
      It's the URL that GitLab uses to access the Kubernetes API. Kubernetes
      exposes several APIs, we want the "base" URL that is common to all of them,
      e.g., `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`.
    - **CA certificate** (optional) -
      If the API is using a self-signed TLS certificate, you'll also need to include
      the `ca.crt` contents here.
    - **Token** -
      GitLab authenticates against Kubernetes using service tokens, which are
Thong Kuah's avatar
Thong Kuah включено в состав коммита
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
      scoped to a particular `namespace`.
      **The token used should belong to a service account with
      [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
      privileges.** To create this service account:

      1. Create a `gitlab` service account in the `default` namespace:

          ```bash
          kubectl create -f - <<EOF
            apiVersion: v1
            kind: ServiceAccount
            metadata:
              name: gitlab
              namespace: default
          EOF
          ```
      1. Create a cluster role binding to give the `gitlab` service account
         `cluster-admin` privileges:

          ```bash
          kubectl create -f - <<EOF
          kind: ClusterRoleBinding
          apiVersion: rbac.authorization.k8s.io/v1
          metadata:
            name: gitlab-cluster-admin
          subjects:
          - kind: ServiceAccount
            name: gitlab
            namespace: default
          roleRef:
            kind: ClusterRole
            name: cluster-admin
            apiGroup: rbac.authorization.k8s.io
          EOF
          ```
      NOTE: **Note:**
      For GKE clusters, you will need the
      `container.clusterRoleBindings.create` permission to create a cluster
      role binding. You can follow the [Google Cloud
      documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access)
      to grant access.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
141
142
    - **Project namespace** (optional) - You don't have to fill it in; by leaving
      it blank, GitLab will create one for you. Also:
Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
143
144
145
146
147
148
149
150
       - Each project should have a unique namespace.
       - The project namespace is not necessarily the namespace of the secret, if
         you're using a secret with broader permissions, like the secret from `default`.
       - You should **not** use `default` as the project namespace.
       - If you or someone created a secret specifically for the project, usually
         with limited permissions, the secret's namespace and project namespace may
         be the same.

Pascal Borreli's avatar
Pascal Borreli включено в состав коммита
151
1. Finally, click the **Create Kubernetes cluster** button.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
152

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
153
154
After a couple of minutes, your cluster will be ready to go. You can now proceed
to install some [pre-defined applications](#installing-applications).
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
155

Jason Colyer's avatar
Jason Colyer включено в состав коммита
156
157
To determine the:

Jason Colyer's avatar
Jason Colyer включено в состав коммита
158
- API URL, run `kubectl cluster-info | grep 'Kubernetes master' | awk '/http/ {print $NF}'`.
Jason Colyer's avatar
Jason Colyer включено в состав коммита
159
- Token:
Jason Colyer's avatar
Jason Colyer включено в состав коммита
160
  1. List the secrets by running: `kubectl get secrets`. Note the name of the secret you need the token for.
Thong Kuah's avatar
Thong Kuah включено в состав коммита
161
162
  1. Get the token for the appropriate secret by running: `kubectl get secret <SECRET_NAME> -o jsonpath="{['data']['token']}" | base64 --decode`.
- CA certificate, run `kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode`.
Jason Colyer's avatar
Jason Colyer включено в состав коммита
163

164
165
166
167
168
169
170
171
## Security implications

CAUTION: **Important:**
The whole cluster security is based on a model where [developers](../../permissions.md)
are trusted, so **only trusted users should be allowed to control your clusters**.

The default cluster configuration grants access to a wide set of
functionalities needed to successfully build and deploy a containerized
Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
172
application. Bear in mind that the same credentials are used for all the
173
174
applications running on the cluster.

Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
175
176
177
178
179
180
## Access controls

When creating a cluster in GitLab, you will be asked if you would like to create an
[Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster, or
a [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) one.

Evan Read's avatar
Evan Read включено в состав коммита
181
182
183
184
185
NOTE: **Note:**
[RBAC](#role-based-access-control-rbac) is recommended and the GitLab default.

Whether [ABAC](#attribute-based-access-control-abac) or [RBAC](#role-based-access-control-rbac) is enabled,
GitLab will create the necessary service accounts and privileges in order to install and run
Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
186
187
[GitLab managed applications](#installing-applications):

Thong Kuah's avatar
Thong Kuah включено в состав коммита
188
189
190
- If GitLab is creating the cluster, a `gitlab` service account with
  `cluster-admin` privileges will be created in the `default` namespace,
  which will be used by GitLab to manage the newly created cluster.
Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
191

Dylan Griffith's avatar
Dylan Griffith включено в состав коммита
192
193
194
195
- A project service account with [`edit`
  privileges](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
  will be created in the project namespace (also created by GitLab), which will
  be used in [deployment jobs](#deployment-variables).
Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235

  NOTE: **Note:**
  Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/51716) in GitLab 11.5.

- When you install Helm Tiller into your cluster, the `tiller` service account
  will be created with `cluster-admin` privileges in the `gitlab-managed-apps`
  namespace. This service account will be added to the installed Helm Tiller and will
  be used by Helm to install and run [GitLab managed applications](#installing-applications).
  Helm Tiller will also create additional service accounts and other resources for each
  installed application. Consult the documentation of the Helm charts for each application
  for details.

If you are [adding an existing Kubernetes cluster](#adding-an-existing-kubernetes-cluster),
ensure the token of the account has administrator privileges for the cluster.

The following sections summarize which resources will be created on ABAC/RBAC clusters.

### Attribute-based access control (ABAC)

| Name              | Kind                 | Details                           | Created when                      |
| ---               | ---                  | ---                               | ---                               |
| `gitlab`          | `ServiceAccount`     | `default` namespace               | Creating a new GKE Cluster        |
| `gitlab-token`    | `Secret`             | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster        |
| `tiller`          | `ServiceAccount`     | `gitlab-managed-apps` namespace   | Installing Helm Tiller            |
| `tiller-admin`    | `ClusterRoleBinding` | `cluster-admin` roleRef           | Installing Helm Tiller            |
| Project namespace | `ServiceAccount`     | Uses namespace of Project         | Creating/Adding a new GKE Cluster |
| Project namespace | `Secret`             | Token for project ServiceAccount  | Creating/Adding a new GKE Cluster |

### Role-based access control (RBAC)

| Name                | Kind                 | Details                           | Created when                      |
| ---                 | ---                  | ---                               | ---                               |
| `gitlab`            | `ServiceAccount`     | `default` namespace               | Creating a new GKE Cluster        |
| `gitlab-admin`      | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef           | Creating a new GKE Cluster |
| `gitlab-token`      | `Secret`             | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster        |
| `tiller`            | `ServiceAccount`     | `gitlab-managed-apps` namespace   | Installing Helm Tiller            |
| `tiller-admin`      | `ClusterRoleBinding` | `cluster-admin` roleRef           | Installing Helm Tiller            |
| Project namespace   | `ServiceAccount`     | Uses namespace of Project         | Creating/Adding a new GKE Cluster |
| Project namespace   | `Secret`             | Token for project ServiceAccount  | Creating/Adding a new GKE Cluster |
| Project namespace   | `RoleBinding`        | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef                  | Creating/Adding a new GKE Cluster |
Thong Kuah's avatar
Thong Kuah включено в состав коммита
236

237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
### Security of GitLab Runners

GitLab Runners have the [privileged mode](https://docs.gitlab.com/runner/executors/docker.html#the-privileged-mode)
enabled by default, which allows them to execute special commands and running
Docker in Docker. This functionality is needed to run some of the [Auto DevOps]
jobs. This implies the containers are running in privileged mode and you should,
therefore, be aware of some important details.

The privileged flag gives all capabilities to the running container, which in
turn can do almost everything that the host can do. Be aware of the
inherent security risk associated with performing `docker run` operations on
arbitrary images as they effectively have root access.

If you don't want to use GitLab Runner in privileged mode, first make sure that
you don't have it installed via the applications, and then use the
[Runner's Helm chart](../../../install/kubernetes/gitlab_runner_chart.md) to
install it manually.

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
255
256
## Installing applications

Alexander Tanayno's avatar
Alexander Tanayno включено в состав коммита
257
NOTE: **Note:**
Alexander Tanayno's avatar
Alexander Tanayno включено в состав коммита
258
259
Before starting the installation of applications, make sure that time is synchronized
between your GitLab server and your Kubernetes cluster. Otherwise, installation could fail
Alexander Tanayno's avatar
Alexander Tanayno включено в состав коммита
260
and you may get errors like `Error: remote error: tls: bad certificate` 
Alexander Tanayno's avatar
Alexander Tanayno включено в состав коммита
261
in the `stdout` of pods created by GitLab in your Kubernetes cluster.
Alexander Tanayno's avatar
Alexander Tanayno включено в состав коммита
262

Thong Kuah's avatar
Thong Kuah включено в состав коммита
263
264
265
GitLab provides a one-click install for various applications which can
be added directly to your configured cluster. Those applications are
needed for [Review Apps](../../../ci/review_apps/index.md) and
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
266
267
268
269
270
271
272
273
274
[deployments](../../../ci/environments.md). You can install them after you
[create a cluster](#adding-and-creating-a-new-gke-cluster-via-gitlab).

To see a list of available applications to install:

1. Navigate to your project's **Operations > Kubernetes**.
1. Select your cluster.

Install Helm Tiller first because it's used to install other applications.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
275

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
276
NOTE: **Note:**
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
277
278
As of GitLab 11.6, Helm Tiller will be upgraded to the latest version supported
by GitLab before installing any of the applications.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
279

Thong Kuah's avatar
Thong Kuah включено в состав коммита
280
281
282
283
| Application | GitLab version | Description | Helm Chart |
| ----------- | :------------: | ----------- | --------------- |
| [Helm Tiller](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a |
| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) |
Amit Rathi's avatar
Amit Rathi включено в состав коммита
284
| [Cert Manager](http://docs.cert-manager.io/en/latest/) | 11.6+ | Cert Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) |
Thong Kuah's avatar
Thong Kuah включено в состав коммита
285
286
| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) |
| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](https://about.gitlab.com/features/gitlab-ci-cd/), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) |
Mike Lewis's avatar
Mike Lewis включено в состав коммита
287
| [JupyterHub](http://jupyter.org/) | 11.0+ | [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. We use a [custom Jupyter image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) that installs additional useful packages on top of the base Jupyter. You will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). More information on creating executable runbooks can be found in [our Nurtch documentation](runbooks/index.md#nurtch-executable-runbooks). **Note**: Authentication will be enabled for any user of the GitLab server via OAuth2. HTTPS will be supported in a future release. | [jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) |
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
288
| [Knative](https://cloud.google.com/knative) | 11.5+ | Knative provides a platform to create, deploy, and manage serverless workloads from a Kubernetes cluster. It is used in conjunction with, and includes [Istio](https://istio.io) to provide an external IP address for all programs hosted by Knative. You will be prompted to enter a wildcard domain where your applications will be exposed. Configure your DNS server to use the external IP address for that domain. For any application created and installed, they will be accessible as `<program_name>.<kubernetes_namespace>.<domain_name>`. This will require your kubernetes cluster to have [RBAC enabled](#role-based-access-control-rbac). | [knative/knative](https://storage.googleapis.com/triggermesh-charts)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
289

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
290
291
292
293
294
295
296
297
With the exception of Knative, the applications will be installed in a dedicated
namespace called `gitlab-managed-apps`.

CAUTION: **Caution:**
If you have an existing Kubernetes cluster with Tiller already installed,
you should be careful as GitLab cannot detect it. In this case, installing
Tiller via the applications will result in the cluster having it twice, which
can lead to confusion during deployments.
Dylan Griffith's avatar
Dylan Griffith включено в состав коммита
298

Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
299
300
301
## Getting the external IP address

NOTE: **Note:**
Dylan Griffith's avatar
Dylan Griffith включено в состав коммита
302
303
304
305
With the following procedure, a load balancer must be installed in your cluster
to obtain the external IP address. You can use either
[Ingress](#installing-applications), or Knative's own load balancer
([Istio](https://istio.io)) if using [Knative](#installing-applications).
Chris Baumbauer's avatar
Chris Baumbauer включено в состав коммита
306

Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
307
308
309
In order to publish your web application, you first need to find the external IP
address associated to your load balancer.

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
310
### Let GitLab fetch the IP address
Dylan Griffith's avatar
Dylan Griffith включено в состав коммита
311
312
313

> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17052) in GitLab 10.6.

Dylan Griffith's avatar
Dylan Griffith включено в состав коммита
314
If you [installed Ingress or Knative](#installing-applications),
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
315
316
317
you should see the Ingress IP address on this same page within a few minutes.
If you don't see this, GitLab might not be able to determine the IP address of
your ingress application in which case you should manually determine it.
Dylan Griffith's avatar
Dylan Griffith включено в состав коммита
318
319
320

### Manually determining the IP address

Mike Lewis's avatar
Mike Lewis включено в состав коммита
321
If the cluster is on GKE, click the **Google Kubernetes Engine** link in the
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
322
323
**Advanced settings**, or go directly to the
[Google Kubernetes Engine dashboard](https://console.cloud.google.com/kubernetes/)
Mike Lewis's avatar
Mike Lewis включено в состав коммита
324
and select the proper project and cluster. Then click **Connect** and execute
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
325
326
327
328
the `gcloud` command in a local terminal or using the **Cloud Shell**.

If the cluster is not on GKE, follow the specific instructions for your
Kubernetes provider to configure `kubectl` with the right credentials.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
329
330
331
The output of the following examples will show the external IP address of your
cluster. This information can then be used to set up DNS entries and forwarding
rules that allow external access to your deployed applications.
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
332

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
333
334
If you installed the Ingress [via the **Applications**](#installing-applications),
run the following command:
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
335
336
337
338
339

```bash
kubectl get svc --namespace=gitlab-managed-apps ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip} '
```

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
340
For Istio/Knative, the command will be different:
danielgruesso's avatar
danielgruesso включено в состав коммита
341

Chris Baumbauer's avatar
Chris Baumbauer включено в состав коммита
342
343
344
345
```bash
kubectl get svc --namespace=istio-system knative-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip} '
```

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
346
Some Kubernetes clusters return a hostname instead, like [Amazon EKS](https://aws.amazon.com/eks/). For these platforms, run:
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
347
348

```bash
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
349
kubectl get service ingress-nginx-ingress-controller -n gitlab-managed-apps -o jsonpath="{.status.loadBalancer.ingress[0].hostname}".
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
350
351
```

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
352
Otherwise, you can list the IP addresses of all load balancers:
Joshua Lambert's avatar
Joshua Lambert включено в состав коммита
353

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
354
355
356
```bash
kubectl get svc --all-namespaces -o jsonpath='{range.items[?(@.status.loadBalancer.ingress)]}{.status.loadBalancer.ingress[*].ip} '
```
Fabio Busatto's avatar
Fabio Busatto включено в состав коммита
357

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
358
359
360
361
362
363
364
365
### Using a static IP

By default, an ephemeral external IP address is associated to the cluster's load
balancer. If you associate the ephemeral IP with your DNS and the IP changes,
your apps will not be able to be reached, and you'd have to change the DNS
record again. In order to avoid that, you should change it into a static
reserved IP.

Mike Lewis's avatar
Mike Lewis включено в состав коммита
366
Read how to [promote an ephemeral external IP address in GKE](https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address#promote_ephemeral_ip).
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
367
368
369
370
371
372
373
374
375

### Pointing your DNS at the cluster IP

Once you've set up the static IP, you should associate it to a [wildcard DNS
record](https://en.wikipedia.org/wiki/Wildcard_DNS_record), in order to be able
to reach your apps. This heavily depends on your domain provider, but in case
you aren't sure, just create an A record with a wildcard host like
`*.example.com.`.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
376
377
378
379
380
381
382
383
384
385
386
387
## Multiple Kubernetes clusters **[PREMIUM]**

> Introduced in [GitLab Premium][ee] 10.3.

With GitLab Premium, you can associate more than one Kubernetes clusters to your
project. That way you can have different clusters for different environments,
like dev, staging, production, etc.

Simply add another cluster, like you did the first time, and make sure to
[set an environment scope](#setting-the-environment-scope) that will
differentiate the new cluster with the rest.

Thong Kuah's avatar
Thong Kuah включено в состав коммита
388
## Setting the environment scope **[PREMIUM]**
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
389

Thong Kuah's avatar
Thong Kuah включено в состав коммита
390
391
392
393
394
When adding more than one Kubernetes clusters to your project, you need
to differentiate them with an environment scope. The environment scope
associates clusters with [environments](../../../ci/environments.md)
similar to how the [environment-specific
variables](../../../ci/variables/README.md#limiting-environment-scopes-of-variables)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
395
396
397
398
399
work.

The default environment scope is `*`, which means all jobs, regardless of their
environment, will use that cluster. Each scope can only be used by a single
cluster in a project, and a validation error will occur if otherwise.
Shinya Maeda's avatar
Shinya Maeda включено в состав коммита
400
Also, jobs that don't have an environment keyword set will not be able to access any cluster.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
401
402
403

---

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
404
For example, let's say the following Kubernetes clusters exist in a project:
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
405

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
406
407
408
409
410
| Cluster     | Environment scope |
| ----------- | ----------------- |
| Development | `*`               |
| Staging     | `staging`         |
| Production  | `production`      |
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426

And the following environments are set in [`.gitlab-ci.yml`](../../../ci/yaml/README.md):

```yaml
stages:
- test
- deploy

test:
  stage: test
  script: sh test

deploy to staging:
  stage: deploy
  script: make deploy
  environment:
Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
427
    name: staging
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
428
429
430
431
432
433
    url: https://staging.example.com/

deploy to production:
  stage: deploy
  script: make deploy
  environment:
Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
434
    name: production
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
435
436
437
438
439
440
441
442
443
    url: https://example.com/
```

The result will then be:

- The development cluster will be used for the "test" job.
- The staging cluster will be used for the "deploy to staging" job.
- The production cluster will be used for the "deploy to production" job.

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
444
445
446
447
## Deployment variables

The Kubernetes cluster integration exposes the following
[deployment variables](../../../ci/variables/README.md#deployment-variables) in the
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
448
449
450
451
452
GitLab CI/CD build environment.

| Variable | Description |
| -------- | ----------- |
| `KUBE_URL` | Equal to the API URL. |
Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
453
| `KUBE_TOKEN` | The Kubernetes token of the [project service account](#access-controls). |
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
454
| `KUBE_NAMESPACE` | The Kubernetes namespace is auto-generated if not specified. The default value is `<project_name>-<project_id>`. You can overwrite it to use different one if needed, otherwise the `KUBE_NAMESPACE` variable will receive the default value. |
Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
455
456
| `KUBE_CA_PEM_FILE` | Path to a file containing PEM data. Only present if a custom CA bundle was specified. |
| `KUBE_CA_PEM` | (**deprecated**) Raw PEM data. Only if a custom CA bundle was specified. |
🤖 GitLab Bot 🤖's avatar
🤖 GitLab Bot 🤖 включено в состав коммита
457
| `KUBECONFIG` | Path to a file containing `kubeconfig` for this deployment. CA bundle would be embedded if specified. This config also embeds the same token defined in `KUBE_TOKEN` so you likely will only need this variable. This variable name is also automatically picked up by `kubectl` so you won't actually need to reference it explicitly if using `kubectl`. |
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
458

Mayra Cabrera's avatar
Mayra Cabrera включено в состав коммита
459
460
461
462
NOTE: **NOTE:**
Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main
service account of the cluster integration.

🤖 GitLab Bot 🤖's avatar
🤖 GitLab Bot 🤖 включено в состав коммита
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
### Troubleshooting missing `KUBECONFIG` or `KUBE_TOKEN`

GitLab will create a new service account specifically for your CI builds. The
new service account is created when the cluster is added to the project.
Sometimes there may be errors that cause the service account creation to fail.

In such instances, your build will not be passed the `KUBECONFIG` or
`KUBE_TOKEN` variables and, if you are using Auto DevOps, your Auto DevOps
pipelines will no longer trigger a `production` deploy build. You will need to
check the [logs](../../../administration/logs.md) to debug why the service
account creation failed.

A common reason for failure is that the token you gave GitLab did not have
[`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles)
privileges as GitLab expects.

Another common problem for why these variables are not being passed to your
builds is that they must have a matching
[`environment:name`](../../../ci/environments.md#defining-environments). If
your build has no `environment:name` set, it will not be passed the Kubernetes
credentials.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
485
486
487
488
489
490
491
492
## Monitoring your Kubernetes cluster **[ULTIMATE]**

> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4701) in [GitLab Ultimate][ee] 10.6.

When [Prometheus is deployed](#installing-applications), GitLab will automatically monitor the cluster's health. At the top of the cluster settings page, CPU and Memory utilization is displayed, along with the total amount available. Keeping an eye on cluster resources can be important, if the cluster runs out of memory pods may be shutdown or fail to start.

![Cluster Monitoring](https://docs.gitlab.com/ee/user/project/clusters/img/k8s_cluster_monitoring.png)

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
493
## Enabling or disabling the Kubernetes cluster integration
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
494
495

After you have successfully added your cluster information, you can enable the
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
496
Kubernetes cluster integration:
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
497

Mike Lewis's avatar
Mike Lewis включено в состав коммита
498
1. Click the **Enabled/Disabled** switch
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
499
500
501
502
1. Hit **Save** for the changes to take effect

You can now start using your Kubernetes cluster for your deployments.

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
503
To disable the Kubernetes cluster integration, follow the same procedure.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
504

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
505
## Removing the Kubernetes cluster integration
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
506
507

NOTE: **Note:**
Mark Chao's avatar
Mark Chao включено в состав коммита
508
You need Maintainer [permissions] and above to remove a Kubernetes cluster integration.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
509
510
511
512
513
514

NOTE: **Note:**
When you remove a cluster, you only remove its relation to GitLab, not the
cluster itself. To remove the cluster, you can do so by visiting the GKE
dashboard or using `kubectl`.

Mike Lewis's avatar
Mike Lewis включено в состав коммита
515
To remove the Kubernetes cluster integration from your project, simply click the
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
516
**Remove integration** button. You will then be able to follow the procedure
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
517
and add a Kubernetes cluster again.
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
518

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
519
520
521
522
523
## View Kubernetes pod logs from GitLab **[ULTIMATE]**

Learn how to easily
[view the logs of running pods in connected Kubernetes clusters](https://docs.gitlab.com/ee/user/project/clusters/kubernetes_pod_logs.html).

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
524
525
526
527
## What you can get with the Kubernetes integration

Here's what you can do with GitLab if you enable the Kubernetes integration.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
528
### Deploy Boards **[PREMIUM]**
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
529
530
531
532
533
534
535

GitLab's Deploy Boards offer a consolidated view of the current health and
status of each CI [environment](../../../ci/environments.md) running on Kubernetes,
displaying the status of the pods in the deployment. Developers and other
teammates can view the progress and status of a rollout, pod by pod, in the
workflow they already use without any need to access Kubernetes.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
536
[Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
537

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
538
### Canary Deployments **[PREMIUM]**
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
539
540
541
542
543

Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cluster-administration/manage-deployment/#canary-deployments)
and visualize your canary deployments right inside the Deploy Board, without
the need to leave GitLab.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
544
[Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
545
546
547
548
549
550

### Kubernetes monitoring

Automatically detect and monitor Kubernetes metrics. Automatic monitoring of
[NGINX ingress](../integrations/prometheus_library/nginx.md) is also supported.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
551
[Read more about Kubernetes monitoring](../integrations/prometheus_library/kubernetes.md)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
552
553
554
555
556
557
558
559
560

### Auto DevOps

Auto DevOps automatically detects, builds, tests, deploys, and monitors your
applications.

To make full use of Auto DevOps(Auto Deploy, Auto Review Apps, and Auto Monitoring)
you will need the Kubernetes project integration enabled.

Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
561
[Read more about Auto DevOps](../../../topics/autodevops/index.md)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
562
563
564
565

### Web terminals

NOTE: **Note:**
Mark Chao's avatar
doc    
Mark Chao включено в состав коммита
566
Introduced in GitLab 8.15. You must be the project owner or have `maintainer` permissions
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
567
568
569
570
571
572
573
574
575
576
to use terminals. Support is limited to the first container in the
first pod of your environment.

When enabled, the Kubernetes service adds [web terminal](../../../ci/environments.md#web-terminals)
support to your [environments](../../../ci/environments.md). This is based on the `exec` functionality found in
Docker and Kubernetes, so you get a new shell session within your existing
containers. To use this integration, you should deploy to Kubernetes using
the deployment variables above, ensuring any pods you create are labelled with
`app=$CI_ENVIRONMENT_SLUG`. GitLab will do the rest!

danielgruesso's avatar
danielgruesso включено в состав коммита
577
578
### Integrating Amazon EKS cluster with GitLab

Mike Lewis's avatar
Mike Lewis включено в состав коммита
579
- Learn how to [connect and deploy to an Amazon EKS cluster](eks_and_gitlab/index.md).
danielgruesso's avatar
danielgruesso включено в состав коммита
580
581
582
583

### Serverless

- [Run serverless workloads on Kubernetes with Knative.](serverless/index.md)
Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
584

Achilleas Pipinellis's avatar
Achilleas Pipinellis включено в состав коммита
585
[permissions]: ../../permissions.md
Marcel Amirault's avatar
Marcel Amirault включено в состав коммита
586
[ee]: https://about.gitlab.com/pricing/
587
[Auto DevOps]: ../../../topics/autodevops/index.md