README.md 11.9 KB
Newer Older
1
# MicroK8s
Konstantinos Tsakalozos's avatar
Konstantinos Tsakalozos committed
2

3
![](https://img.shields.io/badge/Kubernetes-1.13-326de6.svg) ![Build Status](https://travis-ci.org/ubuntu/microk8s.svg)
4

5
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="200px">Kubernetes in a [snap](https://snapcraft.io/) that you can run locally.
6
7
8

## User Guide

9
Snaps are frequently updated to match each release of Kubernetes. The quickest way to get started is to install directly from the snap store. You can install MicroK8s and let it update to the latest stable upstream Kubernetes release with:
10
11

```
12
snap install microk8s --classic
13
14
```

15
Alternatively, you can select a MicroK8s channel that will follow a specific Kubernetes release series. For example, you install MicroK8s and let it follow the `v1.12` series with:
16
17
18
19
20

```
snap install microk8s --classic --channel=1.12/stable
```

21
You can read more on the MicroK8s release channels in the [Release Channels and Upgrades](docs/release-channels.md) doc.
22

23
At any point you can check MicroK8s' availability with:
24
25
26
27
28
29
30
31
32
33
34

```
microk8s.status
```

During installation you can use the `--wait-ready` flag to wait for the kubernetes services to initialise:

```
microk8s.status --wait-ready
```

35
> In order to install MicroK8s make sure
36
> - port 8080 is not used and
37
> - if you have AppArmor enabled (check with `sudo apparmor_status`) you do not have any other [dockerd installed](docs/dockerd.md). You can use the dockerd coming with MicroK8s.
38
39
40

### Accessing Kubernetes

41
To avoid colliding with a `kubectl` already installed and to avoid overwriting any existing Kubernetes configuration file, MicroK8s adds a `microk8s.kubectl` command, configured to exclusively access the new MicroK8s install. When following instructions online, make sure to prefix `kubectl` with `microk8s.`.
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

```
microk8s.kubectl get nodes
microk8s.kubectl get services
```

If you do not already have a `kubectl` installed you can alias `microk8s.kubectl` to `kubectl` using the following command

```
snap alias microk8s.kubectl kubectl
```

This measure can be safely reverted at anytime by doing

```
snap unalias kubectl
```
59
If you already have `kubectl` installed and you want to use it to access the MicroK8s deployment you can export the cluster's config with:
60
61

```
62
microk8s.kubectl config view --raw > $HOME/.kube/config
63
64
```

65
Note: The API server on port 8080 is listening on all network interfaces. In its kubeconfig file MicroK8s is using the loopback interface, as you can see with `microk8s.kubectl config view`. The `microk8s.config` command will output a kubeconfig with the host machine's IP (instead of the 127.0.0.1) as the API server endpoint.
66

67

68
69
### Kubernetes Addons

70
MicroK8s installs a barebones upstream Kubernetes. This means just the api-server, controller-manager, scheduler, kubelet, cni, kube-proxy are installed and run. Additional services like kube-dns and dashboard can be run using the `microk8s.enable` command
71
72
73
74
75
76
77
78
79
80
81

```
microk8s.enable dns dashboard
```

These addons can be disabled at anytime using the `disable` command

```
microk8s.disable dashboard dns
```

82
With `microk8s.status` you can see the list of available addons and which ones are currently enabled. You can find the addon manifests and/or scripts under `${SNAP}/actions/`, with `${SNAP}` pointing by default to `/snap/microk8s/current`.
83

84
#### List of Available Addons
85
- **dns**: Deploy kube dns. This addon may be required by others thus we recommend you always enable it. In environments where the external dns servers `8.8.8.8` and `8.8.4.4` are blocked you will need to update the upstream dns servers in `microk8s.kubectl -n kube-system edit configmap/kube-dns` after enabling the addon.
86
- **dashboard**: Deploy kubernetes dashboard as well as grafana and influxdb. To access grafana point your browser to the url reported by `microk8s.kubectl cluster-info`.
87
- **storage**: Create a default storage class. This storage class makes use of the hostpath-provisioner pointing to a directory on the host. Persistent volumes are created under `${SNAP_COMMON}/default-storage`. Upon disabling this addon you will be asked if you want to delete the persistent volumes created.
88
- **ingress**: Create an ingress controller.
89
- **gpu**: Expose GPU(s) to MicroK8s by enabling the nvidia-docker runtime and nvidia-device-plugin-daemonset. Requires NVIDIA drivers to already be installed on the host system.
Cody Logan's avatar
Cody Logan committed
90
- **istio**: Deploy the core [Istio](https://istio.io/) services. You can use the `microk8s.istioctl` command to manage your deployments.
91
- **registry**: Deploy a docker private registry and expose it on `localhost:32000`. The storage addon will be enabled as part of this addon. To [use the registry](docs/registry.md) you can use the `microk8s.docker` command.
92
- **metrics-server**: Deploy the [Metrics Server](https://kubernetes.io/docs/tasks/debug-application-cluster/core-metrics-pipeline/#metrics-server).
93
- **prometheus**: Deploy the [Prometheus Operator](https://github.com/coreos/prometheus-operator) v0.25.
94

95
### Stopping and Restarting MicroK8s
bkcsfi's avatar
bkcsfi committed
96

97
You may wish to temporarily shutdown MicroK8s when not in use without un-installing it.
bkcsfi's avatar
bkcsfi committed
98

99
MicroK8s can be shutdown with:
bkcsfi's avatar
bkcsfi committed
100
101

```
102
microk8s.stop
bkcsfi's avatar
bkcsfi committed
103
104
```

105
MicroK8s can be restarted later with:
bkcsfi's avatar
bkcsfi committed
106
107

```
108
microk8s.start
bkcsfi's avatar
bkcsfi committed
109
110
```

111
### Removing MicroK8s
112

113
Before removing MicroK8s, use `microk8s.reset` to stop all running pods.
114
115
116
117
118

```
microk8s.reset
snap remove microk8s
```
bkcsfi's avatar
bkcsfi committed
119

120
### Configuring MicroK8s Services
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
The following systemd services will be running in your system:
- **snap.microk8s.daemon-apiserver**, is the [kube-apiserver](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) daemon started using the arguments in `${SNAP_DATA}/args/kube-apiserver`
- **snap.microk8s.daemon-controller-manager**, is the [kube-controller-manager](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-controller-manager/) daemon started using the arguments in `${SNAP_DATA}/args/kube-controller-manager`
- **snap.microk8s.daemon-scheduler**, is the [kube-scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) daemon started using the arguments in `${SNAP_DATA}/args/kube-scheduler`
- **snap.microk8s.daemon-kubelet**, is the [kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/) daemon started using the arguments in `${SNAP_DATA}/args/kubelet`
- **snap.microk8s.daemon-proxy**, is the [kube-proxy](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/) daemon started using the arguments in `${SNAP_DATA}/args/kube-proxy`
- **snap.microk8s.daemon-docker**, is the [docker](https://docs.docker.com/engine/reference/commandline/dockerd/) daemon started using the arguments in `${SNAP_DATA}/args/dockerd`
- **snap.microk8s.daemon-etcd**, is the [etcd](https://coreos.com/etcd/docs/latest/v2/configuration.html) daemon started using the arguments in `${SNAP_DATA}/args/etcd`

Normally, `${SNAP_DATA}` points to `/var/snap/microk8s/current`.

To reconfigure a service you will need to edit the corresponding file and then restart the respective daemon. For example:
```
echo '--config-file=/path-to-my/daemon.json' | sudo tee -a /var/snap/microk8s/current/args/dockerd
sudo systemctl restart snap.microk8s.daemon-docker.service
```

138
139
### Deploy Behind a Proxy

140
To let MicroK8s use a proxy enter the proxy details in `${SNAP_DATA}/args/dockerd-env` and restart the docker daemon service with:
141
142
143
144
145
```
sudo systemctl restart snap.microk8s.daemon-docker.service
```


146
## Troubleshooting
147

148
To troubleshoot a non-functional MicroK8s deployment, start by running the `microk8s.inspect` command. This command performs a set of tests against MicroK8s and collects traces and logs in a report tarball. In case any of the aforementioned daemons are failing you will be urged to look at the respective logs with `journalctl -u snap.microk8s.<daemon>.service`. `microk8s.inspect` may also make suggestions on potential issues it may find. If you do not manage to resolve the issue you are facing please file a [bug](https://github.com/ubuntu/microk8s/issues) attaching the inspection report tarball.
149
150
151
152

Some common problems and solutions are listed below.

### My dns and dashboard pods are CrashLooping.
153
The [Kubenet](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) network plugin used by MicroK8s creates a `cbr0` interface when the first pod is created. If you have `ufw` enabled, you'll need to allow traffic on this interface:
154
155

`sudo ufw allow in on cbr0 && sudo ufw allow out on cbr0`
156

157
### My pods can't reach the internet or each other (but my MicroK8s host machine can).
158
159
160
161
162
163
164
165
166
Make sure packets to/from the pod network interface can be forwarded
to/from the default interface on the host:

`sudo iptables -P FORWARD ACCEPT`

or, if using `ufw`:

`sudo ufw default allow routed`

167
168
169
170
171
172
The microk8s inspect command can be used to check the firewall configuration:

`microk8s.inspect`

A warning will be shown if the firewall is not forwarding traffic.

173
174
### My host machine changed IP and now MicroK8s is not working properly.
The host machine IP may change whenever you switch places with your laptop or you go through a suspend/resume cycle. The kubernetes API server advertises this IP (taken from the default interface) to all kubernetes cluster members. Services such as DNS and the dashboard will lose connectivity to API server in case the host IP changes. You will need to restart MicroK8s whenever this happens:
175
```
176
177
microk8s.stop
microk8s.start
178
```
179

180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
### My log collector is not collecting any logs.
By default docker container logs are located in `/var/lib/docker/containers/{id}/{id}-json.log` but microk8s is packaged with snap and it uses it's own docker. So the logs are located in `/var/snap/microk8s/common/var/lib/docker/containers/{id}/{id}-json.log`. You have to mount this location in your log collector for that to work. Following is an example diff for [fluent-bit](https://raw.githubusercontent.com/fluent/fluent-bit-kubernetes-logging/master/output/elasticsearch/fluent-bit-ds.yaml):

```diff
@@ -36,6 +36,9 @@
         - name: varlibdockercontainers
           mountPath: /var/lib/docker/containers
           readOnly: true
+        - name: varlibdockercontainers
+          mountPath: /var/snap/microk8s/common/var/lib/docker/containers/
+          readOnly: true
         - name: fluent-bit-config
           mountPath: /fluent-bit/etc/
       terminationGracePeriodSeconds: 10
@@ -45,7 +48,7 @@
           path: /var/log
       - name: varlibdockercontainers
         hostPath:
-          path: /var/lib/docker/containers
+          path: /var/snap/microk8s/common/var/lib/docker/containers/
       - name: fluent-bit-config
         configMap:
           name: fluent-bit-config
```

205
## Building from source
206

Tim Van Steenburgh's avatar
Tim Van Steenburgh committed
207
Build the snap with:
208
```
Tim Van Steenburgh's avatar
Tim Van Steenburgh committed
209
snapcraft
210
211
```

Tim Van Steenburgh's avatar
Tim Van Steenburgh committed
212
213
### Building for specific versions

214
215
216
You can set the following environment variables prior to building:
 - KUBE_VERSION: kubernetes release to package. Defaults to latest stable.
 - ETCD_VERSION: version of etcd. Defaults to v3.3.4.
217
 - CNI_VERSION: version of CNI tools. Defaults to v0.7.1.
218
 - KUBE_TRACK: kubernetes release series (e.g., 1.10) to package. Defaults to latest stable.
Konstantinos Tsakalozos's avatar
Konstantinos Tsakalozos committed
219
 - ISTIO_VERSION: istio release. Defaults to v1.0.0.
220
221
222

For example:
```
Tim Van Steenburgh's avatar
Tim Van Steenburgh committed
223
KUBE_VERSION=v1.9.6 snapcraft
224
225
```

Tim Van Steenburgh's avatar
Tim Van Steenburgh committed
226
227
### Faster builds

228
229
230
231
232
To speed-up a build you can reuse the binaries already downloaded from a previous build. Binaries are placed under `parts/microk8s/build/build/kube_bins`. All you need to do is to make a copy of this directory and have the `KUBE_SNAP_BINS` environment variable point to it. Try this for example:
```
> snapcraft
... this build will take a long time and will download all binaries ...
> cp -r parts/microk8s/build/build/kube_bins .
233
> export KUBE_SNAP_BINS=$PWD/kube_bins/v1.10.3/
234
> snapcraft clean
235
236
237
238
> snapcraft
... this build will be much faster and will reuse binaries in KUBE_SNAP_BINS

```
239

240
### Installing the snap
241
```
242
snap install microk8s_v1.10.3_amd64.snap --classic --dangerous
243
```
244
245
246

<p align="center">
  <img src="https://assets.ubuntu.com/v1/9309d097-MicroK8s_SnapStore_icon.svg" width="150px">
247
</p>