Lost ssh access to a node

Have you lost ssh access to one of your Kubernetes nodes? Why do you even need ssh access to nodes in the first place? Well, maybe something is stuck, or you need to see a config with your own eyes… I don’t know and I don’t care, they are your servers, not mine…

I’m assuming you have admin level into kubernetes API.

Talk is cheap, show me the code®:

$ NODE_NAME=master-0

$ kubectl create -n kube-system -f - <<EOF

apiVersion: v1

kind: Pod

metadata:

name: root-shell-$NODE_NAME

namespace: kube-system

spec:

nodeName: $NODE_NAME

containers:

- command:

- /bin/cat

image: alpine:3

name: root-shell

tty: true

stdin: true

volumeMounts:

- mountPath: /host

name: hostroot

hostNetwork: true

hostPID: true

hostIPC: true

tolerations:

- effect: NoSchedule

operator: Exists

- effect: NoExecute

operator: Exists

volumes:

- hostPath:

path: /

name: hostroot

EOF

This pod will create a privileged POD into the node master-0 (change it to your node name) running /bin/cat forever. Now you simply exec into it and change the host’s root to pod’s root:$ kubectl -n kube-system exec -it root-shell-$NODE_NAME chroot /host /bin/bash

[root@master-0 /]# id

uid=0(root) gid=0(root) groups=0(root)

Profit!

PS: Here is a DaemonSet for the lazy

$ kubectl create -n kube-system -f - <<EOF

apiVersion: apps/v1

kind: DaemonSet

metadata:

name: root-shell

namespace: kube-system

spec:

revisionHistoryLimit: 0

selector:

matchLabels:

app: root-shell

template:

metadata:

labels:

app: root-shell

spec:

terminationGracePeriodSeconds: 0

containers:

- command:

- /bin/cat

image: alpine:3

name: root-shell

tty: true

stdin: true

volumeMounts:

- mountPath: /host

name: hostroot

hostNetwork: true

tolerations:

- effect: NoSchedule

operator: Exists

- effect: NoExecute

operator: Exists

volumes:

- hostPath:

path: /

name: hostroot

EOF

Social

Contact us

Almeda Campinas 802, CJ 12, Jardim Paulista,

São Paulo - SP, 01404-001

Opportunities

Our content

Social

Contact us

Almeda Campinas 802, CJ 12, Jardim Paulista,

São Paulo - SP, 01404-001

Opportunities

Our content

Social

Contact us

Almeda Campinas 802, CJ 12, Jardim Paulista,

São Paulo - SP, 01404-001

Opportunities

Our content