You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/blog/_posts/2020-01-15-Kubernetes-on-MIPS.md
+61-61Lines changed: 61 additions & 61 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -19,55 +19,55 @@ For many years, to enrich the ecology of the open-source community, we have been
19
19
20
20
Over the years, we have been actively participating in the Kubernetes community and have rich experience in the using and optimization of Kubernetes technology. Recently, we tried to adapt the MIPS architecture platform for Kubernetes and achieved a new a stage on that journey. The team has completed migration and adaptation of Kubernetes and related components, built not only a stable and highly available MIPS cluster but also completed the conformance test for Kubernetes v1.16.2.
21
21
22
-
22
+
23
23
24
-
*Figure 1 Kubernetes on MIPS*
24
+
_Figure 1 Kubernetes on MIPS_
25
25
26
26
## K8S-MIPS component build
27
27
28
28
Almost all native cloud components related to Kubernetes do not provide a MIPS version installation package or image. The prerequisite of deploying Kubernetes on the MIPS platform is to compile and build all required components on the mips64el platform. These components include:
29
29
30
-
- golang
31
-
- docker-ce
32
-
- hyperkube
33
-
- pause
34
-
- etcd
35
-
- calico
36
-
- coredns
37
-
- metrics-server
30
+
- golang
31
+
- docker-ce
32
+
- hyperkube
33
+
- pause
34
+
- etcd
35
+
- calico
36
+
- coredns
37
+
- metrics-server
38
38
39
39
Thanks to the excellent design of Golang and its good support for the MIPS platform, the compilation processes of the above cloud native components are greatly simplified. First of all, we compiled Golang on the latest stable version for the mips64el platform, and then we compiled most of the above components with source code.
40
40
41
-
During the compilation processes, we inevitably encountered many platform compatibility problems, such as a Golang system call compatibility problem (syscall), typecasting of syscall. Stat_t from uint32 to uint64, patching for EpollEvent, and so on.
41
+
During the compilation processes, we inevitably encountered many platform compatibility problems, such as a Golang system call compatibility problem (syscall), typecasting of syscall. Stat_t from uint32 to uint64, patching for EpollEvent, and so on.
42
42
43
43
To build K8S-MIPS components, we used cross-compilation technology. Our process involved integrating a QEMU tool to translate MIPS CPU instructions and modifying the build script of Kubernetes and E2E image script of Kubernetes, Hyperkube, and E2E test images on MIPS architecture.
44
44
45
45
After successfully building the above components, we use tools such as kubespray and kubeadm to complete kubernetes cluster construction.
| metrics-server for CKE on MIPS | 0.3.2 |`registry.inspurcloud.cn/library/cke/kubernetes/metrics-server-mips64el:v0.3.2`|
52
52
| etcd for CKE on MIPS | 3.2.26 |`registry.inspurcloud.cn/library/cke/etcd/etcd-mips64el:v3.2.26`|
53
53
| pause for CKE on MIPS | 3.1 |`registry.inspurcloud.cn/library/cke/kubernetes/pause-mips64el:3.1`|
54
54
| hyperkube for CKE on MIPS | 1.14.3 |`registry.inspurcloud.cn/library/cke/kubernetes/hyperkube-mips64el:v1.14.3`|
55
-
| coredns for CKE on MIPS | 1.6.5 |`registry.inspurcloud.cn/library/cke/kubernetes/coredns-mips64el:v1.6.5`|
55
+
| coredns for CKE on MIPS | 1.6.5 |`registry.inspurcloud.cn/library/cke/kubernetes/coredns-mips64el:v1.6.5`|
56
56
| calico for CKE on MIPS | 3.8.0 |`registry.inspurcloud.cn/library/cke/calico/cni-mips64el:v3.8.0``registry.inspurcloud.cn/library/cke/calico/ctl-mips64el:v3.8.0``registry.inspurcloud.cn/library/cke/calico/node-mips64el:v3.8.0``registry.inspurcloud.cn/library/cke/calico/kube-controllers-mips64el:v3.8.0`|
57
57
58
58
**Note**: CKE is a Kubernetes-based cloud container engine launched by Inspur
@@ -81,52 +81,52 @@ Once the test has started, it launches several pods for various end-to-end tests
81
81
82
82
The first step is to find all needed images for the test. We can run `sonobuoy images-p e2e` command to list all images, or we can find those images in [/test/utils/image/manifest.go](https://github.com/kubernetes/kubernetes/blob/master/test/utils/image/manifest.go). Although Kubernetes officially has a complete Makefile and shell-script that provides commands for building test images, there are still a number of architecture-related issues that have not been resolved, such as the incompatibilities of base images and dependencies. So we cannot directly build mips64el architecture images by executing these commands.
83
83
84
-
Most test images are in golang, then compiled into binaries and built as Docker image based on the corresponding Dockerfile. These images are easy to build. But note that most images are using alpine as their base image, which does not officially support mips64el architecture for now. For this moment, we are unable to make mips64el version of [alpine](https://www.alpinelinux.org/), so we have to replace the alpine to existing MIPS images, such as Debian-stretch, fedora, ubuntu. Replacing the base image also requires replacing the command to install the dependencies, even the version of these dependencies.
84
+
Most test images are in golang, then compiled into binaries and built as Docker image based on the corresponding Dockerfile. These images are easy to build. But note that most images are using alpine as their base image, which does not officially support mips64el architecture for now. For this moment, we are unable to make mips64el version of [alpine](https://www.alpinelinux.org/), so we have to replace the alpine to existing MIPS images, such as Debian-stretch, fedora, ubuntu. Replacing the base image also requires replacing the command to install the dependencies, even the version of these dependencies.
85
85
86
86
Some images are not in `kubernetes/test/images`, such as `gcr.io/google-samples/gb-frontend:v6`. There is no clear documentation explaining where these images are locaated, though we found the source code in repository [github.com/GoogleCloudPlatform/kubernetes-engine-samples](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples). We soon ran into new problems: to build these google sample images, we have to build the base image it uses, even the base image of the base images, such as `php:5-apache`, `redis`, and `perl`.
87
87
88
88
After a long process of building an image, we finished with about four dozen images, including the images used by the test pod, and the base images. The last step before we run the tests is to place all those images into every node in the cluster and make sure the Pod image pull policy is `imagePullPolicy: ifNotPresent`.
Finally, we ran the tests and got the test result, include `e2e.log`, which showed that all test cases passed. Additionally, we submitted our test result to [k8s-conformance](https://github.com/cncf/k8s-conformance) as a [pull request](https://github.com/cncf/k8s-conformance/pull/779).
126
126
127
-
127
+
128
128
129
-
*Figure 5 Pull request for conformance test results*
129
+
_Figure 5 Pull request for conformance test results_
130
130
131
131
## What's next
132
132
@@ -136,8 +136,8 @@ In the future, we plan to actively contribute our experience and achievements to
136
136
137
137
Contribution plan：
138
138
139
-
* contribute the source of e2e test images for MIPS
140
-
* contribute the source of hyperkube for MIPS
141
-
* contribute the source of deploy tools like kubeadm for MIPS
139
+
- contribute the source of e2e test images for MIPS
140
+
- contribute the source of hyperkube for MIPS
141
+
- contribute the source of deploy tools like kubeadm for MIPS
0 commit comments