Skip to content

Traefik return 404 page not found - Kubernetes on Cloudstack 4.17.2.0 #7517

Closed
Closed
Traefik return 404 page not found - Kubernetes on Cloudstack 4.17.2.0#7517
Milestone
unplanned

Description

@tuanhoangth1603
tuanhoangth1603
opened on May 11, 2023

Hello,
I created a Kubernetes cluster on Cloudstack and used this guide (https://docs.cloudstack.apache.org/en/latest/plugins/cloudstack-kubernetes-provider.html) to deploy Traefik as an ingress controller.

  1. According to the guide:
    On successfully deploying the yaml file, a new Public IP Address in the same network as the cluster will be created. It will automatically have the firewall and port forwarding rules configured to distribute any traffic amongst the cluster worker nodes

However, upon inspecting the LB rules, I noticed that the control node was also added as a backend server ?
image

  1. Furthermore, I created and deployed a file that defines a basic deployment, service, and ingress. However, when I curl http://k8s.lab.com.vn/, the response is "404 page not found". Is there a mistake in my YAML file?
    Thanks for your help!
---
apiVersion: v1
kind: Service
metadata:
  name: app
spec:
  selector:
    app: app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
  labels:
    app: app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
      - name: app
        image: nginx:latest
        ports:
        - containerPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: traefik-app-ingress
spec:
  rules:
    - host: k8s.lab.com.vn
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name:  app
                port:
                  number: 80

Activity

  1. tuanhoangth1603
    closed this as completedon May 13, 2023

  2. weizhouapache commented on May 18, 2023

    @weizhouapache
    weizhouapache
    on May 18, 2023
    Member

    @tuanhoangth1603
    have you solved the issue ?

  3. tuanhoangth1603 commented on May 18, 2023

    @tuanhoangth1603
    tuanhoangth1603
    on May 18, 2023
    Author

    I follow this guide and the issue was solved.
    https://doc.traefik.io/traefik/getting-started/install-traefik/

  4. tuanhoangth1603
    reopened this on Aug 21, 2024

  5. tuanhoangth1603 commented on Aug 21, 2024

    @tuanhoangth1603
    tuanhoangth1603
    on Aug 21, 2024
    Author

    Hello,

    Today I following this procedure to deploy k8s with traefik ingress.
    I am facing an error with traefik ingress: "Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value"

    • Cloudstack version: 4.19.0.1
    • K8S: Version: v1.26.0
    • traefik: v3.1.2

    Here is full describe of service:

    Name:                     traefik
Namespace:                default
Labels:                   app.kubernetes.io/instance=traefik-default
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=traefik
                          helm.sh/chart=traefik-30.1.0
Annotations:              meta.helm.sh/release-name: traefik
                          meta.helm.sh/release-namespace: default
                          service.beta.kubernetes.io/cloudstack-load-balancer-proxy-protocol: enabled
Selector:                 app.kubernetes.io/instance=traefik-default,app.kubernetes.io/name=traefik
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.107.132.3
IPs:                      10.107.132.3
Port:                     http  80/TCP
TargetPort:               http/TCP
NodePort:                 http  30893/TCP
Endpoints:                <none>
Port:                     https  443/TCP
TargetPort:               https/TCP
NodePort:                 https  32256/TCP
Endpoints:                <none>
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type     Reason                  Age                  From                Message
  ----     ------                  ----                 ----                -------
  Normal   EnsuringLoadBalancer    58s (x6 over 2m14s)  service-controller  Ensuring load balancer
  Warning  SyncLoadBalancerFailed  58s (x6 over 2m14s)  service-controller  Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value

  6. weizhouapache commented on Aug 21, 2024

    @weizhouapache
    weizhouapache
    on Aug 21, 2024
    Member

    any logs on cloudstack management server and in the pod cloud-controller-manager ?

  7. tuanhoangth1603 commented on Aug 21, 2024

    @tuanhoangth1603
    tuanhoangth1603
    on Aug 21, 2024
    Author

    any logs on cloudstack management server and in the pod cloud-controller-manager ?

    Thanks for your reply. Here is logs of CCM

    W0821 15:11:05.221926       1 client_config.go:552] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0821 15:11:05.420413       1 controllermanager.go:120] Version: v0.0.0-master+$Format:%h$
I0821 15:11:05.422112       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0821 15:11:05.422169       1 shared_informer.go:223] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0821 15:11:05.422223       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0821 15:11:05.422240       1 shared_informer.go:223] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0821 15:11:05.618715       1 secure_serving.go:178] Serving securely on [::]:10258
I0821 15:11:05.618813       1 leaderelection.go:242] attempting to acquire leader lease  kube-system/cloud-controller-manager...
I0821 15:11:05.619351       1 tlsconfig.go:240] Starting DynamicServingCertificateController
I0821 15:11:05.918307       1 shared_informer.go:230] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0821 15:11:05.918307       1 shared_informer.go:230] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0821 15:11:06.018794       1 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"cloud-controller-manager", UID:"79917607-9b73-462e-bf49-76a5cb88c24b", APIVersion:"v1", ResourceVersion:"785", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' cloud-controller-manager-574bcb86c-fn5r5_957b1aed-2d27-4dac-b6af-418d983f7572 became leader
E0821 15:11:06.018661       1 event.go:316] Could not construct reference to: '&v1.Lease{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cloud-controller-manager", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"33584655-e620-4127-9ce5-e1ac08e95df8", ResourceVersion:"786", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63859849865, loc:(*time.Location)(0x30c2040)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"cloudstack-ccm", Operation:"Update", APIVersion:"coordination.k8s.io/v1", Time:(*v1.Time)(0xc0001a6340), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0001a6380)}}}, Spec:v1.LeaseSpec{HolderIdentity:(*string)(nil), LeaseDurationSeconds:(*int32)(nil), AcquireTime:(*v1.MicroTime)(nil), RenewTime:(*v1.MicroTime)(nil), LeaseTransitions:(*int32)(nil)}}' due to: 'no kind is registered for the type v1.Lease in scheme "pkg/runtime/scheme.go:101"'. Will not report event: 'Normal' 'LeaderElection' 'cloud-controller-manager-574bcb86c-fn5r5_957b1aed-2d27-4dac-b6af-418d983f7572 became leader'
I0821 15:11:06.118337       1 leaderelection.go:252] successfully acquired lease kube-system/cloud-controller-manager
I0821 15:11:06.120413       1 node_controller.go:110] Sending events to api server.
I0821 15:11:06.120516       1 controllermanager.go:247] Started "cloud-node"
I0821 15:11:06.220667       1 node_lifecycle_controller.go:78] Sending events to api server
I0821 15:11:06.220758       1 controllermanager.go:247] Started "cloud-node-lifecycle"
I0821 15:11:06.319991       1 controllermanager.go:247] Started "service"
I0821 15:11:06.320047       1 core.go:101] Will not configure cloud provider routes for allocate-node-cidrs: false, configure-cloud-routes: true.
W0821 15:11:06.320064       1 controllermanager.go:244] Skipping "route"
I0821 15:11:06.418280       1 controller.go:208] Starting service controller
I0821 15:11:06.418313       1 shared_informer.go:223] Waiting for caches to sync for service
I0821 15:11:06.918866       1 shared_informer.go:230] Caches are synced for service
E0821 15:11:08.719857       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:11:08.721984       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-control-1917579d77d": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:11:08.818466       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:11:08.821324       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a2e0b": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:11:08.823046       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:11:08.920077       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a899c": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:16:09.221094       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:16:09.319975       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-control-1917579d77d": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:16:09.321029       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:16:09.323385       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a2e0b": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:16:09.324513       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:16:09.327037       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a899c": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:21:09.922209       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:21:10.019894       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-control-1917579d77d": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:21:10.021178       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:21:10.118357       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a2e0b": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:21:10.120241       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:21:10.122821       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a899c": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
I0821 15:25:38.304728       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3023", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:25:38.519534       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:25:38.519646       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3023", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:25:43.520304       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3026", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:25:43.521805       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:25:43.521878       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3026", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:25:53.522501       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3026", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:25:53.524087       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:25:53.524171       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3026", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
E0821 15:26:10.619873       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:26:10.622562       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-control-1917579d77d": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:26:10.718288       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:26:10.721020       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a2e0b": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:26:10.722174       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:26:10.724422       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a899c": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
I0821 15:26:11.190439       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:26:11.191841       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:26:11.191884       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:26:13.524803       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:26:13.526348       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:26:13.526489       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:27:33.527261       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:27:33.721428       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:27:33.818172       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:30:13.723275       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Normal' reason: 'EnsuringLoadBalancer' Ensuring load balancer
E0821 15:30:13.921406       1 controller.go:244] error processing service default/traefik (will retry): failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
I0821 15:30:14.018213       1 event.go:278] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"traefik", UID:"9b64629a-10b0-4dd7-a7c1-d768d6f502bd", APIVersion:"v1", ResourceVersion:"3141", FieldPath:""}): type: 'Warning' reason: 'SyncLoadBalancerFailed' Error syncing load balancer: failed to ensure load balancer: error retrieving load balancer rules: invalid character '<' looking for beginning of value
E0821 15:31:10.921484       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:31:11.119953       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-control-1917579d77d": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:31:11.121277       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:31:11.123845       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a2e0b": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value
E0821 15:31:11.125066       1 node_controller.go:237] error retrieving instance ID: invalid character '<' looking for beginning of value
E0821 15:31:11.127270       1 node_controller.go:245] Error getting node addresses for node "tuanlab-k8s-node-191757a899c": error fetching node by provider ID: error retrieving node addresses: invalid character '<' looking for beginning of value, and error by node name: error retrieving node addresses: invalid character '<' looking for beginning of value

  8. tuanhoangth1603 commented on Aug 21, 2024

    @tuanhoangth1603
    tuanhoangth1603
    on Aug 21, 2024
    Author

    and I also found this log in mgmt to be helpful

    2024-08-21 22:09:17,133 WARN  [c.c.k.c.u.KubernetesClusterUtil] (API-Job-Executor-47:ctx-4b9b2287 job-16540 ctx-d327ad2e) (logid:584f4248) API endpoint for Kubernetes cluster : tuanlab-k8s not available
javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
	at java.base/sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1701)
	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1519)
	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:456)
	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:427)
	at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:201)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1613)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1541)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
	at com.cloud.kubernetes.cluster.utils.KubernetesClusterUtil.isKubernetesClusterServerRunning(KubernetesClusterUtil.java:239)
	at com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterStartWorker.startKubernetesClusterOnCreate(KubernetesClusterStartWorker.java:546)
	at com.cloud.kubernetes.cluster.KubernetesClusterManagerImpl.startKubernetesCluster(KubernetesClusterManagerImpl.java:1321)
	at org.apache.cloudstack.api.command.user.kubernetes.cluster.CreateKubernetesClusterCmd.execute(CreateKubernetesClusterCmd.java:294)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:172)
	at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:112)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:654)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
	at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:602)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
	at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478)
	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160)
	... 28 more

  9. tuanhoangth1603 commented on Aug 22, 2024

    @tuanhoangth1603
    tuanhoangth1603
    on Aug 22, 2024
    Author

    Hi @weizhouapache,

    Do you have any suggestion for this issue ?

  10. weizhouapache commented on Aug 22, 2024

    @weizhouapache
    weizhouapache
    on Aug 22, 2024
    Member

    any logs on cloudstack management server and in the pod cloud-controller-manager ?

    @tuanhoangth1603
    it looks like failed to get the cks nodes/vms "tuanlab-k8s-control-1917579d77d" and "tuanlab-k8s-node-191757a899c"

    can you try with cloudstack-kubernetes-provider v1.1.0 ?

    1. delete cloudstack-kubernetes-provider 1.0.0
      kubectl delete -f
      https://raw.githubusercontent.com/apache/cloudstack-kubernetes-provider/main/deployment.yaml

    2. install 1.1.0
      kubectl apply -f
      https://github.com/apache/cloudstack-kubernetes-provider/releases/download/v1.1.0/deployment.yaml

  11. tuanhoangth1603 commented on Aug 22, 2024

    @tuanhoangth1603
    tuanhoangth1603
    on Aug 22, 2024
    Author

    Thanks for you idea.
    Sorry, but are you mistaken? Deleting the main release and installing version 1.1.0 are the same. I have checked the two manifest deployment.yaml files, and they are exactly the same.

  12. weizhouapache commented on Aug 22, 2024

    @weizhouapache
    weizhouapache
    on Aug 22, 2024
    Member

    Thanks for you idea.
    Sorry, but are you mistaken? Deleting the main release and installing version 1.1.0 are the same. I have checked the two manifest deployment.yaml files, and they are exactly the same.

    It is used to recreate ccm with latest 1.1.0

  13. weizhouapache commented on Aug 22, 2024

    @weizhouapache
    weizhouapache
    on Aug 22, 2024
    Member

    Thanks for you idea.
    Sorry, but are you mistaken? Deleting the main release and installing version 1.1.0 are the same. I have checked the two manifest deployment.yaml files, and they are exactly the same.

    It is used to recreate ccm with latest 1.1.0
    If you have already installed 1.1.0, ignore it

  14. DaanHoogland
    added this to the unplanned milestone on Nov 8, 2024
  15. apache
    locked and limited conversation to collaborators on Nov 8, 2024
  16. DaanHoogland
    converted this issue into a discussion #9906 on Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions