Windows containers troubleshooting
Download as PPTX, PDF0 likes868 views
This document discusses common issues with running Windows containers in Kubernetes and provides troubleshooting steps. It covers issues with DNS resolution, version mismatches between container images and host OS, and pausing container images. It also provides guidance on choosing specific Windows versions in container images, accessing Windows Server Core containers via RDP, and open questions around configuring pause images and debugging HTTP traffic issues.
![Versions matching
• Container image must match host
• How to check:
• Windows version: major.minor.build.revision ( 10.0.14393.103 )
• Build changes when new version published, revision when Windows updates are applied
• Actually if build numbers are different it blocking from start, for patches it might start
1) Use ‘ver’ inside commang prompt
C:>ver
Microsoft Windows [Version 10.0.16299.125]
2) Read registry
PS C:Usersabokov> (Get-ItemProperty 'HKLM:SOFTWAREMicrosoftWindows
NTCurrentVersion').BuildLabEx
17763.1.amd64fre.rs5_release.180914-1434](https://image.slidesharecdn.com/windowscontainerstroubleshooting-190726215302/85/Windows-containers-troubleshooting-4-320.jpg)
Windows containers troubleshooting
- 1. Windows containers troubleshooting Alexey Bokov, Microsoft, Commercial Software Engineering
- 2. Common troubles 1) Windows Pod are failed to resolve DNS 2) Versions problems 3) Pause image problems
- 3. Windows Pods are failed to resolve DNS After Windows Node rebooted, Host Network Servicy Policy need to be cleaned up # On Windows Node Start-BitsTransfer -Source https://raw.githubusercontent.com/Microsoft/SDN/master/Kubernetes/windows/hns.psm1 Import-Module .hns.psm1 Stop-Service kubeproxy Stop-Service kubelet Get-HnsNetwork | ? Name -eq l2Bridge | Remove-HnsNetwork Get-HnsPolicyList | Remove-HnsPolicyList Start-Service kubelet Start-Service kubeproxy
- 4. Versions matching • Container image must match host • How to check: • Windows version: major.minor.build.revision ( 10.0.14393.103 ) • Build changes when new version published, revision when Windows updates are applied • Actually if build numbers are different it blocking from start, for patches it might start 1) Use ‘ver’ inside commang prompt C:>ver Microsoft Windows [Version 10.0.16299.125] 2) Read registry PS C:Usersabokov> (Get-ItemProperty 'HKLM:SOFTWAREMicrosoftWindows NTCurrentVersion').BuildLabEx 17763.1.amd64fre.rs5_release.180914-1434
- 5. Choose version to use There’s no ‘latest’ tag anymore for Microsoft Windows images You need to specify: FROM mcr.microsoft.com/windows/nanoserver:1809-KB4493509 or FROM mcr.microsoft.com/windows/nanoserver:10.0.17763.437 For ServerCore: FROM mcr.microsoft.com/windows/servercore:ltsc2019
- 6. Windows Server servicing channels LTSC – Long Term Servicing Channel ( 5 mainstream support + 5 years extended), release every 2-3 years. Currently it’s Windows Server 2019 SAC - Semi-Annual Servicing Channel (18 months of support), 2 releases per year, current Windows Server 1903 ( 2019, March ) Long-Term Servicing Channel (Windows Server 2019) Semi-Annual Channel (Windows Server Recommended scenarios General purpose file servers, Microsoft and non-Microsoft workloads, traditional apps, infrastructure roles, software-defined Datacenter, and hyper-converged infrastructure Containerized applications, container hosts, and application scenarios benefiting from faster innovation New releases Every 2–3 years Every 6 months Support 5 years of mainstream support, plus 5 years of extended support 18 months Editions All available Windows Server editions Standard and Datacenter editions Who can use All customers through all channels Software Assurance and cloud customers only Installation options Server Core and Server with Desktop Experience Server Core for container host and image and Nano Server container image
- 7. Versions matching In k8s you may check it with ‘$kubectl describe node 38519acs9010’: .. System Info: Machine ID: 38519acs9010 System UUID: Boot ID: Kernel Version: 10.0 14393 (14393.1715.amd64fre.rs1_release_inmarket.170906-1810) OS Image: Operating System: windows Architecture: amd64 ..
- 8. Image naming Containers on Windows Server 1709 should use images with 1709 tags, e.g. microsoft/aspnet:4.7.2-windowsservercore-1709 microsoft/windowsservercore:1709 microsoft/iis:windowsservercore-1709 Containers on Windows Server 1803 should use images with 1803 tags, e.g. microsoft/aspnet:4.7.2-windowsservercore-1803 microsoft/windowsservercore:1803 microsoft/iis:windowsservercore-1803
- 9. Access to Windows ServerCore Container via RDP (dev/qa only!) Windows Server code has it but disable, to enable set this to 1 HKLMSystemCurrentControlSetControlTerminal ServerTemporaryALiC FROM microsoft/windowsservercore:1709_KB4074588 RUN net user /add abokov RUN net user abokov Abokov!2.718281828 RUN net localgroup "Remote Desktop Users" abokov /add RUN net localgroup "Administrators" abokov /add RUN cmd /k reg add "HKLMSystemCurrentControlSetControlTerminal Server" /v TemporaryALiC /t REG_DWORD /d 1 Or run: cscript C:WindowsSystem32Scregedit.wsf /ar 0
- 10. RDP in K8s (dev/qa only!)# rdp.yaml apiVersion: v1 kind: Service metadata: name: rdp spec: type: LoadBalancer ports: - protocol: TCP port: 3389 targetPort: 3389 --- kind: Endpoints apiVersion: v1 metadata: name: rdp subsets: - addresses: - ip: <node-ip> ports: - port: 3389 $ kubectl create -f rdp.yaml $ kubectl get svc rdp NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE rdp LoadBalancer 10.0.99.149 52.52.52.52 3389:32008/TCP 5m Connect via mstsc.exe -v 52.52.52.52
- 11. Open Questions to talk 1) Configuring pause image 2) Debugging http traffic ( lost packets or web server goes down ) – any alternatives to tcpdump/fiddler 3) Super common topic: vhd -> docker ( or containerize all the things )
- 12. What has gone well • Microsoft leadership in sig-windows • Microsoft engineers respond to bug reports in a timely manner • Meetings with Windows container and container networking teams have been extremely productive Struggles we've had • Configuring HNS and CNI properly • Long-standing Windows platform issues with no timeline for resolution • Development process for Windows CNI plugins • Tracking windows issues Troubleshooting • Debugging Kubernetes test failures is time-consuming • Often we can get something working or develop some workaround, but we don't understand why