Your pods are running. You can kubectl exec into them. But the moment you try to reach one pod from another — or expose your app to the internet — everything falls apart. Welcome to Kubernetes networking, the layer that confuses everyone, including people who've been running clusters for years.
The good news: there are only three abstractions you need to understand. Services give pods a stable network identity. Ingress routes external HTTP traffic into your cluster. NetworkPolicies decide who is allowed to talk to whom. Master these three and 90% of your networking pain disappears.
1. Why Pod IPs Are Useless (and Services Fix That)
A pod's IP address is ephemeral. Restart the pod, get a new IP. Scale from 3 replicas to 10, and you now have 7 new IPs to keep track of. Nothing in your code should ever hardcode a pod IP.
A Service is a stable virtual IP (ClusterIP) in front of a set of pods. The Service selector matches pod labels, and the kube-proxy component programs iptables or IPVS rules to load-balance traffic across the matching pods.
apiVersion: v1
kind: Service
metadata:
name: api
spec:
selector:
app: api
ports:
- port: 80 # port clients use
targetPort: 8080 # port the container listens on
Now any pod in the cluster can reach your API at http://api:80. That's it. The Service name becomes a DNS name — Kubernetes runs a built-in DNS service (CoreDNS) that resolves Service names automatically.
2. Service Types: When to Use What
You'll see four Service types in the wild. Each solves a different problem:
| Type | Reachable From | Use Case |
|---|---|---|
ClusterIP | Inside cluster only | Default. Service-to-service communication. |
NodePort | Outside via nodeIP:port | Quick dev testing, or when you don't have a cloud load balancer. |
LoadBalancer | Public internet | Cloud clusters (AWS, GCP, Azure). Creates a real LB. |
Headless | Via individual pod DNS | StatefulSets, databases that need direct pod addressing. |
For a stateful database, you typically want a headless Service (clusterIP: None). DNS then returns the IP of every matching pod, so your app can connect to db-0.redis directly instead of going through a load balancer — essential when each replica holds different data.
3. Ingress: One Entry Point for HTTP Traffic
If you create a LoadBalancer Service for every microservice, you pay for a cloud load balancer per service. Expensive and messy. Ingress gives you a single entry point that routes requests based on hostname and path.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: main
spec:
ingressClassName: nginx
rules:
- host: api.example.com
http:
paths:
- path: /v1
pathType: Prefix
backend:
service:
name: api
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 80
Requests to api.example.com/v1/* go to the api Service; everything else goes to web. Most Ingress controllers (nginx-ingress, Traefik, AWS ALB Ingress Controller) also handle TLS termination — you attach a Secret containing the certificate and the controller takes care of the rest:
spec:
tls:
- hosts:
- api.example.com
secretName: api-tls
One Ingress, one IP, any number of hostnames and paths. This is how real production clusters expose dozens of services with a single load balancer.
4. NetworkPolicies: Default Allow Is a Security Hole
By default, every pod can talk to every other pod. In a production cluster, that's a security nightmare — a compromised frontend can reach your database directly. NetworkPolicies are firewall rules enforced at the CNI layer (Calico, Cilium, Weave).
A deny-by-default policy for the db tier looks like this:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: db-allow-api-only
spec:
podSelector:
matchLabels:
app: db
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: api
ports:
- protocol: TCP
port: 5432
Once this policy exists, the db pods accept TCP traffic on port 5432 only from pods labeled app: api. Everything else is dropped. Add one policy per tier and your cluster goes from flat open network to least-privilege segmentation.
NetworkPolicies are additive per pod — if multiple policies select the same pod, their rules combine (OR). A default-deny policy needs an empty podSelector: {} plus explicit allow rules for everything that should get through.
5. The Traffic Flow, End to End
Here's the complete path of a request in a typical setup:
- Client hits
https://api.example.com→ DNS resolves to the cloud load balancer → forwards to the Ingress controller pod. - The Ingress controller matches the hostname/path against its rules and proxies to the
apiService's ClusterIP. - kube-proxy rewrites the destination to a healthy backend pod's IP using iptables/IPVS rules, distributing load across replicas.
- The pod receives the request on
targetPort. If the pod is later restarted or scaled, only the Service's endpoint list changes — clients never notice.
Debugging tip: when traffic "doesn't work", isolate the hop. kubectl get endpoints api tells you if the Service found any pods. kubectl describe pod reveals NetworkPolicy blocks via events. And kubectl run netshoot --rm -it --image nicolaka/netshoot gives you a Swiss-army pod with curl, dig, tcpdump, and nc for testing from inside the cluster.
6. Production Checklist
- Never hardcode pod IPs. Always address services by DNS name.
- Prefer Ingress over one LoadBalancer per service. One entry point, one TLS cert, one cost.
- Enable NetworkPolicies early. Retrofitting them into a cluster where everything already talks is painful; do it at day one.
- Check
kubectl get endpointsfirst. Most "networking broken" reports are actually a selector typo — the Service matches zero pods. - Use headless Services for stateful workloads and normal ClusterIP for stateless ones.
Summary
Kubernetes networking isn't magic — it's three layers with clear jobs. Services stabilize pod addressing and load-balance internally. Ingress is the single front door for external HTTP traffic, with routing and TLS handled at one point. NetworkPolicies turn your cluster's default-open network into a segmented, least-privilege one. Understand which abstraction solves which problem, and the tangle of IPs, ports, and proxies becomes a boring, predictable system you can debug in minutes.