Files
Reverse_proxy_Nginx/Nginx.kube.yaml
T
pi-ux-ce 93f8d41aac
Gitea Actions Demo / Explore-Gitea-Actions (push) Failing after 16m7s
fix(reverse-proxy): preserve real client IP in nginx logs
Nginx logs showed internal cluster IPs instead of public client IPs.
Root cause was three stacked issues in the traffic path from MetalLB
LoadBalancer VIP to the reverse-proxy pods:

1. Service was using externalTrafficPolicy: Cluster, allowing
   kube-proxy to forward traffic cross-node and SNAT the source IP
   in the process. Changed to externalTrafficPolicy: Local so traffic
   is only handled by pods on the node that received it.

2. reverse-proxy DaemonSet had a pinned  in
   the pod template, forcing all replicas onto a single node instead
   of scheduling one per node as expected for a DaemonSet. Removed
   nodeName so the DaemonSet scheduler places one pod per eligible
   node (both control-plane nodes, tolerations already in place).

3. Even with the above fixed, flannel's CNI masquerade rule was
   still SNAT'ing traffic to the pod network address (10.244.x.x)
   before it reached nginx, since traffic into the pod overlay
   network gets masqueraded regardless of kube-proxy's policy.
   Switched the DaemonSet to hostNetwork: true (with
   dnsPolicy: ClusterFirstWithHostNet) so nginx binds directly to
   the node's network namespace and bypasses the overlay masquerade
   entirely.

Result: nginx access logs now show the real public client IP for
every request instead of node/pod-internal addresses.
2026-07-23 10:06:43 +01:00

199 lines
5.2 KiB
YAML

apiVersion: v1
kind: Namespace
metadata:
name: reverse-proxy
---
apiVersion: apps/v1
#kind: Deployment
kind: DaemonSet
metadata:
name: reverse-proxy
namespace: reverse-proxy
annotations:
reloader.stakater.com/search: "true"
spec:
#replicas: 1
selector:
matchLabels:
app: reverse-proxy
template:
metadata:
labels:
app: reverse-proxy
spec:
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet # required when hostNetwork: true
#nodeName: kube-master
# Allows the sidecar to see and signal the Nginx process
shareProcessNamespace: true
containers:
#- image: nginx:alpine
- image: gitea.tryton.geeraert.eu/pi-ux-ce/geoip2-nginx:1.29.6
imagePullPolicy: IfNotPresent
name: reverse-proxy
ports:
- containerPort: 80
name: rp-port-http
- containerPort: 443
name: rp-port-https
volumeMounts:
- name: nginx-sites-terraform
mountPath: /etc/nginx/conf.d/
- mountPath: /var/log/nginx/
name: reverse-proxy-log
- name: extra-sites-terraform
mountPath: /etc/nginx/extra.conf.d/
- name: deny-ip-terraform
mountPath: /etc/nginx/deny.d/
- name: blacklisted-sessions-map
mountPath: /etc/nginx/blacklisted.d/
- name: geoip-allow-terraform
mountPath: /etc/nginx/geoip.d/
- name: blacklisted-memory-volume
mountPath: /etc/nginx/blacklisted.memory.d/
- name: reverse-proxy-ssl
mountPath: /etc/nginx/ssl/
- mountPath: /etc/nginx/
name: nginx-configfile
- name: proxy-cache-path-volume
mountPath: /tmp/cache/
- name: secret-htpasswd
mountPath: /authentication-htpasswd/
- name: myprivatecacrt
mountPath: /etc/nginx/certs/
resources:
limits:
cpu: 250m
memory: 250Mi
- name: watcher
image: alpine:latest
command: ["/bin/sh", "-c"]
args:
- |
apk add --no-cache inotify-tools;
# Ensure file exists so inotify doesn't fail
touch /etc/nginx/blacklisted.d/blacklisted-sessions.conf;
echo "Watcher started on /etc/nginx/blacklisted.d/blacklisted-sessions.conf"
while inotifywait -e modify /etc/nginx/blacklisted.d/blacklisted-sessions.conf; do
echo "Change detected, waiting 5 seconds to batch updates..."
sleep 5;
# Check if Nginx is actually running before signaling
if pgrep nginx > /dev/null; then
echo "Reloading Nginx..."
pkill -HUP nginx
fi
done
volumeMounts:
- name: blacklisted-memory-volume
mountPath: /etc/nginx/blacklisted.d/
tolerations:
# these tolerations are to have the daemonset runnable on control plane nodes
# remove them if your control plane nodes should not run pods
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
volumes:
- name: reverse-proxy-log
persistentVolumeClaim:
claimName: reverse-proxy-log
- name: reverse-proxy-ssl
hostPath:
path: /opt/kubernetes/Reverse_proxy_Nginx/certbot/conf/
- name: nginx-configfile
configMap:
name: nginx-config
defaultMode: 0555
- name: nginx-sites-terraform
configMap:
name: sites-terraform
defaultMode: 0555
- name: extra-sites-terraform
configMap:
name: extra-sites-terraform
defaultMode: 0555
- name: deny-ip-terraform
configMap:
name: deny-ip-terraform
defaultMode: 0555
- name: geoip-allow-terraform
configMap:
name: geoip-allow
defaultMode: 0555
- name: proxy-cache-path-volume
emptyDir:
medium: Memory
- name: secret-htpasswd
secret:
secretName: htpasswd
- name: blacklisted-sessions-map
persistentVolumeClaim:
claimName: blacklisted-sessions-map
- name: blacklisted-memory-volume
emptyDir:
medium: Memory
- name: myprivatecacrt
configMap:
name: myprivatecacrt
defaultMode: 0555
---
apiVersion: v1
kind: Service
metadata:
name: reverse-proxy
namespace: reverse-proxy
spec:
externalTrafficPolicy: Local
ports:
- name: http
port: 80
targetPort: rp-port-http
protocol: TCP
nodePort: 30080
- name: https
port: 443
targetPort: rp-port-https
protocol: TCP
nodePort: 30443
type: NodePort
selector:
app: reverse-proxy
---
apiVersion: v1
kind: Service
metadata:
name: reverse-proxy-load
namespace: reverse-proxy
annotations:
spec:
ipFamilyPolicy: PreferDualStack
ports:
- name: http
port: 80
targetPort: rp-port-http
protocol: TCP
- name: https
port: 443
targetPort: rp-port-https
protocol: TCP
type: LoadBalancer
externalTrafficPolicy: Local
selector:
app: reverse-proxy