1185 lines
33 KiB
Markdown
1185 lines
33 KiB
Markdown
# Terraform Kubernetes Deployer
|
|
|
|
A Terraform module that deploys containerised applications on Kubernetes with a single, unified variable block. It handles namespaces, deployments, services, persistent volumes, databases, Redis, backups, secrets (via Bitwarden Secrets Manager), reverse-proxy nginx config, and more — all from one `kubernetes` map in your `tfvars` file.
|
|
|
|
---
|
|
|
|
## How it works
|
|
|
|
Every application is declared as a key inside the `kubernetes` map. The key name is used as the default namespace, deployment name, and service selector. Terraform then automatically creates:
|
|
|
|
- A **Kubernetes namespace** (unless you attach to an existing one with `connected_namespace`)
|
|
- A **Deployment** for the application
|
|
- A **Service** (if `service_port` is specified)
|
|
- **PersistentVolumeClaims** (if `pvc` is specified)
|
|
- A **database Deployment + Service + PVC + backup CronJob** (if `database` is specified)
|
|
- A **Redis Deployment + Service + PVC** (if `redis` is specified)
|
|
- **Kubernetes Secrets / ConfigMaps** imported from Bitwarden Secrets Manager (if `secret_import` / `config_map_import` is specified)
|
|
- An **nginx reverse-proxy config** entry (if `domain_name` is specified)
|
|
- Any additional raw Kubernetes manifests (if `extra_manifest` is specified)
|
|
|
|
---
|
|
|
|
## Prerequisites
|
|
|
|
- Terraform `> 0.8.0`
|
|
- Bitwarden Secrets Manager provider (`bitwarden/bitwarden-secrets` `0.5.4-pre`)
|
|
- A running Kubernetes cluster
|
|
- NFS storage classes: `nfs-storage-intelligent` (default) and `nfs-storage-fastaccess` (used for backups and fast-access volumes)
|
|
- MetalLB (required only when using `service_type = "LoadBalancer"`)
|
|
- A Stakater Reloader or equivalent (used for the nginx ConfigMap annotation)
|
|
|
|
---
|
|
|
|
## Full variable reference
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
<app-key> = {
|
|
# --- Identity ---
|
|
name = optional(string) # Override deployment name (defaults to map key)
|
|
image = string # REQUIRED. Docker image
|
|
namespace = optional(string) # Override namespace (defaults to map key)
|
|
connected_namespace = optional(string, "") # Attach to an existing namespace instead of creating one
|
|
enabled = optional(bool, true) # Disable everything (deployments, PVCs, services, secrets)
|
|
replicas = optional(number, 1) # Number of pod replicas
|
|
|
|
# --- Networking ---
|
|
service_port = optional(list(number)) # Expose these ports via a ClusterIP (or other) service
|
|
service_type = optional(string, "ClusterIP") # "ClusterIP" | "LoadBalancer" | "NodePort"
|
|
service_name = optional(string) # Override service name
|
|
domain_name = optional(string) # Generate an nginx reverse-proxy config for this subdomain
|
|
dns_nameservers = optional(string) # Custom DNS server for the pod (e.g. "1.1.1.1")
|
|
|
|
# --- Compute & Security ---
|
|
command = optional(list(string)) # Override container entrypoint
|
|
requirements = optional(map(string)) # CPU/memory limits & requests
|
|
security_context = optional(bool, false) # Run container as privileged
|
|
service_account_name = optional(string) # Attach a ServiceAccount to the pod
|
|
|
|
# --- Liveness probes ---
|
|
liveness_tcp_socket = optional(number) # TCP port to probe
|
|
liveness_http_get = optional(string) # HTTP port to probe
|
|
liveness_http_get_path = optional(string) # HTTP path (defaults to "/")
|
|
|
|
# --- Environment ---
|
|
environment = optional(map(object({ # Per-variable env config (value or secret ref)
|
|
value = optional(string)
|
|
value_from = optional(object({
|
|
secret_key_ref = object({
|
|
name = string
|
|
key = optional(string, null)
|
|
})
|
|
}))
|
|
})))
|
|
|
|
# --- Secrets & ConfigMaps ---
|
|
secret_ref = optional(list(string)) # Mount entire Kubernetes Secrets as env
|
|
config_map_ref = optional(list(string)) # Mount entire Kubernetes ConfigMaps as env
|
|
secret_import = optional(list(string)) # Pull Bitwarden secret IDs → create K8s Secrets
|
|
config_map_import = optional(list(string)) # Pull Bitwarden secret IDs → create K8s ConfigMaps
|
|
extra_manifest = optional(list(string)) # Paths to raw YAML manifests to apply
|
|
|
|
# --- Volumes ---
|
|
pvc = optional(map(object({
|
|
name = string
|
|
pvc = optional(bool, false) # Create/use a PVC
|
|
config_map = optional(bool, false) # Mount a ConfigMap as a volume
|
|
secret = optional(bool, false) # Mount a Secret as a volume
|
|
host_path = optional(string) # Mount a host path
|
|
pvc_size = optional(string) # PVC size (e.g. "2Gi")
|
|
storage_class_name = optional(string, "nfs-storage-intelligent")
|
|
annotations_fileType = optional(string) # PVC annotation
|
|
existing_pvc = optional(bool, false) # Reuse an existing PVC (skip creation)
|
|
mount_path = string # REQUIRED. Where to mount in container
|
|
sub_path = optional(string) # Sub-path inside the volume
|
|
})))
|
|
|
|
# --- Database (auto-provisions DB deployment, service, PVC, backup CronJob, Bitwarden secrets) ---
|
|
database = optional(map(string))
|
|
# database keys:
|
|
# engine - "postgres" | "mysql" | "sqlserver" (default: postgres)
|
|
# image - DB container image (default: postgres:latest)
|
|
# port - DB port (default: 5432)
|
|
# mountPath - DB data mount path (default: /var/lib/postgresql/)
|
|
# username - DB username (default: map key)
|
|
# name - DB name (default: map key)
|
|
# host - External DB hostname (skips creating DB deployment)
|
|
# map_name - Custom env var name for DB name in the mapped secret
|
|
# map_host - Custom env var name for DB host in the mapped secret
|
|
# map_password - Custom env var name for DB password in the mapped secret
|
|
# map_username - Custom env var name for DB username in the mapped secret
|
|
# shunt - "true" to skip creating the DB deployment (use when the container IS the DB)
|
|
database_args = optional(list(string)) # Args passed to the DB container
|
|
database_command = optional(list(string)) # Command override for DB container
|
|
|
|
# --- Redis (auto-provisions Redis deployment, service, PVC) ---
|
|
redis = optional(map(string))
|
|
# redis keys:
|
|
# image - Redis image (default: docker.io/library/redis:7)
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Feature guide
|
|
|
|
### 1. Minimal deployment
|
|
|
|
The smallest possible entry — just an image. Terraform will create a namespace called `myapp`, a deployment, and nothing else.
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 2. Enabling and disabling an application
|
|
|
|
Use `enabled = false` to destroy all resources for an application while keeping it in the tfvars file for reference. To keep the namespace, PVCs, and secrets but simply stop the pods, set `replicas = 0` instead.
|
|
|
|
**Minimal — disable everything:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
enabled = false
|
|
}
|
|
}
|
|
```
|
|
|
|
**Stop pods only (keep storage):**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
replicas = 0
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 3. Service / networking
|
|
|
|
Add `service_port` to expose the application via a Kubernetes Service. Multiple ports are supported. The default service type is `ClusterIP`.
|
|
|
|
**Minimal — single port:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
service_port = [80]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — multiple ports, LoadBalancer with MetalLB:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "analogic/poste.io:latest"
|
|
service_port = [25, 80, 110, 143, 443, 465, 587, 993]
|
|
service_type = "LoadBalancer"
|
|
}
|
|
}
|
|
```
|
|
|
|
> When `service_type = "LoadBalancer"`, the service is automatically annotated with `metallb.universe.tf/ip-allocated-from-pool: first-pool`.
|
|
|
|
---
|
|
|
|
### 4. Namespace management
|
|
|
|
By default, Terraform creates a namespace matching the map key. Use `namespace` to give it a different name, or `connected_namespace` to deploy into an existing namespace without creating a new one.
|
|
|
|
**Minimal — custom namespace name:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
namespace = "webservers"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — attach to an existing namespace (no new namespace created):**
|
|
```hcl
|
|
kubernetes = {
|
|
celery-worker = {
|
|
image = "myrepo/django:latest"
|
|
connected_namespace = "django"
|
|
command = ["celery", "-A", "core", "worker"]
|
|
secret_ref = ["django-bitwarden-credentials"]
|
|
}
|
|
}
|
|
```
|
|
|
|
> `connected_namespace` is useful for sidecars or worker processes that share the namespace of a primary application.
|
|
|
|
---
|
|
|
|
### 5. Environment variables
|
|
|
|
Two styles are supported: a direct `value`, or a reference to a key inside a Kubernetes Secret (`value_from`).
|
|
|
|
**Minimal — direct value:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
environment = {
|
|
NGINX_PORT = { value = "80" }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — mix of direct values and secret references:**
|
|
```hcl
|
|
kubernetes = {
|
|
paperless = {
|
|
image = "ghcr.io/paperless-ngx/paperless-ngx:2.18.4"
|
|
service_port = [8000]
|
|
environment = {
|
|
PAPERLESS_URL = {
|
|
value = "https://paperless.example.com"
|
|
}
|
|
PAPERLESS_TIKA_ENABLED = {
|
|
value = "1"
|
|
}
|
|
PAPERLESS_REDIS = {
|
|
value = "redis://paperless-redis-svc:6379"
|
|
}
|
|
POSTGRES_USER = {
|
|
value_from = {
|
|
secret_key_ref = {
|
|
name = "paperless-credentials"
|
|
key = "POSTGRES_USER"
|
|
}
|
|
}
|
|
}
|
|
POSTGRES_PASSWORD = {
|
|
value_from = {
|
|
secret_key_ref = {
|
|
name = "paperless-credentials"
|
|
key = "POSTGRES_PASSWORD"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 6. Secrets and ConfigMaps as environment
|
|
|
|
Mount an entire Kubernetes Secret or ConfigMap as environment variables (all keys become env vars).
|
|
|
|
**Minimal — mount a single secret:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
secret_ref = ["my-app-credentials"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — multiple secrets and configmaps:**
|
|
```hcl
|
|
kubernetes = {
|
|
bookstack = {
|
|
image = "linuxserver/bookstack:latest"
|
|
service_port = [80, 443]
|
|
secret_ref = ["bookstack-credentials", "bookstack-extra-secret"]
|
|
config_map_ref = ["bookstack-config", "global-config"]
|
|
database = {
|
|
engine = "mysql"
|
|
image = "lscr.io/linuxserver/mariadb"
|
|
port = 3306
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 7. Importing secrets and ConfigMaps from Bitwarden Secrets Manager
|
|
|
|
Use `secret_import` or `config_map_import` with Bitwarden secret UUIDs. Terraform fetches the secret from Bitwarden Secrets Manager and creates the corresponding Kubernetes resource. The Bitwarden secret's `note` field becomes the Kubernetes resource name, the `key` field (split by `__`) provides the namespace, and the `value` field contains YAML-encoded key-value pairs.
|
|
|
|
**Minimal — import one secret:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
secret_import = ["9fc5a725-75f5-4bab-b595-b474017146c3"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — import multiple secrets and configmaps:**
|
|
```hcl
|
|
kubernetes = {
|
|
fail2ban = {
|
|
image = "gitea.example.com/myorg/fail2ban:latest"
|
|
config_map_import = [
|
|
"9d8ef71a-de74-478b-aa86-b3ee0160dc43",
|
|
"50814edb-d190-41cb-8950-b3ee01622dcf",
|
|
]
|
|
secret_import = ["d1c2b8c9-71a0-4e47-9d9d-b402018376b6"]
|
|
environment = {
|
|
PUID = { value = "1000" }
|
|
TZ = { value = "Etc/UTC" }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 8. Extra raw Kubernetes manifests
|
|
|
|
Apply additional Kubernetes YAML files alongside the Terraform-managed resources. Useful for resources Terraform doesn't manage natively (custom ServiceAccounts, Roles, CronJobs, etc.).
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
extra_manifest = ["/opt/kubernetes/myapp/secret.yaml"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — multiple manifests:**
|
|
```hcl
|
|
kubernetes = {
|
|
fail2ban = {
|
|
image = "gitea.example.com/myorg/fail2ban:latest"
|
|
extra_manifest = [
|
|
"/opt/kubernetes/fail2ban/fail2ban.RoleBinding.yaml",
|
|
"/opt/kubernetes/fail2ban/fail2ban.Role.yaml",
|
|
"/opt/kubernetes/fail2ban/fail2ban.ServiceAccount.yaml",
|
|
]
|
|
service_account_name = "kubectl-service-account"
|
|
security_context = true
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 9. Volumes
|
|
|
|
The `pvc` block supports four volume types. You can declare multiple volumes per application.
|
|
|
|
#### 9a. PVC (Persistent Volume Claim)
|
|
|
|
Terraform creates the PVC automatically. Default storage class is `nfs-storage-intelligent`.
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
pvc = {
|
|
data = {
|
|
name = "myapp-data"
|
|
pvc = true
|
|
pvc_size = "5Gi"
|
|
mount_path = "/data"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — multiple PVCs with custom storage class, sub-path, and annotation:**
|
|
```hcl
|
|
kubernetes = {
|
|
stash = {
|
|
image = "stashapp/stash:v0.30.1"
|
|
service_port = [9999]
|
|
pvc = {
|
|
data = {
|
|
name = "stash-data"
|
|
pvc = true
|
|
pvc_size = "500Gi"
|
|
mount_path = "/data"
|
|
storage_class_name = "nfs-storage-fastaccess"
|
|
annotations_fileType = "Video"
|
|
}
|
|
metadata = {
|
|
name = "stash-metadata"
|
|
pvc = true
|
|
pvc_size = "10Gi"
|
|
mount_path = "/metadata"
|
|
sub_path = "stash"
|
|
}
|
|
config = {
|
|
name = "stash-config"
|
|
pvc = true
|
|
pvc_size = "1Gi"
|
|
mount_path = "/root/.stash"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 9b. Host path
|
|
|
|
Mount a directory from the Kubernetes node directly into the container. No PVC is created.
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
pvc = {
|
|
node-data = {
|
|
host_path = "/data/myapp"
|
|
name = "node-data"
|
|
mount_path = "/data"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — mixing PVC and host path:**
|
|
```hcl
|
|
kubernetes = {
|
|
jellyfin = {
|
|
image = "jellyfin/jellyfin:latest"
|
|
service_port = [8096]
|
|
pvc = {
|
|
config = {
|
|
name = "jellyfin-config"
|
|
pvc = true
|
|
pvc_size = "10Gi"
|
|
mount_path = "/config"
|
|
}
|
|
media = {
|
|
host_path = "/data/media"
|
|
name = "jellyfin-media"
|
|
mount_path = "/data"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 9c. ConfigMap as volume
|
|
|
|
Mount a Kubernetes ConfigMap as a directory inside the container.
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
pvc = {
|
|
nginx-conf = {
|
|
config_map = true
|
|
name = "my-nginx-config"
|
|
mount_path = "/etc/nginx/conf.d"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — fail2ban jail and filter configs from ConfigMaps:**
|
|
```hcl
|
|
kubernetes = {
|
|
fail2ban = {
|
|
image = "gitea.example.com/myorg/fail2ban:latest"
|
|
pvc = {
|
|
fail2ban-jail = {
|
|
config_map = true
|
|
name = "fail2ban-nginx-http-auth-jail"
|
|
mount_path = "/config/fail2ban/jail.d/"
|
|
}
|
|
fail2ban-filter = {
|
|
config_map = true
|
|
name = "fail2ban-nginx-unauthorized-filter"
|
|
mount_path = "/config/fail2ban/filter.d/"
|
|
}
|
|
fail2ban-logs = {
|
|
host_path = "/data/kubernetes/reverse-proxy/reverse-proxy-log/"
|
|
name = "nginx-fail2ban-logs"
|
|
mount_path = "/remotelogs/nginx/"
|
|
}
|
|
fail2ban-config = {
|
|
pvc = true
|
|
pvc_size = "2Gi"
|
|
name = "fail2ban-config"
|
|
mount_path = "/config"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 9d. Secret as volume
|
|
|
|
Mount a Kubernetes Secret as a directory inside the container.
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
pvc = {
|
|
my-secret-vol = {
|
|
secret = true
|
|
name = "my-tls-secret"
|
|
mount_path = "/certs"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — kubeconfig secret mounted alongside a host path:**
|
|
```hcl
|
|
kubernetes = {
|
|
django = {
|
|
image = "myrepo/django:latest"
|
|
service_port = [8000]
|
|
namespace = "django"
|
|
pvc = {
|
|
kubeconfig-secret = {
|
|
secret = true
|
|
name = "kubeconfig"
|
|
mount_path = "/kubeconfig"
|
|
}
|
|
stash-data = {
|
|
host_path = "/ssd/stash/data-pvc/"
|
|
name = "stash-django-pvc"
|
|
mount_path = "/stash"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 9e. Reusing an existing PVC
|
|
|
|
Set `existing_pvc = true` to mount a PVC that already exists without Terraform trying to create it. Useful when two applications share storage.
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
echo = {
|
|
image = "ealen/echo-server:latest"
|
|
service_port = [80]
|
|
namespace = "echo"
|
|
pvc = {
|
|
shared-config = {
|
|
name = "echo-config"
|
|
pvc = true
|
|
pvc_size = "10Mi"
|
|
mount_path = "/app/configs"
|
|
}
|
|
}
|
|
}
|
|
|
|
echo2 = {
|
|
image = "ealen/echo-server:latest"
|
|
service_port = [80]
|
|
connected_namespace = "echo"
|
|
pvc = {
|
|
shared-config = {
|
|
name = "echo-config"
|
|
pvc = true
|
|
pvc_size = "10Mi"
|
|
mount_path = "/app/configs"
|
|
existing_pvc = true # do not recreate; reuse what echo created
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 10. Database
|
|
|
|
Adding a `database` block automatically provisions: a DB deployment, a DB service (`<key>-db-svc`), a DB PVC (`<key>-db-pvc`), a backup PVC (`<key>-db-backup`), a daily backup CronJob, and a Kubernetes Secret (`<key>-db-terraform`) containing all common DB env vars in multiple naming conventions (so most apps just work without extra config).
|
|
|
|
The generated secret (`<key>-db-terraform`) contains: `DB_DATABASE_NAME`, `DB_DATABASE`, `POSTGRES_DB`, `MYSQL_DATABASE`, `PGDATABASE`, `DB_HOST`, `PGHOST`, `POSTGRES_HOST`, `DB_PASSWORD`, `MYSQL_PASSWORD`, `MYSQL_ROOT_PASSWORD`, `POSTGRES_PASSWORD`, `PGPASSWORD`, `DB_USERNAME`, `POSTGRES_USER`, `MYSQL_USER`, `PGUSER`, `DATABASE_URL`.
|
|
|
|
#### Minimal — default PostgreSQL:
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "myrepo/myapp:latest"
|
|
database = {}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Full — MySQL with custom image, port, credentials, and custom env var mapping:
|
|
```hcl
|
|
kubernetes = {
|
|
bookstack = {
|
|
image = "linuxserver/bookstack:latest"
|
|
service_port = [80]
|
|
database = {
|
|
engine = "mysql"
|
|
image = "lscr.io/linuxserver/mariadb"
|
|
port = 3306
|
|
mountPath = "/config"
|
|
username = "bookstack_user"
|
|
name = "bookstackapp"
|
|
map_name = "DB_DATABASE"
|
|
map_host = "DB_HOST"
|
|
map_password = "DB_PASS"
|
|
map_username = "DB_USER"
|
|
}
|
|
secret_ref = ["bookstack-credentials"]
|
|
config_map_ref = ["bookstack-config"]
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Using an external database (no DB deployment created)
|
|
|
|
Set `host` to point to an external hostname. Terraform will still generate the Bitwarden secrets and the Kubernetes secret, but will not create a DB deployment or service.
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "myrepo/myapp:latest"
|
|
database = {
|
|
host = "my-external-postgres.example.com"
|
|
port = 5432
|
|
name = "myapp_db"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Database with args (e.g. MongoDB replica set)
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
overleaf = {
|
|
image = "tuetenk0pp/sharelatex-full:latest"
|
|
service_port = [80]
|
|
database = {
|
|
image = "mongo:8.0"
|
|
port = 27017
|
|
mountPath = "/data/db"
|
|
}
|
|
database_args = ["--replSet", "overleaf"]
|
|
redis = { image = "redis:7.4" }
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Database `shunt` — container is itself the DB
|
|
|
|
When the application container IS the database (e.g. Bitwarden's bundled MSSQL), set `shunt = "true"` to skip creating a separate DB deployment while still generating backup jobs and Bitwarden secrets.
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
bitwarden-mssql = {
|
|
connected_namespace = "bitwarden"
|
|
image = "ghcr.io/bitwarden/mssql:2026.1.0"
|
|
service_port = [1433, 5000]
|
|
database = {
|
|
shunt = "true"
|
|
engine = "sqlserver"
|
|
username = "sa"
|
|
host = "bitwarden-mssql-svc"
|
|
name = "bitwarden_vault"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 11. Redis
|
|
|
|
Adding a `redis` block automatically provisions a Redis deployment (`<key>-redis`), service (`<key>-redis-svc`), and PVC (`<key>-redis-pvc`).
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "myrepo/myapp:latest"
|
|
database = {}
|
|
redis = {}
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — custom Redis image:**
|
|
```hcl
|
|
kubernetes = {
|
|
nextcloud = {
|
|
image = "nextcloud:29.0.3-apache"
|
|
service_port = [80]
|
|
namespace = "next"
|
|
database = { image = "docker.io/library/postgres:14" }
|
|
redis = { image = "docker.io/library/redis:7" }
|
|
environment = {
|
|
REDIS_HOST = { value = "nextcloud-redis-svc" }
|
|
NEXTCLOUD_ADMIN_USER = { value = "admin" }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 12. Liveness probes
|
|
|
|
Two probe types are supported. Only one should be used per application.
|
|
|
|
**Minimal — TCP socket:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "redis:latest"
|
|
service_port = [6379]
|
|
liveness_tcp_socket = 6379
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — HTTP GET with custom path:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "myrepo/myapp:latest"
|
|
service_port = [8080]
|
|
liveness_http_get = 8080
|
|
liveness_http_get_path = "/healthz"
|
|
requirements = {
|
|
limits-cpu = "500m"
|
|
limits-memory = "512Mi"
|
|
requests-cpu = "100m"
|
|
requests-memory = "128Mi"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 13. Resource requirements (CPU / memory)
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
requirements = {
|
|
limits-cpu = "500m"
|
|
limits-memory = "512Mi"
|
|
requests-cpu = "50m"
|
|
requests-memory = "128Mi"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 14. Custom entrypoint command
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "myrepo/myapp:latest"
|
|
command = ["python", "manage.py", "runserver"]
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — Gotenberg with multiple flags:**
|
|
```hcl
|
|
kubernetes = {
|
|
gotenberg = {
|
|
image = "gotenberg/gotenberg:8"
|
|
service_port = [3000]
|
|
connected_namespace = "paperless"
|
|
command = [
|
|
"gotenberg",
|
|
"--chromium-disable-javascript=true",
|
|
"--chromium-allow-list=file:///tmp/.*",
|
|
"--chromium-auto-start",
|
|
]
|
|
requirements = {
|
|
limits-cpu = "1"
|
|
limits-memory = "1000Mi"
|
|
requests-cpu = "500m"
|
|
requests-memory = "512Mi"
|
|
}
|
|
environment = {
|
|
CHROMIUM_PAPER_WIDTH = { value = "595" }
|
|
CHROMIUM_PAPER_HEIGHT = { value = "842" }
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 15. Security context (privileged)
|
|
|
|
Set `security_context = true` to run the container as privileged. Required for applications that manipulate network interfaces or kernel modules (e.g. VPN clients, fail2ban).
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
gluetun = {
|
|
image = "qmcgaw/gluetun:latest"
|
|
service_port = [8888]
|
|
security_context = true
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — Gluetun WireGuard VPN with HTTP proxy:**
|
|
```hcl
|
|
kubernetes = {
|
|
gluetun = {
|
|
image = "qmcgaw/gluetun:latest"
|
|
service_port = [8888]
|
|
security_context = true
|
|
dns_nameservers = "1.1.1.1"
|
|
secret_import = ["5527ba11-07e3-415b-8d1f-b3ee0154eb56"]
|
|
environment = {
|
|
VPN_SERVICE_PROVIDER = { value = "protonvpn" }
|
|
VPN_TYPE = { value = "wireguard" }
|
|
SERVER_COUNTRIES = { value = "Netherlands" }
|
|
FREE_ONLY = { value = "on" }
|
|
HTTPPROXY = { value = "on" }
|
|
HTTPPROXY_LISTENING_ADDRESS = { value = "0.0.0.0:8888" }
|
|
FIREWALL_INPUT_PORTS = { value = "8888,8080,9696" }
|
|
WIREGUARD_PRIVATE_KEY = {
|
|
value_from = {
|
|
secret_key_ref = {
|
|
name = "gluetun-wireguard-secret"
|
|
key = "WIREGUARD_PRIVATE_KEY"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
requirements = {
|
|
limits-cpu = "500m"
|
|
limits-memory = "512Mi"
|
|
requests-cpu = "50m"
|
|
requests-memory = "64Mi"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 16. Custom DNS nameservers
|
|
|
|
Override the DNS resolver for the pod. Useful for containers that route through a VPN.
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
dns_nameservers = "1.1.1.1"
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 17. Service account
|
|
|
|
Attach a Kubernetes ServiceAccount to the pod. The ServiceAccount must already exist (or be created via `extra_manifest`).
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
service_account_name = "my-service-account"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — fail2ban with RBAC manifests and service account:**
|
|
```hcl
|
|
kubernetes = {
|
|
fail2ban = {
|
|
image = "gitea.example.com/myorg/fail2ban:latest"
|
|
service_account_name = "kubectl-service-account"
|
|
security_context = true
|
|
extra_manifest = [
|
|
"/opt/kubernetes/fail2ban/fail2ban.RoleBinding.yaml",
|
|
"/opt/kubernetes/fail2ban/fail2ban.Role.yaml",
|
|
"/opt/kubernetes/fail2ban/fail2ban.ServiceAccount.yaml",
|
|
]
|
|
config_map_import = [
|
|
"9d8ef71a-de74-478b-aa86-b3ee0160dc43",
|
|
"50814edb-d190-41cb-8950-b3ee01622dcf",
|
|
]
|
|
environment = {
|
|
PUID = { value = "1000" }
|
|
TZ = { value = "Etc/UTC" }
|
|
}
|
|
pvc = {
|
|
config = {
|
|
pvc = true
|
|
pvc_size = "2Gi"
|
|
name = "fail2ban-config"
|
|
mount_path = "/config"
|
|
}
|
|
logs = {
|
|
host_path = "/data/kubernetes/reverse-proxy/reverse-proxy-log/"
|
|
name = "nginx-fail2ban-logs"
|
|
mount_path = "/remotelogs/nginx/"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 18. Nginx reverse proxy (domain_name)
|
|
|
|
Setting `domain_name` generates an nginx upstream + server block in the `extra-sites-terraform` ConfigMap in the `reverse-proxy` namespace. The generated config proxies `<domain_name>.tryton.geeraert.eu` to `<key>-svc.<namespace>.svc:<first port>`. This requires Stakater Reloader (or equivalent) to pick up the ConfigMap change.
|
|
|
|
**Minimal:**
|
|
```hcl
|
|
kubernetes = {
|
|
myapp = {
|
|
image = "nginx:latest"
|
|
service_port = [80]
|
|
domain_name = "myapp"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Full — Paperless with domain, database, and Redis:**
|
|
```hcl
|
|
kubernetes = {
|
|
paperless = {
|
|
image = "ghcr.io/paperless-ngx/paperless-ngx:2.18.4"
|
|
service_port = [8000]
|
|
domain_name = "paperless"
|
|
database = {
|
|
image = "postgres:15.6"
|
|
mountPath = "/var/lib/postgresql/data"
|
|
}
|
|
redis = { image = "redis:7.4.1" }
|
|
environment = {
|
|
PAPERLESS_URL = { value = "https://paperless.example.com" }
|
|
PAPERLESS_REDIS = { value = "redis://paperless-redis-svc:6379" }
|
|
PAPERLESS_TIKA_ENABLED = { value = "1" }
|
|
PAPERLESS_TIKA_GOTENBERG_ENDPOINT = { value = "http://gotenberg-svc.paperless.svc:3000" }
|
|
PAPERLESS_TIKA_ENDPOINT = { value = "http://tika-svc.paperless.svc:9998" }
|
|
}
|
|
pvc = {
|
|
data = {
|
|
pvc = true
|
|
name = "paperless-data"
|
|
pvc_size = "50Gi"
|
|
mount_path = "/usr/src/paperless/data"
|
|
}
|
|
media = {
|
|
pvc = true
|
|
name = "paperless-media"
|
|
pvc_size = "200Gi"
|
|
mount_path = "/usr/src/paperless/media"
|
|
}
|
|
consume = {
|
|
pvc = true
|
|
name = "paperless-consume"
|
|
pvc_size = "10Gi"
|
|
mount_path = "/usr/src/paperless/consume"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Complete full example
|
|
|
|
The following deploys a full-featured application with every available option set. It demonstrates: custom namespace, multiple ports, LoadBalancer service, PostgreSQL with custom image and credential mapping, Redis, environment variables (direct + secret refs), mounted secrets and configmaps, a host path volume, liveness probe, resource limits, custom entrypoint, privileged security context, custom DNS, service account, extra manifests, and Bitwarden secret imports.
|
|
|
|
```hcl
|
|
kubernetes = {
|
|
my-full-app = {
|
|
# --- Identity ---
|
|
name = "my-full-app"
|
|
image = "myrepo/myapp:1.2.3"
|
|
namespace = "myapp-namespace"
|
|
enabled = true
|
|
replicas = 2
|
|
|
|
# --- Networking ---
|
|
service_port = [8080, 8443]
|
|
service_type = "LoadBalancer"
|
|
domain_name = "myapp"
|
|
dns_nameservers = "1.1.1.1"
|
|
|
|
# --- Entrypoint & security ---
|
|
command = ["python", "manage.py", "runserver", "0.0.0.0:8080"]
|
|
security_context = true
|
|
service_account_name = "myapp-service-account"
|
|
|
|
# --- Liveness ---
|
|
liveness_http_get = 8080
|
|
liveness_http_get_path = "/healthz"
|
|
|
|
# --- Resources ---
|
|
requirements = {
|
|
limits-cpu = "1000m"
|
|
limits-memory = "1Gi"
|
|
requests-cpu = "200m"
|
|
requests-memory = "256Mi"
|
|
}
|
|
|
|
# --- Database ---
|
|
database = {
|
|
engine = "postgres"
|
|
image = "docker.io/library/postgres:16"
|
|
port = 5432
|
|
mountPath = "/var/lib/postgresql/data"
|
|
username = "myapp_user"
|
|
name = "myapp_db"
|
|
map_name = "APP_DB_NAME"
|
|
map_host = "APP_DB_HOST"
|
|
map_password = "APP_DB_PASS"
|
|
map_username = "APP_DB_USER"
|
|
}
|
|
|
|
# --- Redis ---
|
|
redis = { image = "docker.io/library/redis:7" }
|
|
|
|
# --- Environment ---
|
|
environment = {
|
|
APP_ENV = { value = "production" }
|
|
APP_URL = { value = "https://myapp.example.com" }
|
|
APP_SECRET_KEY = {
|
|
value_from = {
|
|
secret_key_ref = {
|
|
name = "myapp-credentials"
|
|
key = "SECRET_KEY"
|
|
}
|
|
}
|
|
}
|
|
DB_PASSWORD = {
|
|
value_from = {
|
|
secret_key_ref = {
|
|
name = "myapp-db-terraform"
|
|
key = "DB_PASSWORD"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
# --- Bulk secret/configmap mounts ---
|
|
secret_ref = ["myapp-credentials", "myapp-extra-secret"]
|
|
config_map_ref = ["myapp-config", "global-config"]
|
|
|
|
# --- Bitwarden imports ---
|
|
secret_import = ["9fc5a725-75f5-4bab-b595-b474017146c3"]
|
|
config_map_import = ["9d8ef71a-de74-478b-aa86-b3ee0160dc43"]
|
|
|
|
# --- Extra manifests ---
|
|
extra_manifest = [
|
|
"/opt/kubernetes/myapp/myapp-role.yaml",
|
|
"/opt/kubernetes/myapp/myapp-rolebinding.yaml",
|
|
]
|
|
|
|
# --- Volumes ---
|
|
pvc = {
|
|
app-data = {
|
|
pvc = true
|
|
name = "myapp-data"
|
|
pvc_size = "50Gi"
|
|
mount_path = "/app/data"
|
|
storage_class_name = "nfs-storage-fastaccess"
|
|
}
|
|
app-config = {
|
|
pvc = true
|
|
name = "myapp-config"
|
|
pvc_size = "1Gi"
|
|
mount_path = "/app/config"
|
|
sub_path = "prod"
|
|
}
|
|
node-media = {
|
|
host_path = "/data/media"
|
|
name = "node-media"
|
|
mount_path = "/media"
|
|
}
|
|
nginx-conf = {
|
|
config_map = true
|
|
name = "myapp-nginx-config"
|
|
mount_path = "/etc/nginx/conf.d"
|
|
}
|
|
tls-certs = {
|
|
secret = true
|
|
name = "myapp-tls"
|
|
mount_path = "/certs"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Database backup
|
|
|
|
When a `database` block is present, a daily backup CronJob is automatically created (`<key>-db-backup`, runs at 00:01 UTC). Backups are written to the `<key>-db-backup` PVC (storage class: `nfs-storage-fastaccess`, 1Gi). The backup command and image are selected based on the `engine` field:
|
|
|
|
| Engine | Image | Command |
|
|
|-------------|-------------------------------|-------------------------------------------------------|
|
|
| `postgres` | `alpine/psql` | `pg_dump \| gzip -9 > /backup/<name>-DB-<date>.sql.gz` |
|
|
| `mysql` | `mysql` | `mysqldump ... \| gzip -9 > /backup/<name>-DB-<date>.sql.gz` |
|
|
| `sqlserver` | `mcr.microsoft.com/mssql-tools` | `sqlcmd BACKUP DATABASE ... TO DISK` |
|
|
|
|
---
|
|
|
|
## Naming conventions
|
|
|
|
Given a map key of `myapp` with no overrides:
|
|
|
|
| Resource | Name |
|
|
|---------------------------|-----------------------------------|
|
|
| Namespace | `myapp` |
|
|
| Deployment | `myapp` |
|
|
| Service | `myapp-svc` |
|
|
| DB Deployment | `myapp-db` |
|
|
| DB Service | `myapp-db-svc` |
|
|
| DB PVC | `myapp-db-pvc` |
|
|
| DB Backup PVC | `myapp-db-backup` |
|
|
| DB Backup CronJob | `myapp-db-backup` |
|
|
| DB Secret (all var names) | `myapp-db-terraform` |
|
|
| DB Secret (mapped names) | `myapp-db-mapped-terraform` |
|
|
| Redis Deployment | `myapp-redis` |
|
|
| Redis Service | `myapp-redis-svc` |
|
|
| Redis PVC | `myapp-redis-pvc` |
|
|
| PVC (from `pvc` block) | `<pvc.name>-pvc` |
|