Referring to TLS secret from other namespace (i.e. not the namespace in which ingress is created)

[amit-kumar-4]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see https://kubernetes.io/docs/tasks/debug-application-cluster/troubleshooting/.): No

What keywords did you search in NGINX Ingress controller issues before filing this one? (If you have found any duplicates, you should instead reply there.): "Secret namespace"

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): Feature

NGINX Ingress controller version: "0.10.2"

Kubernetes version (use kubectl version): v1.8.7

Environment:

• Cloud provider or hardware configuration: Azure
• OS (e.g. from /etc/os-release): Ubuntu 16.04.3
• Kernel (e.g. uname -a): 4.11.0-1016-azure
• Install tools: Helm Chart v0.9.5 from https://github.com/kubernetes/charts/blob/master/stable/nginx-ingress/
• Others:

What happened:
When trying to use a TLS certificate using <namespace>/<secretName> pattern in tls section of ingress definition, Nginx controller still tries to get the details from the namespace where ingress was created.

What you expected to happen:
When TLS secret referred when creating an ingress is of pattern <namespace>/<secretName>, ingress controller shouldn't check only in ingress' namespace.

How to reproduce it (as minimally and precisely as possible):

1. Create 2 namespaces, say secret-store and ingress-store.
2. Create a secret containing a TLS certificate and key in secret-store namespace, say my-tls.
3. Create an ingress in ingress-store namespace with TLS enabled and in the .spec.tls.hosts[].secretName field put secret-store/my-tls to refer to the secret in secret-store namespace.
4. Check logs of ingress controller, line similar to below will be printed, indicating secret was never searched in secret-store namespace:

W0305 11:39:26.826578       6 backend_ssl.go:49] error obtaining PEM from secret ingress-store/secret-store/my-tls: error retrieving secret ingress-store/secret-store/my-tls: secret ingress-store/secret-store/my-tls was not found

Anything else we need to know:
Initial glance on the code suggests that in below snippet, in the if block, we check if the secret has a / in it, and try to extract the secret from the namespace provided in ingress .spec.tls definition, it could work.

ingress-nginx/internal/ingress/controller/controller.go

Lines 1000 to 1005 in 164bb7b

	key := fmt.Sprintf("%v/%v", ing.Namespace, tlsSecretName)
	cert, err := n.store.GetLocalSecret(key)
	if err != nil {
	glog.Warningf("ssl certificate \"%v\" does not exist in local store", key)
	continue
	}

[mikebryant]

The standard security model for Kubernetes is that secrets are only accessible from within the current Namespace.

As per the API definition, the secretName field is purely a name reference, and cannot cross namespaces.

[amit-kumar-4]

True enough, but I think there is a catch-22 when trying to use wildcard certificate in cluster and keeping the certificate private key secure.
If I don't use the way mentioned above, I'll be exposing the private key for the wildcard certificate for anyone who has access to read secrets in the namespace, which will be many usually.

The standard security model for Kubernetes is that secrets are only accessible from within the current Namespace.

Since it will be the Ingress controller reading the secret, rather than something in the ingress namespace, I think your statement will still remain true.

[aledbf]

Closing. This should be allowed in the Ingress spec first. We are are not going to break this rule.

[sheerun]

@aledbf Is there a followup issue for changing the spec?

[aledbf]

@sheerun not yet

[mattalberts]

@aledbf @sheerun @aku105
During my evaluation of 0.14.0 and 0.15.0, I noticed that the ingress spec (as consumed by the nginx controller) would allow a mini-version of the issue described above without any patching (supported now).

• configure the controller with --default-ssl-certificate set to your wildcard cert
• define all ingress spec with hosts but do not specify the secretName

The ingress will be registered :80,:443 and use the default (wildcard certificate).

[Nowaker]

Thanks @mattalberts. While this is far from ideal (you can only have one default certificate), it's better than nothing.

That said, the spec should be updated to address the issue of using secrets from different namespaces. Wildcard certificates should be stored in the ingress controller's namespace, and Ingress resources from different namespaces should be allowed to reference them. What is the process of bringing up the spec discussion? Thanks! @sheerun @aledbf

[caleblloyd]

I look forward to an official fix coming in terms of a spec change that allows for referring to Secrets across namespace boundaries.

In the meantime, I am using a deployment that runs 2 kubectl containers - one to watch for new namespaces and copy the TLS Secret, and one to watch the TLS Secret for changes and apply to all namespaces. The code is here: ingress-cert-reflector.yml and I also wrote a corresponding blog post with detailed instructions.

[FossPrime]

UPDATE: in nginxinc/nginx-ingress default-ssl-certificate is default-server-tls-secret

---

@mattalberts I've tried to drop default-ssl-certificate in my controller, but I can't find where it goes

Dropping it in the spec gives ValidationError(DaemonSet.spec): unknown field "default-ssl-certificate" in io.k8s.api.extensions.v1beta1.DaemonSetSpec

my 1.2.0 controller spec

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  generation: 8
  labels:
    app: nginx-ingress
  name: nginx-ingress
  namespace: nginx-ingress
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nginx-ingress
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx-ingress
    spec:
      containers:
      - args:
        - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
        - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
        - -v=3
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: nginx/nginx-ingress:1.2.0
        imagePullPolicy: IfNotPresent
        name: nginx-ingress
        ports:
        - containerPort: 80
          hostPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          hostPort: 443
          name: https
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: nginx-ingress
      serviceAccountName: nginx-ingress
      terminationGracePeriodSeconds: 30
  templateGeneration: 8
  updateStrategy:
    type: OnDelete