failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden

[AndreasBergmeier6176]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

• The documentation does not mention anything about my problem
• There are no open or closed issues that are related to my problem

Description

I have two Google Artifact Registries:

• foo: for fetching base images from
• bar: for pushing images to

Now docker build works, if I replace foo by docker.io.
When however I run docker build using foo I get an error:

europe-west1-docker.pkg.dev/foo/ar/python:3.9-slim: failed to authorize: failed to fetch anonymous token: unexpected status: 403 Forbidden

github-deployer@bar.iam.gserviceaccount.com has Artifact Registry Reader permissions on europe-west1-docker.pkg.dev/foo/ar/python

So it seems like I cannot use the WIP access_token for accessing foo. But how would I then use WIP to login for foo?

Expected behaviour

Would be good if either it worked or at least the error message would state clearly why it doesn't work.

Actual behaviour

See above

Repository URL

No response

Workflow run URL

No response

YAML workflow

- id: auth
      uses: "google-github-actions/auth@v1"
      with:
        project_id: bar
        retries: 10
        service_account: 'github-deployer@bar.iam.gserviceaccount.com'
        token_format: access_token
        workload_identity_provider: projects/${{ inputs.project_number }}/locations/global/workloadIdentityPools/github/providers/oidc
    - uses: google-github-actions/setup-gcloud@v1
    - run: |
        gcloud auth configure-docker -q europe-west1-docker.pkg.dev
    - uses: docker/login-action@v3
      with:
        registry: europe-west1-docker.pkg.dev
        username: oauth2accesstoken
        password: "${{ steps.auth.outputs.access_token }}"

Workflow logs

No response

BuildKit logs

No response

Additional info

No response

[ying-jeanne]

Hi I am not sure the problem I am having is related to this. I have a github action that use the action to login to docker artifect, it was working fine with v2, but since we update to v3 2 weeks ago, I am having this. just in case you can't see this is the error

Run docker/login-action@v3
  with:
    ecr: auto
    logout: true
Error: Username and password required

and this is my github workflow
https://github.com/grafana/mimir/blob/main/.github/workflows/push-mimir-build-image.yml#L34

[crazy-max]

@ying-jeanne This is not related, see #29 (comment).

[tbernacchi]

Same here.

I'm following these steps https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

And when I've tried to pull a image from my private Artifact Registry on my GKE/k8s cluster I'm getting this:

unpack image "us-central1-docker.pkg.dev/org/containers/images/mongo-backup:1.0.4": failed to resolve reference "us-central1-docker.pkg.dev/org/containers/images/mongo-backup:1.0.4": failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://us-central1-docker.pkg.dev/v2/token?scope=repository%3Aorg%2Fcontainers%2Fimages%2Fmongo-backup%3Apull&service=us-central1-docker.pkg.dev: 403 Forbidden

Any ideias how to solve this? Any help will be appreciate!

Thank you!

[ktraister]

I had a similar problem with a new service I set up on K3s, and I realized I forgot to add this to my new manifests:

      imagePullSecrets:                                                                                                                       
        - name: ghcrcred

Good luck!

[BenjaminSsempala]

If you're deploying to a Kubernetes cluster outside of GKE or without using Workload Identity, you'll need to manually create and configure an ImagePullSecret to authenticate with the Google Container Registry. First, generate a Docker config by logging in with an access token:

gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://eu.gcr.io

Then, create the Kubernetes secret using your service account key file:

kubectl create secret docker-registry gcr-json-key \
  --docker-server=eu.gcr.io \
  --docker-username=_json_key \
  --docker-password="$(cat your-key-file.json)" \
  --docker-email=your-email@example.com

You can either reference this secret in your pod specs under imagePullSecrets, or set it as the default for all pods within a namespace by patching the default service account:

kubectl patch serviceaccount default -n <your-namespace> \
  -p '{"imagePullSecrets": [{"name": "gcr-json-key"}]}'

To apply this across all namespaces, use a loop to patch each one like so:

for ns in $(kubectl get ns --no-headers -o custom-columns=":metadata.name"); do
  kubectl patch serviceaccount default -n "$ns" \
    -p '{"imagePullSecrets": [{"name": "gcr-json-key"}]}' --type='merge'
done

This ensures all default service accounts in all namespaces are configured to use your private registry credentials by default, simplifying image pulls across your cluster.