docker stack can't publish udp and tcp for the same port

[lneva-fastly]

Description

Using docker stack deploy, it is not possible to publish the same port as both UDP and TCP. An example where this is important is DNS.

Steps to reproduce the issue:

1. Deploy the following stack:

version: '3.7'
services:
  consul:
    image: consul
    command: agent -dev
    ports:
      # Port 8600 is the consul DNS query interface.
      # Mapping this to port 53 allows us to use this as our nameserver.
      - target: 8600
        published: 53
        protocol: tcp
      - target: 8600
        published: 53
        protocol: udp
    networks:
      - consul
networks:
  consul:

Describe the results you received:
Only the second port (53/udp) is published.

Describe the results you expected:
Both ports are published.

Additional information you deem important (e.g. issue happens only occasionally):

This may only occur if you specify multiple docker-compose yaml files that are merged by docker stack deploy. I haven't tested with just a single compose file.

Output of docker version:

19.03.8 client and server

Additional details:

The problem is in this function:

cli/cli/compose/loader/merge.go

Line 114 in 22acbbc

m[p.Published] = p

When merging, ports are placed into a map keyed on the port number. If the same port is published twice (udp and tcp), the second protocol wins because it overwrites the first. This map should instead be keyed on (port, protocol).

[lneva-fastly]

As a note, this problem does not occur if you run docker service create manually.

[ngrilly]

I have the exact same issue, and came to the same conclusion.

This may only occur if you specify multiple docker-compose yaml files that are merged by docker stack deploy. I haven't tested with just a single compose file.

This doesn't occur with a single compose file.

It's possible to reproduce the issue with docker stack config only, without deploying with docker stack deploy:

$ cat <<EOF >> bug-compose.yml
version: "3.5"
services:
  test:
    ports:
      - published: 80
        target: 80
        protocol: tcp
      - published: 443
        target: 443
        protocol: tcp
      - published: 443
        target: 443
        protocol: udp
EOF
$ cat <<EOF >> bug-compose.prod.yml
version: "3.5"
services:
  test:
    environment:
      TEST: test
EOF
$ docker stack config -c bug-compose.yml -c bug-compose.prod.yml
version: "3.5"
services:
  test:
    environment:
      TEST: test
    ports:
    - target: 80
      published: 80
      protocol: tcp
    - target: 443
      published: 443
      protocol: udp

One temporary workaround is put the compose file with the ports at the end:

$ docker stack config -c bug-compose.prod.yml -c bug-compose.yml     
version: "3.5"
services:
  test:
    environment:
      TEST: test
    ports:
    - target: 80
      published: 80
      protocol: tcp
    - target: 443
      published: 443
      protocol: tcp
    - target: 443
      published: 443
      protocol: udp

The solution would be to fix toServicePortConfigsMap in github.com/docker/cli/cli/compose/loader/merge.go by using a composite key (published port + protocol, instead of published port only), like in github.com/moby/swarmkit/manager/scheduler.HostPortFilter.Check.