docker-compose up -d fails when network already exists.

[reenberg]

VIC version:

vic-v1.2.1-dev-b37a80f8

Deployment details:

I have deployed the VCH with client client and bridge network. --tls-server-key, --tls-server-cert, --tls-ca and --tls-cert-path, with https://vic_address as a insecure registry.

Steps to reproduce:

Running docker-compose up -d on a project where one network is already existing, throws an AttributeError. This could for example be due to the fact that another project is using this network as an "external" and thus the network was not deleted on docker-compose down. Of if you run docker-compose up -d without calling docker-compose down (one might just have stopped a single service, etc).

Note: This only happens as long as the network is not defined as external.

Actual behavior:

$ eval  $(cat vch.env) docker-compose up -d             
Traceback (most recent call last):                                                                          
  File "/usr/local/bin/docker-compose", line 11, in <module>                                                
    sys.exit(main())                                                                                        
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 68, in main                      
    command()                                                                                              
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 121, in perform_command
    handler(command, command_options)                                                                             
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 938, in up
    scale_override=parse_scale_args(options['--scale']),                 
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 419, in up
    self.initialize()                                                    
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 468, in initialize
    self.networks.initialize()                                           
  File "/usr/local/lib/python3.5/dist-packages/compose/network.py", line 257, in initialize
    network.ensure()                                                                         
  File "/usr/local/lib/python3.5/dist-packages/compose/network.py", line 62, in ensure
    check_remote_network_config(data, self)                              
  File "/usr/local/lib/python3.5/dist-packages/compose/network.py", line 191, in check_remote_network_config
    for k in set.union(set(remote_labels.keys()), set(local_labels.keys())):
AttributeError: 'NoneType' object has no attribute 'keys'                

Expected behavior:

No errors, just create whatever services, networks or volumes that are not already created.

Logs:

I can't supply any log files (company policy -- sorry) but I can see from the docker-personality.log, that when trying to bring up the compose project, it first warns about error getting config from source: config not modified, then informs about finding various volumes, and then at the end warns about the same. These are the only new lines in that log file. Other log files doesn't seem to have any related content when trying to run the command.

[dbarkelew]

Following workaround works for me.
docker-compose down && docker-compose up -d

[reenberg]

well, obvious that will work and I should perhaps have added that. It works because down will delete/remove the network (along with any running containers that is part of the compose file, and not to mention unnamed volumes). This is not necessarily always what you really want to do, if avoidable (could be a db service that other containers use).

And it will not always work. You may have external containers using the network. In this case the network will not be deleted, leaving you with all containers (in the compose file) down.

[mariolenz]

We're running into the same problem with VIC 1.4.3. This is a very annoying issue, especially since docker-compose down isn't an option for us.

[mariolenz]

On a normal Linux container host, when I docker network inspect the bridge network I get an empty value for labels: "Labels": {}. When I do this with VIC I get "Labels": null. I'm pretty sure that this is the problem.

Would it be possible for you to make VIC return an empty {} instead of null? This should fix my problem.

edit:

I think that the JSON Null results in a Python None for remote_labels. This would explain the following errror:

File "/usr/local/lib/python3.5/dist-packages/compose/network.py", line 191, in check_remote_network_config
    for k in set.union(set(remote_labels.keys()), set(local_labels.keys())):
AttributeError: 'NoneType' object has no attribute 'keys'

[mariolenz]

@yuyangbj Did you have time to look into this issue? Can you confirm my hypothesis that the root cause is the null value returned for Labels? Or do you think it is something else?

[yuyangbj]

@mariolenz yes, I agree with you totally. I am working on the fix.

[yuyangbj]

I checked the latest build. Labels will show empty object.

yangyu-a01:bin yangyu$ docker --tls inspect fac6672e9371
[
{
"Name": "bridge",
"Id": "fac6672e937158e7f5a4623a033238b9021b4571a88c0920c263370bc67a3f4f",
"Created": "2019-08-22T10:31:42.011212429Z",
"Scope": "",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "",
"Options": {},
"Config": [
{
"Subnet": "172.16.0.0/16",
"Gateway": "172.16.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Containers": {
"709c1f8fc84ae243d87e9d2172ec767d8a7ff8b4b8e104462fb78205d4ea0f26": {
"Name": "webserver1",
"EndpointID": "709c1f8fc84ae243d87e9d2172ec767d8a7ff8b4b8e104462fb78205d4ea0f26",
"MacAddress": "",
"IPv4Address": "172.16.0.3/16",
"IPv6Address": ""
},
"dc42bbf417a44d1d3afb3bd0c622a8335f293fda819108493d27494740a2a65a": {
"Name": "webserver",
"EndpointID": "dc42bbf417a44d1d3afb3bd0c622a8335f293fda819108493d27494740a2a65a",
"MacAddress": "",
"IPv4Address": "172.16.0.4/16",
"IPv6Address": ""
},
"e88459df6bb62f4b6445aeea539fbe497e7a937f3f63d81be69f977f9d445a4b": {
"Name": "webserver2",
"EndpointID": "e88459df6bb62f4b6445aeea539fbe497e7a937f3f63d81be69f977f9d445a4b",
"MacAddress": "",
"IPv4Address": "172.16.0.2/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}

Piece of screenshot for docker ps:

       },
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": true,
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "NGINX_VERSION=1.17.3",
            "NJS_VERSION=0.3.5",
            "PKG_RELEASE=1~buster"
        ],
        "Cmd": [
            "nginx",
            "-g",
            "daemon off;"
        ],
        "ArgsEscaped": true,
        "Image": "nginx",
        "Volumes": {},
        "WorkingDir": "",
        "Entrypoint": null,
        "OnBuild": null,
        "Labels": {}
    },

[mariolenz]

I checked the latest build. Labels will show empty object.

This sounds promising. I'd like to test it and give you some feedback if this fixes our problems, but I'm not sure if I will find the time... we're very short-staffed at the moment :-/

Anyway, thanks for the info :-)

[yuyangbj]

I am waiting your response to see if we can close it. Thanks.

[yuyangbj]

I submit a PR to fix this issue. Currently you guys can set version to 3.0, that will help you avoid this problem