Add Docker registry

[mcopik]

We are wrapping up the support for using Docker images and rFaaS (#10). However, we only use DockerHub, which has multiple restrictions - throttling and network latency. We want to locally deploy a Docker registry.

• Add scripts for deploying the Docker registry and generating JSON configuration.
• Add configuration of Docker registry in the rFaaS executor manager configuration.
• Add polling Docker images on container allocation in the execution manager - only when the image is not present already.
• Support registry credentials.

[BhuvanAde]

Hey @mcopik, Can you please assign me this issue?

[mcopik]

@BhuvanAde Absolutely, please feel free to start on the issue :-)

Let me know if you have any questions or concerns - happy to help.

[mattnappo]

Hi @mcopik. I have made a great deal of progress on this issue so far (as well as some progress on #10). However, I am wondering what the docker network configuration used during testing looks like. I see that in the hard-coded part of the code where the docker_rdma_sriov binary is invoked (here), the docker network mynet is specified. I presume that this network is a user-defined bridge network that was used during testing/evaluation. Do you remember how this network was defined?

UPDATE: docker network create -d sriov --subnet=<subnet> -o netdevice=<device> mynet seemed to work.

[mcopik]

@mattnappo Great to hear that - thanks so much for the help! Can you please open a draft PR? I was going to work on the same thing (fixing container selection and customizing the hardcoded options), and I do not want to repeat your work.

Yes, you found the correct thing - this is the command. I'm afraid that our documentation does not contain references to the installation instructions for the Mellanox plugin - I need to fix that.

In addition to using the SR-IOV plugin, which is unavailable on many platforms and can be a bit fragile, we also used standard Docker with host networking - I will add it as another customization.

[mattnappo]

@mcopik Here is the draft pr. I plan on finishing up the PR within the next few days. Also: I was able to setup a rFaas cluster on an AWS VPC and perform tests. I built a custom kernel to use Soft-RoCE since my cluster did not have RDMA-capable NICs. If you were wondering if rFaas works over Soft-RoCE, it does!

[mcopik]

@mattnappo Thanks, glad to hear it works! :-) I will review the PR soon.

Did you manage to run SR-IOV on top of SoftRoCe? Or did you try Docker host networking mode?

[mattnappo]

Yes, I ran SR-IOV on SoftRoCe. I can try to get it working with host networking.