Hortonworks HDP 2.2, supervisord and consul form Hadoop clusters of configurable layout
1.2K
https://github.com/sroegner/docker-builds/tree/master/doop
The doop image is meant to run real clusters of containers, with dedicated containers per role.
Example:
run with
container_name=doop-cluster-0
docker run -d --name=consul-leader docker.io/sroegner/doop \
/usr/sbin/consul agent -server -bootstrap-expect=1 \
-data-dir=/var/lib/consul/data -config-dir=/etc/consul-leader
# need the IP of this node as it acts as our DNS server
dns_ip=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' consul-leader)
docker run -d --dns-search=node.doop.local \
--hostname=namenode --dns=$dns_ip \
-e="SVCLIST=namenode,secondarynamenode" \
--name=${container_name} --link=consul-leader:consul-leader \
docker.io/sroegner/doop /usr/bin/supervisord -n
docker run -d --dns-search=node.doop.local --hostname=resourcemanager \
--dns=$dns_ip -e="SVCLIST=resourcemanager" \
--name=${container_name}-rm --link=consul-leader:consul-leader \
docker.io/sroegner/doop /usr/bin/supervisord -n
docker run -d --dns-search=node.doop.local --hostname=datanode0 \
--dns=$dns_ip -e="SVCLIST=datanode,nodemanager" \
--name=${container_name}-dn0 \
--link=consul-leader:consul-leader docker.io/sroegner/doop \
/usr/bin/supervisord -n
docker run -d --dns-search=node.doop.local --hostname=datanode1 \
--dns=$dns_ip -e="SVCLIST=datanode,nodemanager" \
--name=${container_name}-dn1 \
--link=consul-leader:consul-leader docker.io/sroegner/doop \
/usr/bin/supervisord -n
docker run -d --dns-search=node.doop.local --hostname=zookeeper --dns=$dns_ip \
-e="SVCLIST=zookeeper" --name=${container_name}-zk0 \
--link=consul-leader:consul-leader docker.io/sroegner/doop \
/usr/bin/supervisord -n
You can then look at the consul status page on any of the containers at port 8500, all hadoop ports are exposed on their respective container IPs. DNS is on port 53 of the consul-leader container.
Content type
Image
Digest
sha256:73d4bffa3…
Size
631.1 MB
Last updated
over 10 years ago
docker pull sroegner/doop