Sign inSign up

sroegner/doop

By sroegner

Updated over 10 years ago

Hortonworks HDP 2.2, supervisord and consul form Hadoop clusters of configurable layout

Image
0

1.2K

sroegner/doop repository overview

https://github.com/sroegner/docker-builds/tree/master/doop

The doop image is meant to run real clusters of containers, with dedicated containers per role.

Example:

  • consul-leader
  • namenode (and secondary namenode)
  • resourcemanager
  • zookeeper
  • nodemanager + datanode (start as many as needed)

run with

container_name=doop-cluster-0
docker run -d --name=consul-leader docker.io/sroegner/doop \
              /usr/sbin/consul agent -server -bootstrap-expect=1 \
              -data-dir=/var/lib/consul/data -config-dir=/etc/consul-leader

# need the IP of this node as it acts as our DNS server
dns_ip=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' consul-leader)

docker run -d --dns-search=node.doop.local \
              --hostname=namenode --dns=$dns_ip \
              -e="SVCLIST=namenode,secondarynamenode" \
              --name=${container_name} --link=consul-leader:consul-leader \
              docker.io/sroegner/doop /usr/bin/supervisord -n

docker run -d --dns-search=node.doop.local --hostname=resourcemanager \
              --dns=$dns_ip -e="SVCLIST=resourcemanager" \
              --name=${container_name}-rm --link=consul-leader:consul-leader \
              docker.io/sroegner/doop /usr/bin/supervisord -n

docker run -d --dns-search=node.doop.local --hostname=datanode0 \
              --dns=$dns_ip -e="SVCLIST=datanode,nodemanager" \
              --name=${container_name}-dn0 \
              --link=consul-leader:consul-leader docker.io/sroegner/doop \
              /usr/bin/supervisord -n

docker run -d --dns-search=node.doop.local --hostname=datanode1 \
              --dns=$dns_ip -e="SVCLIST=datanode,nodemanager" \
              --name=${container_name}-dn1 \
              --link=consul-leader:consul-leader docker.io/sroegner/doop \
              /usr/bin/supervisord -n

docker run -d --dns-search=node.doop.local --hostname=zookeeper --dns=$dns_ip \
              -e="SVCLIST=zookeeper" --name=${container_name}-zk0 \
              --link=consul-leader:consul-leader docker.io/sroegner/doop \
              /usr/bin/supervisord -n

You can then look at the consul status page on any of the containers at port 8500, all hadoop ports are exposed on their respective container IPs. DNS is on port 53 of the consul-leader container.

Tag summary

Content type

Image

Digest

sha256:73d4bffa3

Size

631.1 MB

Last updated

over 10 years ago

docker pull sroegner/doop