Configuring Makefiles to build and run Docker Images using Docker, Docker Compose and Dockerfile

by John Vincent

Posted on May 28, 2021

Makefiles are used to build and run Docker images in Docker containers.

This is part of a series of discussions regarding Deploying TaskMuncher, a React and Node application, to a Multi-Container Docker Environment at AWS using Dockerhub and Travis CI

For more details, please see Overview of Create Multi-Container Docker TaskMuncher Application at AWS

Overview

The following are listings of the Makefiles provided without explanations.

It is understood this document requires of the reader a leap of faith however all will be revealed in later documents in this series.

Docker Config Folder Structure

The structure of the ./config folder is as follows.

Makefile

Create file Makefile is base folder of the repository.

MAKEFILE_DIR := $(dir $(lastword $(MAKEFILE_LIST)))

include $(MAKEFILE_DIR)/make/dev.mk
include $(MAKEFILE_DIR)/make/devprod.mk

include $(MAKEFILE_DIR)/make/dev-compose.mk
include $(MAKEFILE_DIR)/make/devprod-compose.mk

APPLICATION := taskmuncher

list:
	docker ps -all
	docker images
	docker network ls

#
# all
#
all-build: all-dev-build all-devprod-build compose-dev-build compose-devprod-build

all-clean: all-dev-clean all-devprod-clean compose-dev-clean compose-devprod-clean
	docker system prune -f
	docker rmi mongo
	docker ps -all
	docker images

#
# all unit tests
#
all-unit-tests: client-dev-build client-dev-jest server-dev-build server-dev-jest

Notice

all-build - build all targets
all-clean - removes all resources created by all-build
the references to 4 other Makefiles.

Makefile for the Development Environment

Notice

all-dev-build - build all targets
all-dev-clean - removes all resources created by this Makefile
all-dev-containers-clean - remove all containers created by this Makefile
all-dev-images-clean - remove all containers created by this Makefile

All images and containers are explicitly named using a naming convention. Thus it is possible to easily understand the origins of a listing of images and containers.

This is a best practice. Allowing Docker to use its default naming rules is a recipe for chaos.

Create file make/dev.mk

#
# dev
#

all-dev-build: client-dev-build server-dev-build \
			nginx-dev-build \
			mongo-client-dev-build mongodb-dev-build \
			dockercfg-dev-build

all-dev-clean: all-dev-images-clean all-dev-containers-clean

all-dev-containers-clean:
	docker rm -f taskmuncher-client-dev
	docker rm -f taskmuncher-server-dev
	docker rm -f taskmuncher-mongo-client-dev
	docker rm -f taskmuncher-mongodb-dev
	docker rm -f taskmuncher-dockercfg-dev
	docker rm -f taskmuncher-nginx-dev

all-dev-images-clean:
	docker rmi -f taskmuncher-client-dev-image
	docker rmi -f taskmuncher-server-dev-image
	docker rmi -f taskmuncher-mongo-client-dev-image
	docker rmi -f taskmuncher-mongodb-dev-image
	docker rmi -f taskmuncher-dockercfg-dev-image
	docker rmi -f taskmuncher-nginx-dev-image

#
# client
#

client-dev-build:
	docker build -f ./config/client/dev/Dockerfile \
		-t taskmuncher-client-dev-image:latest \
		.

client-dev-sh:
	docker run --rm --name taskmuncher-client-dev \
		-it \
		taskmuncher-client-dev-image \
		sh

client-dev-run:
	docker run \
		--rm \
		--name taskmuncher-client-dev \
		-it \
		-p 8040:8040 \
		-p 3110:3110 \
		taskmuncher-client-dev-image:latest

client-dev-jest:
	docker run --rm --name taskmuncher-client-dev \
		-it taskmuncher-client-dev-image \
		sh -c "npm run test -- --coverage"

#
# server
#

server-dev-build:
	docker build -f ./config/server/dev/Dockerfile \
	-t taskmuncher-server-dev-image:latest \
	.

server-dev-sh:
	docker run --rm --name taskmuncher-server-dev \
		-it taskmuncher-server-dev-image \
		sh

server-dev-run:
	docker run \
		--rm \
		--name taskmuncher-server-dev \
		-it \
		-p 3110:3110 \
		taskmuncher-server-dev-image:latest

server-dev-jest:
	docker run --rm --name taskmuncher-server-dev \
		-it taskmuncher-server-dev-image \
		sh -c "npm run test -- --coverage"

#
# nginx
#

nginx-dev-build:
	docker build -f ./config/nginx/Dockerfile \
		-t taskmuncher-nginx-dev-image:latest \
		.

nginx-dev-sh:
	docker run --rm --name taskmuncher-nginx-dev \
		-it \
		taskmuncher-nginx-dev-image \
		sh

nginx-dev-run:
	docker run \
		--rm \
		--name taskmuncher-nginx-dev \
		-it \
		-p 8100:8100 \
		taskmuncher-nginx-dev-image:latest

#
# mongo-client
#

mongo-client-dev-build:
	docker build -f ./config/mongo-client/Dockerfile \
		-t taskmuncher-mongo-client-dev-image:latest \
		.

mongo-client-dev-sh:
	docker run --rm --name taskmuncher-mongo-client-dev \
		-it taskmuncher-mongo-client-dev-image \
		sh

mongo-client-dev-run:
	docker run \
		--rm \
		--name taskmuncher-mongo-client-dev \
		-it \
		taskmuncher-mongo-client-dev-image:latest

#
# mongodb
#

mongodb-dev-build:
	docker build -f ./config/mongodb/Dockerfile \
		-t taskmuncher-mongodb-dev-image:latest \
		.

mongodb-dev-sh:
	docker run --rm --name taskmuncher-mongodb-dev \
		-it taskmuncher-mongodb-dev-image \
		sh

mongodb-dev-run:
	docker run \
		--rm \
		--name taskmuncher-mongodb-dev \
		-it \
		-p 27017:27017 \
		taskmuncher-mongodb-dev-image:latest

mongodb-dev-connect:
	docker exec \
		-it \
		taskmuncher-mongodb-dev \
		bash

#
# dockercfg
#

dockercfg-dev-build:
	docker build -f ./config/dockercfg/Dockerfile \
		-t taskmuncher-dockercfg-dev-image:latest \
		.

dockercfg-dev-sh:
	docker run --rm --name taskmuncher-dockercfg-dev \
		-it taskmuncher-dockercfg-dev-image \
		sh

dockercfg-dev-run:
	docker run \
		--rm \
		--name taskmuncher-dockercfg-dev \
		-it \
		taskmuncher-dockercfg-dev-image:latest

Discuss Makefile for the Development Environment

Notice there are 6 components.

client - React application
server - Node/Express application
nginx - Nginx server for proxying requests
mongodb - MongoDB Database
mongo-client - Node application to seed the MongoDB Database
dockercfg - used to create a .dockercfg file. See Allow AWS to authenticate with a Dockerhub private repository

Notice that for each component there are usually these targets

*-dev-build - build Docker image
*-dev-sh - starts with a shell
*-dev-run - run the image in a named container
*-dev-jest - run units tests in a named container

Standard -dev-build

client-dev-build:
	docker build -f ./config/client/dev/Dockerfile \
		-t taskmuncher-client-dev-image:latest \
		.

Use docker to create docker image taskmuncher-client-dev-image using instructions in Dockerfile named ./config/client/dev/Dockerfile using . as the context.

Standard -dev-sh

client-dev-sh:
	docker run --rm --name taskmuncher-client-dev \
		-it \
		taskmuncher-client-dev-image \
		sh

Run docker image taskmuncher-client-dev-image from local docker repository, in a docker container to be named taskmuncher-client-dev which will be removed when the container terminates, start with a shell sh. Also

-i attach to stdin
-t ensure text output is in a nice format

Standard -dev-run

client-dev-run:
	docker run \
		--rm \
		--name taskmuncher-client-dev \
		-it \
		-p 8040:8040 \
		-p 3110:3110 \
		taskmuncher-client-dev-image:latest

-i attach to stdin
-t ensure text output is in a nice format
-p exposes and maps ports

Standard -dev-jest

client-dev-jest:
	docker run --rm --name taskmuncher-client-dev \
		-it taskmuncher-client-dev-image \
		sh -c "npm run test -- --coverage"

Run latest docker image taskmuncher-client-dev-image from local docker repository, in a docker container to be named taskmuncher-client-dev which will be removed when the container terminates, start with a shell sh and execute the command npm run test -- --coverage. This is the npm command to run jest unit tests.

Notice mongodb-dev-connect. This allows a shell to connect to a running instance of a MongoDB database.

Makefile for the Production Application running in a Development Environment

The goal is to produce the Production Application and deploy it in a Development environment so the application may be thoroughly tested and all configuration issues resolved before an attempt is made to deploy to production.

Attempting to resolve these issues in a Production environment, for example at AWS, is an extremely expensive, time consuming and frustrating experience which in the end will probably yield a weak application. It is just not worth it.

Always verify the Production Application in a Development environment before proceeding. This is another best practice.

Create file make/devprod.mk

#
# devprod
#

all-devprod-build: client-devprod-build server-devprod-build \
				mongo-client-devprod-build nginx-devprod-build

all-devprod-clean: all-devprod-images-clean all-devprod-containers-clean

all-devprod-containers-clean:
	docker rm -f taskmuncher-client-devprod
	docker rm -f taskmuncher-server-devprod
	docker rm -f taskmuncher-mongo-client-devprod
	docker rm -f taskmuncher-nginx-devprod

all-devprod-images-clean:
	docker rmi -f taskmuncher-client-devprod-image
	docker rmi -f taskmuncher-server-devprod-image
	docker rmi -f taskmuncher-mongo-client-devprod-image
	docker rmi -f taskmuncher-nginx-devprod-image

#
# client
#

client-devprod-build:
	docker build -f ./config/client/devprod/Dockerfile \
		-t taskmuncher-client-devprod-image:latest \
		.

client-devprod-sh:
	docker run --rm --name taskmuncher-client-devprod \
		-it \
		taskmuncher-client-devprod-image \
		sh

client-devprod-run:
	docker run \
		--rm \
		--name taskmuncher-client-devprod \
		-it \
		-p 8040:8040 \
		-p 3110:3110 \
		taskmuncher-client-devprod-image:latest

#
# server
#

server-devprod-build:
	docker build -f ./config/server/dev/Dockerfile \
	-t taskmuncher-server-devprod-image:latest \
	.

server-devprod-sh:
	docker run --rm --name taskmuncher-server-devprod \
		-it taskmuncher-server-devprod-image \
		sh

server-devprod-run:
	docker run --rm --name taskmuncher-server-devprod \
		-it -p 3110:3110 \
		taskmuncher-server-devprod-image:latest

#
# mongo-client
#

mongo-client-devprod-build:
	docker build -f ./config/mongo-client/Dockerfile \
		-t taskmuncher-mongo-client-devprod-image:latest \
		.

mongo-client-devprod-sh:
	docker run --rm --name taskmuncher-mongo-client-devprod \
		-it taskmuncher-mongo-client-devprod-image \
		sh

mongo-client-devprod-run:
	docker run \
		--rm \
		--name taskmuncher-mongo-client-devprod \
		-it \
		taskmuncher-mongo-client-devprod-image:latest

#
# nginx
#

nginx-devprod-build:
	docker build -f ./config/nginx/Dockerfile \
	-t taskmuncher-nginx-devprod-image:latest \
	.

nginx-devprod-sh:
	docker run --rm --name taskmuncher-nginx-devprod \
		-it taskmuncher-nginx-devprod-image \
		sh

nginx-devprod-run:
	docker run \
		--rm \
		--name taskmuncher-nginx-devprod \
		-it \
		taskmuncher-nginx-devprod-image:latest

Discuss Makefile for the Production Application running in a Development Environment

This is very similar in structure except for the addition of a Nginx component.

Makefile using Docker Compose for a Development Environment

Docker Compose is a tool that allows for the configuration of application services running multi-container Docker applications.

Docker Compose configures the networking so the components may have the required access to other components.

This is a tremendous feature allowing for the entire environment to be configured or, if you like, composed.

Create file make/dev-compose.mk

#
# compose dev
#
compose-dev-build:
	docker-compose -f ./docker-compose-dev.yml \
		-p taskmuncher-dev build

compose-dev-up:
	docker-compose -f ./docker-compose-dev.yml -p taskmuncher-dev up

compose-dev-down:
	docker-compose -f ./docker-compose-dev.yml -p taskmuncher-dev down

compose-dev-clean: compose-dev-down
	docker rm -f taskmuncher-dev

	docker rmi -f taskmuncher-client-dev
	docker rmi -f taskmuncher-api-dev
	docker rmi -f taskmuncher-mongo-client-dev
	docker rmi -f taskmuncher-nginx-dev

	docker rmi -f mongo

Notice

compose-dev-build - build Docker images using ./docker-compose-dev.yml
compose-dev-up - builds, (re)creates, starts, and attaches to containers for a service. Unless they are already running, this command also starts any linked services.
compose-dev-down - stops containers and removes containers, networks, volumes, and images created by up.
compose-dev-clean - stop the container and remove any containers and images created by the build

Docker Compose YAML file for a Development Environment

Note that each service is actually a hostname which can be accessed by any other container that was created by docker-compose.

Create file ./docker-compose-dev.yml

version: '3'
services:

  mongodb:
    image: 'mongo:latest'
    volumes: 
      - ./mongo_data:/data/db
    ports:
      - 27017:27017

  api:
    image: taskmuncher-api-dev
    depends_on:
      - mongodb
    build:
      dockerfile: ./config/server/dev/Dockerfile
      context: .

  client:
    image: taskmuncher-client-dev
    stdin_open: true
    build:
      dockerfile: ./config/client/dev/Dockerfile
      context: .
    environment:
      - CHOKIDAR_USEPOLLING=true 

  nginx:
    image: taskmuncher-nginx-dev
    depends_on:
      - api
      - client
    restart: always
    build:
      dockerfile: ./config/nginx/Dockerfile
      context: .
    ports:
      - '8100:8100'

  mongo-client:
    image: taskmuncher-mongo-client-dev
    # container_name: taskmuncher-mongo-client
    build:
      dockerfile: ./config/mongo-client/Dockerfile
      context: .
    depends_on:
      - mongodb

Let's discuss ports. The general form is host port: container port

	ports:
		- 80:8100

exposes port 8100 to other Docker containers on the same network (for inter-container communication) and port 80 to the host.

Let's discuss each service

  mongodb:
    image: 'mongo:latest'
    volumes: 
      - ./mongo_data:/data/db
    ports:
      - 27017:27017

Service name mongodb

build from the latest version of mongo in the Dockerhub repository
mount ./mongo_data from local file system to /data/db in the running mongodb container
- this allows for MongoDB data to be saved when the Docker container terminates.
expose port 27017

Adding ./mongo_data to .gitignore and .dockerignore may be a requirement as neither a github repository nor a Docker image should persist MongoDB data.

  api:
    image: taskmuncher-api-dev
    depends_on:
      - mongodb
    build:
      dockerfile: ./config/server/dev/Dockerfile
      context: .

Service name api

build image taskmuncher-api-dev using Dockerfile ./config/server/dev/Dockerfile and context .
depends_on expresses dependency between services. Services are started in dependency order, however each depends_on does not wait for a service to be ready before it starts the next service.

  client:
    image: taskmuncher-client-dev
    stdin_open: true
    build:
      dockerfile: ./config/client/dev/Dockerfile
      context: .
    environment:
      - CHOKIDAR_USEPOLLING=true

Service name client

build image taskmuncher-client-dev using Dockerfile ./config/client/dev/Dockerfile and context .
CHOKIDAR_USEPOLLING=true appears to be required to allow webpack to hot swap React.

  nginx:
    image: taskmuncher-nginx-dev
    depends_on:
      - api
      - client
    restart: always
    build:
      dockerfile: ./config/nginx/Dockerfile
      context: .
    ports:
      - '8100:8100'

Service name nginx

build image taskmuncher-nginx-dev using Dockerfile ./config/nginx/Dockerfile and context .
depends_on - allow other services to start first
restart: always - restart service if it terminates
maps host port 8100 to container port 8100

  mongo-client:
    image: taskmuncher-mongo-client-dev
    build:
      dockerfile: ./config/mongo-client/Dockerfile
      context: .
    depends_on:
      - mongodb

Service name mongo-client

build image taskmuncher-mongo-client-dev using Dockerfile ./config/mongo-client/Dockerfile and context .
depends_on - allow other services to start first

Makefile using Docker Compose for a Production Application in a Development Environment

The following uses the same rules as described above.

Create file make/devprod-compose.mk

#
# compose devprod
#
compose-devprod-build:
	docker-compose -f ./docker-compose-devprod.yml \
		-p taskmuncher-devprod build

compose-devprod-up:
	docker-compose -f ./docker-compose-devprod.yml \
		-p taskmuncher-devprod up

compose-devprod-down:
	docker-compose -f ./docker-compose-devprod.yml \
		-p taskmuncher-devprod down

compose-devprod-config:
	docker-compose -f ./docker-compose-devprod.yml -p taskmuncher-devprod config

compose-devprod-ps:
	docker-compose -f ./docker-compose-devprod.yml -p taskmuncher-devprod ps

compose-devprod-clean: compose-devprod-down
	docker rm -f taskmuncher-devprod

	docker rmi -f taskmuncher-client-devprod
	docker rmi -f taskmuncher-api-devprod
	docker rmi -f taskmuncher-mongo-client-devprod
	docker rmi -f taskmuncher-nginx-devprod

	docker rmi -f mongo

Create file docker-compose-devprod.yml

version: '3'
services:

  mongodb:
    image: 'mongo:latest'
    volumes: 
      - ./mongo_data:/data/db
    ports:
      - 27017:27017

  api:
    image: taskmuncher-api-devprod
    depends_on:
      - mongodb
    build:
      dockerfile: ./config/server/dev/Dockerfile
      context: .

  client:
    image: taskmuncher-client-devprod
    stdin_open: true
    build:
      dockerfile: ./config/client/devprod/Dockerfile
      context: .

  nginx:
    image: taskmuncher-nginx-devprod
    depends_on:
      - api
      - client
    restart: always
    build:
      dockerfile: ./config/nginx/Dockerfile
      context: .
    ports:
      - '8100:8100'

  mongo-client:
    image: taskmuncher-mongo-client-devprod
    build:
      dockerfile: ./config/mongo-client/Dockerfile
      context: .
    depends_on:
      - mongodb

Docker BasicsCreating and Running a Docker Development Environment