Understanding the Behavior of ARG and ENV for Setting Environment Variables in Docker

Overview

1. RUN export
   • Only usable within RUN export
   • Not usable elsewhere
2. ARG
   • Once set, usable throughout the build process, within docker image build
   • Not usable in CMD, ENTRYPOINT, or within docker container run
3. ENV
   • Once set, usable thereafter

Sample Code

This will work with copy-paste.

Dockerfile

#
# Display environment variables JAPANESE, ENGLISH, CHINESE
#
FROM python

ARG JAPANESE
ARG ENGLISH

# 1. RUN export
RUN export CHINESE="你好，世界！" \
  && python -c "import os; print(*map(os.environ.get, ('JAPANESE', 'ENGLISH', 'CHINESE')), sep=' | ')"
#
# こんにちは、世界！ | Hello, world! | 你好，世界！
#

# 2. ARG
RUN export CHINESE="你好，世界！"
RUN python -c "import os; print(*map(os.environ.get, ('JAPANESE', 'ENGLISH', 'CHINESE')), sep=' | ')"
#
# こんにちは、世界！ | Hello, world! | None
#
#   CHINESE is not saved as an environment variable
#   Different shell session for each RUN command

# 3. ENV
ENV JAPANESE="こんにちは、世界！"
CMD python -c "import os; print(*map(os.environ.get, ('JAPANESE', 'ENGLISH', 'CHINESE')), sep=' | ')"
#
# こんにちは、世界！ | None | None
#
#   ENGLISH is not saved as an environment variable
#   Environment variables set with ARG command are not applied in CMD

#
# Build and run the above Dockerfile
#
docker image build \
  --build-arg JAPANESE="こんにちは、世界！" \
  --build-arg ENGLISH="Hello, world!" \
  --no-cache \
  --progress=plain \
  --tag app \
  .
docker container run app

Supplementary

• I used the python command instead of echo to display environment variables because I wanted to verify if they could be referenced inside a file like main.py outside of the Dockerfile.
• The code under #1 is equivalent to the code under #2

#1
import os; print(*map(os.environ.get, ('JAPANESE', 'ENGLISH', 'CHINESE')), sep=' | ')

#2
import os
print(
  os.environ.get('JAPANESE'),
  os.environ.get('ENGLISH'),
  os.environ.get('CHINESE'),
  sep=' | '
)

Behavior

1. RUN export Command

• Even when using RUN export ... or RUN source ..., the values of the environment variables set are not available in the next RUN command
• According to ChatGPT, this is because "environment variables set within a RUN command in a Dockerfile using source or export exist only within the shell session in which that RUN command is executed."
• I wondered why we don't just use RUN export but rather the Docker-specific ENV command. Turns out, this is why...
• In my case, I had a lot of environment variables set in set_env_vars.sh, so I wanted to use RUN source set_env_vars.sh. I managed to work around it using a command chain with &&.
• Workarounds:
  1. Example inside a RUN command:
     • RUN export MY_VAR=value && echo $MY_VAR
  2. Example inside a CMD command:
     • CMD source set_env_vars.sh && python main.py
  3. Example inside an ENTRYPOINT command:
     • ENTRYPOINT entrypoint.sh — inside entrypoint.sh, you can utilize the export, source commands

2. ARG Command

• To reference a value specified with ARG in CMD, set it with ENV
• A value specified with ARG cannot be referenced in CMD
• Values specified with ARG can only be referenced during the build process (they can be referenced during docker container build but not during docker container run)

Questions

I asked ChatGPT some questions and have summarized the answers below.

1. Why is there an ENV command?

Why did Docker introduce the ENV command? Why does Docker separate shell sessions for each RUN command? What would be the downside if there was no ENV command and shell sessions were not separated, allowing environment variables set with RUN export to be available afterwards?

ChatGPT explains:

Why are shell sessions separated for each RUN command?

1. Security and Isolation: Docker aims to isolate processes within containers to enhance security. By having each RUN command maintain a separate shell session, it prevents unwanted state carryover, ensuring that each step starts from a predictable and reproducible state.
2. Reproducibility and Predictability: Starting a new shell session for each RUN command ensures that the build process is consistent and not influenced by previous commands, which allows for accurate predictions and ease in debugging complex build processes.
3. Maintenance and Debugging Ease: Since each RUN command is independent, it is easier to identify and address issues at specific stages of the build. It also allows for targeted modifications and rebuilding of specific RUN commands as needed.

2. Why is there an ARG command?

Why did Docker introduce the ARG command? Why can't environment variables set with ARG be referenced in CMD? What would be the downside if there was no ARG command and values specified with --build-arg were directly assigned to environment variables set with ENV?

ChatGPT explains:
Integrating ARG and ENV into one would expose build-time settings unnecessarily at runtime, posing a risk. For instance, if credentials used during the build remain in the runtime image, it increases security risks. Moreover, build parameters and runtime settings often differ, so keeping them separate allows for clearer management and more flexible container configuration and operation by users.

3. Is there one shell session or two?

I am a beginner with Linux and do not understand shell sessions well. Which of the following is a closer description of how Docker's shell sessions work?

1. Is there only one shell session? Do ENV and ARG set environment variables that are written to this shell session when RUN commands are executed?
2. Are there two shell sessions? Is there a global shell session operating in addition to a local shell session created at the time of RUN?

ChatGPT explains:

Each RUN command initiates a new shell session, which is temporary and ends once the RUN command completes. Environment variables set with ENV are automatically imported into these temporary shell sessions.

There is no concept of a 'global shell session' within Docker. Instead, environment variables set during the image build (using ENV) are saved in the image metadata and can be used in any container derived from that image.