SageMakerでNotebook Jobの実行に使えるコンテナイメージ(CPUタイプ)に、タグ0-cpuと1-cpuの2種類があったため、その差を調べた。
（参考）Amazon SageMaker Deploper Guide - Supported URI tags
(2024/9/19時点の記述)
The following list shows the tags you can include in your image URI.
1-cpu
1-gpu
0-cpu
0-gpuThe following examples show URIs with various tag formats:
542918446943.dkr.ecr.us-west-2.amazonaws.com/sagemaker-distribution-prod:1-cpu
542918446943.dkr.ecr.us-west-2.amazonaws.com/sagemaker-distribution-prod:0-gpu
調査した背景は、0-cpuのコンテナで実行したところpandas.read_csv()がs3fsのエラーが出てなかなか解消せず、1-cpuのコンテナではそのエラーが出なかったため。その差がどこにあるかを念のため確認したかった。
なお上記のAWSドキュメントには違いは掲載されていない。（2024/9/19時点）
(追加情報 2024/9/19)
GithubのAmazon SageMaker Distribution - Image tagsによると、イメージのタグは<AMAZON_SAGEMAKER_DISTRIBUTION_VERSION_NUMBER>-<CPU_OR_GPU> という形式。
イメージの中身の違い (Dockerfileの差異)
ECRからイメージをローカルにダウンロードして、docker history --no-truncコマンドでDockerfileを逆作成して中身を比較してみる。
環境：
- macOS Sonoma 14.6.1
- Docker desktop 25.0.3
- aws-cli/2.15.36
-
AWS CLIでawsへの認証情報をセットしておく。
環境に合わせた方法で認証情報をセットする。
-
docker loginでログインする。
aws ecr get-login-password --region ap-northeast-1 | docker login --username AWS --password-stdin 010972774902.dkr.ecr.ap-northeast-1.amazonaws.com
-
ECRから目的のイメージをdocker pullする。
docker pull 010972774902.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-distribution-prod:0-cpu docker pull 010972774902.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-distribution-prod:1-cpu
-
ローカルにpullできたイメージを確認
docker image ls REPOSITORY TAG IMAGE ID CREATED SIZE 010972774902.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-distribution-prod 1-cpu 9441f662f574 6 weeks ago 7.78GB 010972774902.dkr.ecr.ap-northeast-1.amazonaws.com/sagemaker-distribution-prod 0-cpu 56e714fbc4f4 7 months ago 4.9GB1-cpuの方が新しく、サイズも大きい。
-
各イメージの元になったDockerfileを生成して、diffで比較する
docker history --no-trunc 56e714fbc4f4 | tac | tr -s ' ' | cut -d " " -f 5- | sed 's,^/bin/sh -c #(nop) ,,g' | sed 's,^/bin/sh -c,RUN,g' | sed 's, && ,\n & ,g' | sed 's,\s*[0-9]*[\.]*[0-9]*\s*[kMG]*B\s*$,,g' > cpu-0.txt docker history --no-trunc 9441f662f574 | tac | tr -s ' ' | cut -d " " -f 5- | sed 's,^/bin/sh -c #(nop) ,,g' | sed 's,^/bin/sh -c,RUN,g' | sed 's, && ,\n & ,g' | sed 's,\s*[0-9]*[\.]*[0-9]*\s*[kMG]*B\s*$,,g' > cpu-1.txt diff cpu-0.txt cpu-1.txt(参考) Stackoverflow - How to generate a Dockerfile from an image?
0-cpuタグのコンテナイメージの中身
ARG RELEASE 0B
ARG LAUNCHPAD_BUILD_ARCH 0B
LABEL org.opencontainers.image.ref.name=ubuntu 0B
LABEL org.opencontainers.image.version=22.04 0B
ADD file:99224b1f237763b3053ca27ea5641f9a801c21154c7ccbff2c099654cc6db942 in / 77.9MB
CMD ["/bin/bash"] 0B
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 0B buildkit.dockerfile.v0
ENV ENV_NAME=base 0B buildkit.dockerfile.v0
ENV MAMBA_ROOT_PREFIX=/opt/conda 0B buildkit.dockerfile.v0
ENV MAMBA_EXE=/bin/micromamba 0B buildkit.dockerfile.v0
COPY /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt # buildkit 209kB buildkit.dockerfile.v0
COPY /tmp/bin/micromamba /bin/micromamba # buildkit 14.2MB buildkit.dockerfile.v0
RUN /bin/sh -c { grep -q '^ID=alpine$' /etc/os-release
&& apk add --no-cache bash; } || true # buildkit 0B buildkit.dockerfile.v0
ARG MAMBA_USER=mambauser 0B buildkit.dockerfile.v0
ARG MAMBA_USER_ID=57439 0B buildkit.dockerfile.v0
ARG MAMBA_USER_GID=57439 0B buildkit.dockerfile.v0
ENV MAMBA_USER=mambauser 0B buildkit.dockerfile.v0
ENV MAMBA_USER_ID=57439 0B buildkit.dockerfile.v0
ENV MAMBA_USER_GID=57439 0B buildkit.dockerfile.v0
COPY _dockerfile_initialize_user_accounts.sh /usr/local/bin/_dockerfile_initialize_user_accounts.sh # buildkit 981B buildkit.dockerfile.v0
COPY _dockerfile_setup_root_prefix.sh /usr/local/bin/_dockerfile_setup_root_prefix.sh # buildkit 112B buildkit.dockerfile.v0
RUN |3 MAMBA_USER=mambauser MAMBA_USER_ID=57439 MAMBA_USER_GID=57439 /bin/sh -c /usr/local/bin/_dockerfile_initialize_user_accounts.sh
&& /usr/local/bin/_dockerfile_setup_root_prefix.sh # buildkit 18.6MB buildkit.dockerfile.v0
USER mambauser 0B buildkit.dockerfile.v0
WORKDIR /tmp 0B buildkit.dockerfile.v0
COPY _apptainer_shell.sh /usr/local/bin/_apptainer_shell.sh # buildkit 49B buildkit.dockerfile.v0
COPY _entrypoint.sh /usr/local/bin/_entrypoint.sh # buildkit 866B buildkit.dockerfile.v0
COPY _activate_current_env.sh /usr/local/bin/_activate_current_env.sh # buildkit 1.07kB buildkit.dockerfile.v0
ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"] 0B buildkit.dockerfile.v0
CMD ["/bin/bash"] 0B buildkit.dockerfile.v0
COPY _dockerfile_shell.sh /usr/local/bin/_dockerfile_shell.sh # buildkit 222B buildkit.dockerfile.v0
SHELL [/usr/local/bin/_dockerfile_shell.sh] 0B buildkit.dockerfile.v0
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG CUDA_MAJOR_MINOR_VERSION= 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG ENV_IN_FILENAME 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG ARG_BASED_ENV_IN_FILENAME 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG NB_USER=sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG NB_UID=1000 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG NB_GID=100 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV SAGEMAKER_LOGGING_DIR=/var/log/sagemaker/ 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV STUDIO_LOGGING_DIR=/var/log/studio/ 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER root 0B
|5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh usermod "--login=${NB_USER}" "--home=/home/${NB_USER}" --move-home "-u ${NB_UID}" "${MAMBA_USER}"
&& groupmod "--new-name=${NB_USER}" --non-unique "-g ${NB_GID}" "${MAMBA_USER}"
&& echo "${NB_USER}" > "/etc/arg_mamba_user"
&& : 18.6MB
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV MAMBA_USER=sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV USER=sagemaker-user 0B
|5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh apt-get update
&& apt-get install -y --no-install-recommends sudo gettext-base wget curl unzip git rsync build-essential openssh-client
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata krb5-user libkrb5-dev libsasl2-dev libsasl2-modules
&& chmod g+w /etc/passwd
&& echo "ALL ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
&& touch /etc/krb5.conf.lock
&& chown ${NB_USER}:${MAMBA_USER} /etc/krb5.conf*
&& curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
&& unzip awscliv2.zip
&& sudo ./aws/install
&& rm -rf aws awscliv2.zip
&& : 582MB
|5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh echo "source /usr/local/bin/_activate_current_env.sh" | tee --append /etc/profile 629B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY --chown=sagemaker-user:sagemaker-usermulti:8abe73ab7ad7e79fe399b3eb80b9a5dd0186437df6b513d752eda5a460901c7a in /tmp/ 52.3kB
|5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh if [[ -z $ARG_BASED_ENV_IN_FILENAME ]] ; then echo 'No ARG_BASED_ENV_IN_FILENAME passed' ; else envsubst < /tmp/$ARG_BASED_ENV_IN_FILENAME | tee --append /tmp/$ENV_IN_FILENAME ; fi 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG CONDA_OVERRIDE_CUDA= 0B
|6 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh micromamba install -y --name base --file /tmp/$ENV_IN_FILENAME
&& micromamba clean --all --yes --force-pkgs-dirs
&& rm -rf /tmp/*.in 4.17GB
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG MAMBA_DOCKERFILE_ACTIVATE=1 0B
|7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh sudo ln -s $(which python3) /usr/bin/python 22B
|7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh install-glue-kernels
&& SITE_PACKAGES=$(pip show aws-glue-sessions | grep Location | awk '{print $2}')
&& jupyter-kernelspec install $SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_pyspark --user
&& jupyter-kernelspec install $SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_spark --user
&& mv /home/sagemaker-user/.local/share/jupyter/kernels/glue_pyspark /opt/conda/share/jupyter/kernels
&& mv /home/sagemaker-user/.local/share/jupyter/kernels/glue_spark /opt/conda/share/jupyter/kernels
&& sed -i '/if not store_history and (/i\ if "sm_analytics_runtime_check" in code:\n return await self._complete_cell()\n' "$SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_kernel_base/BaseKernel.py" 466kB
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:8552c645d5ea804d01cafbd84feb957764792ba462f6acbffef2f4ea5f8899c7 in /opt/conda/share/jupyter/kernels/glue_pyspark/kernel.json 370B
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:3aab153e4748900d5efe73f35e36a963705a12d60bbd433073d494c1e493d0c9 in /opt/conda/share/jupyter/kernels/glue_spark/kernel.json 356B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER root 0B
|7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh HOME_DIR="/home/${NB_USER}/licenses"
&& mkdir -p ${HOME_DIR}
&& curl -o ${HOME_DIR}/oss_compliance.zip https://aws-dlinfra-utilities.s3.amazonaws.com/oss_compliance.zip
&& unzip ${HOME_DIR}/oss_compliance.zip -d ${HOME_DIR}/
&& cp ${HOME_DIR}/oss_compliance/test/testOSSCompliance /usr/local/bin/testOSSCompliance
&& chmod +x /usr/local/bin/testOSSCompliance
&& chmod +x ${HOME_DIR}/oss_compliance/generate_oss_compliance.sh
&& ${HOME_DIR}/oss_compliance/generate_oss_compliance.sh ${HOME_DIR} python
&& rm -rf ${HOME_DIR}/oss_compliance* 5.84MB
|7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p $SAGEMAKER_LOGGING_DIR
&& chmod a+rw $SAGEMAKER_LOGGING_DIR
&& mkdir -p ${STUDIO_LOGGING_DIR}
&& chown ${NB_USER}:${MAMBA_USER} ${STUDIO_LOGGING_DIR} 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV PATH=/opt/conda/bin:/opt/conda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) WORKDIR /home/sagemaker-user 0B
|7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh pip install "krb5>=0.5.1,<0.6"
&& pip show krb5 | grep Require | xargs -i sh -c '[ $(echo {} | cut -d: -f2 | wc -w) -eq 0 ] ' 9.32MB
|7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh SYSTEM_PYTHON_PATH=$(python3 -c "from __future__ import print_function;import sysconfig; print(sysconfig.get_paths().get('purelib'))")
&& jupyter-kernelspec remove -f -y sparkrkernel
&& cp -a ${SYSTEM_PYTHON_PATH}/sagemaker_studio_analytics_extension/patches/configuration.py ${SYSTEM_PYTHON_PATH}/sparkmagic/utils/
&& cp -a ${SYSTEM_PYTHON_PATH}/sagemaker_studio_analytics_extension/patches/reliablehttpclient.py ${SYSTEM_PYTHON_PATH}/sparkmagic/livyclientlib/reliablehttpclient.py
&& sed -i 's= "python"= "/opt/conda/bin/python"=g' /opt/conda/share/jupyter/kernels/pysparkkernel/kernel.json /opt/conda/share/jupyter/kernels/sparkkernel/kernel.json
&& sed -i 's="Spark"="SparkMagic Spark"=g' /opt/conda/share/jupyter/kernels/sparkkernel/kernel.json
&& sed -i 's="PySpark"="SparkMagic PySpark"=g' /opt/conda/share/jupyter/kernels/pysparkkernel/kernel.json 14kB
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV SHELL=/bin/bash 0B
SIZE COMMENT
1-cpuタグのコンテナイメージの中身
ARG RELEASE 0B
ARG LAUNCHPAD_BUILD_ARCH 0B
LABEL org.opencontainers.image.ref.name=ubuntu 0B
LABEL org.opencontainers.image.version=22.04 0B
ADD file:d5da92199726e42da09a6f75a778befb607fe3f79e4afaf7ef5188329b26b386 in / 77.9MB
CMD ["/bin/bash"] 0B
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8 0B buildkit.dockerfile.v0
ENV ENV_NAME=base 0B buildkit.dockerfile.v0
ENV MAMBA_ROOT_PREFIX=/opt/conda 0B buildkit.dockerfile.v0
ENV MAMBA_EXE=/bin/micromamba 0B buildkit.dockerfile.v0
COPY /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt # buildkit 209kB buildkit.dockerfile.v0
COPY /tmp/bin/micromamba /bin/micromamba # buildkit 14.3MB buildkit.dockerfile.v0
RUN /bin/sh -c { grep -q '^ID=alpine$' /etc/os-release
&& apk add --no-cache bash; } || true # buildkit 0B buildkit.dockerfile.v0
ARG MAMBA_USER=mambauser 0B buildkit.dockerfile.v0
ARG MAMBA_USER_ID=57439 0B buildkit.dockerfile.v0
ARG MAMBA_USER_GID=57439 0B buildkit.dockerfile.v0
ENV MAMBA_USER=mambauser 0B buildkit.dockerfile.v0
ENV MAMBA_USER_ID=57439 0B buildkit.dockerfile.v0
ENV MAMBA_USER_GID=57439 0B buildkit.dockerfile.v0
COPY _dockerfile_initialize_user_accounts.sh /usr/local/bin/_dockerfile_initialize_user_accounts.sh # buildkit 981B buildkit.dockerfile.v0
COPY _dockerfile_setup_root_prefix.sh /usr/local/bin/_dockerfile_setup_root_prefix.sh # buildkit 112B buildkit.dockerfile.v0
RUN |3 MAMBA_USER=mambauser MAMBA_USER_ID=57439 MAMBA_USER_GID=57439 /bin/sh -c /usr/local/bin/_dockerfile_initialize_user_accounts.sh
&& /usr/local/bin/_dockerfile_setup_root_prefix.sh # buildkit 18.6MB buildkit.dockerfile.v0
USER mambauser 0B buildkit.dockerfile.v0
WORKDIR /tmp 0B buildkit.dockerfile.v0
COPY _apptainer_shell.sh /usr/local/bin/_apptainer_shell.sh # buildkit 49B buildkit.dockerfile.v0
COPY _entrypoint.sh /usr/local/bin/_entrypoint.sh # buildkit 866B buildkit.dockerfile.v0
COPY _activate_current_env.sh /usr/local/bin/_activate_current_env.sh # buildkit 1.07kB buildkit.dockerfile.v0
ENTRYPOINT ["/usr/local/bin/_entrypoint.sh"] 0B buildkit.dockerfile.v0
CMD ["/bin/bash"] 0B buildkit.dockerfile.v0
COPY _dockerfile_shell.sh /usr/local/bin/_dockerfile_shell.sh # buildkit 222B buildkit.dockerfile.v0
SHELL [/usr/local/bin/_dockerfile_shell.sh] 0B buildkit.dockerfile.v0
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG CUDA_MAJOR_MINOR_VERSION= 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG ENV_IN_FILENAME 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG ARG_BASED_ENV_IN_FILENAME 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG AMZN_BASE=/opt/amazon/sagemaker 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG DB_ROOT_DIR=/opt/db 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG NB_USER=sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG NB_UID=1000 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG NB_GID=100 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG FIPS_VALIDATED_SSL=3.0.8 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV SAGEMAKER_LOGGING_DIR=/var/log/sagemaker/ 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV STUDIO_LOGGING_DIR=/var/log/studio/ 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV EDITOR=nano 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER root 0B
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh usermod "--login=${NB_USER}" "--home=/home/${NB_USER}" --move-home "-u ${NB_UID}" "${MAMBA_USER}"
&& groupmod "--new-name=${NB_USER}" --non-unique "-g ${NB_GID}" "${MAMBA_USER}"
&& echo "${NB_USER}" > "/etc/arg_mamba_user"
&& : 18.6MB
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV MAMBA_USER=sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV USER=sagemaker-user 0B
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh apt-get update
&& apt-get upgrade -y
&& apt-get install -y --no-install-recommends sudo gettext-base wget curl unzip git rsync build-essential openssh-client nano cron less mandoc
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata krb5-user libkrb5-dev libsasl2-dev libsasl2-modules
&& chmod g+w /etc/passwd
&& echo "ALL ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
&& touch /etc/krb5.conf.lock
&& chown ${NB_USER}:${MAMBA_USER} /etc/krb5.conf*
&& curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
&& unzip awscliv2.zip
&& sudo ./aws/install
&& rm -rf aws awscliv2.zip
&& : 601MB
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh echo "source /usr/local/bin/_activate_current_env.sh" | tee --append /etc/profile 629B
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data
&& chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data 0B
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /opt/amazon/sagemaker/user-data
&& chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/user-data 0B
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p ${DIRECTORY_TREE_STAGE_DIR} 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY dir:6a956536e3a61429f745fd780621c2ec5f4ab5fb6e80c90372cca9b293c5c941 in /opt/amazon/sagemaker/dir-staging/ 8.18kB
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh rsync -a ${DIRECTORY_TREE_STAGE_DIR}/ /
&& rm -rf ${DIRECTORY_TREE_STAGE_DIR} 8.18kB
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /etc/code-editor/extensions
&& while IFS= read -r url || [ -n "$url" ]; do echo "Downloading extension from ${url}..."
&& wget --no-check-certificate -P /etc/code-editor/extensions "${url}"; done < /etc/code-editor/extensions.txt 41.8MB
/usr/local/bin/_dockerfile_shell.sh #(nop) USER sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY --chown=sagemaker-user:sagemaker-usermulti:1afafb3a3057d9f95abfb629607799db849d4826f2a2695823eb57bc870c4af4 in /tmp/ 5.94kB
|9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh if [[ -z $ARG_BASED_ENV_IN_FILENAME ]] ; then echo 'No ARG_BASED_ENV_IN_FILENAME passed' ; else envsubst < /tmp/$ARG_BASED_ENV_IN_FILENAME | tee --append /tmp/$ENV_IN_FILENAME ; fi 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG CONDA_OVERRIDE_CUDA= 0B
|10 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh micromamba install -y --name base --file /tmp/$ENV_IN_FILENAME
&& micromamba clean --all --yes --force-pkgs-dirs
&& rm -rf /tmp/*.in 6.77GB
/usr/local/bin/_dockerfile_shell.sh #(nop) ARG MAMBA_DOCKERFILE_ACTIVATE=1 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh sudo ln -s $(which python3) /usr/bin/python 22B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh npm i -g npm 36.9MB
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh extensionloc=/opt/amazon/sagemaker/sagemaker-code-editor-server-data/extensions
&& mkdir -p "${extensionloc}"
&& for ext in /etc/code-editor/extensions/*.vsix; do echo "Installing extension ${ext}..."; sagemaker-code-editor --install-extension "${ext}" --extensions-dir "${extensionloc}" --server-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-server-data --user-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-user-data; done
&& cp /etc/code-editor/code_editor_machine_settings.json /opt/amazon/sagemaker/sagemaker-code-editor-server-data/data/Machine/settings.json 187MB
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh install-glue-kernels
&& SITE_PACKAGES=$(pip show aws-glue-sessions | grep Location | awk '{print $2}')
&& jupyter-kernelspec install $SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_pyspark --user
&& jupyter-kernelspec install $SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_spark --user
&& mv /home/sagemaker-user/.local/share/jupyter/kernels/glue_pyspark /opt/conda/share/jupyter/kernels
&& mv /home/sagemaker-user/.local/share/jupyter/kernels/glue_spark /opt/conda/share/jupyter/kernels
&& sed -i '/if not store_history and (/i\ if "sm_analytics_runtime_check" in code:\n return await self._complete_cell()\n' "$SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_kernel_base/BaseKernel.py" 456kB
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:6ce696974ceb19d70abac766700ae13bbe61f94f2b434f61825f28384cbd497c in /opt/conda/share/jupyter/kernels/glue_pyspark/kernel.json 363B
/usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:95c75cfdbd090ad4a6535597c0b1d017a6555be7f1a43f9a338474886b8729d8 in /opt/conda/share/jupyter/kernels/glue_spark/kernel.json 357B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh jupyter labextension disable @jupyter/collaboration-extension 110B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER root 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh HOME_DIR="/home/${NB_USER}/licenses"
&& mkdir -p ${HOME_DIR}
&& curl -o ${HOME_DIR}/oss_compliance.zip https://aws-dlinfra-utilities.s3.amazonaws.com/oss_compliance.zip
&& unzip ${HOME_DIR}/oss_compliance.zip -d ${HOME_DIR}/
&& cp ${HOME_DIR}/oss_compliance/test/testOSSCompliance /usr/local/bin/testOSSCompliance
&& chmod +x /usr/local/bin/testOSSCompliance
&& chmod +x ${HOME_DIR}/oss_compliance/generate_oss_compliance.sh
&& ${HOME_DIR}/oss_compliance/generate_oss_compliance.sh ${HOME_DIR} python
&& rm -rf ${HOME_DIR}/oss_compliance* 7.52MB
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p $SAGEMAKER_LOGGING_DIR
&& chmod a+rw $SAGEMAKER_LOGGING_DIR
&& mkdir -p ${STUDIO_LOGGING_DIR}
&& chown ${NB_USER}:${MAMBA_USER} ${STUDIO_LOGGING_DIR} 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh rm -rf /etc/code-editor 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /var/run/supervisord
&& chmod a+rw /var/run/supervisord 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p $DB_ROOT_DIR
&& chmod a+rw $DB_ROOT_DIR 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) USER sagemaker-user 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV PATH=/opt/conda/bin:/opt/conda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 0B
/usr/local/bin/_dockerfile_shell.sh #(nop) WORKDIR /home/sagemaker-user 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh INSTALLED_SSL=$(micromamba list | grep openssl | tr -s ' ' | cut -d ' ' -f 3 | head -n 1)
&& curl -L https://www.openssl.org/source/openssl-$FIPS_VALIDATED_SSL.tar.gz > openssl-$FIPS_VALIDATED_SSL.tar.gz
&& curl -L https://www.openssl.org/source/openssl-$INSTALLED_SSL.tar.gz > openssl-$INSTALLED_SSL.tar.gz
&& tar -xf openssl-$FIPS_VALIDATED_SSL.tar.gz
&& tar -xf openssl-$INSTALLED_SSL.tar.gz
&& cd openssl-$FIPS_VALIDATED_SSL
&& ./Configure enable-fips --prefix=/opt/conda --openssldir=/opt/conda/ssl
&& make
&& cd ../openssl-$INSTALLED_SSL
&& ./Configure enable-fips --prefix=/opt/conda --openssldir=/opt/conda/ssl
&& make
&& cp ../openssl-$FIPS_VALIDATED_SSL/providers/fips.so providers/.
&& cp ../openssl-$FIPS_VALIDATED_SSL/providers/fipsmodule.cnf providers/.
&& make tests
&& cd ../openssl-$FIPS_VALIDATED_SSL
&& make install_fips
&& cd ..
&& rm -rf ./openssl-* 1.86MB
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh cp /opt/conda/ssl/openssl.cnf /opt/conda/ssl/openssl-fips.cnf
&& sed -i "s:# .include fipsmodule.cnf:.include /opt/conda/ssl/fipsmodule.cnf:" /opt/conda/ssl/openssl-fips.cnf
&& sed -i 's:# fips = fips_sect:fips = fips_sect:' /opt/conda/ssl/openssl-fips.cnf 12.3kB
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV OPENSSL_MODULES=/opt/conda/lib64/ossl-modules/ 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh pip install "krb5>=0.5.1,<0.6"
&& pip show krb5 | grep Require | xargs -i sh -c '[ $(echo {} | cut -d: -f2 | wc -w) -eq 0 ] ' 0B
|11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh SYSTEM_PYTHON_PATH=$(python3 -c "from __future__ import print_function;import sysconfig; print(sysconfig.get_paths().get('purelib'))")
&& jupyter-kernelspec remove -f -y sparkrkernel
&& cp -a ${SYSTEM_PYTHON_PATH}/sagemaker_studio_analytics_extension/patches/configuration.py ${SYSTEM_PYTHON_PATH}/sparkmagic/utils/
&& cp -a ${SYSTEM_PYTHON_PATH}/sagemaker_studio_analytics_extension/patches/reliablehttpclient.py ${SYSTEM_PYTHON_PATH}/sparkmagic/livyclientlib/reliablehttpclient.py
&& sed -i 's= "python"= "/opt/conda/bin/python"=g' /opt/conda/share/jupyter/kernels/pysparkkernel/kernel.json /opt/conda/share/jupyter/kernels/sparkkernel/kernel.json
&& sed -i 's="Spark"="SparkMagic Spark"=g' /opt/conda/share/jupyter/kernels/sparkkernel/kernel.json
&& sed -i 's="PySpark"="SparkMagic PySpark"=g' /opt/conda/share/jupyter/kernels/pysparkkernel/kernel.json 14kB
/usr/local/bin/_dockerfile_shell.sh #(nop) ENV SHELL=/bin/bash 0B
SIZE COMMENT
両者のdiff
5c5
< ADD file:99224b1f237763b3053ca27ea5641f9a801c21154c7ccbff2c099654cc6db942 in / 77.9MB
---
> ADD file:d5da92199726e42da09a6f75a778befb607fe3f79e4afaf7ef5188329b26b386 in / 77.9MB
12c12
< COPY /tmp/bin/micromamba /bin/micromamba # buildkit 14.2MB buildkit.dockerfile.v0
---
> COPY /tmp/bin/micromamba /bin/micromamba # buildkit 14.3MB buildkit.dockerfile.v0
36a37,39
> /usr/local/bin/_dockerfile_shell.sh #(nop) ARG AMZN_BASE=/opt/amazon/sagemaker 0B
> /usr/local/bin/_dockerfile_shell.sh #(nop) ARG DB_ROOT_DIR=/opt/db 0B
> /usr/local/bin/_dockerfile_shell.sh #(nop) ARG DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging 0B
39a43
> /usr/local/bin/_dockerfile_shell.sh #(nop) ARG FIPS_VALIDATED_SSL=3.0.8 0B
41a46
> /usr/local/bin/_dockerfile_shell.sh #(nop) ENV EDITOR=nano 0B
43c48
< |5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh usermod "--login=${NB_USER}" "--home=/home/${NB_USER}" --move-home "-u ${NB_UID}" "${MAMBA_USER}"
---
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh usermod "--login=${NB_USER}" "--home=/home/${NB_USER}" --move-home "-u ${NB_UID}" "${MAMBA_USER}"
49,50c54,56
< |5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh apt-get update
< && apt-get install -y --no-install-recommends sudo gettext-base wget curl unzip git rsync build-essential openssh-client
---
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh apt-get update
> && apt-get upgrade -y
> && apt-get install -y --no-install-recommends sudo gettext-base wget curl unzip git rsync build-essential openssh-client nano cron less mandoc
60,61c66,78
< && : 582MB
< |5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh echo "source /usr/local/bin/_activate_current_env.sh" | tee --append /etc/profile 629B
---
> && : 601MB
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh echo "source /usr/local/bin/_activate_current_env.sh" | tee --append /etc/profile 629B
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data
> && chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/sagemaker-code-editor-server-data /opt/amazon/sagemaker/sagemaker-code-editor-user-data 0B
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /opt/amazon/sagemaker/user-data
> && chown $MAMBA_USER:$MAMBA_USER /opt/amazon/sagemaker/user-data 0B
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p ${DIRECTORY_TREE_STAGE_DIR} 0B
> /usr/local/bin/_dockerfile_shell.sh #(nop) COPY dir:6a956536e3a61429f745fd780621c2ec5f4ab5fb6e80c90372cca9b293c5c941 in /opt/amazon/sagemaker/dir-staging/ 8.18kB
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh rsync -a ${DIRECTORY_TREE_STAGE_DIR}/ /
> && rm -rf ${DIRECTORY_TREE_STAGE_DIR} 8.18kB
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /etc/code-editor/extensions
> && while IFS= read -r url || [ -n "$url" ]; do echo "Downloading extension from ${url}..."
> && wget --no-check-certificate -P /etc/code-editor/extensions "${url}"; done < /etc/code-editor/extensions.txt 41.8MB
63,64c80,81
< /usr/local/bin/_dockerfile_shell.sh #(nop) COPY --chown=sagemaker-user:sagemaker-usermulti:8abe73ab7ad7e79fe399b3eb80b9a5dd0186437df6b513d752eda5a460901c7a in /tmp/ 52.3kB
< |5 CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh if [[ -z $ARG_BASED_ENV_IN_FILENAME ]] ; then echo 'No ARG_BASED_ENV_IN_FILENAME passed' ; else envsubst < /tmp/$ARG_BASED_ENV_IN_FILENAME | tee --append /tmp/$ENV_IN_FILENAME ; fi 0B
---
> /usr/local/bin/_dockerfile_shell.sh #(nop) COPY --chown=sagemaker-user:sagemaker-usermulti:1afafb3a3057d9f95abfb629607799db849d4826f2a2695823eb57bc870c4af4 in /tmp/ 5.94kB
> |9 AMZN_BASE=/opt/amazon/sagemaker CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh if [[ -z $ARG_BASED_ENV_IN_FILENAME ]] ; then echo 'No ARG_BASED_ENV_IN_FILENAME passed' ; else envsubst < /tmp/$ARG_BASED_ENV_IN_FILENAME | tee --append /tmp/$ENV_IN_FILENAME ; fi 0B
66c83
< |6 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh micromamba install -y --name base --file /tmp/$ENV_IN_FILENAME
---
> |10 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh micromamba install -y --name base --file /tmp/$ENV_IN_FILENAME
68c85
< && rm -rf /tmp/*.in 4.17GB
---
> && rm -rf /tmp/*.in 6.77GB
70,71c87,93
< |7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh sudo ln -s $(which python3) /usr/bin/python 22B
< |7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh install-glue-kernels
---
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh sudo ln -s $(which python3) /usr/bin/python 22B
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh npm i -g npm 36.9MB
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh extensionloc=/opt/amazon/sagemaker/sagemaker-code-editor-server-data/extensions
> && mkdir -p "${extensionloc}"
> && for ext in /etc/code-editor/extensions/*.vsix; do echo "Installing extension ${ext}..."; sagemaker-code-editor --install-extension "${ext}" --extensions-dir "${extensionloc}" --server-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-server-data --user-data-dir /opt/amazon/sagemaker/sagemaker-code-editor-user-data; done
> && cp /etc/code-editor/code_editor_machine_settings.json /opt/amazon/sagemaker/sagemaker-code-editor-server-data/data/Machine/settings.json 187MB
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh install-glue-kernels
77,79c99,102
< && sed -i '/if not store_history and (/i\ if "sm_analytics_runtime_check" in code:\n return await self._complete_cell()\n' "$SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_kernel_base/BaseKernel.py" 466kB
< /usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:8552c645d5ea804d01cafbd84feb957764792ba462f6acbffef2f4ea5f8899c7 in /opt/conda/share/jupyter/kernels/glue_pyspark/kernel.json 370B
< /usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:3aab153e4748900d5efe73f35e36a963705a12d60bbd433073d494c1e493d0c9 in /opt/conda/share/jupyter/kernels/glue_spark/kernel.json 356B
---
> && sed -i '/if not store_history and (/i\ if "sm_analytics_runtime_check" in code:\n return await self._complete_cell()\n' "$SITE_PACKAGES/aws_glue_interactive_sessions_kernel/glue_kernel_base/BaseKernel.py" 456kB
> /usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:6ce696974ceb19d70abac766700ae13bbe61f94f2b434f61825f28384cbd497c in /opt/conda/share/jupyter/kernels/glue_pyspark/kernel.json 363B
> /usr/local/bin/_dockerfile_shell.sh #(nop) COPY file:95c75cfdbd090ad4a6535597c0b1d017a6555be7f1a43f9a338474886b8729d8 in /opt/conda/share/jupyter/kernels/glue_spark/kernel.json 357B
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh jupyter labextension disable @jupyter/collaboration-extension 110B
81c104
< |7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh HOME_DIR="/home/${NB_USER}/licenses"
---
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh HOME_DIR="/home/${NB_USER}/licenses"
89,90c112,113
< && rm -rf ${HOME_DIR}/oss_compliance* 5.84MB
< |7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p $SAGEMAKER_LOGGING_DIR
---
> && rm -rf ${HOME_DIR}/oss_compliance* 7.52MB
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p $SAGEMAKER_LOGGING_DIR
93a117,121
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh rm -rf /etc/code-editor 0B
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p /var/run/supervisord
> && chmod a+rw /var/run/supervisord 0B
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh mkdir -p $DB_ROOT_DIR
> && chmod a+rw $DB_ROOT_DIR 0B
97,99c125,149
< |7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh pip install "krb5>=0.5.1,<0.6"
< && pip show krb5 | grep Require | xargs -i sh -c '[ $(echo {} | cut -d: -f2 | wc -w) -eq 0 ] ' 9.32MB
< |7 CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= ENV_IN_FILENAME=cpu.env.out MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh SYSTEM_PYTHON_PATH=$(python3 -c "from __future__ import print_function;import sysconfig; print(sysconfig.get_paths().get('purelib'))")
---
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh INSTALLED_SSL=$(micromamba list | grep openssl | tr -s ' ' | cut -d ' ' -f 3 | head -n 1)
> && curl -L https://www.openssl.org/source/openssl-$FIPS_VALIDATED_SSL.tar.gz > openssl-$FIPS_VALIDATED_SSL.tar.gz
> && curl -L https://www.openssl.org/source/openssl-$INSTALLED_SSL.tar.gz > openssl-$INSTALLED_SSL.tar.gz
> && tar -xf openssl-$FIPS_VALIDATED_SSL.tar.gz
> && tar -xf openssl-$INSTALLED_SSL.tar.gz
> && cd openssl-$FIPS_VALIDATED_SSL
> && ./Configure enable-fips --prefix=/opt/conda --openssldir=/opt/conda/ssl
> && make
> && cd ../openssl-$INSTALLED_SSL
> && ./Configure enable-fips --prefix=/opt/conda --openssldir=/opt/conda/ssl
> && make
> && cp ../openssl-$FIPS_VALIDATED_SSL/providers/fips.so providers/.
> && cp ../openssl-$FIPS_VALIDATED_SSL/providers/fipsmodule.cnf providers/.
> && make tests
> && cd ../openssl-$FIPS_VALIDATED_SSL
> && make install_fips
> && cd ..
> && rm -rf ./openssl-* 1.86MB
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh cp /opt/conda/ssl/openssl.cnf /opt/conda/ssl/openssl-fips.cnf
> && sed -i "s:# .include fipsmodule.cnf:.include /opt/conda/ssl/fipsmodule.cnf:" /opt/conda/ssl/openssl-fips.cnf
> && sed -i 's:# fips = fips_sect:fips = fips_sect:' /opt/conda/ssl/openssl-fips.cnf 12.3kB
> /usr/local/bin/_dockerfile_shell.sh #(nop) ENV OPENSSL_MODULES=/opt/conda/lib64/ossl-modules/ 0B
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh pip install "krb5>=0.5.1,<0.6"
> && pip show krb5 | grep Require | xargs -i sh -c '[ $(echo {} | cut -d: -f2 | wc -w) -eq 0 ] ' 0B
> |11 AMZN_BASE=/opt/amazon/sagemaker CONDA_OVERRIDE_CUDA= CUDA_MAJOR_MINOR_VERSION= DB_ROOT_DIR=/opt/db DIRECTORY_TREE_STAGE_DIR=/opt/amazon/sagemaker/dir-staging ENV_IN_FILENAME=cpu.env.in FIPS_VALIDATED_SSL=3.0.8 MAMBA_DOCKERFILE_ACTIVATE=1 NB_GID=100 NB_UID=1000 NB_USER=sagemaker-user /usr/local/bin/_dockerfile_shell.sh SYSTEM_PYTHON_PATH=$(python3 -c "from __future__ import print_function;import sysconfig; print(sysconfig.get_paths().get('purelib'))")
diffの結果を確認したが、単純比較できないくらいかなり差があった。
含まれるpythonライブラリの違い (pip listの結果比較)
1-cpuと0-cpuのコンテナイメージを使って、単純にpip listの結果表示のみを行うNotebook Jobを実行し、出力結果を比較。
結果:
0-cpuタグのコンテナイメージでのpip list結果
Package Version
------------------------------------ ----------------
absl-py 2.1.0
aiofiles 22.1.0
aiohttp 3.9.1
aiosignal 1.3.1
aiosqlite 0.19.0
ansiwrap 0.8.4
anyio 4.2.0
archspec 0.2.2
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
astunparse 1.6.3
async-timeout 4.0.3
attrs 23.2.0
autovizwidget 0.21.0
aws-glue-sessions 1.0.4
Babel 2.14.0
backcall 0.2.0
bcrypt 4.1.2
beautifulsoup4 4.12.3
black 24.1.1
bleach 6.1.0
blinker 1.7.0
boltons 23.1.1
boto3 1.34.36
botocore 1.34.36
Brotli 1.1.0
cached-property 1.5.2
cachetools 5.3.2
certifi 2024.2.2
cffi 1.16.0
charset-normalizer 3.3.2
click 8.1.7
cloudpickle 2.2.1
colorama 0.4.6
comm 0.2.1
conda 23.11.0
conda-libmamba-solver 24.1.0
conda-package-handling 2.2.0
conda_package_streaming 0.9.0
contextlib2 21.6.0
contourpy 1.1.1
cryptography 42.0.2
cycler 0.12.1
debugpy 1.8.0
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.8
distro 1.9.0
docker 7.0.0
entrypoints 0.4
exceptiongroup 1.2.0
executing 2.0.1
fastjsonschema 2.19.1
filelock 3.13.1
flatbuffers 23.5.26
fonttools 4.48.1
fqdn 1.5.1
frozenlist 1.4.1
future 0.18.3
gast 0.5.4
gmpy2 2.1.2
google-auth 2.27.0
google-auth-oauthlib 1.0.0
google-pasta 0.2.0
greenlet 3.0.3
grpcio 1.54.3
gssapi 1.8.3
h5py 3.10.0
hdijupyterutils 0.21.0
idna 3.6
importlib-metadata 6.10.0
importlib-resources 6.1.1
ipykernel 6.29.1
ipython 8.12.2
ipython-genutils 0.2.0
ipywidgets 7.8.0
isoduration 20.11.0
jedi 0.19.1
Jinja2 3.1.3
jmespath 1.0.1
joblib 1.3.2
json5 0.9.14
jsonpatch 1.33
jsonpointer 2.4
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter 1.0.0
jupyter_client 7.4.9
jupyter-console 6.6.3
jupyter_core 5.7.1
jupyter-events 0.9.0
jupyter_server 2.12.5
jupyter_server_fileid 0.9.1
jupyter_server_terminals 0.5.2
jupyter_server_ydoc 0.8.0
jupyter-ydoc 0.2.4
jupyterlab 3.6.7
jupyterlab_pygments 0.3.0
jupyterlab_server 2.25.2
jupyterlab-widgets 1.1.7
keras 2.13.1
kiwisolver 1.4.5
krb5 0.5.1
libmambapy 1.5.6
Markdown 3.5.2
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.7.3
matplotlib-inline 0.1.6
mdurl 0.1.2
menuinst 2.0.2
mistune 3.0.2
mock 5.1.0
mpmath 1.3.0
multidict 6.0.5
multiprocess 0.70.16
munkres 1.1.4
mypy-extensions 1.0.0
nbclassic 1.0.0
nbclient 0.8.0
nbconvert 7.15.0
nbformat 5.9.2
nest_asyncio 1.6.0
networkx 3.1
nose 1.3.7
notebook 6.5.6
notebook_shim 0.2.3
numpy 1.24.4
oauthlib 3.2.2
opt-einsum 3.3.0
overrides 7.7.0
packaging 23.2
pandas 2.0.3
pandocfilters 1.5.0
papermill 2.4.0
paramiko 3.4.0
parso 0.8.3
pathos 0.3.2
pathspec 0.12.1
pexpect 4.9.0
pickleshare 0.7.5
pillow 10.2.0
pip 23.3.2
pkgutil_resolve_name 1.3.10
platformdirs 4.2.0
plotly 5.18.0
pluggy 1.4.0
ply 3.11
pooch 1.8.0
pox 0.3.4
ppft 1.7.6.8
prometheus-client 0.19.0
prompt-toolkit 3.0.42
protobuf 4.21.12
psutil 5.9.8
ptyprocess 0.7.0
pure-eval 0.2.2
pure-sasl 0.6.2
pyasn1 0.5.1
pyasn1-modules 0.3.0
pycosat 0.6.6
pycparser 2.21
Pygments 2.17.2
PyHive 0.7.0
PyJWT 2.8.0
PyNaCl 1.5.0
pyOpenSSL 24.0.0
pyparsing 3.1.1
PyQt5 5.15.9
PyQt5-sip 12.12.2
PySocks 1.7.1
pyspnego 0.9.1
python-dateutil 2.8.2
python-json-logger 2.0.7
pytz 2024.1
pyu2f 0.1.5
PyYAML 6.0.1
pyzmq 24.0.1
qtconsole 5.5.1
QtPy 2.4.1
referencing 0.33.0
requests 2.31.0
requests-kerberos 0.14.0
requests-oauthlib 1.3.1
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.7.0
rpds-py 0.17.1
rsa 4.9
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.8
s3transfer 0.10.0
sagemaker 2.206.0
sagemaker-headless-execution-driver 0.0.12
sagemaker-kernel-wrapper 0.0.2
sagemaker-studio-analytics-extension 0.0.21
sagemaker-studio-sparkmagic-lib 0.1.4
sasl 0.3.1
schema 0.7.5
scikit-learn 1.3.2
scipy 1.10.1
Send2Trash 1.8.2
setuptools 69.0.3
sip 6.7.12
six 1.16.0
smdebug-rulesconfig 1.0.1
sniffio 1.3.0
soupsieve 2.5
sparkmagic 0.21.0
SQLAlchemy 2.0.24
stack-data 0.6.2
sympy 1.12
tabulate 0.9.0
tblib 2.0.0
tenacity 8.2.3
tensorboard 2.13.0
tensorboard-data-server 0.7.0
tensorflow 2.13.1
tensorflow-estimator 2.13.0
termcolor 2.4.0
terminado 0.18.0
textwrap3 0.9.2
threadpoolctl 3.2.0
thrift 0.19.0
thrift-sasl 0.4.3
tinycss2 1.2.1
toml 0.10.2
tomli 2.0.1
torch 2.0.0.post101
torchvision 0.15.2a0+ab7b3e6
tornado 6.3.3
tqdm 4.66.1
traitlets 5.14.1
types-python-dateutil 2.8.19.20240106
typing_extensions 4.5.0
typing-utils 0.1.0
tzdata 2023.4
unicodedata2 15.1.0
uri-template 1.3.0
urllib3 1.26.18
wcwidth 0.2.13
webcolors 1.13
webencodings 0.5.1
websocket-client 1.7.0
Werkzeug 3.0.1
wheel 0.42.0
widgetsnbextension 3.6.6
wrapt 1.16.0
xgboost 1.7.6
y-py 0.5.9
yarl 1.9.4
ypy-websocket 0.8.2
zipp 3.17.0
zstandard 0.22.0
1-cpuタグのコンテナイメージでのpip list結果
Package Version
--------------------------------------- ----------------
absl-py 2.1.0
accelerate 0.21.0
aiobotocore 2.13.1
aiohttp 3.9.5
aioitertools 0.11.0
aiosignal 1.3.1
aiosqlite 0.19.0
altair 5.3.0
amazon-q-developer-jupyterlab-ext 3.2.0
amazon_sagemaker_jupyter_ai_q_developer 1.0.7
amazon_sagemaker_jupyter_scheduler 3.1.2
amazon-sagemaker-sql-editor 0.1.10
amazon-sagemaker-sql-execution 0.1.3
amazon-sagemaker-sql-magic 0.1.3
annotated-types 0.7.0
ansi2html 1.9.2
ansicolors 1.1.8
antlr4-python3-runtime 4.9.3
anyio 4.4.0
archspec 0.2.3
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asn1crypto 1.5.1
astroid 3.2.4
asttokens 2.4.1
astunparse 1.6.3
async-lru 2.0.4
async-timeout 4.0.3
attrs 23.2.0
autogluon 0.8.3
autogluon.common 0.8.3
autogluon.core 0.8.3
autogluon.features 0.8.3
autogluon.multimodal 0.8.3
autogluon.tabular 0.8.3
autogluon.timeseries 0.8.3
autopep8 2.0.4
autovizwidget 0.21.0
aws-embedded-metrics 3.2.0
aws-glue-sessions 1.0.6
Babel 2.14.0
bcrypt 4.2.0
beautifulsoup4 4.12.3
binaryornot 0.4.4
bleach 6.1.0
blinker 1.8.2
blis 0.7.10
boltons 24.0.0
boto3 1.34.131
botocore 1.34.131
Brotli 1.1.0
cached-property 1.5.2
cachetools 5.4.0
catalogue 2.0.10
catboost 1.2.5
certifi 2024.7.4
cffi 1.16.0
chardet 5.2.0
charset-normalizer 3.3.2
click 8.1.7
cloudpathlib 0.18.1
cloudpickle 2.2.1
colorama 0.4.6
comm 0.2.2
conda 23.11.0
conda-libmamba-solver 24.7.0
conda-package-handling 2.3.0
conda_package_streaming 0.10.0
confection 0.1.4
contextlib2 21.6.0
contourpy 1.2.1
cookiecutter 2.6.0
croniter 1.4.1
cryptography 43.0.0
cycler 0.12.1
cymem 2.0.8
cytoolz 0.12.3
dash 2.17.1
dask 2024.7.1
dataclasses-json 0.6.7
datasets 2.20.0
debugpy 1.8.5
decorator 5.1.1
deepmerge 1.1.1
defusedxml 0.7.1
dill 0.3.8
diskcache 5.6.3
distributed 2024.7.1
distro 1.9.0
docker 7.1.0
docstring-to-markdown 0.15
entrypoints 0.4
evaluate 0.4.1
exceptiongroup 1.2.2
executing 2.0.1
faiss-cpu 1.8.0
fastai 2.7.16
fastapi 0.110.3
fastcore 1.5.55
fastdownload 0.0.7
fastjsonschema 2.20.0
fastprogress 1.0.3
filelock 3.15.4
flake8 7.0.0
Flask 3.0.3
flatbuffers 24.3.25
fonttools 4.53.1
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.6.0
future 1.0.0
gast 0.5.5
gdown 5.2.0
gitdb 4.0.11
GitPython 3.1.43
gluonts 0.13.7
gmpy2 2.1.5
google-auth 2.32.0
google-auth-oauthlib 1.2.1
google-pasta 0.2.0
graphviz 0.20.3
greenlet 3.0.3
grpcio 1.59.3
gssapi 1.8.3
h11 0.14.0
h2 4.1.0
h5py 3.11.0
hdijupyterutils 0.21.0
hpack 4.0.0
httpcore 1.0.5
httpx 0.27.0
huggingface_hub 0.24.5
hyperframe 6.0.1
idna 3.7
imagecodecs-lite 2019.12.3
imageio 2.34.2
importlib-metadata 6.10.0
importlib_resources 6.4.0
ipykernel 6.29.5
ipython 8.26.0
ipywidgets 8.1.3
isoduration 20.11.0
isort 5.13.2
itsdangerous 2.2.0
jedi 0.19.1
Jinja2 3.1.4
jmespath 1.0.1
joblib 1.4.2
json5 0.9.25
jsonpatch 1.33
jsonpath-ng 1.6.1
jsonpointer 3.0.0
jsonschema 4.17.3
jupyter 1.0.0
jupyter_ai 2.20.0
jupyter_ai_magics 2.20.0
jupyter_client 8.6.2
jupyter_collaboration 1.1.0
jupyter-console 6.6.3
jupyter_core 5.7.2
jupyter-dash 0.4.2
jupyter-events 0.6.3
jupyter-lsp 2.2.5
jupyter_scheduler 2.7.1
jupyter_server 2.10.0
jupyter_server_fileid 0.9.2
jupyter-server-mathjax 0.2.6
jupyter_server_proxy 4.3.0
jupyter_server_terminals 0.5.3
jupyter-ydoc 1.1.1
jupyterlab 4.1.6
jupyterlab_git 0.50.1
jupyterlab-lsp 5.0.3
jupyterlab_pygments 0.3.0
jupyterlab_server 2.24.0
jupyterlab_widgets 3.0.11
keras 2.15.0
kiwisolver 1.4.5
krb5 0.5.1
langchain 0.2.5
langchain-aws 0.1.15
langchain-community 0.2.11
langchain-core 0.2.28
langchain-text-splitters 0.2.2
langcodes 3.4.0
langsmith 0.1.98
language_data 1.2.0
libmambapy 1.5.8
lightgbm 3.3.5
lightning-utilities 0.11.6
llvmlite 0.43.0
locket 1.0.0
lxml 4.9.4
marisa-trie 1.1.0
Markdown 3.6
markdown-it-py 3.0.0
MarkupSafe 2.1.5
marshmallow 3.21.3
matplotlib 3.9.1
matplotlib-inline 0.1.7
mccabe 0.7.0
mdurl 0.1.2
menuinst 2.1.1
mistune 3.0.2
ml-dtypes 0.2.0
mlforecast 0.7.3
mock 5.1.0
model-index 0.1.11
mpmath 1.3.0
msgpack 1.0.8
multidict 6.0.5
multiprocess 0.70.16
munkres 1.1.4
murmurhash 1.0.10
mypy-extensions 1.0.0
nbclient 0.10.0
nbconvert 7.16.4
nbdime 4.0.1
nbformat 5.10.4
nest_asyncio 1.6.0
networkx 3.3
nlpaug 1.1.11
nltk 3.8.1
nose 1.3.7
notebook 7.1.3
notebook_shim 0.2.4
nptyping 2.4.1
numba 0.60.0
numpy 1.26.4
oauthlib 3.2.2
omegaconf 2.2.3
openmim 0.3.7
opt-einsum 3.3.0
ordered-set 4.1.0
orjson 3.10.6
overrides 7.7.0
packaging 24.1
pandas 2.1.4
pandocfilters 1.5.0
papermill 2.6.0
paramiko 3.4.0
parso 0.8.4
partd 1.4.2
pathos 0.3.2
patsy 0.5.6
pexpect 4.9.0
pickleshare 0.7.5
pillow 10.4.0
pip 23.3.2
pkgutil_resolve_name 1.3.10
platformdirs 4.2.2
plotly 5.23.0
pluggy 1.5.0
ply 3.11
pox 0.3.4
ppft 1.7.6.8
preshed 3.0.9
prometheus_client 0.20.0
prompt_toolkit 3.0.47
protobuf 4.24.4
psutil 5.9.8
ptyprocess 0.7.0
pure_eval 0.2.3
pure-sasl 0.6.2
pyarrow 15.0.0
pyarrow-hotfix 0.6
pyasn1 0.6.0
pyasn1_modules 0.4.0
PyAthena 3.8.3
pycodestyle 2.11.1
pycosat 0.6.6
pycparser 2.22
pydantic 1.10.17
pydantic_core 2.18.4
pydocstyle 6.3.0
pyflakes 3.2.0
Pygments 2.18.0
PyHive 0.7.0
PyJWT 2.9.0
pylint 3.2.6
PyNaCl 1.5.0
pyOpenSSL 24.2.1
pyparsing 3.1.2
pyrsistent 0.20.0
PySide6 6.7.2
PySocks 1.7.1
pyspnego 0.11.1
pytesseract 0.3.10
python-dateutil 2.9.0
python-json-logger 2.0.7
python-lsp-jsonrpc 1.1.2
python-lsp-server 1.11.0
python-slugify 8.0.4
pytoolconfig 1.2.5
pytorch-lightning 2.0.9
pytorch-metric-learning 1.7.3
pytz 2023.3
pyu2f 0.1.5
PyWavelets 1.6.0
PyYAML 6.0.1
pyzmq 26.0.3
qtconsole 5.5.2
QtPy 2.4.1
redshift_connector 2.1.2
regex 2024.7.24
requests 2.32.3
requests-kerberos 0.14.0
requests-oauthlib 2.0.0
responses 0.18.0
retrying 1.3.3
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.7.1
rope 1.13.0
rsa 4.9
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.8
s3transfer 0.10.2
safetensors 0.4.4
sagemaker 2.227.0
sagemaker-headless-execution-driver 0.0.13
sagemaker-jupyterlab-emr-extension 0.3.2
sagemaker-jupyterlab-extension 0.3.2
sagemaker-jupyterlab-extension-common 0.1.19
sagemaker-kernel-wrapper 0.0.2
sagemaker-studio-analytics-extension 0.1.2
sagemaker-studio-sparkmagic-lib 0.1.4
sasl 0.3.1
schema 0.7.7
scikit-image 0.19.3
scikit-learn 1.4.2
SciPy 1.11.4
scramp 1.4.4
Send2Trash 1.8.3
sentencepiece 0.1.99
seqeval 1.2.2
setuptools 72.1.0
shellingham 1.5.4
shiboken6 6.7.2
simpervisor 1.0.0
six 1.16.0
smart_open 7.0.4
smdebug-rulesconfig 1.0.1
smmap 5.0.0
sniffio 1.3.1
snowballstemmer 2.2.0
sortedcontainers 2.4.0
soupsieve 2.5
spacy 3.7.5
spacy-legacy 3.0.12
spacy-loggers 1.0.5
sparkmagic 0.21.0
SQLAlchemy 2.0.30
sqlparse 0.5.1
srsly 2.4.8
stack-data 0.6.2
starlette 0.37.2
statsforecast 1.4.0
statsmodels 0.14.2
supervisor 4.2.5
sympy 1.13.0
tabulate 0.9.0
tblib 2.0.0
tenacity 8.5.0
tensorboard 2.15.2
tensorboard-data-server 0.7.0
tensorflow 2.15.0
tensorflow_estimator 2.15.0
termcolor 2.4.0
terminado 0.18.1
text-unidecode 1.3
thinc 8.2.5
threadpoolctl 3.5.0
thrift 0.20.0
thrift-sasl 0.4.3
tifffile 2020.6.3
timm 0.9.16
tinycss2 1.3.0
tokenizers 0.19.1
tomli 2.0.1
tomlkit 0.13.0
toolz 0.12.1
torch 2.0.0.post104
torchmetrics 1.0.3
torchvision 0.15.2a0+ab7b3e6
tornado 6.4.1
tqdm 4.66.5
traitlets 5.14.3
transformers 4.40.2
truststore 0.8.0
typer 0.9.4
types-python-dateutil 2.9.0.20240316
typing_extensions 4.12.2
typing-inspect 0.9.0
typing-utils 0.1.0
typish 1.9.3
tzdata 2024.1
ujson 5.10.0
unicodedata2 15.1.0
uri-template 1.3.0
urllib3 1.26.19
uvicorn 0.30.4
wasabi 1.1.2
wcwidth 0.2.13
weasel 0.4.1
webcolors 24.6.0
webencodings 0.5.1
websocket-client 1.8.0
Werkzeug 3.0.3
whatthepatch 1.0.6
wheel 0.44.0
widgetsnbextension 4.0.11
window_ops 0.0.15
wrapt 1.14.1
xgboost 1.7.6
xxhash 3.4.1
y-py 0.6.2
yapf 0.40.1
yarl 1.9.4
ypy-websocket 0.12.4
zict 3.0.0
zipp 3.19.2
zstandard 0.23.0
両者のdiff
4,5c4,7
< aiofiles 22.1.0
< aiohttp 3.9.1
---
> accelerate 0.21.0
> aiobotocore 2.13.1
> aiohttp 3.9.5
> aioitertools 0.11.0
8,10c10,22
< ansiwrap 0.8.4
< anyio 4.2.0
< archspec 0.2.2
---
> altair 5.3.0
> amazon-q-developer-jupyterlab-ext 3.2.0
> amazon_sagemaker_jupyter_ai_q_developer 1.0.7
> amazon_sagemaker_jupyter_scheduler 3.1.2
> amazon-sagemaker-sql-editor 0.1.10
> amazon-sagemaker-sql-execution 0.1.3
> amazon-sagemaker-sql-magic 0.1.3
> annotated-types 0.7.0
> ansi2html 1.9.2
> ansicolors 1.1.8
> antlr4-python3-runtime 4.9.3
> anyio 4.4.0
> archspec 0.2.3
13a26,27
> asn1crypto 1.5.1
> astroid 3.2.4
15a30
> async-lru 2.0.4
17a33,40
> autogluon 0.8.3
> autogluon.common 0.8.3
> autogluon.core 0.8.3
> autogluon.features 0.8.3
> autogluon.multimodal 0.8.3
> autogluon.tabular 0.8.3
> autogluon.timeseries 0.8.3
> autopep8 2.0.4
19c42,43
< aws-glue-sessions 1.0.4
---
> aws-embedded-metrics 3.2.0
> aws-glue-sessions 1.0.6
21,22c45
< backcall 0.2.0
< bcrypt 4.1.2
---
> bcrypt 4.2.0
24c47
< black 24.1.1
---
> binaryornot 0.4.4
26,29c49,53
< blinker 1.7.0
< boltons 23.1.1
< boto3 1.34.36
< botocore 1.34.36
---
> blinker 1.8.2
> blis 0.7.10
> boltons 24.0.0
> boto3 1.34.131
> botocore 1.34.131
32,33c56,59
< cachetools 5.3.2
< certifi 2024.2.2
---
> cachetools 5.4.0
> catalogue 2.0.10
> catboost 1.2.5
> certifi 2024.7.4
34a61
> chardet 5.2.0
36a64
> cloudpathlib 0.18.1
39c67
< comm 0.2.1
---
> comm 0.2.2
41,43c69,72
< conda-libmamba-solver 24.1.0
< conda-package-handling 2.2.0
< conda_package_streaming 0.9.0
---
> conda-libmamba-solver 24.7.0
> conda-package-handling 2.3.0
> conda_package_streaming 0.10.0
> confection 0.1.4
45,46c74,77
< contourpy 1.1.1
< cryptography 42.0.2
---
> contourpy 1.2.1
> cookiecutter 2.6.0
> croniter 1.4.1
> cryptography 43.0.0
48c79,85
< debugpy 1.8.0
---
> cymem 2.0.8
> cytoolz 0.12.3
> dash 2.17.1
> dask 2024.7.1
> dataclasses-json 0.6.7
> datasets 2.20.0
> debugpy 1.8.5
49a87
> deepmerge 1.1.1
51a90,91
> diskcache 5.6.3
> distributed 2024.7.1
53c93,94
< docker 7.0.0
---
> docker 7.1.0
> docstring-to-markdown 0.15
55c96,97
< exceptiongroup 1.2.0
---
> evaluate 0.4.1
> exceptiongroup 1.2.2
57,60c99,110
< fastjsonschema 2.19.1
< filelock 3.13.1
< flatbuffers 23.5.26
< fonttools 4.48.1
---
> faiss-cpu 1.8.0
> fastai 2.7.16
> fastapi 0.110.3
> fastcore 1.5.55
> fastdownload 0.0.7
> fastjsonschema 2.20.0
> fastprogress 1.0.3
> filelock 3.15.4
> flake8 7.0.0
> Flask 3.0.3
> flatbuffers 24.3.25
> fonttools 4.53.1
63,67c113,122
< future 0.18.3
< gast 0.5.4
< gmpy2 2.1.2
< google-auth 2.27.0
< google-auth-oauthlib 1.0.0
---
> fsspec 2023.6.0
> future 1.0.0
> gast 0.5.5
> gdown 5.2.0
> gitdb 4.0.11
> GitPython 3.1.43
> gluonts 0.13.7
> gmpy2 2.1.5
> google-auth 2.32.0
> google-auth-oauthlib 1.2.1
68a124
> graphviz 0.20.3
70c126
< grpcio 1.54.3
---
> grpcio 1.59.3
72c128,130
< h5py 3.10.0
---
> h11 0.14.0
> h2 4.1.0
> h5py 3.11.0
74c132,139
< idna 3.6
---
> hpack 4.0.0
> httpcore 1.0.5
> httpx 0.27.0
> huggingface_hub 0.24.5
> hyperframe 6.0.1
> idna 3.7
> imagecodecs-lite 2019.12.3
> imageio 2.34.2
76,80c141,144
< importlib-resources 6.1.1
< ipykernel 6.29.1
< ipython 8.12.2
< ipython-genutils 0.2.0
< ipywidgets 7.8.0
---
> importlib_resources 6.4.0
> ipykernel 6.29.5
> ipython 8.26.0
> ipywidgets 8.1.3
81a146,147
> isort 5.13.2
> itsdangerous 2.2.0
83c149
< Jinja2 3.1.3
---
> Jinja2 3.1.4
85,86c151,152
< joblib 1.3.2
< json5 0.9.14
---
> joblib 1.4.2
> json5 0.9.25
88,90c154,156
< jsonpointer 2.4
< jsonschema 4.21.1
< jsonschema-specifications 2023.12.1
---
> jsonpath-ng 1.6.1
> jsonpointer 3.0.0
> jsonschema 4.17.3
92c158,161
< jupyter_client 7.4.9
---
> jupyter_ai 2.20.0
> jupyter_ai_magics 2.20.0
> jupyter_client 8.6.2
> jupyter_collaboration 1.1.0
94,101c163,176
< jupyter_core 5.7.1
< jupyter-events 0.9.0
< jupyter_server 2.12.5
< jupyter_server_fileid 0.9.1
< jupyter_server_terminals 0.5.2
< jupyter_server_ydoc 0.8.0
< jupyter-ydoc 0.2.4
< jupyterlab 3.6.7
---
> jupyter_core 5.7.2
> jupyter-dash 0.4.2
> jupyter-events 0.6.3
> jupyter-lsp 2.2.5
> jupyter_scheduler 2.7.1
> jupyter_server 2.10.0
> jupyter_server_fileid 0.9.2
> jupyter-server-mathjax 0.2.6
> jupyter_server_proxy 4.3.0
> jupyter_server_terminals 0.5.3
> jupyter-ydoc 1.1.1
> jupyterlab 4.1.6
> jupyterlab_git 0.50.1
> jupyterlab-lsp 5.0.3
103,105c178,180
< jupyterlab_server 2.25.2
< jupyterlab-widgets 1.1.7
< keras 2.13.1
---
> jupyterlab_server 2.24.0
> jupyterlab_widgets 3.0.11
> keras 2.15.0
108,109c183,198
< libmambapy 1.5.6
< Markdown 3.5.2
---
> langchain 0.2.5
> langchain-aws 0.1.15
> langchain-community 0.2.11
> langchain-core 0.2.28
> langchain-text-splitters 0.2.2
> langcodes 3.4.0
> langsmith 0.1.98
> language_data 1.2.0
> libmambapy 1.5.8
> lightgbm 3.3.5
> lightning-utilities 0.11.6
> llvmlite 0.43.0
> locket 1.0.0
> lxml 4.9.4
> marisa-trie 1.1.0
> Markdown 3.6
112,113c201,204
< matplotlib 3.7.3
< matplotlib-inline 0.1.6
---
> marshmallow 3.21.3
> matplotlib 3.9.1
> matplotlib-inline 0.1.7
> mccabe 0.7.0
115c206
< menuinst 2.0.2
---
> menuinst 2.1.1
116a208,209
> ml-dtypes 0.2.0
> mlforecast 0.7.3
117a211
> model-index 0.1.11
118a213
> msgpack 1.0.8
121a217
> murmurhash 1.0.10
123,126c219,222
< nbclassic 1.0.0
< nbclient 0.8.0
< nbconvert 7.15.0
< nbformat 5.9.2
---
> nbclient 0.10.0
> nbconvert 7.16.4
> nbdime 4.0.1
> nbformat 5.10.4
128c224,226
< networkx 3.1
---
> networkx 3.3
> nlpaug 1.1.11
> nltk 3.8.1
130,132c228,232
< notebook 6.5.6
< notebook_shim 0.2.3
< numpy 1.24.4
---
> notebook 7.1.3
> notebook_shim 0.2.4
> nptyping 2.4.1
> numba 0.60.0
> numpy 1.26.4
133a234,235
> omegaconf 2.2.3
> openmim 0.3.7
134a237,238
> ordered-set 4.1.0
> orjson 3.10.6
136,137c240,241
< packaging 23.2
< pandas 2.0.3
---
> packaging 24.1
> pandas 2.1.4
139c243
< papermill 2.4.0
---
> papermill 2.6.0
141c245,246
< parso 0.8.3
---
> parso 0.8.4
> partd 1.4.2
143c248
< pathspec 0.12.1
---
> patsy 0.5.6
146c251
< pillow 10.2.0
---
> pillow 10.4.0
149,151c254,256
< platformdirs 4.2.0
< plotly 5.18.0
< pluggy 1.4.0
---
> platformdirs 4.2.2
> plotly 5.23.0
> pluggy 1.5.0
153d257
< pooch 1.8.0
156,158c260,263
< prometheus-client 0.19.0
< prompt-toolkit 3.0.42
< protobuf 4.21.12
---
> preshed 3.0.9
> prometheus_client 0.20.0
> prompt_toolkit 3.0.47
> protobuf 4.24.4
161c266
< pure-eval 0.2.2
---
> pure_eval 0.2.3
163,164c268,273
< pyasn1 0.5.1
< pyasn1-modules 0.3.0
---
> pyarrow 15.0.0
> pyarrow-hotfix 0.6
> pyasn1 0.6.0
> pyasn1_modules 0.4.0
> PyAthena 3.8.3
> pycodestyle 2.11.1
166,167c275,280
< pycparser 2.21
< Pygments 2.17.2
---
> pycparser 2.22
> pydantic 1.10.17
> pydantic_core 2.18.4
> pydocstyle 6.3.0
> pyflakes 3.2.0
> Pygments 2.18.0
169c282,283
< PyJWT 2.8.0
---
> PyJWT 2.9.0
> pylint 3.2.6
171,174c285,288
< pyOpenSSL 24.0.0
< pyparsing 3.1.1
< PyQt5 5.15.9
< PyQt5-sip 12.12.2
---
> pyOpenSSL 24.2.1
> pyparsing 3.1.2
> pyrsistent 0.20.0
> PySide6 6.7.2
176,177c290,292
< pyspnego 0.9.1
< python-dateutil 2.8.2
---
> pyspnego 0.11.1
> pytesseract 0.3.10
> python-dateutil 2.9.0
179c294,300
< pytz 2024.1
---
> python-lsp-jsonrpc 1.1.2
> python-lsp-server 1.11.0
> python-slugify 8.0.4
> pytoolconfig 1.2.5
> pytorch-lightning 2.0.9
> pytorch-metric-learning 1.7.3
> pytz 2023.3
180a302
> PyWavelets 1.6.0
182,183c304,305
< pyzmq 24.0.1
< qtconsole 5.5.1
---
> pyzmq 26.0.3
> qtconsole 5.5.2
185,186c307,309
< referencing 0.33.0
< requests 2.31.0
---
> redshift_connector 2.1.2
> regex 2024.7.24
> requests 2.32.3
188c311,313
< requests-oauthlib 1.3.1
---
> requests-oauthlib 2.0.0
> responses 0.18.0
> retrying 1.3.3
191,192c316,317
< rich 13.7.0
< rpds-py 0.17.1
---
> rich 13.7.1
> rope 1.13.0
196,198c321,327
< s3transfer 0.10.0
< sagemaker 2.206.0
< sagemaker-headless-execution-driver 0.0.12
---
> s3transfer 0.10.2
> safetensors 0.4.4
> sagemaker 2.227.0
> sagemaker-headless-execution-driver 0.0.13
> sagemaker-jupyterlab-emr-extension 0.3.2
> sagemaker-jupyterlab-extension 0.3.2
> sagemaker-jupyterlab-extension-common 0.1.19
200c329
< sagemaker-studio-analytics-extension 0.0.21
---
> sagemaker-studio-analytics-extension 0.1.2
203,208c332,343
< schema 0.7.5
< scikit-learn 1.3.2
< scipy 1.10.1
< Send2Trash 1.8.2
< setuptools 69.0.3
< sip 6.7.12
---
> schema 0.7.7
> scikit-image 0.19.3
> scikit-learn 1.4.2
> SciPy 1.11.4
> scramp 1.4.4
> Send2Trash 1.8.3
> sentencepiece 0.1.99
> seqeval 1.2.2
> setuptools 72.1.0
> shellingham 1.5.4
> shiboken6 6.7.2
> simpervisor 1.0.0
209a345
> smart_open 7.0.4
211c347,350
< sniffio 1.3.0
---
> smmap 5.0.0
> sniffio 1.3.1
> snowballstemmer 2.2.0
> sortedcontainers 2.4.0
212a352,354
> spacy 3.7.5
> spacy-legacy 3.0.12
> spacy-loggers 1.0.5
214c356,358
< SQLAlchemy 2.0.24
---
> SQLAlchemy 2.0.30
> sqlparse 0.5.1
> srsly 2.4.8
216c360,364
< sympy 1.12
---
> starlette 0.37.2
> statsforecast 1.4.0
> statsmodels 0.14.2
> supervisor 4.2.5
> sympy 1.13.0
219,220c367,368
< tenacity 8.2.3
< tensorboard 2.13.0
---
> tenacity 8.5.0
> tensorboard 2.15.2
222,223c370,371
< tensorflow 2.13.1
< tensorflow-estimator 2.13.0
---
> tensorflow 2.15.0
> tensorflow_estimator 2.15.0
225,228c373,377
< terminado 0.18.0
< textwrap3 0.9.2
< threadpoolctl 3.2.0
< thrift 0.19.0
---
> terminado 0.18.1
> text-unidecode 1.3
> thinc 8.2.5
> threadpoolctl 3.5.0
> thrift 0.20.0
230,231c379,382
< tinycss2 1.2.1
< toml 0.10.2
---
> tifffile 2020.6.3
> timm 0.9.16
> tinycss2 1.3.0
> tokenizers 0.19.1
233c384,387
< torch 2.0.0.post101
---
> tomlkit 0.13.0
> toolz 0.12.1
> torch 2.0.0.post104
> torchmetrics 1.0.3
235,239c389,397
< tornado 6.3.3
< tqdm 4.66.1
< traitlets 5.14.1
< types-python-dateutil 2.8.19.20240106
< typing_extensions 4.5.0
---
> tornado 6.4.1
> tqdm 4.66.5
> traitlets 5.14.3
> transformers 4.40.2
> truststore 0.8.0
> typer 0.9.4
> types-python-dateutil 2.9.0.20240316
> typing_extensions 4.12.2
> typing-inspect 0.9.0
241c399,401
< tzdata 2023.4
---
> typish 1.9.3
> tzdata 2024.1
> ujson 5.10.0
244c404,406
< urllib3 1.26.18
---
> urllib3 1.26.19
> uvicorn 0.30.4
> wasabi 1.1.2
246c408,409
< webcolors 1.13
---
> weasel 0.4.1
> webcolors 24.6.0
248,252c411,417
< websocket-client 1.7.0
< Werkzeug 3.0.1
< wheel 0.42.0
< widgetsnbextension 3.6.6
< wrapt 1.16.0
---
> websocket-client 1.8.0
> Werkzeug 3.0.3
> whatthepatch 1.0.6
> wheel 0.44.0
> widgetsnbextension 4.0.11
> window_ops 0.0.15
> wrapt 1.14.1
254c419,421
< y-py 0.5.9
---
> xxhash 3.4.1
> y-py 0.6.2
> yapf 0.40.1
256,258c423,426
< ypy-websocket 0.8.2
< zipp 3.17.0
< zstandard 0.22.0
\ No newline at end of file
---
> ypy-websocket 0.12.4
> zict 3.0.0
> zipp 3.19.2
> zstandard 0.23.0
\ No newline at end of file
バージョンが異なるのは想定通りだが、1-cpuのイメージの方が基本的には新しい。
今回の差の調査で気にしていたのがS3回りだが、0-cpuと1-cpuどちらもs3fsは含まれておらず、1-cpuにs3transfer 0.10.2が含まれていた。pandasでread_csv('s3://xxxxx')やread_parquet('s3://xxxx')などのS3URI指定でのデータ読み込み時の挙動の差異は、このs3transferあたりにある可能性がある。
考察
1-cpuのタグのイメージの方が7.78GBと大きいことと、更新日が新しいこと(6 weeks ago vs 7 months ago)から、バージョンが新しい1-cpuタグの方を基本的には使うべし。(2024/9/19時点）
ちなみに、pandasでのS3URI指定でのreadではエラーが出ないのも1-cpuタグの方のイメージ(2024/9/19時点）。