はじめに

AI。流行ってますねー。
イラスト生成に、chatGPT、なんか本当にシンギュラリティとか来ちゃいそうですが。
で、画像生成AIの「Stable Diffusion」ですがWEBサイト経由やアプリなんかで利用できますが、
動かせる環境とかマシンがあればチャレジしたいじゃないですかー。

というわけで、やってみた。

まずベースですが、この記事に記載されている内容を補完するものです。

忙しい人向けに要点をまとめ

Stable Diffusionを動かすのに必要な要件は
Python 3.10以上です。

(1)Pythonをインストールしたら、Stable Diffusionのフォークをクローンします。

git clone -b apple-silicon-mps-support https://github.com/bfirsh/stable-diffusion.git
cd stable-diffusion
mkdir -p models/ldm/stable-diffusion-v1/

(2)依存関係を解消するためにVirtualenvをインストールして実行

pip install virtualenv
python3 -m virtualenv venv
source venv/bin/activate

(3)依存関係のインストール

pip install -r requirements.txt

(4)sd-v1-4.ckptのダウンロードと配置
下記リンクより「sd-v1-4.ckpt」を取得して(1)で作成したディレクトリに「model.ckpt」というファイル名で配置します。

以下のスクリーンショットのようになっていればOKです。

(5)その他必要なパッケージのインストール
gigazineの解説記事では「model.ckpt」の配置の後で、すぐにtxt2img.py を実行するように記載されていますが
当然、このままでは動きません。
その為、必要なパッケージ群をインストールしていきます。
なおログでえらいことになりますので、必要なコマンドの羅列のみとします。

pip install diffusers
pip install opencv-python 
pip install torch torchvision torchaudio 
pip install omegaconf
pip install imwatermark
pip install invisible-watermark 
pip install einops
pip install pytorch_lightning
pip install transformers 
pip install taming-transformers-rom1504
pip install clip
pip install kornia

(6)実行してみる。
gigazineの記事にあったサンプルのおまじないを唱えてみます。
3行に見えてますが、バックスラッシュで折り返しているので実際のコマンドはワンライナーです。

stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms

「宇宙空間に惑星のように浮いている美味しそうな赤いリンゴ」と指定していますね。

実行中の様子です。
（少し時間がかかります）

最終的にこのようなメッセージが出力されたら成功です。

出力されたファイルがこちら「grid-0000.png」

出力画像

アップルマークが自己主張気味？

他の指定も試してみましょう。

stable-diffusion % python scripts/txt2img.py \
--prompt "grandma overtakes supercar" \                                                            
--n_samples 1 --n_iter 1 --plms

スーパーカーを追い抜くおばあちゃん

はたして？

出力画像

スーパーカーは出たけど、追い抜くおばあちゃんまでは無理か？

というわけで無事、Stable Diffusionを動かす事に成功しました。
良かったらお試し下さい。

最後にトライアンドエラーしたログを掲載して終わりたいと思います。
それでは、また〜。

おまけ（トライエラーの様子です）

~ % pip install diffusers
Collecting diffusers
  Using cached diffusers-0.13.0-py3-none-any.whl (716 kB)
Collecting regex!=2019.12.17
  Using cached regex-2022.10.31-cp310-cp310-macosx_10_9_x86_64.whl (293 kB)
Requirement already satisfied: filelock in ./.pyenv/versions/3.10.7/lib/python3.10/site-packages (from diffusers) (3.9.0)
Collecting Pillow
  Using cached Pillow-9.4.0-2-cp310-cp310-macosx_10_10_x86_64.whl (3.3 MB)
Collecting importlib-metadata
  Using cached importlib_metadata-6.0.0-py3-none-any.whl (21 kB)
Collecting huggingface-hub>=0.10.0
  Using cached huggingface_hub-0.12.1-py3-none-any.whl (190 kB)
Collecting requests
  Using cached requests-2.28.2-py3-none-any.whl (62 kB)
Requirement already satisfied: numpy in ./.pyenv/versions/3.10.7/lib/python3.10/site-packages (from diffusers) (1.23.5)
Collecting packaging>=20.9
  Using cached packaging-23.0-py3-none-any.whl (42 kB)
Collecting typing-extensions>=3.7.4.3
  Using cached typing_extensions-4.5.0-py3-none-any.whl (27 kB)
Collecting tqdm>=4.42.1
  Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting pyyaml>=5.1
  Using cached PyYAML-6.0-cp310-cp310-macosx_10_9_x86_64.whl (197 kB)
Collecting zipp>=0.5
  Using cached zipp-3.14.0-py3-none-any.whl (6.7 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 kB 5.7 MB/s eta 0:00:00
Collecting charset-normalizer<4,>=2
  Using cached charset_normalizer-3.0.1-cp310-cp310-macosx_10_9_x86_64.whl (124 kB)
Installing collected packages: charset-normalizer, zipp, urllib3, typing-extensions, tqdm, regex, pyyaml, Pillow, packaging, idna, certifi, requests, importlib-metadata, huggingface-hub, diffusers
Successfully installed Pillow-9.4.0 certifi-2022.12.7 charset-normalizer-3.0.1 diffusers-0.13.0 huggingface-hub-0.12.1 idna-3.4 importlib-metadata-6.0.0 packaging-23.0 pyyaml-6.0 regex-2022.10.31 requests-2.28.2 tqdm-4.64.1 typing-extensions-4.5.0 urllib3-1.26.14 zipp-3.14.0




stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 2, in <module>
    import cv2
ModuleNotFoundError: No module named 'cv2'



stable-diffusion % pip install opencv-python 
Collecting opencv-python
  Downloading opencv_python-4.7.0.68-cp37-abi3-macosx_10_13_x86_64.whl (51.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.7/51.7 MB 26.3 MB/s eta 0:00:00
Requirement already satisfied: numpy>=1.21.2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from opencv-python) (1.23.5)
Installing collected packages: opencv-python
Successfully installed opencv-python-4.7.0.68
stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 3, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'



stable-diffusion % pip install torch torchvision torchaudio 
Collecting torch
  Downloading torch-1.13.1-cp310-none-macosx_10_9_x86_64.whl (135.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 135.3/135.3 MB 18.1 MB/s eta 0:00:00
Collecting torchvision
  Downloading torchvision-0.14.1-cp310-cp310-macosx_10_9_x86_64.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 31.2 MB/s eta 0:00:00
Collecting torchaudio
  Downloading torchaudio-0.13.1-cp310-cp310-macosx_10_9_x86_64.whl (3.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.3/3.3 MB 40.7 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torch) (4.5.0)
Requirement already satisfied: requests in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision) (2.28.2)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision) (9.4.0)
Requirement already satisfied: numpy in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision) (1.23.5)
Requirement already satisfied: certifi>=2017.4.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision) (2022.12.7)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision) (1.26.14)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision) (3.0.1)
Requirement already satisfied: idna<4,>=2.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision) (3.4)
Installing collected packages: torch, torchvision, torchaudio
Successfully installed torch-1.13.1 torchaudio-0.13.1 torchvision-0.14.1


stable-diffusion % python scripts/txt2img.py \             
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 5, in <module>
    from omegaconf import OmegaConf
ModuleNotFoundError: No module named 'omegaconf'


stable-diffusion % pip install omegaconf
Collecting omegaconf
  Downloading omegaconf-2.3.0-py3-none-any.whl (79 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.5/79.5 kB 4.0 MB/s eta 0:00:00
Requirement already satisfied: PyYAML>=5.1.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from omegaconf) (6.0)
Collecting antlr4-python3-runtime==4.9.*
  Downloading antlr4-python3-runtime-4.9.3.tar.gz (117 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.0/117.0 kB 14.0 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Installing collected packages: antlr4-python3-runtime, omegaconf
  DEPRECATION: antlr4-python3-runtime is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for antlr4-python3-runtime ... done
Successfully installed antlr4-python3-runtime-4.9.3 omegaconf-2.3.0


stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 8, in <module>
    from imwatermark import WatermarkEncoder
ModuleNotFoundError: No module named 'imwatermark'


stable-diffusion % pip install imwatermark    
Collecting imwatermark
  Downloading imWatermark-0.0.2-py3-none-any.whl (2.8 kB)
Requirement already satisfied: numpy in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from imwatermark) (1.23.5)
Collecting imhist
  Downloading imhist-0.0.4-py3-none-any.whl (2.8 kB)
Installing collected packages: imhist, imwatermark
Successfully installed imhist-0.0.4 imwatermark-0.0.2


stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 8, in <module>
    from imwatermark import WatermarkEncoder
ModuleNotFoundError: No module named 'imwatermark'


stable-diffusion % pip install imwatermark           
Requirement already satisfied: imwatermark in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (0.0.2)
Requirement already satisfied: numpy in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from imwatermark) (1.23.5)
Requirement already satisfied: imhist in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from imwatermark) (0.0.4)


stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 8, in <module>
    from imwatermark import WatermarkEncoder
ModuleNotFoundError: No module named 'imwatermark'


stable-diffusion % pip install invisible-watermark                    
Collecting invisible-watermark
  Using cached invisible_watermark-0.1.5-py3-none-any.whl (1.6 MB)
Requirement already satisfied: Pillow>=6.0.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from invisible-watermark) (9.4.0)
Collecting PyWavelets>=1.1.1
  Using cached PyWavelets-1.4.1-cp310-cp310-macosx_10_13_x86_64.whl (4.4 MB)
Collecting onnxruntime
  Using cached onnxruntime-1.14.0-cp310-cp310-macosx_11_0_x86_64.whl (6.6 MB)
Requirement already satisfied: opencv-python>=4.1.0.25 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from invisible-watermark) (4.7.0.68)
Collecting onnx
  Using cached onnx-1.13.0-cp310-cp310-macosx_10_12_x86_64.whl (12.7 MB)
Requirement already satisfied: numpy>=1.17.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from invisible-watermark) (1.23.5)
Requirement already satisfied: torch in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from invisible-watermark) (1.13.1)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from onnx->invisible-watermark) (4.5.0)
Collecting protobuf<4,>=3.20.2
  Using cached protobuf-3.20.3-py2.py3-none-any.whl (162 kB)
Requirement already satisfied: packaging in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from onnxruntime->invisible-watermark) (23.0)
Collecting coloredlogs
  Using cached coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)
Collecting flatbuffers
  Using cached flatbuffers-23.1.21-py2.py3-none-any.whl (26 kB)
Collecting sympy
  Downloading sympy-1.11.1-py3-none-any.whl (6.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 36.2 MB/s eta 0:00:00
Collecting humanfriendly>=9.1
  Using cached humanfriendly-10.0-py2.py3-none-any.whl (86 kB)
Collecting mpmath>=0.19
  Downloading mpmath-1.2.1-py3-none-any.whl (532 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 532.6/532.6 kB 37.8 MB/s eta 0:00:00
Installing collected packages: mpmath, flatbuffers, sympy, PyWavelets, protobuf, humanfriendly, onnx, coloredlogs, onnxruntime, invisible-watermark
Successfully installed PyWavelets-1.4.1 coloredlogs-15.0.1 flatbuffers-23.1.21 humanfriendly-10.0 invisible-watermark-0.1.5 mpmath-1.2.1 onnx-1.13.0 onnxruntime-1.14.0 protobuf-3.20.3 sympy-1.11.1


stable-diffusion % python scripts/txt2img.py \                        
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 10, in <module>
    from einops import rearrange
ModuleNotFoundError: No module named 'einops'


stable-diffusion % pip install einops             
Collecting einops
  Downloading einops-0.6.0-py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 kB 2.9 MB/s eta 0:00:00
Installing collected packages: einops
Successfully installed einops-0.6.0


stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 13, in <module>
    from pytorch_lightning import seed_everything
ModuleNotFoundError: No module named 'pytorch_lightning'


stable-diffusion % pip install pytorch_lightning
Collecting pytorch_lightning
  Downloading pytorch_lightning-1.9.2-py3-none-any.whl (826 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 826.2/826.2 kB 15.3 MB/s eta 0:00:00
Requirement already satisfied: tqdm>=4.57.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch_lightning) (4.64.1)
Collecting torchmetrics>=0.7.0
  Downloading torchmetrics-0.11.1-py3-none-any.whl (517 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 517.2/517.2 kB 23.6 MB/s eta 0:00:00
Requirement already satisfied: torch>=1.10.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch_lightning) (1.13.1)
Collecting fsspec[http]>2021.06.0
  Using cached fsspec-2023.1.0-py3-none-any.whl (143 kB)
Requirement already satisfied: numpy>=1.17.2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch_lightning) (1.23.5)
Collecting lightning-utilities>=0.6.0.post0
  Downloading lightning_utilities-0.6.0.post0-py3-none-any.whl (18 kB)
Requirement already satisfied: typing-extensions>=4.0.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch_lightning) (4.5.0)
Requirement already satisfied: PyYAML>=5.4 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch_lightning) (6.0)
Requirement already satisfied: packaging>=17.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch_lightning) (23.0)
Collecting aiohttp!=4.0.0a0,!=4.0.0a1
  Using cached aiohttp-3.8.4-cp310-cp310-macosx_10_9_x86_64.whl (358 kB)
Requirement already satisfied: requests in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from fsspec[http]>2021.06.0->pytorch_lightning) (2.28.2)
Collecting aiosignal>=1.1.2
  Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting async-timeout<5.0,>=4.0.0a3
  Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch_lightning) (3.0.1)
Collecting frozenlist>=1.1.1
  Using cached frozenlist-1.3.3-cp310-cp310-macosx_10_9_x86_64.whl (35 kB)
Collecting attrs>=17.3.0
  Using cached attrs-22.2.0-py3-none-any.whl (60 kB)
Collecting multidict<7.0,>=4.5
  Using cached multidict-6.0.4-cp310-cp310-macosx_10_9_x86_64.whl (29 kB)
Collecting yarl<2.0,>=1.0
  Using cached yarl-1.8.2-cp310-cp310-macosx_10_9_x86_64.whl (61 kB)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->fsspec[http]>2021.06.0->pytorch_lightning) (1.26.14)
Requirement already satisfied: idna<4,>=2.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->fsspec[http]>2021.06.0->pytorch_lightning) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->fsspec[http]>2021.06.0->pytorch_lightning) (2022.12.7)
Installing collected packages: multidict, lightning-utilities, fsspec, frozenlist, attrs, async-timeout, yarl, torchmetrics, aiosignal, aiohttp, pytorch_lightning
Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 attrs-22.2.0 frozenlist-1.3.3 fsspec-2023.1.0 lightning-utilities-0.6.0.post0 multidict-6.0.4 pytorch_lightning-1.9.2 torchmetrics-0.11.1 yarl-1.8.2



stable-diffusion % python scripts/txt2img.py \  
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 32, in <module>
    from diffusers.pipelines.stable_diffusion.safety_checker import StableDiffusionSafetyChecker
  File "/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/safety_checker.py", line 18, in <module>
    from transformers import CLIPConfig, CLIPVisionModel, PreTrainedModel
ModuleNotFoundError: No module named 'transformers'



stable-diffusion % pip install transformers     
Collecting transformers
  Downloading transformers-4.26.1-py3-none-any.whl (6.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 32.0 MB/s eta 0:00:00
Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (0.12.1)
Requirement already satisfied: tqdm>=4.27 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (4.64.1)
Requirement already satisfied: filelock in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (3.9.0)
Requirement already satisfied: numpy>=1.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (1.23.5)
Requirement already satisfied: requests in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (2.28.2)
Requirement already satisfied: packaging>=20.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (23.0)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1
  Downloading tokenizers-0.13.2-cp310-cp310-macosx_10_11_x86_64.whl (3.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 42.7 MB/s eta 0:00:00
Requirement already satisfied: pyyaml>=5.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (6.0)
Requirement already satisfied: regex!=2019.12.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from transformers) (2022.10.31)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.11.0->transformers) (4.5.0)
Requirement already satisfied: certifi>=2017.4.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->transformers) (2022.12.7)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->transformers) (3.0.1)
Requirement already satisfied: idna<4,>=2.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->transformers) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->transformers) (1.26.14)
Installing collected packages: tokenizers, transformers
Successfully installed tokenizers-0.13.2 transformers-4.26.1


stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
Downloading (…)rocessor_config.json: 100%|███████████████████████████████████████████████████████████| 342/342 [00:00<00:00, 89.7kB/s]
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Downloading (…)lve/main/config.json: 100%|███████████████████████████████████████████████████████| 4.55k/4.55k [00:00<00:00, 1.41MB/s]
Downloading (…)"pytorch_model.bin";: 100%|███████████████████████████████████████████████████████| 1.22G/1.22G [00:19<00:00, 62.3MB/s]
Global seed set to 42
Loading model from models/ldm/stable-diffusion-v1/model.ckpt
Global Step: 470000
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 359, in <module>
    main()
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 251, in main
    model = load_model_from_config(config, f"{opt.ckpt}")
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 65, in load_model_from_config
    model = instantiate_from_config(config.model)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 93, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
  File "/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/diffusion/ddpm.py", line 24, in <module>
    from ldm.models.autoencoder import VQModelInterface, IdentityFirstStage, AutoencoderKL
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/autoencoder.py", line 6, in <module>
    from taming.modules.vqvae.quantize import VectorQuantizer2 as VectorQuantizer
ModuleNotFoundError: No module named 'taming'


stable-diffusion % pip install taming-transformers
Collecting taming-transformers
  Downloading taming_transformers-0.0.1-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.6/45.6 kB 3.1 MB/s eta 0:00:00
Requirement already satisfied: torchvision in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers) (0.14.1)
Requirement already satisfied: torch in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers) (1.13.1)
Requirement already satisfied: pytorch-lightning>=1.0.8 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers) (1.9.2)
Requirement already satisfied: omegaconf>=2.0.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers) (2.3.0)
Requirement already satisfied: tqdm in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers) (4.64.1)
Requirement already satisfied: numpy in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers) (1.23.5)
Requirement already satisfied: PyYAML>=5.1.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from omegaconf>=2.0.0->taming-transformers) (6.0)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from omegaconf>=2.0.0->taming-transformers) (4.9.3)
Requirement already satisfied: torchmetrics>=0.7.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers) (0.11.1)
Requirement already satisfied: typing-extensions>=4.0.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers) (4.5.0)
Requirement already satisfied: lightning-utilities>=0.6.0.post0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers) (0.6.0.post0)
Requirement already satisfied: packaging>=17.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers) (23.0)
Requirement already satisfied: fsspec[http]>2021.06.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers) (2023.1.0)
Requirement already satisfied: requests in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision->taming-transformers) (2.28.2)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision->taming-transformers) (9.4.0)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (3.8.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers) (1.26.14)
Requirement already satisfied: idna<4,>=2.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers) (3.0.1)
Requirement already satisfied: certifi>=2017.4.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers) (2022.12.7)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (4.0.2)
Requirement already satisfied: multidict<7.0,>=4.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (1.8.2)
Requirement already satisfied: frozenlist>=1.1.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (1.3.3)
Requirement already satisfied: attrs>=17.3.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (22.2.0)
Requirement already satisfied: aiosignal>=1.1.2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers) (1.3.1)
Installing collected packages: taming-transformers
Successfully installed taming-transformers-0.0.1
stable-diffusion % python scripts/txt2img.py \    
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Global seed set to 42
Loading model from models/ldm/stable-diffusion-v1/model.ckpt
Global Step: 470000
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 359, in <module>
    main()
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 251, in main
    model = load_model_from_config(config, f"{opt.ckpt}")
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 65, in load_model_from_config
    model = instantiate_from_config(config.model)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 93, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
  File "/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/diffusion/ddpm.py", line 24, in <module>
    from ldm.models.autoencoder import VQModelInterface, IdentityFirstStage, AutoencoderKL
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/autoencoder.py", line 6, in <module>
    from taming.modules.vqvae.quantize import VectorQuantizer2 as VectorQuantizer
ImportError: cannot import name 'VectorQuantizer2' from 'taming.modules.vqvae.quantize' (/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/taming/modules/vqvae/quantize.py)



stable-diffusion % pip install taming-transformers-rom1504 
Collecting taming-transformers-rom1504
  Downloading taming_transformers_rom1504-0.0.6-py3-none-any.whl (51 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.5/51.5 kB 3.5 MB/s eta 0:00:00
Requirement already satisfied: tqdm in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers-rom1504) (4.64.1)
Requirement already satisfied: numpy in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers-rom1504) (1.23.5)
Requirement already satisfied: torchvision in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers-rom1504) (0.14.1)
Requirement already satisfied: pytorch-lightning>=1.0.8 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers-rom1504) (1.9.2)
Requirement already satisfied: torch in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers-rom1504) (1.13.1)
Requirement already satisfied: omegaconf>=2.0.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from taming-transformers-rom1504) (2.3.0)
Requirement already satisfied: PyYAML>=5.1.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from omegaconf>=2.0.0->taming-transformers-rom1504) (6.0)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from omegaconf>=2.0.0->taming-transformers-rom1504) (4.9.3)
Requirement already satisfied: lightning-utilities>=0.6.0.post0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers-rom1504) (0.6.0.post0)
Requirement already satisfied: torchmetrics>=0.7.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers-rom1504) (0.11.1)
Requirement already satisfied: typing-extensions>=4.0.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers-rom1504) (4.5.0)
Requirement already satisfied: fsspec[http]>2021.06.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers-rom1504) (2023.1.0)
Requirement already satisfied: packaging>=17.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from pytorch-lightning>=1.0.8->taming-transformers-rom1504) (23.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision->taming-transformers-rom1504) (9.4.0)
Requirement already satisfied: requests in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torchvision->taming-transformers-rom1504) (2.28.2)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (3.8.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers-rom1504) (1.26.14)
Requirement already satisfied: idna<4,>=2.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers-rom1504) (3.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers-rom1504) (3.0.1)
Requirement already satisfied: certifi>=2017.4.17 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from requests->torchvision->taming-transformers-rom1504) (2022.12.7)
Requirement already satisfied: aiosignal>=1.1.2 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (1.3.1)
Requirement already satisfied: frozenlist>=1.1.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (1.3.3)
Requirement already satisfied: yarl<2.0,>=1.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (1.8.2)
Requirement already satisfied: attrs>=17.3.0 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (22.2.0)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (4.0.2)
Requirement already satisfied: multidict<7.0,>=4.5 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]>2021.06.0->pytorch-lightning>=1.0.8->taming-transformers-rom1504) (6.0.4)
Installing collected packages: taming-transformers-rom1504
Successfully installed taming-transformers-rom1504-0.0.6



stable-diffusion % python scripts/txt2img.py \             
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Global seed set to 42
Loading model from models/ldm/stable-diffusion-v1/model.ckpt
Global Step: 470000
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/pytorch_lightning/utilities/distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 359, in <module>
    main()
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 251, in main
    model = load_model_from_config(config, f"{opt.ckpt}")
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 65, in load_model_from_config
    model = instantiate_from_config(config.model)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/diffusion/ddpm.py", line 461, in __init__
    self.instantiate_cond_stage(cond_stage_config)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/diffusion/ddpm.py", line 519, in instantiate_cond_stage
    model = instantiate_from_config(config)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 93, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
  File "/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/modules/encoders/modules.py", line 4, in <module>
    import clip
ModuleNotFoundError: No module named 'clip'



stable-diffusion % pip install clip                        
Collecting clip
  Downloading clip-0.2.0.tar.gz (5.5 kB)
  Preparing metadata (setup.py) ... done
Installing collected packages: clip
  DEPRECATION: clip is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for clip ... done
Successfully installed clip-0.2.0



stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Global seed set to 42
Loading model from models/ldm/stable-diffusion-v1/model.ckpt
Global Step: 470000
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/pytorch_lightning/utilities/distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Traceback (most recent call last):
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 359, in <module>
    main()
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 251, in main
    model = load_model_from_config(config, f"{opt.ckpt}")
  File "/Users/gatuwo/SD/stable-diffusion/scripts/txt2img.py", line 65, in load_model_from_config
    model = instantiate_from_config(config.model)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/diffusion/ddpm.py", line 461, in __init__
    self.instantiate_cond_stage(cond_stage_config)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/models/diffusion/ddpm.py", line 519, in instantiate_cond_stage
    model = instantiate_from_config(config)
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 85, in instantiate_from_config
    return get_obj_from_str(config["target"])(**config.get("params", dict()))
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/util.py", line 93, in get_obj_from_str
    return getattr(importlib.import_module(module, package=None), cls)
  File "/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/gatuwo/SD/stable-diffusion/scripts/../ldm/modules/encoders/modules.py", line 7, in <module>
    import kornia
ModuleNotFoundError: No module named 'kornia'



stable-diffusion % pip install kornia         
Collecting kornia
  Downloading kornia-0.6.10-py2.py3-none-any.whl (612 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 612.0/612.0 kB 10.2 MB/s eta 0:00:00
Requirement already satisfied: torch>=1.9.1 in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from kornia) (1.13.1)
Requirement already satisfied: packaging in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from kornia) (23.0)
Requirement already satisfied: typing-extensions in /Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages (from torch>=1.9.1->kornia) (4.5.0)
Installing collected packages: kornia
Successfully installed kornia-0.6.10



stable-diffusion % python scripts/txt2img.py \
--prompt "a red juicy apple floating in outer space, like a planet" \
--n_samples 1 --n_iter 1 --plms
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Global seed set to 42
Loading model from models/ldm/stable-diffusion-v1/model.ckpt
Global Step: 470000
/Users/gatuwo/.pyenv/versions/3.10.7/lib/python3.10/site-packages/pytorch_lightning/utilities/distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Downloading (…)olve/main/vocab.json: 100%|█████████████████████████████████████████████████████████| 961k/961k [00:00<00:00, 1.02MB/s]
Downloading (…)olve/main/merges.txt: 100%|██████████████████████████████████████████████████████████| 525k/525k [00:00<00:00, 593kB/s]
Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████████████| 389/389 [00:00<00:00, 128kB/s]
Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████████████████████| 905/905 [00:00<00:00, 221kB/s]
Downloading (…)lve/main/config.json: 100%|███████████████████████████████████████████████████████| 4.52k/4.52k [00:00<00:00, 1.38MB/s]
Downloading (…)"pytorch_model.bin";: 100%|███████████████████████████████████████████████████████| 1.71G/1.71G [00:26<00:00, 64.2MB/s]
Some weights of the model checkpoint at openai/clip-vit-large-patch14 were not used when initializing CLIPTextModel: ['vision_model.encoder.layers.22.self_attn.q_proj.bias', 'vision_model.encoder.layers.12.layer_norm1.bias', 'vision_model.encoder.layers.21.self_attn.out_proj.weight', 'vision_model.encoder.layers.4.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.self_attn.q_proj.bias', 'vision_model.encoder.layers.11.mlp.fc1.bias', 'vision_model.encoder.layers.13.layer_norm1.weight', 'vision_model.encoder.layers.9.layer_norm2.weight', 'vision_model.encoder.layers.23.self_attn.v_proj.bias', 'vision_model.encoder.layers.12.self_attn.out_proj.weight', 'vision_model.encoder.layers.22.layer_norm1.weight', 'vision_model.encoder.layers.1.self_attn.v_proj.weight', 'vision_model.encoder.layers.6.layer_norm1.weight', 'vision_model.encoder.layers.18.self_attn.k_proj.weight', 'vision_model.encoder.layers.6.self_attn.out_proj.bias', 'vision_model.encoder.layers.1.self_attn.v_proj.bias', 'vision_model.encoder.layers.19.layer_norm2.weight', 'vision_model.encoder.layers.6.self_attn.k_proj.bias', 'vision_model.encoder.layers.9.layer_norm2.bias', 'vision_model.encoder.layers.10.mlp.fc2.bias', 'vision_model.encoder.layers.19.layer_norm1.bias', 'vision_model.encoder.layers.4.layer_norm2.weight', 'vision_model.encoder.layers.16.mlp.fc1.weight', 'vision_model.encoder.layers.1.self_attn.out_proj.weight', 'vision_model.encoder.layers.11.layer_norm2.weight', 'vision_model.encoder.layers.0.self_attn.v_proj.weight', 'vision_model.encoder.layers.8.layer_norm2.weight', 'vision_model.encoder.layers.7.self_attn.v_proj.bias', 'vision_model.encoder.layers.2.mlp.fc1.weight', 'vision_model.encoder.layers.23.mlp.fc2.bias', 'vision_model.encoder.layers.15.mlp.fc1.weight', 'vision_model.encoder.layers.22.self_attn.k_proj.bias', 'vision_model.encoder.layers.0.layer_norm1.bias', 'vision_model.encoder.layers.16.self_attn.out_proj.weight', 'vision_model.encoder.layers.15.mlp.fc2.weight', 'vision_model.encoder.layers.6.mlp.fc2.bias', 'vision_model.encoder.layers.1.mlp.fc1.weight', 'vision_model.encoder.layers.2.self_attn.v_proj.weight', 'vision_model.encoder.layers.19.self_attn.q_proj.bias', 'vision_model.encoder.layers.10.self_attn.q_proj.bias', 'vision_model.encoder.layers.1.mlp.fc1.bias', 'vision_model.encoder.layers.23.self_attn.k_proj.weight', 'vision_model.encoder.layers.6.layer_norm1.bias', 'vision_model.encoder.layers.21.layer_norm1.weight', 'vision_model.encoder.layers.11.self_attn.q_proj.weight', 'vision_model.encoder.layers.15.self_attn.out_proj.weight', 'vision_model.encoder.layers.2.self_attn.k_proj.weight', 'vision_model.encoder.layers.3.self_attn.v_proj.weight', 'vision_model.encoder.layers.12.self_attn.v_proj.weight', 'vision_model.encoder.layers.5.mlp.fc1.weight', 'vision_model.encoder.layers.13.self_attn.k_proj.weight', 'vision_model.encoder.layers.12.mlp.fc2.bias', 'vision_model.encoder.layers.19.layer_norm1.weight', 'logit_scale', 'vision_model.encoder.layers.12.self_attn.q_proj.weight', 'vision_model.encoder.layers.17.mlp.fc1.bias', 'vision_model.encoder.layers.9.mlp.fc1.weight', 'vision_model.encoder.layers.2.mlp.fc1.bias', 'vision_model.encoder.layers.9.mlp.fc1.bias', 'vision_model.encoder.layers.15.self_attn.q_proj.bias', 'vision_model.encoder.layers.21.self_attn.q_proj.weight', 'vision_model.encoder.layers.11.self_attn.k_proj.bias', 'vision_model.encoder.layers.17.mlp.fc1.weight', 'vision_model.encoder.layers.2.layer_norm2.bias', 'vision_model.encoder.layers.5.self_attn.k_proj.bias', 'vision_model.encoder.layers.16.self_attn.k_proj.weight', 'vision_model.encoder.layers.15.self_attn.k_proj.bias', 'vision_model.encoder.layers.21.self_attn.v_proj.bias', 'vision_model.encoder.layers.15.layer_norm2.weight', 'vision_model.embeddings.position_embedding.weight', 'vision_model.encoder.layers.4.mlp.fc1.weight', 'vision_model.encoder.layers.1.self_attn.q_proj.bias', 'vision_model.encoder.layers.14.self_attn.q_proj.weight', 'vision_model.encoder.layers.12.self_attn.out_proj.bias', 'vision_model.encoder.layers.17.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.out_proj.bias', 'vision_model.encoder.layers.6.self_attn.q_proj.bias', 'vision_model.encoder.layers.0.self_attn.q_proj.bias', 'vision_model.encoder.layers.17.self_attn.q_proj.bias', 'vision_model.encoder.layers.22.mlp.fc2.weight', 'vision_model.encoder.layers.14.self_attn.out_proj.bias', 'vision_model.encoder.layers.1.self_attn.k_proj.weight', 'vision_model.encoder.layers.15.self_attn.v_proj.weight', 'vision_model.encoder.layers.19.self_attn.out_proj.weight', 'vision_model.embeddings.position_ids', 'vision_model.encoder.layers.12.mlp.fc1.bias', 'vision_model.encoder.layers.21.mlp.fc2.weight', 'vision_model.encoder.layers.20.self_attn.v_proj.bias', 'vision_model.encoder.layers.12.layer_norm2.bias', 'vision_model.encoder.layers.22.mlp.fc1.weight', 'vision_model.encoder.layers.5.self_attn.q_proj.weight', 'vision_model.encoder.layers.16.layer_norm2.bias', 'vision_model.encoder.layers.23.self_attn.out_proj.weight', 'vision_model.encoder.layers.10.mlp.fc2.weight', 'vision_model.encoder.layers.3.self_attn.q_proj.weight', 'vision_model.encoder.layers.13.mlp.fc1.weight', 'vision_model.encoder.layers.8.self_attn.k_proj.weight', 'vision_model.encoder.layers.20.layer_norm1.bias', 'vision_model.encoder.layers.5.mlp.fc2.weight', 'vision_model.encoder.layers.17.self_attn.q_proj.weight', 'vision_model.encoder.layers.13.self_attn.out_proj.bias', 'vision_model.encoder.layers.3.layer_norm2.weight', 'vision_model.encoder.layers.22.layer_norm1.bias', 'vision_model.encoder.layers.1.layer_norm1.bias', 'vision_model.encoder.layers.4.self_attn.v_proj.bias', 'vision_model.encoder.layers.15.self_attn.out_proj.bias', 'vision_model.encoder.layers.4.self_attn.v_proj.weight', 'vision_model.encoder.layers.3.self_attn.out_proj.bias', 'vision_model.encoder.layers.4.self_attn.q_proj.weight', 'vision_model.encoder.layers.12.self_attn.k_proj.bias', 'vision_model.encoder.layers.5.layer_norm2.weight', 'vision_model.encoder.layers.4.mlp.fc1.bias', 'vision_model.encoder.layers.11.self_attn.v_proj.weight', 'vision_model.encoder.layers.18.mlp.fc1.weight', 'vision_model.encoder.layers.3.self_attn.q_proj.bias', 'vision_model.encoder.layers.21.layer_norm2.weight', 'vision_model.encoder.layers.22.self_attn.k_proj.weight', 'vision_model.encoder.layers.19.self_attn.k_proj.bias', 'vision_model.encoder.layers.10.layer_norm2.bias', 'vision_model.encoder.layers.0.self_attn.q_proj.weight', 'vision_model.encoder.layers.19.self_attn.out_proj.bias', 'vision_model.encoder.layers.5.self_attn.q_proj.bias', 'vision_model.encoder.layers.9.self_attn.out_proj.bias', 'vision_model.encoder.layers.23.self_attn.k_proj.bias', 'vision_model.encoder.layers.8.self_attn.k_proj.bias', 'vision_model.encoder.layers.7.layer_norm2.weight', 'vision_model.encoder.layers.3.mlp.fc1.bias', 'vision_model.encoder.layers.5.self_attn.out_proj.weight', 'vision_model.encoder.layers.14.self_attn.v_proj.weight', 'vision_model.encoder.layers.16.layer_norm1.weight', 'vision_model.encoder.layers.6.layer_norm2.bias', 'vision_model.encoder.layers.14.self_attn.v_proj.bias', 'vision_model.encoder.layers.2.self_attn.out_proj.bias', 'vision_model.encoder.layers.4.mlp.fc2.weight', 'vision_model.encoder.layers.9.self_attn.k_proj.bias', 'vision_model.encoder.layers.10.self_attn.k_proj.bias', 'vision_model.encoder.layers.8.mlp.fc1.bias', 'vision_model.encoder.layers.3.layer_norm1.weight', 'vision_model.encoder.layers.14.self_attn.q_proj.bias', 'vision_model.encoder.layers.8.mlp.fc1.weight', 'vision_model.encoder.layers.18.layer_norm1.bias', 'vision_model.encoder.layers.14.self_attn.k_proj.bias', 'vision_model.encoder.layers.7.layer_norm1.weight', 'vision_model.encoder.layers.1.mlp.fc2.bias', 'vision_model.encoder.layers.21.mlp.fc2.bias', 'vision_model.encoder.layers.0.mlp.fc1.weight', 'vision_model.encoder.layers.20.mlp.fc1.weight', 'vision_model.encoder.layers.13.self_attn.out_proj.weight', 'vision_model.encoder.layers.13.mlp.fc2.bias', 'vision_model.encoder.layers.18.layer_norm2.bias', 'vision_model.encoder.layers.20.mlp.fc1.bias', 'vision_model.encoder.layers.9.mlp.fc2.bias', 'vision_model.encoder.layers.13.self_attn.q_proj.weight', 'vision_model.encoder.layers.20.self_attn.v_proj.weight', 'vision_model.encoder.layers.5.layer_norm2.bias', 'vision_model.encoder.layers.17.layer_norm2.weight', 'vision_model.encoder.layers.11.self_attn.out_proj.bias', 'vision_model.encoder.layers.5.mlp.fc1.bias', 'vision_model.encoder.layers.10.self_attn.v_proj.weight', 'vision_model.embeddings.patch_embedding.weight', 'vision_model.encoder.layers.21.layer_norm2.bias', 'vision_model.encoder.layers.4.self_attn.q_proj.bias', 'vision_model.encoder.layers.1.layer_norm1.weight', 'vision_model.encoder.layers.4.self_attn.k_proj.bias', 'vision_model.encoder.layers.14.mlp.fc1.bias', 'vision_model.encoder.layers.12.mlp.fc1.weight', 'vision_model.encoder.layers.6.self_attn.out_proj.weight', 'vision_model.encoder.layers.20.self_attn.out_proj.bias', 'vision_model.encoder.layers.22.self_attn.out_proj.bias', 'vision_model.encoder.layers.0.self_attn.k_proj.weight', 'vision_model.encoder.layers.20.layer_norm2.weight', 'vision_model.encoder.layers.16.self_attn.v_proj.weight', 'vision_model.encoder.layers.9.self_attn.q_proj.weight', 'vision_model.encoder.layers.14.layer_norm1.weight', 'vision_model.encoder.layers.14.mlp.fc2.bias', 'vision_model.encoder.layers.2.self_attn.k_proj.bias', 'vision_model.encoder.layers.19.mlp.fc2.bias', 'vision_model.encoder.layers.5.self_attn.v_proj.weight', 'vision_model.post_layernorm.bias', 'vision_model.encoder.layers.3.mlp.fc1.weight', 'vision_model.encoder.layers.13.self_attn.q_proj.bias', 'vision_model.encoder.layers.23.layer_norm2.weight', 'vision_model.encoder.layers.10.layer_norm1.bias', 'vision_model.encoder.layers.11.self_attn.out_proj.weight', 'vision_model.encoder.layers.8.mlp.fc2.weight', 'vision_model.encoder.layers.6.layer_norm2.weight', 'vision_model.encoder.layers.8.self_attn.q_proj.weight', 'vision_model.encoder.layers.3.self_attn.out_proj.weight', 'vision_model.encoder.layers.8.mlp.fc2.bias', 'vision_model.encoder.layers.7.self_attn.out_proj.weight', 'vision_model.encoder.layers.20.self_attn.out_proj.weight', 'vision_model.encoder.layers.7.self_attn.out_proj.bias', 'vision_model.encoder.layers.5.self_attn.out_proj.bias', 'vision_model.encoder.layers.11.layer_norm1.weight', 'vision_model.encoder.layers.14.layer_norm2.bias', 'vision_model.encoder.layers.12.mlp.fc2.weight', 'text_projection.weight', 'vision_model.encoder.layers.23.layer_norm1.weight', 'vision_model.encoder.layers.9.layer_norm1.bias', 'vision_model.encoder.layers.16.self_attn.q_proj.bias', 'vision_model.encoder.layers.16.self_attn.k_proj.bias', 'vision_model.encoder.layers.17.self_attn.v_proj.weight', 'vision_model.encoder.layers.21.mlp.fc1.weight', 'vision_model.encoder.layers.23.self_attn.q_proj.weight', 'vision_model.encoder.layers.17.layer_norm1.bias', 'vision_model.encoder.layers.2.layer_norm2.weight', 'vision_model.encoder.layers.21.self_attn.k_proj.weight', 'vision_model.encoder.layers.0.layer_norm2.bias', 'vision_model.encoder.layers.2.self_attn.v_proj.bias', 'vision_model.encoder.layers.7.mlp.fc1.weight', 'vision_model.encoder.layers.19.mlp.fc2.weight', 'vision_model.encoder.layers.14.self_attn.out_proj.weight', 'vision_model.encoder.layers.23.layer_norm1.bias', 'vision_model.encoder.layers.11.self_attn.v_proj.bias', 'vision_model.encoder.layers.15.mlp.fc1.bias', 'vision_model.encoder.layers.9.layer_norm1.weight', 'vision_model.encoder.layers.6.self_attn.k_proj.weight', 'vision_model.encoder.layers.8.layer_norm1.bias', 'vision_model.encoder.layers.20.layer_norm1.weight', 'vision_model.encoder.layers.22.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.v_proj.bias', 'vision_model.encoder.layers.20.self_attn.k_proj.bias', 'vision_model.encoder.layers.20.mlp.fc2.bias', 'vision_model.encoder.layers.20.layer_norm2.bias', 'vision_model.encoder.layers.8.self_attn.v_proj.weight', 'vision_model.encoder.layers.8.self_attn.out_proj.weight', 'vision_model.encoder.layers.16.layer_norm1.bias', 'vision_model.encoder.layers.19.mlp.fc1.weight', 'vision_model.encoder.layers.2.self_attn.q_proj.bias', 'vision_model.encoder.layers.4.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.self_attn.k_proj.weight', 'vision_model.encoder.layers.22.layer_norm2.weight', 'vision_model.encoder.layers.12.self_attn.v_proj.bias', 'vision_model.encoder.layers.0.self_attn.k_proj.bias', 'vision_model.encoder.layers.6.mlp.fc1.weight', 'vision_model.encoder.layers.23.mlp.fc1.weight', 'vision_model.encoder.layers.5.self_attn.k_proj.weight', 'vision_model.encoder.layers.5.mlp.fc2.bias', 'vision_model.encoder.layers.3.self_attn.k_proj.bias', 'vision_model.encoder.layers.18.mlp.fc2.weight', 'vision_model.encoder.layers.9.self_attn.out_proj.weight', 'vision_model.encoder.layers.8.layer_norm1.weight', 'vision_model.encoder.layers.18.self_attn.out_proj.bias', 'vision_model.encoder.layers.14.layer_norm1.bias', 'vision_model.post_layernorm.weight', 'vision_model.encoder.layers.13.self_attn.v_proj.bias', 'vision_model.encoder.layers.6.mlp.fc1.bias', 'vision_model.encoder.layers.20.mlp.fc2.weight', 'vision_model.encoder.layers.14.mlp.fc1.weight', 'vision_model.encoder.layers.11.mlp.fc1.weight', 'vision_model.encoder.layers.7.self_attn.k_proj.weight', 'vision_model.encoder.layers.22.mlp.fc2.bias', 'vision_model.encoder.layers.12.layer_norm2.weight', 'vision_model.encoder.layers.1.self_attn.k_proj.bias', 'vision_model.encoder.layers.15.layer_norm1.bias', 'vision_model.encoder.layers.17.self_attn.v_proj.bias', 'vision_model.encoder.layers.19.self_attn.v_proj.weight', 'vision_model.encoder.layers.13.layer_norm2.bias', 'vision_model.encoder.layers.0.mlp.fc1.bias', 'vision_model.encoder.layers.4.mlp.fc2.bias', 'vision_model.encoder.layers.11.layer_norm2.bias', 'vision_model.encoder.layers.15.self_attn.q_proj.weight', 'vision_model.pre_layrnorm.weight', 'vision_model.encoder.layers.0.self_attn.out_proj.bias', 'vision_model.encoder.layers.22.self_attn.q_proj.weight', 'vision_model.encoder.layers.4.layer_norm2.bias', 'vision_model.encoder.layers.10.self_attn.k_proj.weight', 'vision_model.encoder.layers.16.self_attn.q_proj.weight', 'vision_model.encoder.layers.7.mlp.fc2.bias', 'vision_model.encoder.layers.2.self_attn.q_proj.weight', 'vision_model.encoder.layers.0.layer_norm1.weight', 'vision_model.encoder.layers.5.self_attn.v_proj.bias', 'vision_model.encoder.layers.23.mlp.fc2.weight', 'vision_model.encoder.layers.16.mlp.fc2.bias', 'vision_model.encoder.layers.4.layer_norm1.weight', 'vision_model.encoder.layers.11.mlp.fc2.bias', 'vision_model.encoder.layers.7.mlp.fc2.weight', 'vision_model.encoder.layers.16.layer_norm2.weight', 'vision_model.encoder.layers.3.layer_norm2.bias', 'vision_model.encoder.layers.18.self_attn.v_proj.weight', 'vision_model.encoder.layers.2.mlp.fc2.weight', 'vision_model.encoder.layers.0.mlp.fc2.bias', 'vision_model.encoder.layers.23.self_attn.q_proj.bias', 'vision_model.encoder.layers.5.layer_norm1.bias', 'vision_model.encoder.layers.13.self_attn.k_proj.bias', 'vision_model.encoder.layers.23.self_attn.v_proj.weight', 'vision_model.encoder.layers.17.mlp.fc2.weight', 'vision_model.encoder.layers.18.mlp.fc1.bias', 'vision_model.encoder.layers.18.mlp.fc2.bias', 'vision_model.encoder.layers.17.layer_norm1.weight', 'vision_model.encoder.layers.13.layer_norm2.weight', 'vision_model.encoder.layers.23.mlp.fc1.bias', 'vision_model.encoder.layers.20.self_attn.q_proj.weight', 'vision_model.encoder.layers.7.self_attn.v_proj.weight', 'vision_model.encoder.layers.3.mlp.fc2.bias', 'vision_model.encoder.layers.2.layer_norm1.weight', 'vision_model.encoder.layers.0.mlp.fc2.weight', 'vision_model.encoder.layers.12.self_attn.q_proj.bias', 'vision_model.encoder.layers.0.self_attn.out_proj.weight', 'vision_model.encoder.layers.17.self_attn.k_proj.bias', 'vision_model.encoder.layers.18.self_attn.q_proj.bias', 'vision_model.encoder.layers.3.layer_norm1.bias', 'vision_model.encoder.layers.0.self_attn.v_proj.bias', 'vision_model.encoder.layers.1.layer_norm2.weight', 'vision_model.encoder.layers.10.layer_norm1.weight', 'vision_model.encoder.layers.23.self_attn.out_proj.bias', 'vision_model.encoder.layers.19.layer_norm2.bias', 'vision_model.encoder.layers.9.self_attn.v_proj.weight', 'vision_model.encoder.layers.18.self_attn.k_proj.bias', 'vision_model.encoder.layers.12.layer_norm1.weight', 'vision_model.encoder.layers.15.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.mlp.fc1.bias', 'vision_model.encoder.layers.8.layer_norm2.bias', 'vision_model.encoder.layers.23.layer_norm2.bias', 'vision_model.encoder.layers.7.layer_norm2.bias', 'vision_model.encoder.layers.14.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.self_attn.out_proj.bias', 'vision_model.encoder.layers.2.layer_norm1.bias', 'vision_model.encoder.layers.16.self_attn.out_proj.bias', 'vision_model.encoder.layers.9.self_attn.v_proj.bias', 'vision_model.pre_layrnorm.bias', 'vision_model.encoder.layers.19.mlp.fc1.bias', 'vision_model.encoder.layers.3.mlp.fc2.weight', 'vision_model.encoder.layers.1.self_attn.out_proj.bias', 'vision_model.encoder.layers.15.mlp.fc2.bias', 'vision_model.encoder.layers.1.layer_norm2.bias', 'vision_model.encoder.layers.5.layer_norm1.weight', 'vision_model.encoder.layers.18.layer_norm1.weight', 'vision_model.encoder.layers.9.self_attn.q_proj.bias', 'vision_model.encoder.layers.15.layer_norm1.weight', 'vision_model.encoder.layers.19.self_attn.v_proj.bias', 'vision_model.encoder.layers.10.mlp.fc1.bias', 'vision_model.encoder.layers.13.self_attn.v_proj.weight', 'vision_model.encoder.layers.13.mlp.fc1.bias', 'vision_model.encoder.layers.22.mlp.fc1.bias', 'vision_model.encoder.layers.6.mlp.fc2.weight', 'vision_model.encoder.layers.18.self_attn.v_proj.bias', 'visual_projection.weight', 'vision_model.encoder.layers.17.self_attn.out_proj.weight', 'vision_model.encoder.layers.0.layer_norm2.weight', 'vision_model.encoder.layers.10.self_attn.out_proj.weight', 'vision_model.encoder.layers.16.mlp.fc2.weight', 'vision_model.encoder.layers.7.self_attn.q_proj.bias', 'vision_model.encoder.layers.10.self_attn.v_proj.bias', 'vision_model.encoder.layers.7.self_attn.q_proj.weight', 'vision_model.encoder.layers.22.self_attn.v_proj.bias', 'vision_model.encoder.layers.3.self_attn.k_proj.weight', 'vision_model.encoder.layers.11.self_attn.q_proj.bias', 'vision_model.encoder.layers.18.self_attn.q_proj.weight', 'vision_model.embeddings.class_embedding', 'vision_model.encoder.layers.11.mlp.fc2.weight', 'vision_model.encoder.layers.15.self_attn.v_proj.bias', 'vision_model.encoder.layers.16.mlp.fc1.bias', 'vision_model.encoder.layers.6.self_attn.v_proj.weight', 'vision_model.encoder.layers.9.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.self_attn.k_proj.bias', 'vision_model.encoder.layers.21.layer_norm1.bias', 'vision_model.encoder.layers.10.layer_norm2.weight', 'vision_model.encoder.layers.14.mlp.fc2.weight', 'vision_model.encoder.layers.4.self_attn.out_proj.bias', 'vision_model.encoder.layers.13.mlp.fc2.weight', 'vision_model.encoder.layers.2.self_attn.out_proj.weight', 'vision_model.encoder.layers.19.self_attn.k_proj.weight', 'vision_model.encoder.layers.6.self_attn.q_proj.weight', 'vision_model.encoder.layers.13.layer_norm1.bias', 'vision_model.encoder.layers.6.self_attn.v_proj.bias', 'vision_model.encoder.layers.17.mlp.fc2.bias', 'vision_model.encoder.layers.1.mlp.fc2.weight', 'vision_model.encoder.layers.18.self_attn.out_proj.weight', 'vision_model.encoder.layers.22.self_attn.v_proj.weight', 'vision_model.encoder.layers.10.self_attn.q_proj.weight', 'vision_model.encoder.layers.22.self_attn.out_proj.weight', 'vision_model.encoder.layers.20.self_attn.q_proj.bias', 'vision_model.encoder.layers.19.self_attn.q_proj.weight', 'vision_model.encoder.layers.18.layer_norm2.weight', 'vision_model.encoder.layers.12.self_attn.k_proj.weight', 'vision_model.encoder.layers.21.self_attn.v_proj.weight', 'vision_model.encoder.layers.1.self_attn.q_proj.weight', 'vision_model.encoder.layers.11.self_attn.k_proj.weight', 'vision_model.encoder.layers.9.mlp.fc2.weight', 'vision_model.encoder.layers.7.mlp.fc1.bias', 'vision_model.encoder.layers.8.self_attn.q_proj.bias', 'vision_model.encoder.layers.7.layer_norm1.bias', 'vision_model.encoder.layers.14.layer_norm2.weight', 'vision_model.encoder.layers.7.self_attn.k_proj.bias', 'vision_model.encoder.layers.10.mlp.fc1.weight', 'vision_model.encoder.layers.17.self_attn.out_proj.bias', 'vision_model.encoder.layers.20.self_attn.k_proj.weight', 'vision_model.encoder.layers.10.self_attn.out_proj.bias', 'vision_model.encoder.layers.15.layer_norm2.bias', 'vision_model.encoder.layers.2.mlp.fc2.bias', 'vision_model.encoder.layers.4.layer_norm1.bias', 'vision_model.encoder.layers.11.layer_norm1.bias', 'vision_model.encoder.layers.3.self_attn.v_proj.bias', 'vision_model.encoder.layers.16.self_attn.v_proj.bias']
- This IS expected if you are initializing CLIPTextModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing CLIPTextModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Creating invisible watermark encoder (see https://github.com/ShieldMnt/invisible-watermark)...
Sampling:   0%|                                                                                                 | 0/1 [00:00<?, ?it/sData shape for PLMS sampling is (1, 4, 64, 64)                                                                   | 0/1 [00:00<?, ?it/s]
Running PLMS Sampling with 50 timesteps
PLMS Sampler: 100%|███████████████████████████████████████████████████████████████████████████████████| 50/50 [04:11<00:00,  5.03s/it]
data: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [04:21<00:00, 261.23s/it]
Sampling: 100%|████████████████████████████████████████████████████████████████████████████████████████| 1/1 [04:21<00:00, 261.23s/it]
Your samples are ready and waiting for you here: 
outputs/txt2img-samples 
 
Enjoy.
stable-diffusion %