Waifu Diffusionやーる（Python3.9、Windows10）

はじめに

Waifu Diffusionをやっていきまーす

開発環境

• Windows 10 PC

• Python 3.9

導入

1．クローンします

2．ライブラリをインストールします

pip install -r requirements.txt

requirements.txt

diffusers>=0.5.1
numpy==1.23.4
wandb==0.13.4
torch
torchvision
transformers>=4.21.0
huggingface-hub>=0.10.0
Pillow==9.2.0
tqdm==4.64.1
ftfy==6.1.1
bitsandbytes
pynvml~=11.4.1
psutil~=5.9.0
accelerate==0.13.1
scipy==1.9.3

Successfully installed Pillow-9.2.0 accelerate-0.13.1 bitsandbytes-0.35.4 diffusers-0.10.2 docker-pycreds-0.4.0 huggingface-hub-0.11.1 numpy-1.23.4 pathtools-0.1.2 promise-2.3 psutil-5.9.4 pynvml-11.4.1 scipy-1.9.3 sentry-sdk-1.11.1 setproctitle-1.3.2 shortuuid-1.0.11 tqdm-4.64.1 transformers-4.25.1 wandb-0.13.4

3．下記のプログラムを作成し、実行します

python main.py

main.py

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained(
    'hakurei/waifu-diffusion',
    torch_dtype=torch.float32
).to('cuda')

prompt = "1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt"
with autocast("cuda"):
    image = pipe(prompt, guidance_scale=6)["sample"][0]  
    
image.save("test.png")

4．結果

GPUが足りませんでした

NOTE: Redirects are currently not supported in Windows or MacOs.
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling `transformers.utils.move_cache()`.
Moving 15 files to the new cache system
100%|██████████████████████████████████████████████████████████████████████████████████| 15/15 [00:19<00:00,  1.32s/it]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 550/550 [00:00<00:00, 523kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 342/342 [00:00<00:00, 218kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████| 4.67k/4.67k [00:00<00:00, 2.14MB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████| 1.22G/1.22G [02:58<00:00, 6.83MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 215/215 [00:00<00:00, 189kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 612/612 [00:00<00:00, 585kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████| 492M/492M [01:21<00:00, 6.06MB/s]
Downloading: 100%|███████████████████████████████████████████████████████████████████| 525k/525k [00:01<00:00, 509kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 472/472 [00:00<00:00, 374kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 806/806 [00:00<00:00, 773kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████| 1.06M/1.06M [00:01<00:00, 813kB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 743/743 [00:00<00:00, 345kB/s]
Downloading: 100%|████████████████████████████████████████████████████████████████| 3.44G/3.44G [08:54<00:00, 6.43MB/s]
Downloading: 100%|█████████████████████████████████████████████████████████████████████| 547/547 [00:00<00:00, 200kB/s]
Downloading: 100%|██████████████████████████████████████████████████████████████████| 335M/335M [00:33<00:00, 9.92MB/s]
Fetching 15 files: 100%|███████████████████████████████████████████████████████████████| 15/15 [14:26<00:00, 57.75s/it]
C:\Users\good_\anaconda3\envs\py39\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py:151: FutureWarning: The configuration file of the unet has set the default `sample_size` to smaller than 64 which seems highly unlikely. If your checkpoint is a fine-tuned version of any of the following:
- CompVis/stable-diffusion-v1-4
- CompVis/stable-diffusion-v1-3
- CompVis/stable-diffusion-v1-2
- CompVis/stable-diffusion-v1-1
- runwayml/stable-diffusion-v1-5
- runwayml/stable-diffusion-inpainting
 you should change 'sample_size' to 64 in the configuration file. Please make sure to update the config accordingly as leaving `sample_size=32` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `unet/config.json` file
  deprecate("sample_size<64", "1.0.0", deprecation_message, standard_warn=False)
Traceback (most recent call last):
  File "D:\PythonProjects\waifu-diffusion-main\main.py", line 5, in <module>
    pipe = StableDiffusionPipeline.from_pretrained(
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\diffusers\pipeline_utils.py", line 270, in to
    module.to(torch_device)
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 927, in to
    return self._apply(convert)
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
    module._apply(fn)
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
    module._apply(fn)
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
    module._apply(fn)
  [Previous line repeated 5 more times]
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 602, in _apply
    param_applied = fn(param)
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\torch\nn\modules\module.py", line 925, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 3.37 GiB already allocated; 0 bytes free; 3.43 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

CPUで推論してみました

main.py

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained(
    'hakurei/waifu-diffusion',
    torch_dtype=torch.float32
)

prompt = "1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt"
with autocast("cuda"):
    image = pipe(prompt, guidance_scale=6)["sample"][0]  
    
image.save("test.png")

NOTE: Redirects are currently not supported in Windows or MacOs.
Fetching 15 files: 100%|█████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 3034.66it/s]
C:\Users\good_\anaconda3\envs\py39\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py:151: FutureWarning: The configuration file of the unet has set the default `sample_size` to smaller than 64 which seems highly unlikely. If your checkpoint is a fine-tuned version of any of the following:
- CompVis/stable-diffusion-v1-4
- CompVis/stable-diffusion-v1-3
- CompVis/stable-diffusion-v1-2
- CompVis/stable-diffusion-v1-1
- runwayml/stable-diffusion-v1-5
- runwayml/stable-diffusion-inpainting
 you should change 'sample_size' to 64 in the configuration file. Please make sure to update the config accordingly as leaving `sample_size=32` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `unet/config.json` file
  deprecate("sample_size<64", "1.0.0", deprecation_message, standard_warn=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [16:20<00:00, 19.60s/it]
Traceback (most recent call last):
  File "D:\PythonProjects\waifu-diffusion-main\main.py", line 17, in <module>
    image = pipe(prompt, guidance_scale=6)["sample"][0]
  File "C:\Users\good_\anaconda3\envs\py39\lib\site-packages\diffusers\utils\outputs.py", line 88, in __getitem__
    return inner_dict[k]
KeyError: 'sample'

sampleないよと怒られたので、下記のようにプログラムを編集して実行します

main.py

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline

# pipe = StableDiffusionPipeline.from_pretrained(
#     'hakurei/waifu-diffusion',
#     torch_dtype=torch.float32
# ).to('cuda')

pipe = StableDiffusionPipeline.from_pretrained(
    'hakurei/waifu-diffusion',
    torch_dtype=torch.float32
)

prompt = "1girl, aqua eyes, baseball cap, blonde hair, closed mouth, earrings, green background, hat, hoop earrings, jewelry, looking at viewer, shirt, short hair, simple background, solo, upper body, yellow shirt"
with autocast("cuda"):
    images = pipe(prompt, guidance_scale=10, num_inference_steps=50).images    
images[0].save("test.png")

NOTE: Redirects are currently not supported in Windows or MacOs.
Fetching 15 files: 100%|█████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 4665.18it/s]
C:\Users\good_\anaconda3\envs\py39\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_stable_diffusion.py:151: FutureWarning: The configuration file of the unet has set the default `sample_size` to smaller than 64 which seems highly unlikely. If your checkpoint is a fine-tuned version of any of the following:
- CompVis/stable-diffusion-v1-4
- CompVis/stable-diffusion-v1-3
- CompVis/stable-diffusion-v1-2
- CompVis/stable-diffusion-v1-1
- runwayml/stable-diffusion-v1-5
- runwayml/stable-diffusion-inpainting
 you should change 'sample_size' to 64 in the configuration file. Please make sure to update the config accordingly as leaving `sample_size=32` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `unet/config.json` file
  deprecate("sample_size<64", "1.0.0", deprecation_message, standard_warn=False)
100%|██████████████████████████████████████████████████████████████████████████████████| 50/50 [14:54<00:00, 17.89s/it]

処理に15分ほどかかりました
num_inference_stepsはデフォルトで50なので、少なくすると少ないステップで推論できます

お疲れ様でした