タイトル: DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」を使ってみる

DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」が公開されているので、Google Colab上で使ってみました。

しかし、手順通り、MDETRの「Visual Question Answering(視覚的質問応答)の推論」の部分を実行しても、結果出力データにnan値が多く含まれ、推論結果もnan値となり、推論が出来ませんでした。以降に試行結果を記載します。

下記の記事の通りにMDETRを実行

画像について何でも文章で質問して、その回答(該当する単語)が得られるという、MDETRの機能が面白そうなので、下記の記事にある、MDETRの「Visual Question Answering(視覚的質問応答)の推論」の部分を実行してみます。

複数モダリティを使う「マルチモーダルAI」の紹介
第1回 DETR+RoBERTaを使ったマルチモーダル推論モデル「MDETR」の紹介とColabでの推論
https://www.ogis-ri.co.jp/otc/hiroba/technical/multimodal-ai/part1.html

上記記事の元になっているチュートリアルファイル
https://colab.research.google.com/github/ashkamath/mdetr/blob/colab/notebooks/MDETR_demo.ipynb

Google Colabを用意・GPU利用へ設定変更

新規にGoogle Colabを開いて、GPU利用へ設定を変更

(記事通り) 4-1.必要なパッケージの準備と前処理･後処理の定義

記事の通りに実行してみます。(コード・実行結果のみ抜粋)

!pip install timm transformers

Requirement already satisfied: timm in /usr/local/lib/python3.11/dist-packages (1.0.15)
Requirement already satisfied: transformers in /usr/local/lib/python3.11/dist-packages (4.50.3)
Requirement already satisfied: torch in /usr/local/lib/python3.11/dist-packages (from timm) (2.6.0+cu124)
Requirement already satisfied: torchvision in /usr/local/lib/python3.11/dist-packages (from timm) (0.21.0+cu124)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.11/dist-packages (from timm) (6.0.2)
Requirement already satisfied: huggingface_hub in /usr/local/lib/python3.11/dist-packages (from timm) (0.30.1)
Requirement already satisfied: safetensors in /usr/local/lib/python3.11/dist-packages (from timm) (0.5.3)
Requirement already satisfied: filelock in /usr/local/lib/python3.11/dist-packages (from transformers) (3.18.0)
Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.11/dist-packages (from transformers) (2.0.2)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.11/dist-packages (from transformers) (24.2)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.11/dist-packages (from transformers) (2024.11.6)
Requirement already satisfied: requests in /usr/local/lib/python3.11/dist-packages (from transformers) (2.32.3)
Requirement already satisfied: tokenizers<0.22,>=0.21 in /usr/local/lib/python3.11/dist-packages (from transformers) (0.21.1)
Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.11/dist-packages (from transformers) (4.67.1)
Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub->timm) (2025.3.2)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.11/dist-packages (from huggingface_hub->timm) (4.13.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.11/dist-packages (from requests->transformers) (2025.1.31)
Requirement already satisfied: networkx in /usr/local/lib/python3.11/dist-packages (from torch->timm) (3.4.2)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (3.1.6)
Collecting nvidia-cuda-nvrtc-cu12==12.4.127 (from torch->timm)
  Downloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-runtime-cu12==12.4.127 (from torch->timm)
  Downloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cuda-cupti-cu12==12.4.127 (from torch->timm)
  Downloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cudnn-cu12==9.1.0.70 (from torch->timm)
  Downloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cublas-cu12==12.4.5.8 (from torch->timm)
  Downloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cufft-cu12==11.2.1.3 (from torch->timm)
  Downloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-curand-cu12==10.3.5.147 (from torch->timm)
  Downloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting nvidia-cusolver-cu12==11.6.1.9 (from torch->timm)
  Downloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting nvidia-cusparse-cu12==12.3.1.170 (from torch->timm)
  Downloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
Requirement already satisfied: nvidia-cusparselt-cu12==0.6.2 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (0.6.2)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (12.4.127)
Collecting nvidia-nvjitlink-cu12==12.4.127 (from torch->timm)
  Downloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
Requirement already satisfied: triton==3.2.0 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (3.2.0)
Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.11/dist-packages (from torch->timm) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.11/dist-packages (from sympy==1.13.1->torch->timm) (1.3.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.11/dist-packages (from torchvision->timm) (11.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.11/dist-packages (from jinja2->torch->timm) (3.0.2)
Downloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl (363.4 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m363.4/363.4 MB[0m [31m3.7 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (13.8 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m13.8/13.8 MB[0m [31m34.1 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (24.6 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m24.6/24.6 MB[0m [31m26.1 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (883 kB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m883.7/883.7 kB[0m [31m39.7 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl (664.8 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m664.8/664.8 MB[0m [31m2.6 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl (211.5 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m211.5/211.5 MB[0m [31m5.3 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl (56.3 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m56.3/56.3 MB[0m [31m12.3 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl (127.9 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m127.9/127.9 MB[0m [31m7.4 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl (207.5 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m207.5/207.5 MB[0m [31m5.8 MB/s[0m eta [36m0:00:00[0m
[?25hDownloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl (21.1 MB)
[2K   [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m21.1/21.1 MB[0m [31m98.2 MB/s[0m eta [36m0:00:00[0m
[?25hInstalling collected packages: nvidia-nvjitlink-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, nvidia-cusparse-cu12, nvidia-cudnn-cu12, nvidia-cusolver-cu12
  Attempting uninstall: nvidia-nvjitlink-cu12
    Found existing installation: nvidia-nvjitlink-cu12 12.5.82
    Uninstalling nvidia-nvjitlink-cu12-12.5.82:
      Successfully uninstalled nvidia-nvjitlink-cu12-12.5.82
  Attempting uninstall: nvidia-curand-cu12
    Found existing installation: nvidia-curand-cu12 10.3.6.82
    Uninstalling nvidia-curand-cu12-10.3.6.82:
      Successfully uninstalled nvidia-curand-cu12-10.3.6.82
  Attempting uninstall: nvidia-cufft-cu12
    Found existing installation: nvidia-cufft-cu12 11.2.3.61
    Uninstalling nvidia-cufft-cu12-11.2.3.61:
      Successfully uninstalled nvidia-cufft-cu12-11.2.3.61
  Attempting uninstall: nvidia-cuda-runtime-cu12
    Found existing installation: nvidia-cuda-runtime-cu12 12.5.82
    Uninstalling nvidia-cuda-runtime-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-runtime-cu12-12.5.82
  Attempting uninstall: nvidia-cuda-nvrtc-cu12
    Found existing installation: nvidia-cuda-nvrtc-cu12 12.5.82
    Uninstalling nvidia-cuda-nvrtc-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-nvrtc-cu12-12.5.82
  Attempting uninstall: nvidia-cuda-cupti-cu12
    Found existing installation: nvidia-cuda-cupti-cu12 12.5.82
    Uninstalling nvidia-cuda-cupti-cu12-12.5.82:
      Successfully uninstalled nvidia-cuda-cupti-cu12-12.5.82
  Attempting uninstall: nvidia-cublas-cu12
    Found existing installation: nvidia-cublas-cu12 12.5.3.2
    Uninstalling nvidia-cublas-cu12-12.5.3.2:
      Successfully uninstalled nvidia-cublas-cu12-12.5.3.2
  Attempting uninstall: nvidia-cusparse-cu12
    Found existing installation: nvidia-cusparse-cu12 12.5.1.3
    Uninstalling nvidia-cusparse-cu12-12.5.1.3:
      Successfully uninstalled nvidia-cusparse-cu12-12.5.1.3
  Attempting uninstall: nvidia-cudnn-cu12
    Found existing installation: nvidia-cudnn-cu12 9.3.0.75
    Uninstalling nvidia-cudnn-cu12-9.3.0.75:
      Successfully uninstalled nvidia-cudnn-cu12-9.3.0.75
  Attempting uninstall: nvidia-cusolver-cu12
    Found existing installation: nvidia-cusolver-cu12 11.6.3.83
    Uninstalling nvidia-cusolver-cu12-11.6.3.83:
      Successfully uninstalled nvidia-cusolver-cu12-11.6.3.83
Successfully installed nvidia-cublas-cu12-12.4.5.8 nvidia-cuda-cupti-cu12-12.4.127 nvidia-cuda-nvrtc-cu12-12.4.127 nvidia-cuda-runtime-cu12-12.4.127 nvidia-cudnn-cu12-9.1.0.70 nvidia-cufft-cu12-11.2.1.3 nvidia-curand-cu12-10.3.5.147 nvidia-cusolver-cu12-11.6.1.9 nvidia-cusparse-cu12-12.3.1.170 nvidia-nvjitlink-cu12-12.4.127

import torch
from PIL import Image
import requests
import json
import torchvision.transforms as T
import matplotlib.pyplot as plt
import numpy as np
import torch.nn.functional as F
from skimage.measure import find_contours
from matplotlib.patches import Polygon
from collections import defaultdict
torch.set_grad_enabled(False);

# カラーマップ
COLORS = [[0.000, 0.447, 0.741], [0.850, 0.325, 0.098], [0.929, 0.694, 0.125],
          [0.494, 0.184, 0.556], [0.466, 0.674, 0.188], [0.301, 0.745, 0.933]]

# インプット画像の前処理
transform = T.Compose([
    T.Resize(800),
    T.ToTensor(),
    T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

def box_cxcywh_to_xyxy(x):
    """
    (center_x, center_y, width, height)から(xmin, ymin, xmax, ymax)に座標変換
    """
    x_c, y_c, w, h = x.unbind(1)
    b = [(x_c - 0.5 * w), (y_c - 0.5 * h),
         (x_c + 0.5 * w), (y_c + 0.5 * h)]
    return torch.stack(b, dim=1)

def rescale_bboxes(out_bbox, size):
    """
    バウンディングボックスのリスケール
    """
    img_w, img_h = size
    b = box_cxcywh_to_xyxy(out_bbox)
    b = b * torch.tensor([img_w, img_h, img_w, img_h], dtype=torch.float32)
    return b

def apply_mask(image, mask, color, alpha=0.5):
    """
    セグメンテーション用の領域を塗りつぶしたマスクを画像に適用

    Parameters
    ----------
    image : numpy.ndarray
      適用元の画像
    mask : tensor
      適用するマスクの領域座標
    color : list
      カラーマップ
    alpha : float
      透過値
    """
    for c in range(3):
        image[:, :, c] = np.where(mask == 1,
                                  image[:, :, c] *
                                  (1 - alpha) + alpha * color[c] * 255,
                                  image[:, :, c])
    return image

def plot_results(pil_img, scores, boxes, labels, masks=None):
    """
    結果表示

    Parameters
    ----------
    pil_img : PIL.Image
      画像
    scores : list
      検出された物体の予測値のリスト
    boxes : list
      検出された物体のバウンディングボックス座標(center_x, center_y, width, height)のリスト
    labels : list
      検出された物体のラベルのリスト
    masks : list
      セグメンテーション用のマスクのリスト
    """
    plt.figure(figsize=(16,10))
    # PIL.Imageをnumpy.ndarrayに変換
    np_image = np.array(pil_img)
    ax = plt.gca()
    colors = COLORS * 100
    if masks is None:
        # マスクが無い場合は、len(scores)のNoneのリストで埋める
        masks = [None for _ in range(len(scores))]
    # リストの長さが違う場合、例外をスロー
    assert len(scores) == len(boxes) == len(labels) == len(masks)
    for s, (xmin, ymin, xmax, ymax), l, mask, c in zip(scores, boxes.tolist(), labels, masks, colors):
        ax.add_patch(plt.Rectangle((xmin, ymin), xmax - xmin, ymax - ymin,
                                   fill=False, color=c, linewidth=3))
        text = f'{l}: {s:0.2f}'
        ax.text(xmin, ymin, text, fontsize=15, bbox=dict(facecolor='white', alpha=0.8))

        if mask is None:
          continue
        # マスク適用
        np_image = apply_mask(np_image, mask, c)
        # マスク部分の輪郭線を描画。maskデータはバイナリなので、中間の0.5の位置で輪郭線を描画する。
        padded_mask = np.zeros((mask.shape[0] + 2, mask.shape[1] + 2), dtype=np.uint8)
        padded_mask[1:-1, 1:-1] = mask
        contours = find_contours(padded_mask, 0.5)
        for verts in contours:
            # (y, x)を(x, y)に反転
            verts = np.fliplr(verts) - 1
            # 輪郭線の内部を塗り潰し
            p = Polygon(verts, facecolor="none", edgecolor=c)
            ax.add_patch(p)

    plt.imshow(np_image)
    plt.axis('off')
    plt.show()

(記事通り) 4-4. Visual Question Answering(視覚的質問応答)の推論

記事の通りに実行していきます。(コード・実行結果のみ抜粋)

import json, requests
answer2id_by_type = json.load(requests.get("https://nyu.box.com/shared/static/j4rnpo8ixn6v0iznno2pim6ffj3jyaj8.json", stream=True).raw)
id2answerbytype = {}
for ans_type in answer2id_by_type.keys():
    curr_reversed_dict = {v: k for k, v in answer2id_by_type[ans_type].items()}
    id2answerbytype[ans_type] = curr_reversed_dict

print(id2answerbytype.keys())

dict_keys(['answer_rel', 'answer_obj', 'answer_global', 'answer_attr', 'answer_cat'])

import pprint
pprint.pprint(id2answerbytype["answer_rel"])

{0: 'no',
 1: 'yes',
 2: 'towel',
 3: 'man',
 4: 'backpack',
 5: 'cabinet',
 6: 'left',
 7: 'purse',
 8: 'soda',
 9: 'right',
 10: 'bag',
 11: 'bricks',
 12: 'street',
 13: 'sign',
 14: 'train',
 15: 'arrow',
 16: 'coat',
 17: 'table',
 18: 'tree',
 19: 'airplane',
 20: 'bed',
 21: 'wetsuit',
 22: 'sky',
 23: 'lettuce',
 24: 'cake',
 25: 'dog',
 26: 'horse',
 27: 'tiles',
 28: 'broccoli',
 29: 'fence',
 30: 'van',
 31: 'boats',
 32: 'bathroom',
 33: 'potato',
 34: 'giraffe',
 35: 'car',
 36: 'end table',
 37: 'wine',
 38: 'shelf',
 39: 'bus',
 40: 'tray',
 41: 'cabinets',
 42: 'truck',
 43: 'flowers',
 44: 'computer mouse',
 45: 'wall',
 46: 'radio',
 47: 'peppers',
 48: 'phone',
 49: 'lady',
 50: 'elephant',
 51: 'floor',
 52: 'skier',
 53: 'helmet',
 54: 'blender',
 55: 'cow',
 56: 'mountain',
 57: 'sofa',
 58: 'coffee table',
 59: 'grass',
 60: 'orange',
 61: 'behind',
 62: 'surfboard',
 63: 'water',
 64: 'carrots',
 65: 'tie',
 66: 'pecan',
 67: 'players',
 68: 'plate',
 69: 'umbrella',
 70: 'frisbee',
 71: 'soup',
 72: 'pizza',
 73: 'chicken',
 74: 'home plate',
 75: 'shirt',
 76: 'building',
 77: 'ground',
 78: 'sandwich',
 79: 'people',
 80: 'fire hydrant',
 81: 'trees',
 82: 'bottle',
 83: 'woman',
 84: 'papers',
 85: 'chair',
 86: 'cup',
 87: 'trailer',
 88: 'desk',
 89: 'celery',
 90: 'pen',
 91: 'bull',
 92: 'roll',
 93: 'sneakers',
 94: 'pillow',
 95: 'flamingo',
 96: 'knee pads',
 97: 'field',
 98: 'side table',
 99: 'beans',
 100: 'monitor',
 101: 'sidewalk',
 102: 'zebra',
 103: 'coffee',
 104: 'giraffes',
 105: 'cat',
 106: 'stop sign',
 107: 'laptop',
 108: 'front',
 109: 'onions',
 110: 'pan',
 111: 'tennis ball',
 112: 'hat',
 113: 'lid',
 114: 'artichokes',
 115: 'branch',
 116: 'glasses',
 117: 'spice',
 118: 'printer',
 119: 'leaves',
 120: 'cap',
 121: 'baseball bat',
 122: 'batter',
 123: 'boat',
 124: 'bear',
 125: 'rock',
 126: 'lake',
 127: 'jacket',
 128: 'train car',
 129: 'bacon',
 130: 'rice',
 131: 'bread',
 132: 'bird',
 133: 'television',
 134: 'bicycle',
 135: 'pants',
 136: 'grapes',
 137: 'beer',
 138: 'spinach',
 139: 'boy',
 140: 'child',
 141: 'tag',
 142: 'suitcase',
 143: 'cheese',
 144: 'soap',
 145: 'box',
 146: 'magnets',
 147: 'seat',
 148: 'refrigerator',
 149: 'bikini',
 150: 'oranges',
 151: 'zoo',
 152: 'keyboard',
 153: 'banana',
 154: 'skateboard',
 155: 'glass',
 156: 'hillside',
 157: 'street light',
 158: 'book',
 159: 'girl',
 160: 'bun',
 161: 'jeans',
 162: 'snow',
 163: 'stew',
 164: 'fries',
 165: 'guy',
 166: 'carriage',
 167: 'pepper',
 168: 'paper',
 169: 'carrot',
 170: 'post',
 171: 'clock',
 172: 'boots',
 173: 'ice',
 174: 'jar',
 175: 'couch',
 176: 'cars',
 177: 'clouds',
 178: 'mirror',
 179: 'onion',
 180: 'cupboard',
 181: 'suit',
 182: 'wagon',
 183: 'remote control',
 184: 'donut',
 185: 'sand',
 186: 'faucet',
 187: 'grapefruit',
 188: 'bookshelf',
 189: 'hot dog',
 190: 'drum',
 191: 'potatoes',
 192: 'bookcase',
 193: 'pole',
 194: 'meat',
 195: 'trunk',
 196: 'drawer',
 197: 'player',
 198: 'cabbage',
 199: 'words',
 200: 'rocks',
 201: 'beach',
 202: 'road',
 203: 'seagull',
 204: 'skirt',
 205: 'pastry',
 206: 'bowl',
 207: 'pasta',
 208: 'tomato',
 209: 'oven',
 210: 'cucumber',
 211: 'eggs',
 212: 'flag',
 213: 'hill',
 214: 'speaker',
 215: 'spots',
 216: 'microwave oven',
 217: 'bench',
 218: 'luggage',
 219: 'controller',
 220: 'olive',
 221: 'uniform',
 222: 'straw',
 223: 'container',
 224: 'sheep',
 225: 'gorilla',
 226: 'blanket',
 227: 'suv',
 228: 'letters',
 229: 'horses',
 230: 'sausage',
 231: 'bush',
 232: 'plant',
 233: 'shorts',
 234: 'goat',
 235: 'wii controller',
 236: 'microwave',
 237: 'bracelet',
 238: 'salad',
 239: 'doors',
 240: 'dish',
 241: 'sauce',
 242: 'teddy bear',
 243: 'ostrich',
 244: 'path',
 245: 'shrimp',
 246: 'radiator',
 247: 'tablecloth',
 248: 'chairs',
 249: 'shoe',
 250: 'window',
 251: 'baskets',
 252: 'cereal',
 253: 'computer',
 254: 'office',
 255: 'cutting board',
 256: 'donuts',
 257: 'apples',
 258: 'pickles',
 259: 'plants',
 260: 'dress',
 261: 'curtains',
 262: 'spatula',
 263: 'onion rings',
 264: 'noodles',
 265: 'scarf',
 266: 'cows',
 267: 'books',
 268: 'stove',
 269: 'envelope',
 270: 'graffiti',
 271: 'cell phone',
 272: 'school bus',
 273: 'wire',
 274: 'tablet',
 275: 'mother',
 276: 'boot',
 277: 'racket',
 278: 'cds',
 279: 'tofu',
 280: 'above',
 281: 'eye glasses',
 282: 'doorway',
 283: 'tv stand',
 284: 'doll',
 285: 'necklace',
 286: 'street sign',
 287: 'ball',
 288: 'parking meter',
 289: 'mushrooms',
 290: 'donkey',
 291: 'hay',
 292: 'blueberries',
 293: 'kettle',
 294: 'screen',
 295: 'bananas',
 296: 'dvd player',
 297: 'computer monitor',
 298: 'toaster',
 299: 'spoon',
 300: 'ducks',
 301: 'turtle',
 302: 'nightstand',
 303: 'kite',
 304: 'headphones',
 305: 'mickey mouse',
 306: 'ski',
 307: 'elephants',
 308: 'strawberries',
 309: 'bathtub',
 310: 'bike',
 311: 'number',
 312: 'camera',
 313: 'egg',
 314: 'picture',
 315: 'door',
 316: 'painting',
 317: 'barrier',
 318: 'jeep',
 319: 'dispenser',
 320: 'platform',
 321: 'powdered sugar',
 322: 'letter',
 323: 'cart',
 324: 'headband',
 325: 'canoe',
 326: 'gentleman',
 327: 'taxi',
 328: 't-shirt',
 329: 'jet',
 330: 'avocado',
 331: 'station',
 332: 'tongs',
 333: 'snack',
 334: 'bookshelves',
 335: 'roof',
 336: 'magazine',
 337: 'calculator',
 338: 'planter',
 339: 'pepperoni',
 340: 'motorcycle',
 341: 'action figure',
 342: 'binder',
 343: 'outlet',
 344: 'lamp',
 345: 'cage',
 346: 'collar',
 347: 'benches',
 348: 'garment',
 349: 'cookie',
 350: 'baby',
 351: 'office chair',
 352: 'flower',
 353: 'light switch',
 354: 'toilet paper',
 355: 'curtain',
 356: 'dishwasher',
 357: 'corn',
 358: 'seed',
 359: 'park',
 360: 'nuts',
 361: 'ocean',
 362: 'mountains',
 363: 'at the camera',
 364: 'pavement',
 365: 'ice cream',
 366: 'coleslaw',
 367: 'salad dressing',
 368: 'vase',
 369: 'dessert',
 370: 'men',
 371: 'canister',
 372: 'baseball players',
 373: 'stuffed animal',
 374: 'basket',
 375: 'lime',
 376: 'goggles',
 377: 'beverage',
 378: 'calf',
 379: 'shelves',
 380: 'napkin',
 381: 'placemat',
 382: 'skis',
 383: 'pastries',
 384: 'lamb',
 385: 'snail',
 386: 'cauliflower',
 387: 'cords',
 388: 'ice cube',
 389: 'toothbrush',
 390: 'customers',
 391: 'logs',
 392: 'cross',
 393: 'bleachers',
 394: 'raspberries',
 395: 'game controller',
 396: 'living room',
 397: 'pizza pan',
 398: 'below',
 399: 'catcher',
 400: 'toilet',
 401: 'frog',
 402: 'cherries',
 403: 'chain',
 404: 'apple',
 405: 'log',
 406: 'pilot',
 407: 'umbrellas',
 408: 'peas',
 409: 'stuffed bear',
 410: 'burner',
 411: 'chocolate',
 412: 'surfer',
 413: 'light bulbs',
 414: 'bedroom',
 415: 'tank top',
 416: 'traffic light',
 417: 'tennis',
 418: 'spectator',
 419: 'cupcakes',
 420: 'airplanes',
 421: 'sailboats',
 422: 'guitar',
 423: 'counter',
 424: 'snowboard',
 425: 'vest',
 426: 'cooler',
 427: 'bar stool',
 428: 'palm tree',
 429: 'hats',
 430: 'closet',
 431: 'speakers',
 432: 'ceiling',
 433: 'pipe',
 434: 'pineapple',
 435: 'dryer',
 436: 'sun',
 437: 'waffles',
 438: 'shore',
 439: 'desktop computer',
 440: 'cats',
 441: 'blouse',
 442: 'butterfly',
 443: 'costume',
 444: 'coffee cup',
 445: 'pot',
 446: 'coffee maker',
 447: 'towels',
 448: 'children',
 449: 'under',
 450: 'air',
 451: 'wii game',
 452: 'garland',
 453: 'dirt',
 454: 'ketchup',
 455: 'cookies',
 456: 'hard drive',
 457: 'bandana',
 458: 'runway',
 459: 'console',
 460: 'mango',
 461: 'piano',
 462: 'wires',
 463: 'tables',
 464: 'rope',
 465: 'sword',
 466: 'grape',
 467: 'bat',
 468: 'knife',
 469: 'air conditioner',
 470: 'squash',
 471: 'baseball',
 472: 'cat food',
 473: 'sticker',
 474: 'fork',
 475: 'buildings',
 476: 'locomotive',
 477: 'snacks',
 478: 'armchair',
 479: 'hallway',
 480: 'chopsticks',
 481: 'tomatoes',
 482: 'dresser',
 483: 'tower',
 484: 'trash can',
 485: 'ottoman',
 486: 'berry',
 487: 'trash bag',
 488: 'umpire',
 489: 'cupcake',
 490: 'women',
 491: 'sweet potato',
 492: 'crown',
 493: 'hot dogs',
 494: 'ice maker',
 495: 'moss',
 496: 'officer',
 497: 'ham',
 498: 'milk',
 499: 'beads',
 500: 'cupboards',
 501: 'pea',
 502: 'kitchen',
 503: 'word',
 504: 'wii',
 505: 'drawers',
 506: 'strawberry',
 507: 'goats',
 508: 'coffee pot',
 509: 'sink',
 510: 'cyclist',
 511: 'controllers',
 512: 'popcorn',
 513: 'frosting',
 514: 'duck',
 515: 'apron',
 516: 'handbag',
 517: 'countertop',
 518: 'dip',
 519: 'restroom',
 520: 'cucumbers',
 521: 'pipes',
 522: 'parrot',
 523: 'shoes',
 524: 'paddle',
 525: 'desk lamp',
 526: 'poster',
 527: 'juice',
 528: 'champagne',
 529: 'olives',
 530: 'fish',
 531: 'angry bird',
 532: 'belt',
 533: 'parking lot',
 534: 'bagels',
 535: 'bushes',
 536: 'skateboarder',
 537: 'mat',
 538: 'cowboy',
 539: 'bicycles',
 540: 'sweater',
 541: 'pineapples',
 542: 'stroller',
 543: 'dining table',
 544: 'earphones',
 545: 'rug',
 546: 'tractor',
 547: 'cherry',
 548: 'wool',
 549: 'skater',
 550: 'chickpeas',
 551: 'utensil',
 552: 'swimsuit',
 553: 'branches',
 554: 'muffin',
 555: 'beets',
 556: 'steps',
 557: 'glove',
 558: 'sushi',
 559: 'drink',
 560: 'watch',
 561: 'remote controls',
 562: 'toast',
 563: 'toppings',
 564: 'logo',
 565: 'salmon',
 566: 'socks',
 567: 'bridge',
 568: 'antenna',
 569: 'puppy',
 570: 'down',
 571: 'parsley',
 572: 'ipod',
 573: 'candies',
 574: 'island',
 575: 'birds',
 576: 'watermelon',
 577: 'xbox controller',
 578: 'lamps',
 579: 'wave',
 580: 'policeman',
 581: 'pear',
 582: 'jersey',
 583: 'water bottle',
 584: 'bags',
 585: 'blinds',
 586: 'pears',
 587: 'crowd',
 588: 'match',
 589: 'stadium',
 590: 'baking sheet',
 591: 'mozzarella',
 592: 'sheet',
 593: 'microphone',
 594: 'polar bear',
 595: 'croissant',
 596: 'minivan',
 597: 'bears',
 598: 'mushroom',
 599: 'computer desk',
 600: 'chandelier',
 601: 'packet',
 602: 'game',
 603: 'stick',
 604: 'spices',
 605: 'cable',
 606: 'bus stop',
 607: 'tissues',
 608: 'tea kettle',
 609: 'forest',
 610: 'candle holder',
 611: 'mixing bowl',
 612: 'forks',
 613: 'dragon',
 614: 'wine glass',
 615: 'fire truck',
 616: 'panda',
 617: 'wine bottle',
 618: 'toy',
 619: 'urinal',
 620: 'gas stove',
 621: 'magnet',
 622: 'porch',
 623: 'soldier',
 624: 'pans',
 625: 'outfit',
 626: 'shower',
 627: 'bucket',
 628: 'buses',
 629: 'mask',
 630: 'lemons',
 631: 'bunny',
 632: 'beds',
 633: 'artichoke',
 634: 'spear',
 635: 'tea',
 636: 'robe',
 637: 'laptops',
 638: 'gate',
 639: 'almonds',
 640: 'dishes',
 641: 'rose',
 642: 'dry-erase board',
 643: 'hotdog bun',
 644: 'owl',
 645: 'liquor',
 646: 'girls',
 647: 'heart',
 648: 'houses',
 649: 'gloves',
 650: 'pocket',
 651: 'yard',
 652: 'drinks',
 653: 'bagel',
 654: 'pajamas',
 655: 'crosswalk',
 656: 'airport',
 657: 'palm trees',
 658: 'video camera',
 659: 'statue',
 660: 'bottles',
 661: 'pedestrian',
 662: 'wines',
 663: 'luggage cart',
 664: 'pens',
 665: 'peach',
 666: 'ski lift',
 667: 'staircase',
 668: 'comforter',
 669: 'hair dryer',
 670: 'tennis balls',
 671: 'hamburger',
 672: 'medicine cabinet',
 673: 'mustard',
 674: 'aquarium',
 675: 'herbs',
 676: 'cakes',
 677: 'waffle',
 678: 'beer bottle',
 679: 'lion',
 680: 'avocados',
 681: 'foil',
 682: 'fudge',
 683: 'dumpster',
 684: 'stuffed dog',
 685: 'patio',
 686: 'heater',
 687: 'daughter',
 688: 'passenger',
 689: 'utensils',
 690: 'bubbles',
 691: 'safety jacket',
 692: 'driver',
 693: 'pigeon',
 694: 'hangar',
 695: 'garage',
 696: 'lemon',
 697: 'oil',
 698: 'boys',
 699: 'sandal',
 700: 'leaf',
 701: 'cards',
 702: 'shop',
 703: 'sandwiches',
 704: 'whisk',
 705: 'mixer',
 706: 'waste basket',
 707: 'skate park',
 708: 'ladder',
 709: 'coffee mug',
 710: 'dining room',
 711: 'ring',
 712: 'kitten',
 713: 'kites',
 714: 'silverware',
 715: 'squirrel',
 716: 'flip flop',
 717: 'power line',
 718: 'stuffed animals',
 719: 'asparagus',
 720: 'cooker',
 721: 'carpet',
 722: 'scissors',
 723: 'macaroni',
 724: 'cracker',
 725: 'stairs',
 726: 'sack',
 727: 'buns',
 728: 'brush',
 729: 'ship',
 730: 'mashed potatoes',
 731: 'cord',
 732: 'pump',
 733: 'frame',
 734: 'toiletries',
 735: 'music',
 736: 'snowboarder',
 737: 'liquid',
 738: 'candy',
 739: 'briefcase',
 740: 'artwork',
 741: 'backyard',
 742: 'soccer player',
 743: 'menu',
 744: 'food truck',
 745: 'sandals',
 746: 'upward',
 747: 'chimney',
 748: 'fans',
 749: 'deck',
 750: 'icing',
 751: 'steam',
 752: 'dough',
 753: 'spectators',
 754: 'light fixture',
 755: 'trucks',
 756: 'peaches',
 757: 'newspaper',
 758: 'crate',
 759: 'stickers',
 760: 'mouse pad',
 761: 'canopy',
 762: 'pie',
 763: 'dog food',
 764: 'tomato sauce',
 765: 'sculpture',
 766: 'chili',
 767: 'american flag',
 768: 'zebras',
 769: 'platter',
 770: 'coke',
 771: 'train station',
 772: 'roadside',
 773: 'earring',
 774: 'river',
 775: 'pitcher',
 776: 'saucer',
 777: 'soap dish',
 778: 'beet',
 779: 'cones',
 780: 'wii controllers',
 781: 'sailboat',
 782: 'bowls',
 783: 'fan',
 784: 'dressing',
 785: 'train tracks',
 786: 'tissue box',
 787: 'cables',
 788: 'gadget',
 789: 'house',
 790: 'hedge',
 791: 'pine trees',
 792: 'swimming pool',
 793: 'magazines',
 794: 'trumpet',
 795: 'turkey',
 796: 'mud',
 797: 'mugs',
 798: 'farm',
 799: 'jockey',
 800: 'dock',
 801: 'can',
 802: 'crab',
 803: 'athlete',
 804: 'marker',
 805: 'smoke',
 806: 'power lines',
 807: 'mattress',
 808: 'watermelons',
 809: 'couple',
 810: 'windows',
 811: 'soccer ball',
 812: 'desert',
 813: 'soda can',
 814: 'trains',
 815: 'net',
 816: 'bird cage',
 817: 'floor lamp',
 818: 'omelette',
 819: 'church',
 820: 'washing machine',
 821: 'granola',
 822: 'shirts',
 823: 'chains',
 824: 'ornament',
 825: 'parent',
 826: 'walkway',
 827: 'sausages',
 828: 'mayonnaise',
 829: 'seal',
 830: 'roast beef',
 831: 'water bottles',
 832: 'tools',
 833: 'puddle',
 834: 'biscuit',
 835: 'couches',
 836: 'suitcases',
 837: 'soft drink',
 838: 'berries',
 839: 'star',
 840: 'symbol',
 841: 'figure',
 842: 'chicken breast',
 843: 'hotel',
 844: 'router',
 845: 'gun',
 846: 'bee',
 847: 'rubber duck',
 848: 'containers',
 849: 'appetizers',
 850: 'raisin',
 851: 'eagle',
 852: 'topping',
 853: 'biker',
 854: 'bomb',
 855: 'decoration',
 856: 'burrito',
 857: 'pig',
 858: 'nut',
 859: 'steak',
 860: 'hammer',
 861: 'paint',
 862: 'cream cheese',
 863: 'tea pot',
 864: 'name tag',
 865: 'dugout',
 866: 'mug',
 867: 'pond',
 868: 'computers',
 869: 'ambulance',
 870: 'railroad',
 871: 'card',
 872: 'aircraft',
 873: 'pizza slice',
 874: 'seagulls',
 875: 'store',
 876: 'tent',
 877: 'sugar',
 878: 'blackberries',
 879: 'candle',
 880: 'numbers',
 881: 'sconce',
 882: 'antelope',
 883: 'bone',
 884: 'coats',
 885: 'cake stand',
 886: 'pizzas',
 887: 'bakery',
 888: 'pine tree',
 889: 'meal',
 890: 'cd',
 891: 'snowsuit',
 892: 'wolf',
 893: 'ceiling light',
 894: 'skateboards',
 895: 'gifts',
 896: 'crane',
 897: 'audience',
 898: 'balcony',
 899: 'fog',
 900: 'gravy',
 901: 'life jacket',
 902: 'heels',
 903: 'sweatshirt',
 904: 'entertainment center',
 905: 'scooter',
 906: 'pillows',
 907: 'walnuts',
 908: 'candles',
 909: 'tool',
 910: 'machine',
 911: 'eggplant',
 912: 'breakfast',
 913: 'basil',
 914: 'vines',
 915: 'passengers',
 916: 'boxes',
 917: 'muffins',
 918: 'blueberry',
 919: 'wolves',
 920: 'paper container',
 921: 'cockpit',
 922: 'skillet',
 923: 'alcohol',
 924: 'label',
 925: 'oatmeal',
 926: 'melon',
 927: 'bus driver',
 928: 'guys',
 929: 'helicopter',
 930: 'projector',
 931: 'chef',
 932: 'cloths',
 933: 'dogs',
 934: 'mound',
 935: 'jumpsuit',
 936: 'monkey',
 937: 'receipt',
 938: 'swan',
 939: 'tape',
 940: 'twigs',
 941: 'pancake',
 942: 'wallet',
 943: 'characters',
 944: 'walls',
 945: 'dvds',
 946: 'lambs',
 947: 'shopping bag',
 948: 'fur',
 949: 'soap dispenser',
 950: 'sea',
 951: 'burger',
 952: 'hair',
 953: 'flip flops',
 954: 'beer can',
 955: 'pouch',
 956: 'trash',
 957: 'package',
 958: 'flatbread',
 959: 'hurdle',
 960: 'terminal',
 961: 'hook',
 962: 'grater',
 963: 'classroom',
 964: 'rooftop',
 965: 'tunnel',
 966: 'dome',
 967: 'food',
 968: 'ladles',
 969: 'control panel',
 970: 'football',
 971: 'balls',
 972: 'melons',
 973: 'clock tower',
 974: 'stone',
 975: 'grill',
 976: 'family',
 977: 'picnic table',
 978: 'toddler',
 979: 'geese',
 980: 'beef',
 981: 'pancakes',
 982: 'cafe',
 983: 'marshmallow',
 984: 'bouquet',
 985: 'stir fry',
 986: 'pizza pie',
 987: 'up',
 988: 'bracelets',
 989: 'yogurt',
 990: 'coconut',
 991: 'charger',
 992: 'wristband',
 993: 'fireplace',
 994: 'pigeons',
 995: 'twig',
 996: 'cowboy hat',
 997: 'flags',
 998: 'gas station',
 999: 'loaf',
 1000: 'visitor',
 1001: 'drape',
 1002: 'lions',
 1003: 'comb',
 1004: 'stump',
 1005: 'tortilla',
 1006: 'bedspread',
 1007: 'games',
 1008: 'heel',
 1009: 'feeder',
 1010: 'drain',
 1011: 'bath towel',
 1012: 'video games',
 1013: 'plates',
 1014: 'produce',
 1015: 'display',
 1016: 'highway',
 1017: 'broom',
 1018: 'map',
 1019: 'salt shaker',
 1020: 'parking sign',
 1021: 'sprinkles',
 1022: 'light bulb',
 1023: 'broth',
 1024: 'whipped cream',
 1025: 'goose',
 1026: 'pizza crust',
 1027: 'photographer',
 1028: 'garden',
 1029: 'trunks',
 1030: 'lighthouse',
 1031: 'earrings',
 1032: 'rain',
 1033: 'lock',
 1034: 'biscuits',
 1035: 'electric toothbrush',
 1036: 'traffic sign',
 1037: 'tissue',
 1038: 'ropes',
 1039: 'flower pot',
 1040: 'napkins',
 1041: 'napkin dispenser',
 1042: 'raspberry',
 1043: 'drapes',
 1044: 'cameras',
 1045: 'banana peel',
 1046: 'wheelchair',
 1047: 'cups',
 1048: 'cone',
 1049: 'market',
 1050: 'figurine',
 1051: 'wildflowers',
 1052: 'blind',
 1053: 'hills',
 1054: 'dish drainer',
 1055: 'bread loaf',
 1056: 'museum',
 1057: 'picture frame',
 1058: 'notebook',
 1059: 'mall',
 1060: 'castle',
 1061: 'alligator',
 1062: 'almond',
 1063: 'face mask',
 1064: 'dress shirt',
 1065: 'meadow',
 1066: 'tangerine',
 1067: 'vacuum',
 1068: 'gym',
 1069: 'father',
 1070: 'fountain',
 1071: 'paper towel',
 1072: 'bread box',
 1073: 'soap bottle',
 1074: 'crust',
 1075: 'sponge',
 1076: 'pillowcase',
 1077: 'worker',
 1078: 'clocks',
 1079: 'pudding',
 1080: 'balloons',
 1081: 'courtyard',
 1082: 'bats',
 1083: 'crumbs',
 1084: 'rhino',
 1085: 'zucchini',
 1086: 'helmets',
 1087: 'butter',
 1088: 'battery',
 1089: 'keyboards',
 1090: 'wardrobe',
 1091: 'soccer',
 1092: 'garlic',
 1093: 'gravel',
 1094: 'pumpkin',
 1095: 'plantains',
 1096: 'back',
 1097: 'knee pad',
 1098: 'deer',
 1099: 'video game',
 1100: 'decorations',
 1101: 'cream',
 1102: 'pikachu',
 1103: 'uniforms',
 1104: 'cans',
 1105: 'cooking pot',
 1106: 'weeds',
 1107: 'money',
 1108: 'syrup',
 1109: 'bell',
 1110: 'roadway',
 1111: 'pedestrians',
 1112: 'sweet potatoes',
 1113: 'cell phones',
 1114: 'beach chair',
 1115: 'restaurant',
 1116: 'waiter',
 1117: 'trays',
 1118: 'gas pump',
 1119: 'motorcycles',
 1120: 'telephone pole',
 1121: 'intersection',
 1122: 'engineer',
 1123: 'spider',
 1124: 'workers',
 1125: 'onion ring',
 1126: 'fire',
 1127: 'mannequins',
 1128: 'toothpaste',
 1129: 'tents',
 1130: 'herd',
 1131: 'stage',
 1132: 'drawing',
 1133: 'traffic lights',
 1134: 'pasture',
 1135: 'daisy',
 1136: 'vine',
 1137: 'outlets',
 1138: 'christmas light',
 1139: 'employee',
 1140: 'monitors',
 1141: 'attic',
 1142: 'kiosk',
 1143: 'fire extinguisher',
 1144: 'satellite dish',
 1145: 'barn',
 1146: 'potato salad',
 1147: 'mailbox',
 1148: 'sea foam',
 1149: 'town',
 1150: 'elevator',
 1151: 'teddy bears',
 1152: 'tiger',
 1153: 'lego',
 1154: 'smoothie',
 1155: 'peanut butter',
 1156: 'serving dish',
 1157: 'pizza oven',
 1158: 'dish soap',
 1159: 'egg yolk',
 1160: 'paintings',
 1161: 'wine glasses',
 1162: 'leggings',
 1163: 'croissants',
 1164: 'coach',
 1165: 'table lamp',
 1166: 'nest',
 1167: 'snow pants',
 1168: 'balloon',
 1169: 'ribs',
 1170: 'pizza box',
 1171: 'shower curtain',
 1172: 'hand soap',
 1173: 'shampoo',
 1174: 'toy car',
 1175: 'buoy',
 1176: 'smoke stack',
 1177: 'hilltop',
 1178: 'orchid',
 1179: 'harbor',
 1180: 'toaster oven',
 1181: 'beneath',
 1182: 'thermometer',
 1183: 'cash register',
 1184: 'plain',
 1185: 'pots',
 1186: 'ladle',
 1187: 'camel',
 1188: 'blossoms',
 1189: 'life preserver',
 1190: 'vending machine',
 1191: 'officers',
 1192: 'hotel room',
 1193: 'step',
 1194: 'ice cubes',
 1195: 'shelter',
 1196: 'raisins',
 1197: 'fruit stand',
 1198: 'away',
 1199: 'scaffolding',
 1200: 'character',
 1201: 'mannequin',
 1202: 'notepad',
 1203: 'horse hoof',
 1204: 'lunch',
 1205: 'crackers',
 1206: 'raincoat',
 1207: 'hose',
 1208: 'pork',
 1209: 'alarm clock',
 1210: 'knives',
 1211: 'cabin',
 1212: 'shoe lace',
 1213: 'jars',
 1214: 'mangoes',
 1215: 'seafood',
 1216: 'price tag',
 1217: 'dolls',
 1218: 'polo shirt',
 1219: 'vitamins',
 1220: 'cricket',
 1221: 'sunglasses',
 1222: 'taco',
 1223: 'bug',
 1224: 'suits',
 1225: 'phones',
 1226: 'tourists',
 1227: 'forward',
 1228: 'orchids',
 1229: 'folding chair',
 1230: 'cigarette',
 1231: 'underneath',
 1232: 'shark',
 1233: 'butter knife',
 1234: 'sour cream',
 1235: 'shampoo bottle',
 1236: 'kiwi',
 1237: 'sheets',
 1238: 'garnish',
 1239: 'sticky notes',
 1240: 'cactus',
 1241: 'cereal box',
 1242: 'donkeys',
 1243: 'bikes',
 1244: 'desserts',
 1245: 'penguin',
 1246: 'tree branch',
 1247: 'pretzel',
 1248: 'casserole',
 1249: 'vests',
 1250: 'rolling pin',
 1251: 'chickens',
 1252: 'snow flakes',
 1253: 'brownie',
 1254: 'dinosaur',
 1255: 'roses',
 1256: 'blazer',
 1257: 'lawn',
 1258: 'shopping cart',
 1259: 'guacamole',
 1260: 'lemonade',
 1261: 'tuna',
 1262: 'wok',
 1263: 'potato chips',
 1264: 'performer',
 1265: 'pomegranate',
 1266: 'printers',
 1267: 'dumplings',
 1268: 'skyscraper',
 1269: 'gourd',
 1270: 'pencil',
 1271: 'water glass',
 1272: 'milkshake',
 1273: 'mountain side',
 1274: 'cheetah',
 1275: 'seeds',
 1276: 'cliff',
 1277: 'chocolate chips',
 1278: 'spoons',
 1279: 'cane',
 1280: 'keypad',
 1281: 'team',
 1282: 'meats',
 1283: 'library',
 1284: 'merchandise',
 1285: 'rabbit',
 1286: 'gown',
 1287: 'vases',
 1288: 'rice cooker',
 1289: 'baking pan',
 1290: 'blankets',
 1291: 'city',
 1292: 'swans',
 1293: 'hand dryer',
 1294: 'glaze',
 1295: 'canisters',
 1296: 'eiffel tower',
 1297: 'pizza cutter',
 1298: 'cotton dessert',
 1299: 'mexican food',
 1300: 'kittens',
 1301: 'feta cheese',
 1302: 'anchovies',
 1303: 'papaya',
 1304: 'lobster',
 1305: 'hummus',
 1306: 'sunflower',
 1307: 'bandage',
 1308: 'beverages',
 1309: 'meatballs',
 1310: 'seaweed',
 1311: 'elmo',
 1312: 'peacock',
 1313: 'antelopes',
 1314: 'boulders',
 1315: 'surfboards',
 1316: 'wallpaper',
 1317: 'cappuccino',
 1318: 'mirrors',
 1319: 'pillars',
 1320: 'food processor',
 1321: 'herb',
 1322: 'ostriches',
 1323: 'fishing pole',
 1324: 'octopus',
 1325: 'rackets',
 1326: 'ovens',
 1327: 'shield',
 1328: 'toilet brush',
 1329: 'temple',
 1330: 'kiwis',
 1331: 'cheesecake',
 1332: 'sign post',
 1333: 'parrots',
 1334: 'student',
 1335: 'french toast',
 1336: 'paper towels',
 1337: 'hamburgers',
 1338: 'carts',
 1339: 'stones',
 1340: 'serving tray',
 1341: 'moose',
 1342: 'shoe laces',
 1343: 'honey',
 1344: 'vendor',
 1345: 'stapler',
 1346: 'food container',
 1347: 'beach umbrella',
 1348: 'chinese food',
 1349: 'pita',
 1350: 'peanuts',
 1351: 'wristwatch',
 1352: 'peanut',
 1353: 'straight',
 1354: 'son',
 1355: 'knife block',
 1356: 'soda bottle',
 1357: 'stuffed bears',
 1358: 'door frame',
 1359: 'moon',
 1360: 'alien',
 1361: 'cake pan',
 1362: 'hospital',
 1363: 'panda bear',
 1364: 'toys',
 1365: 'jackets',
 1366: 'lunch box',
 1367: 'orchard',
 1368: 'cheeseburger',
 1369: 'oreo',
 1370: 'pizza boxes',
 1371: 'pencils',
 1372: 'butterflies',
 1373: 'rifle',
 1374: 'gummy bear',
 1375: 'dolphin',
 1376: 'walnut',
 1377: 'out',
 1378: 'crates',
 1379: 'olive oil',
 1380: 'shops',
 1381: 'hedges',
 1382: 'vegetable',
 1383: 'bubble',
 1384: 'bison',
 1385: 'window frame',
 1386: 'hippo',
 1387: 'sugar packets',
 1388: 'pepper shaker',
 1389: 'shuttle',
 1390: 'drawings',
 1391: 'tourist',
 1392: 'cinnamon roll',
 1393: 'christmas lights',
 1394: 'parmesan cheese',
 1395: 'brownies',
 1396: 'ravioli',
 1397: 'figurines',
 1398: 'baseball mitt',
 1399: 'bedding',
 1400: 'pretzels',
 1401: 'oak tree',
 1402: 'chopstick',
 1403: 'poodle',
 1404: 'farmer',
 1405: 'soldiers',
 1406: 'pasta salad',
 1407: 'spray bottle',
 1408: 'penguins',
 1409: 'pumpkins',
 1410: 'lounge',
 1411: 'fruit',
 1412: 'stars',
 1413: 'flames',
 1414: 'bar stools',
 1415: 'banana bunches',
 1416: 'apple logo',
 1417: 'fence post',
 1418: 'cheese cube',
 1419: 'caramel',
 1420: 'skin',
 1421: 'school',
 1422: 'rhinos',
 1423: 'sharks',
 1424: 'life jackets',
 1425: 'feathers',
 1426: 'whale',
 1427: 'sticks',
 1428: 'milk carton',
 1429: 'diaper',
 1430: 'masks',
 1431: 'entrance',
 1432: 'wig',
 1433: 'towel dispenser',
 1434: 'powder',
 1435: 'dinosaurs',
 1436: 'tags',
 1437: 'chalkboard',
 1438: 'nutella',
 1439: 'fisherman',
 1440: 'grinder',
 1441: 'tree branches',
 1442: 'flour',
 1443: 'snake',
 1444: 'banana bunch',
 1445: 'kimono',
 1446: 'mint',
 1447: 'toothbrushes',
 1448: 'customer',
 1449: 'backpacks',
 1450: 'cathedral',
 1451: 'sunflowers',
 1452: 'auditorium',
 1453: 'salt',
 1454: 'cookbook',
 1455: 'vinegar',
 1456: 'mustard bottle',
 1457: 'snakes',
 1458: 'wine bottles',
 1459: 'robot',
 1460: 'toothpicks',
 1461: 'lobby',
 1462: 'dinner',
 1463: 'hippos',
 1464: 'televisions',
 1465: 'students',
 1466: 'marina',
 1467: 'ahead',
 1468: 'garage door',
 1469: 'theater',
 1470: 'baker',
 1471: 'manhole cover',
 1472: 'gift',
 1473: 'parachute',
 1474: 'ear buds',
 1475: 'cotton candy',
 1476: 'kitchen towel',
 1477: 'swamp',
 1478: 'coin',
 1479: 'apartment building',
 1480: 'picnic tables',
 1481: 'cookie dough',
 1482: 'groceries',
 1483: 'pigs',
 1484: 'necklaces',
 1485: 'lizard',
 1486: 'obstacle',
 1487: 'cigar',
 1488: 'pandas',
 1489: 'leopard',
 1490: 'bartender',
 1491: 'village',
 1492: 'dream catcher',
 1493: 'pistachio',
 1494: 'sideways',
 1495: 'shopping center',
 1496: 'ketchup bottle',
 1497: 'baseball bats',
 1498: 'lily',
 1499: 'street lights',
 1500: 'egg carton',
 1501: 'buoys',
 1502: 'outside',
 1503: 'coconuts',
 1504: 'ornaments',
 1505: 'cages',
 1506: 'undershirt',
 1507: 'mountain peak',
 1508: 'pesto',
 1509: 'goal',
 1510: 'cranberry',
 1511: 'hearts',
 1512: 'bell tower',
 1513: 'owls',
 1514: 'salon',
 1515: 'paint brush',
 1516: 'dresses',
 1517: 'blossom',
 1518: 'beer mug',
 1519: 'cranberries',
 1520: 'athletic shoe',
 1521: 'kangaroo',
 1522: 'lipstick',
 1523: 'panda bears',
 1524: 'shaving cream',
 1525: 'cemetery',
 1526: 'shopper',
 1527: 'egg roll',
 1528: 'juice box',
 1529: 'coffee shop',
 1530: 'coffee beans',
 1531: 'ramekin',
 1532: 'blood',
 1533: 'storage box',
 1534: 'underwear',
 1535: 'beer cans',
 1536: 'taxis',
 1537: 'packages',
 1538: 'scooters',
 1539: 'waitress',
 1540: 'upwards',
 1541: 'waterfall',
 1542: 'coffee cups',
 1543: 'hair clip',
 1544: 'policemen',
 1545: 'batteries',
 1546: 'homes',
 1547: 'antennas',
 1548: 'riding boots',
 1549: 'in the mirror',
 1550: 'snow boots',
 1551: 'jewelry',
 1552: 'pizza tray',
 1553: 'sugar packet',
 1554: 'downward',
 1555: 'apartment',
 1556: 'bird house',
 1557: 'utensil holder',
 1558: 'dragons',
 1559: 'snoopy',
 1560: 'hairbrush',
 1561: 'cinnamon',
 1562: 'amusement park',
 1563: 'parachutes',
 1564: 'boar',
 1565: 'wedding',
 1566: 'supermarket',
 1567: 'tractors',
 1568: 'cookie jar',
 1569: 'blenders',
 1570: 'paper dispenser',
 1571: 'vegetables',
 1572: 'cafeteria',
 1573: 'armor',
 1574: 'towers',
 1575: 'waves',
 1576: 'pocket watch',
 1577: 'dolphins',
 1578: 'spray can',
 1579: 'nightstands',
 1580: 'elbow pad',
 1581: 'seat belt',
 1582: 'scrub brush',
 1583: 'tree leaves',
 1584: 'dragonfly',
 1585: 'mattresses',
 1586: 'appetizer',
 1587: 'dvd players',
 1588: 'swimmer',
 1589: 'stores',
 1590: 'toolbox',
 1591: 'lilies',
 1592: 'lab coat',
 1593: 'unknown'}

# Torch Hubからモデル読込
model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=True, return_postprocessor=False)
model_qa = model_qa.cuda()
model_qa.eval();

/usr/local/lib/python3.11/dist-packages/torch/hub.py:330: UserWarning: You are about to download and run code from an untrusted repository. In a future release, this won't be allowed. To add the repository to your trusted list, change the command to {calling_fn}(..., trust_repo=False) and a command prompt will appear asking for an explicit confirmation of trust, or load(..., trust_repo=True), which will assume that the prompt is to be answered with 'yes'. You can also use load(..., trust_repo='check') which will only prompt for confirmation if the repo is not already trusted. This will eventually be the default behaviour
  warnings.warn(
Downloading: "https://github.com/ashkamath/mdetr/zipball/main" to /root/.cache/torch/hub/main.zip
/usr/local/lib/python3.11/dist-packages/timm/models/_factory.py:126: UserWarning: Mapping deprecated model name tf_efficientnet_b5_ns to current tf_efficientnet_b5.ns_jft_in1k.
  model = create_fn(
/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(



model.safetensors:   0%|          | 0.00/122M [00:00<?, ?B/s]


WARNING:timm.models._builder:Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.



tokenizer_config.json:   0%|          | 0.00/25.0 [00:00<?, ?B/s]



vocab.json:   0%|          | 0.00/899k [00:00<?, ?B/s]



merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]



tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]



config.json:   0%|          | 0.00/481 [00:00<?, ?B/s]


Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
WARNING:huggingface_hub.file_download:Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`



model.safetensors:   0%|          | 0.00/499M [00:00<?, ?B/s]


Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Downloading: "https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth" to /root/.cache/torch/hub/checkpoints/gqa_EB5_checkpoint.pth
100%|██████████| 2.54G/2.54G [03:43<00:00, 12.2MB/s]



---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

<ipython-input-9-6faeb8f28838> in <cell line: 0>()
      1 # Torch Hubからモデル読込
----> 2 model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=True, return_postprocessor=False)
      3 model_qa = model_qa.cuda()
      4 model_qa.eval();


/usr/local/lib/python3.11/dist-packages/torch/hub.py in load(repo_or_dir, model, source, trust_repo, force_reload, verbose, skip_validation, *args, **kwargs)
    645         )
    646 
--> 647     model = _load_local(repo_or_dir, model, *args, **kwargs)
    648     return model
    649 


/usr/local/lib/python3.11/dist-packages/torch/hub.py in _load_local(hubconf_dir, model, *args, **kwargs)
    674 
    675         entry = _load_entry_from_hubconf(hub_module, model)
--> 676         model = entry(*args, **kwargs)
    677 
    678     return model


~/.cache/torch/hub/ashkamath_mdetr_main/hubconf.py in mdetr_efficientnetB5_gqa(pretrained, return_postprocessor)
    180             url="https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth", map_location="cpu", check_hash=True
    181         )
--> 182         model.load_state_dict(checkpoint["model"])
    183     if return_postprocessor:
    184         return model, PostProcess()


/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict, assign)
   2579 
   2580         if len(error_msgs) > 0:
-> 2581             raise RuntimeError(
   2582                 "Error(s) in loading state_dict for {}:\n\t{}".format(
   2583                     self.__class__.__name__, "\n\t".join(error_msgs)


RuntimeError: Error(s) in loading state_dict for MDETR:
	Unexpected key(s) in state_dict: "transformer.text_encoder.embeddings.position_ids".

この「Torch Hubからモデル読込」部分でエラーが発生。

エラーが発生、デバッグ

「Unexpected key(s) in state_dict: "transformer.text_encoder.embeddings.position_ids". 」とは、モデル定義と、学習済みパラメータのデータとの間で、整合が取れないもの(position_ids)がある模様。
以降で色々調べてみます。

Debug: モデル側の変数を見る

学習済みパラメータの読み込み無し「pretrained=False」で、モデル定義のみ読み込んで、モデル側の該当変数の値を見てみます。

model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=False, return_postprocessor=False)

Using cache found in /root/.cache/torch/hub/ashkamath_mdetr_main
WARNING:timm.models._builder:Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.
Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

model_qa.transformer.text_encoder.embeddings.position_ids

tensor([[  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
          14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,
          28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,
          42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,
          56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,
          70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,
          84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,
          98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
         112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
         126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,
         140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,
         154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,
         168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,
         182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,
         196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
         210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
         224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,
         238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
         252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
         266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,
         280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,
         294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,
         308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,
         322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,
         336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,
         350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,
         364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,
         378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,
         392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,
         406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,
         420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,
         434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,
         448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,
         462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,
         476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,
         490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,
         504, 505, 506, 507, 508, 509, 510, 511, 512, 513]])

(学習パラメータではない)「tensor」型で、既に値が入っていました。

学習済みパラメータを別で読み込み、該当変数を見ると、

state_dict = torch.hub.load_state_dict_from_url("https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth", map_location='cpu')

state_dict.keys()

dict_keys(['model', 'model_ema', 'optimizer', 'epoch', 'args'])

state_dict["model"].keys()

odict_keys(['transformer.encoder.layers.0.self_attn.in_proj_weight', 'transformer.encoder.layers.0.self_attn.in_proj_bias', 'transformer.encoder.layers.0.self_attn.out_proj.weight', 'transformer.encoder.layers.0.self_attn.out_proj.bias', 'transformer.encoder.layers.0.linear1.weight', 'transformer.encoder.layers.0.linear1.bias', 'transformer.encoder.layers.0.linear2.weight', 'transformer.encoder.layers.0.linear2.bias', 'transformer.encoder.layers.0.norm1.weight', 'transformer.encoder.layers.0.norm1.bias', 'transformer.encoder.layers.0.norm2.weight', 'transformer.encoder.layers.0.norm2.bias', 'transformer.encoder.layers.1.self_attn.in_proj_weight', 'transformer.encoder.layers.1.self_attn.in_proj_bias', 'transformer.encoder.layers.1.self_attn.out_proj.weight', 'transformer.encoder.layers.1.self_attn.out_proj.bias', 'transformer.encoder.layers.1.linear1.weight', 'transformer.encoder.layers.1.linear1.bias', 'transformer.encoder.layers.1.linear2.weight', 'transformer.encoder.layers.1.linear2.bias', 'transformer.encoder.layers.1.norm1.weight', 'transformer.encoder.layers.1.norm1.bias', 'transformer.encoder.layers.1.norm2.weight', 'transformer.encoder.layers.1.norm2.bias', 'transformer.encoder.layers.2.self_attn.in_proj_weight', 'transformer.encoder.layers.2.self_attn.in_proj_bias', 'transformer.encoder.layers.2.self_attn.out_proj.weight', 'transformer.encoder.layers.2.self_attn.out_proj.bias', 'transformer.encoder.layers.2.linear1.weight', 'transformer.encoder.layers.2.linear1.bias', 'transformer.encoder.layers.2.linear2.weight', 'transformer.encoder.layers.2.linear2.bias', 'transformer.encoder.layers.2.norm1.weight', 'transformer.encoder.layers.2.norm1.bias', 'transformer.encoder.layers.2.norm2.weight', 'transformer.encoder.layers.2.norm2.bias', 'transformer.encoder.layers.3.self_attn.in_proj_weight', 'transformer.encoder.layers.3.self_attn.in_proj_bias', 'transformer.encoder.layers.3.self_attn.out_proj.weight', 'transformer.encoder.layers.3.self_attn.out_proj.bias', 'transformer.encoder.layers.3.linear1.weight', 'transformer.encoder.layers.3.linear1.bias', 'transformer.encoder.layers.3.linear2.weight', 'transformer.encoder.layers.3.linear2.bias', 'transformer.encoder.layers.3.norm1.weight', 'transformer.encoder.layers.3.norm1.bias', 'transformer.encoder.layers.3.norm2.weight', 'transformer.encoder.layers.3.norm2.bias', 'transformer.encoder.layers.4.self_attn.in_proj_weight', 'transformer.encoder.layers.4.self_attn.in_proj_bias', 'transformer.encoder.layers.4.self_attn.out_proj.weight', 'transformer.encoder.layers.4.self_attn.out_proj.bias', 'transformer.encoder.layers.4.linear1.weight', 'transformer.encoder.layers.4.linear1.bias', 'transformer.encoder.layers.4.linear2.weight', 'transformer.encoder.layers.4.linear2.bias', 'transformer.encoder.layers.4.norm1.weight', 'transformer.encoder.layers.4.norm1.bias', 'transformer.encoder.layers.4.norm2.weight', 'transformer.encoder.layers.4.norm2.bias', 'transformer.encoder.layers.5.self_attn.in_proj_weight', 'transformer.encoder.layers.5.self_attn.in_proj_bias', 'transformer.encoder.layers.5.self_attn.out_proj.weight', 'transformer.encoder.layers.5.self_attn.out_proj.bias', 'transformer.encoder.layers.5.linear1.weight', 'transformer.encoder.layers.5.linear1.bias', 'transformer.encoder.layers.5.linear2.weight', 'transformer.encoder.layers.5.linear2.bias', 'transformer.encoder.layers.5.norm1.weight', 'transformer.encoder.layers.5.norm1.bias', 'transformer.encoder.layers.5.norm2.weight', 'transformer.encoder.layers.5.norm2.bias', 'transformer.decoder.layers.0.self_attn.in_proj_weight', 'transformer.decoder.layers.0.self_attn.in_proj_bias', 'transformer.decoder.layers.0.self_attn.out_proj.weight', 'transformer.decoder.layers.0.self_attn.out_proj.bias', 'transformer.decoder.layers.0.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.0.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.0.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.0.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.0.linear1.weight', 'transformer.decoder.layers.0.linear1.bias', 'transformer.decoder.layers.0.linear2.weight', 'transformer.decoder.layers.0.linear2.bias', 'transformer.decoder.layers.0.norm1.weight', 'transformer.decoder.layers.0.norm1.bias', 'transformer.decoder.layers.0.norm3.weight', 'transformer.decoder.layers.0.norm3.bias', 'transformer.decoder.layers.0.norm4.weight', 'transformer.decoder.layers.0.norm4.bias', 'transformer.decoder.layers.1.self_attn.in_proj_weight', 'transformer.decoder.layers.1.self_attn.in_proj_bias', 'transformer.decoder.layers.1.self_attn.out_proj.weight', 'transformer.decoder.layers.1.self_attn.out_proj.bias', 'transformer.decoder.layers.1.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.1.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.1.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.1.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.1.linear1.weight', 'transformer.decoder.layers.1.linear1.bias', 'transformer.decoder.layers.1.linear2.weight', 'transformer.decoder.layers.1.linear2.bias', 'transformer.decoder.layers.1.norm1.weight', 'transformer.decoder.layers.1.norm1.bias', 'transformer.decoder.layers.1.norm3.weight', 'transformer.decoder.layers.1.norm3.bias', 'transformer.decoder.layers.1.norm4.weight', 'transformer.decoder.layers.1.norm4.bias', 'transformer.decoder.layers.2.self_attn.in_proj_weight', 'transformer.decoder.layers.2.self_attn.in_proj_bias', 'transformer.decoder.layers.2.self_attn.out_proj.weight', 'transformer.decoder.layers.2.self_attn.out_proj.bias', 'transformer.decoder.layers.2.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.2.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.2.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.2.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.2.linear1.weight', 'transformer.decoder.layers.2.linear1.bias', 'transformer.decoder.layers.2.linear2.weight', 'transformer.decoder.layers.2.linear2.bias', 'transformer.decoder.layers.2.norm1.weight', 'transformer.decoder.layers.2.norm1.bias', 'transformer.decoder.layers.2.norm3.weight', 'transformer.decoder.layers.2.norm3.bias', 'transformer.decoder.layers.2.norm4.weight', 'transformer.decoder.layers.2.norm4.bias', 'transformer.decoder.layers.3.self_attn.in_proj_weight', 'transformer.decoder.layers.3.self_attn.in_proj_bias', 'transformer.decoder.layers.3.self_attn.out_proj.weight', 'transformer.decoder.layers.3.self_attn.out_proj.bias', 'transformer.decoder.layers.3.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.3.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.3.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.3.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.3.linear1.weight', 'transformer.decoder.layers.3.linear1.bias', 'transformer.decoder.layers.3.linear2.weight', 'transformer.decoder.layers.3.linear2.bias', 'transformer.decoder.layers.3.norm1.weight', 'transformer.decoder.layers.3.norm1.bias', 'transformer.decoder.layers.3.norm3.weight', 'transformer.decoder.layers.3.norm3.bias', 'transformer.decoder.layers.3.norm4.weight', 'transformer.decoder.layers.3.norm4.bias', 'transformer.decoder.layers.4.self_attn.in_proj_weight', 'transformer.decoder.layers.4.self_attn.in_proj_bias', 'transformer.decoder.layers.4.self_attn.out_proj.weight', 'transformer.decoder.layers.4.self_attn.out_proj.bias', 'transformer.decoder.layers.4.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.4.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.4.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.4.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.4.linear1.weight', 'transformer.decoder.layers.4.linear1.bias', 'transformer.decoder.layers.4.linear2.weight', 'transformer.decoder.layers.4.linear2.bias', 'transformer.decoder.layers.4.norm1.weight', 'transformer.decoder.layers.4.norm1.bias', 'transformer.decoder.layers.4.norm3.weight', 'transformer.decoder.layers.4.norm3.bias', 'transformer.decoder.layers.4.norm4.weight', 'transformer.decoder.layers.4.norm4.bias', 'transformer.decoder.layers.5.self_attn.in_proj_weight', 'transformer.decoder.layers.5.self_attn.in_proj_bias', 'transformer.decoder.layers.5.self_attn.out_proj.weight', 'transformer.decoder.layers.5.self_attn.out_proj.bias', 'transformer.decoder.layers.5.cross_attn_image.in_proj_weight', 'transformer.decoder.layers.5.cross_attn_image.in_proj_bias', 'transformer.decoder.layers.5.cross_attn_image.out_proj.weight', 'transformer.decoder.layers.5.cross_attn_image.out_proj.bias', 'transformer.decoder.layers.5.linear1.weight', 'transformer.decoder.layers.5.linear1.bias', 'transformer.decoder.layers.5.linear2.weight', 'transformer.decoder.layers.5.linear2.bias', 'transformer.decoder.layers.5.norm1.weight', 'transformer.decoder.layers.5.norm1.bias', 'transformer.decoder.layers.5.norm3.weight', 'transformer.decoder.layers.5.norm3.bias', 'transformer.decoder.layers.5.norm4.weight', 'transformer.decoder.layers.5.norm4.bias', 'transformer.decoder.norm.weight', 'transformer.decoder.norm.bias', 'transformer.text_encoder.embeddings.position_ids', 'transformer.text_encoder.embeddings.word_embeddings.weight', 'transformer.text_encoder.embeddings.position_embeddings.weight', 'transformer.text_encoder.embeddings.token_type_embeddings.weight', 'transformer.text_encoder.embeddings.LayerNorm.weight', 'transformer.text_encoder.embeddings.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.0.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.0.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.0.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.0.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.0.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.0.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.0.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.0.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.0.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.0.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.0.output.dense.weight', 'transformer.text_encoder.encoder.layer.0.output.dense.bias', 'transformer.text_encoder.encoder.layer.0.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.0.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.1.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.1.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.1.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.1.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.1.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.1.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.1.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.1.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.1.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.1.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.1.output.dense.weight', 'transformer.text_encoder.encoder.layer.1.output.dense.bias', 'transformer.text_encoder.encoder.layer.1.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.1.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.2.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.2.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.2.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.2.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.2.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.2.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.2.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.2.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.2.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.2.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.2.output.dense.weight', 'transformer.text_encoder.encoder.layer.2.output.dense.bias', 'transformer.text_encoder.encoder.layer.2.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.2.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.3.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.3.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.3.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.3.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.3.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.3.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.3.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.3.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.3.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.3.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.3.output.dense.weight', 'transformer.text_encoder.encoder.layer.3.output.dense.bias', 'transformer.text_encoder.encoder.layer.3.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.3.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.4.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.4.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.4.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.4.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.4.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.4.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.4.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.4.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.4.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.4.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.4.output.dense.weight', 'transformer.text_encoder.encoder.layer.4.output.dense.bias', 'transformer.text_encoder.encoder.layer.4.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.4.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.5.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.5.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.5.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.5.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.5.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.5.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.5.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.5.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.5.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.5.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.5.output.dense.weight', 'transformer.text_encoder.encoder.layer.5.output.dense.bias', 'transformer.text_encoder.encoder.layer.5.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.5.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.6.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.6.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.6.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.6.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.6.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.6.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.6.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.6.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.6.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.6.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.6.output.dense.weight', 'transformer.text_encoder.encoder.layer.6.output.dense.bias', 'transformer.text_encoder.encoder.layer.6.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.6.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.7.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.7.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.7.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.7.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.7.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.7.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.7.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.7.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.7.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.7.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.7.output.dense.weight', 'transformer.text_encoder.encoder.layer.7.output.dense.bias', 'transformer.text_encoder.encoder.layer.7.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.7.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.8.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.8.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.8.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.8.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.8.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.8.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.8.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.8.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.8.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.8.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.8.output.dense.weight', 'transformer.text_encoder.encoder.layer.8.output.dense.bias', 'transformer.text_encoder.encoder.layer.8.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.8.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.9.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.9.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.9.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.9.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.9.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.9.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.9.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.9.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.9.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.9.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.9.output.dense.weight', 'transformer.text_encoder.encoder.layer.9.output.dense.bias', 'transformer.text_encoder.encoder.layer.9.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.9.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.10.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.10.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.10.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.10.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.10.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.10.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.10.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.10.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.10.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.10.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.10.output.dense.weight', 'transformer.text_encoder.encoder.layer.10.output.dense.bias', 'transformer.text_encoder.encoder.layer.10.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.10.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.11.attention.self.query.weight', 'transformer.text_encoder.encoder.layer.11.attention.self.query.bias', 'transformer.text_encoder.encoder.layer.11.attention.self.key.weight', 'transformer.text_encoder.encoder.layer.11.attention.self.key.bias', 'transformer.text_encoder.encoder.layer.11.attention.self.value.weight', 'transformer.text_encoder.encoder.layer.11.attention.self.value.bias', 'transformer.text_encoder.encoder.layer.11.attention.output.dense.weight', 'transformer.text_encoder.encoder.layer.11.attention.output.dense.bias', 'transformer.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias', 'transformer.text_encoder.encoder.layer.11.intermediate.dense.weight', 'transformer.text_encoder.encoder.layer.11.intermediate.dense.bias', 'transformer.text_encoder.encoder.layer.11.output.dense.weight', 'transformer.text_encoder.encoder.layer.11.output.dense.bias', 'transformer.text_encoder.encoder.layer.11.output.LayerNorm.weight', 'transformer.text_encoder.encoder.layer.11.output.LayerNorm.bias', 'transformer.text_encoder.pooler.dense.weight', 'transformer.text_encoder.pooler.dense.bias', 'transformer.resizer.fc.weight', 'transformer.resizer.fc.bias', 'transformer.resizer.layer_norm.weight', 'transformer.resizer.layer_norm.bias', 'class_embed.weight', 'class_embed.bias', 'bbox_embed.layers.0.weight', 'bbox_embed.layers.0.bias', 'bbox_embed.layers.1.weight', 'bbox_embed.layers.1.bias', 'bbox_embed.layers.2.weight', 'bbox_embed.layers.2.bias', 'query_embed.weight', 'qa_embed.weight', 'input_proj.weight', 'input_proj.bias', 'backbone.0.body.conv_stem.weight', 'backbone.0.body.bn1.weight', 'backbone.0.body.bn1.bias', 'backbone.0.body.bn1.running_mean', 'backbone.0.body.bn1.running_var', 'backbone.0.body.blocks.0.0.conv_dw.weight', 'backbone.0.body.blocks.0.0.bn1.weight', 'backbone.0.body.blocks.0.0.bn1.bias', 'backbone.0.body.blocks.0.0.bn1.running_mean', 'backbone.0.body.blocks.0.0.bn1.running_var', 'backbone.0.body.blocks.0.0.se.conv_reduce.weight', 'backbone.0.body.blocks.0.0.se.conv_reduce.bias', 'backbone.0.body.blocks.0.0.se.conv_expand.weight', 'backbone.0.body.blocks.0.0.se.conv_expand.bias', 'backbone.0.body.blocks.0.0.conv_pw.weight', 'backbone.0.body.blocks.0.0.bn2.weight', 'backbone.0.body.blocks.0.0.bn2.bias', 'backbone.0.body.blocks.0.0.bn2.running_mean', 'backbone.0.body.blocks.0.0.bn2.running_var', 'backbone.0.body.blocks.0.1.conv_dw.weight', 'backbone.0.body.blocks.0.1.bn1.weight', 'backbone.0.body.blocks.0.1.bn1.bias', 'backbone.0.body.blocks.0.1.bn1.running_mean', 'backbone.0.body.blocks.0.1.bn1.running_var', 'backbone.0.body.blocks.0.1.se.conv_reduce.weight', 'backbone.0.body.blocks.0.1.se.conv_reduce.bias', 'backbone.0.body.blocks.0.1.se.conv_expand.weight', 'backbone.0.body.blocks.0.1.se.conv_expand.bias', 'backbone.0.body.blocks.0.1.conv_pw.weight', 'backbone.0.body.blocks.0.1.bn2.weight', 'backbone.0.body.blocks.0.1.bn2.bias', 'backbone.0.body.blocks.0.1.bn2.running_mean', 'backbone.0.body.blocks.0.1.bn2.running_var', 'backbone.0.body.blocks.0.2.conv_dw.weight', 'backbone.0.body.blocks.0.2.bn1.weight', 'backbone.0.body.blocks.0.2.bn1.bias', 'backbone.0.body.blocks.0.2.bn1.running_mean', 'backbone.0.body.blocks.0.2.bn1.running_var', 'backbone.0.body.blocks.0.2.se.conv_reduce.weight', 'backbone.0.body.blocks.0.2.se.conv_reduce.bias', 'backbone.0.body.blocks.0.2.se.conv_expand.weight', 'backbone.0.body.blocks.0.2.se.conv_expand.bias', 'backbone.0.body.blocks.0.2.conv_pw.weight', 'backbone.0.body.blocks.0.2.bn2.weight', 'backbone.0.body.blocks.0.2.bn2.bias', 'backbone.0.body.blocks.0.2.bn2.running_mean', 'backbone.0.body.blocks.0.2.bn2.running_var', 'backbone.0.body.blocks.1.0.conv_pw.weight', 'backbone.0.body.blocks.1.0.bn1.weight', 'backbone.0.body.blocks.1.0.bn1.bias', 'backbone.0.body.blocks.1.0.bn1.running_mean', 'backbone.0.body.blocks.1.0.bn1.running_var', 'backbone.0.body.blocks.1.0.conv_dw.weight', 'backbone.0.body.blocks.1.0.bn2.weight', 'backbone.0.body.blocks.1.0.bn2.bias', 'backbone.0.body.blocks.1.0.bn2.running_mean', 'backbone.0.body.blocks.1.0.bn2.running_var', 'backbone.0.body.blocks.1.0.se.conv_reduce.weight', 'backbone.0.body.blocks.1.0.se.conv_reduce.bias', 'backbone.0.body.blocks.1.0.se.conv_expand.weight', 'backbone.0.body.blocks.1.0.se.conv_expand.bias', 'backbone.0.body.blocks.1.0.conv_pwl.weight', 'backbone.0.body.blocks.1.0.bn3.weight', 'backbone.0.body.blocks.1.0.bn3.bias', 'backbone.0.body.blocks.1.0.bn3.running_mean', 'backbone.0.body.blocks.1.0.bn3.running_var', 'backbone.0.body.blocks.1.1.conv_pw.weight', 'backbone.0.body.blocks.1.1.bn1.weight', 'backbone.0.body.blocks.1.1.bn1.bias', 'backbone.0.body.blocks.1.1.bn1.running_mean', 'backbone.0.body.blocks.1.1.bn1.running_var', 'backbone.0.body.blocks.1.1.conv_dw.weight', 'backbone.0.body.blocks.1.1.bn2.weight', 'backbone.0.body.blocks.1.1.bn2.bias', 'backbone.0.body.blocks.1.1.bn2.running_mean', 'backbone.0.body.blocks.1.1.bn2.running_var', 'backbone.0.body.blocks.1.1.se.conv_reduce.weight', 'backbone.0.body.blocks.1.1.se.conv_reduce.bias', 'backbone.0.body.blocks.1.1.se.conv_expand.weight', 'backbone.0.body.blocks.1.1.se.conv_expand.bias', 'backbone.0.body.blocks.1.1.conv_pwl.weight', 'backbone.0.body.blocks.1.1.bn3.weight', 'backbone.0.body.blocks.1.1.bn3.bias', 'backbone.0.body.blocks.1.1.bn3.running_mean', 'backbone.0.body.blocks.1.1.bn3.running_var', 'backbone.0.body.blocks.1.2.conv_pw.weight', 'backbone.0.body.blocks.1.2.bn1.weight', 'backbone.0.body.blocks.1.2.bn1.bias', 'backbone.0.body.blocks.1.2.bn1.running_mean', 'backbone.0.body.blocks.1.2.bn1.running_var', 'backbone.0.body.blocks.1.2.conv_dw.weight', 'backbone.0.body.blocks.1.2.bn2.weight', 'backbone.0.body.blocks.1.2.bn2.bias', 'backbone.0.body.blocks.1.2.bn2.running_mean', 'backbone.0.body.blocks.1.2.bn2.running_var', 'backbone.0.body.blocks.1.2.se.conv_reduce.weight', 'backbone.0.body.blocks.1.2.se.conv_reduce.bias', 'backbone.0.body.blocks.1.2.se.conv_expand.weight', 'backbone.0.body.blocks.1.2.se.conv_expand.bias', 'backbone.0.body.blocks.1.2.conv_pwl.weight', 'backbone.0.body.blocks.1.2.bn3.weight', 'backbone.0.body.blocks.1.2.bn3.bias', 'backbone.0.body.blocks.1.2.bn3.running_mean', 'backbone.0.body.blocks.1.2.bn3.running_var', 'backbone.0.body.blocks.1.3.conv_pw.weight', 'backbone.0.body.blocks.1.3.bn1.weight', 'backbone.0.body.blocks.1.3.bn1.bias', 'backbone.0.body.blocks.1.3.bn1.running_mean', 'backbone.0.body.blocks.1.3.bn1.running_var', 'backbone.0.body.blocks.1.3.conv_dw.weight', 'backbone.0.body.blocks.1.3.bn2.weight', 'backbone.0.body.blocks.1.3.bn2.bias', 'backbone.0.body.blocks.1.3.bn2.running_mean', 'backbone.0.body.blocks.1.3.bn2.running_var', 'backbone.0.body.blocks.1.3.se.conv_reduce.weight', 'backbone.0.body.blocks.1.3.se.conv_reduce.bias', 'backbone.0.body.blocks.1.3.se.conv_expand.weight', 'backbone.0.body.blocks.1.3.se.conv_expand.bias', 'backbone.0.body.blocks.1.3.conv_pwl.weight', 'backbone.0.body.blocks.1.3.bn3.weight', 'backbone.0.body.blocks.1.3.bn3.bias', 'backbone.0.body.blocks.1.3.bn3.running_mean', 'backbone.0.body.blocks.1.3.bn3.running_var', 'backbone.0.body.blocks.1.4.conv_pw.weight', 'backbone.0.body.blocks.1.4.bn1.weight', 'backbone.0.body.blocks.1.4.bn1.bias', 'backbone.0.body.blocks.1.4.bn1.running_mean', 'backbone.0.body.blocks.1.4.bn1.running_var', 'backbone.0.body.blocks.1.4.conv_dw.weight', 'backbone.0.body.blocks.1.4.bn2.weight', 'backbone.0.body.blocks.1.4.bn2.bias', 'backbone.0.body.blocks.1.4.bn2.running_mean', 'backbone.0.body.blocks.1.4.bn2.running_var', 'backbone.0.body.blocks.1.4.se.conv_reduce.weight', 'backbone.0.body.blocks.1.4.se.conv_reduce.bias', 'backbone.0.body.blocks.1.4.se.conv_expand.weight', 'backbone.0.body.blocks.1.4.se.conv_expand.bias', 'backbone.0.body.blocks.1.4.conv_pwl.weight', 'backbone.0.body.blocks.1.4.bn3.weight', 'backbone.0.body.blocks.1.4.bn3.bias', 'backbone.0.body.blocks.1.4.bn3.running_mean', 'backbone.0.body.blocks.1.4.bn3.running_var', 'backbone.0.body.blocks.2.0.conv_pw.weight', 'backbone.0.body.blocks.2.0.bn1.weight', 'backbone.0.body.blocks.2.0.bn1.bias', 'backbone.0.body.blocks.2.0.bn1.running_mean', 'backbone.0.body.blocks.2.0.bn1.running_var', 'backbone.0.body.blocks.2.0.conv_dw.weight', 'backbone.0.body.blocks.2.0.bn2.weight', 'backbone.0.body.blocks.2.0.bn2.bias', 'backbone.0.body.blocks.2.0.bn2.running_mean', 'backbone.0.body.blocks.2.0.bn2.running_var', 'backbone.0.body.blocks.2.0.se.conv_reduce.weight', 'backbone.0.body.blocks.2.0.se.conv_reduce.bias', 'backbone.0.body.blocks.2.0.se.conv_expand.weight', 'backbone.0.body.blocks.2.0.se.conv_expand.bias', 'backbone.0.body.blocks.2.0.conv_pwl.weight', 'backbone.0.body.blocks.2.0.bn3.weight', 'backbone.0.body.blocks.2.0.bn3.bias', 'backbone.0.body.blocks.2.0.bn3.running_mean', 'backbone.0.body.blocks.2.0.bn3.running_var', 'backbone.0.body.blocks.2.1.conv_pw.weight', 'backbone.0.body.blocks.2.1.bn1.weight', 'backbone.0.body.blocks.2.1.bn1.bias', 'backbone.0.body.blocks.2.1.bn1.running_mean', 'backbone.0.body.blocks.2.1.bn1.running_var', 'backbone.0.body.blocks.2.1.conv_dw.weight', 'backbone.0.body.blocks.2.1.bn2.weight', 'backbone.0.body.blocks.2.1.bn2.bias', 'backbone.0.body.blocks.2.1.bn2.running_mean', 'backbone.0.body.blocks.2.1.bn2.running_var', 'backbone.0.body.blocks.2.1.se.conv_reduce.weight', 'backbone.0.body.blocks.2.1.se.conv_reduce.bias', 'backbone.0.body.blocks.2.1.se.conv_expand.weight', 'backbone.0.body.blocks.2.1.se.conv_expand.bias', 'backbone.0.body.blocks.2.1.conv_pwl.weight', 'backbone.0.body.blocks.2.1.bn3.weight', 'backbone.0.body.blocks.2.1.bn3.bias', 'backbone.0.body.blocks.2.1.bn3.running_mean', 'backbone.0.body.blocks.2.1.bn3.running_var', 'backbone.0.body.blocks.2.2.conv_pw.weight', 'backbone.0.body.blocks.2.2.bn1.weight', 'backbone.0.body.blocks.2.2.bn1.bias', 'backbone.0.body.blocks.2.2.bn1.running_mean', 'backbone.0.body.blocks.2.2.bn1.running_var', 'backbone.0.body.blocks.2.2.conv_dw.weight', 'backbone.0.body.blocks.2.2.bn2.weight', 'backbone.0.body.blocks.2.2.bn2.bias', 'backbone.0.body.blocks.2.2.bn2.running_mean', 'backbone.0.body.blocks.2.2.bn2.running_var', 'backbone.0.body.blocks.2.2.se.conv_reduce.weight', 'backbone.0.body.blocks.2.2.se.conv_reduce.bias', 'backbone.0.body.blocks.2.2.se.conv_expand.weight', 'backbone.0.body.blocks.2.2.se.conv_expand.bias', 'backbone.0.body.blocks.2.2.conv_pwl.weight', 'backbone.0.body.blocks.2.2.bn3.weight', 'backbone.0.body.blocks.2.2.bn3.bias', 'backbone.0.body.blocks.2.2.bn3.running_mean', 'backbone.0.body.blocks.2.2.bn3.running_var', 'backbone.0.body.blocks.2.3.conv_pw.weight', 'backbone.0.body.blocks.2.3.bn1.weight', 'backbone.0.body.blocks.2.3.bn1.bias', 'backbone.0.body.blocks.2.3.bn1.running_mean', 'backbone.0.body.blocks.2.3.bn1.running_var', 'backbone.0.body.blocks.2.3.conv_dw.weight', 'backbone.0.body.blocks.2.3.bn2.weight', 'backbone.0.body.blocks.2.3.bn2.bias', 'backbone.0.body.blocks.2.3.bn2.running_mean', 'backbone.0.body.blocks.2.3.bn2.running_var', 'backbone.0.body.blocks.2.3.se.conv_reduce.weight', 'backbone.0.body.blocks.2.3.se.conv_reduce.bias', 'backbone.0.body.blocks.2.3.se.conv_expand.weight', 'backbone.0.body.blocks.2.3.se.conv_expand.bias', 'backbone.0.body.blocks.2.3.conv_pwl.weight', 'backbone.0.body.blocks.2.3.bn3.weight', 'backbone.0.body.blocks.2.3.bn3.bias', 'backbone.0.body.blocks.2.3.bn3.running_mean', 'backbone.0.body.blocks.2.3.bn3.running_var', 'backbone.0.body.blocks.2.4.conv_pw.weight', 'backbone.0.body.blocks.2.4.bn1.weight', 'backbone.0.body.blocks.2.4.bn1.bias', 'backbone.0.body.blocks.2.4.bn1.running_mean', 'backbone.0.body.blocks.2.4.bn1.running_var', 'backbone.0.body.blocks.2.4.conv_dw.weight', 'backbone.0.body.blocks.2.4.bn2.weight', 'backbone.0.body.blocks.2.4.bn2.bias', 'backbone.0.body.blocks.2.4.bn2.running_mean', 'backbone.0.body.blocks.2.4.bn2.running_var', 'backbone.0.body.blocks.2.4.se.conv_reduce.weight', 'backbone.0.body.blocks.2.4.se.conv_reduce.bias', 'backbone.0.body.blocks.2.4.se.conv_expand.weight', 'backbone.0.body.blocks.2.4.se.conv_expand.bias', 'backbone.0.body.blocks.2.4.conv_pwl.weight', 'backbone.0.body.blocks.2.4.bn3.weight', 'backbone.0.body.blocks.2.4.bn3.bias', 'backbone.0.body.blocks.2.4.bn3.running_mean', 'backbone.0.body.blocks.2.4.bn3.running_var', 'backbone.0.body.blocks.3.0.conv_pw.weight', 'backbone.0.body.blocks.3.0.bn1.weight', 'backbone.0.body.blocks.3.0.bn1.bias', 'backbone.0.body.blocks.3.0.bn1.running_mean', 'backbone.0.body.blocks.3.0.bn1.running_var', 'backbone.0.body.blocks.3.0.conv_dw.weight', 'backbone.0.body.blocks.3.0.bn2.weight', 'backbone.0.body.blocks.3.0.bn2.bias', 'backbone.0.body.blocks.3.0.bn2.running_mean', 'backbone.0.body.blocks.3.0.bn2.running_var', 'backbone.0.body.blocks.3.0.se.conv_reduce.weight', 'backbone.0.body.blocks.3.0.se.conv_reduce.bias', 'backbone.0.body.blocks.3.0.se.conv_expand.weight', 'backbone.0.body.blocks.3.0.se.conv_expand.bias', 'backbone.0.body.blocks.3.0.conv_pwl.weight', 'backbone.0.body.blocks.3.0.bn3.weight', 'backbone.0.body.blocks.3.0.bn3.bias', 'backbone.0.body.blocks.3.0.bn3.running_mean', 'backbone.0.body.blocks.3.0.bn3.running_var', 'backbone.0.body.blocks.3.1.conv_pw.weight', 'backbone.0.body.blocks.3.1.bn1.weight', 'backbone.0.body.blocks.3.1.bn1.bias', 'backbone.0.body.blocks.3.1.bn1.running_mean', 'backbone.0.body.blocks.3.1.bn1.running_var', 'backbone.0.body.blocks.3.1.conv_dw.weight', 'backbone.0.body.blocks.3.1.bn2.weight', 'backbone.0.body.blocks.3.1.bn2.bias', 'backbone.0.body.blocks.3.1.bn2.running_mean', 'backbone.0.body.blocks.3.1.bn2.running_var', 'backbone.0.body.blocks.3.1.se.conv_reduce.weight', 'backbone.0.body.blocks.3.1.se.conv_reduce.bias', 'backbone.0.body.blocks.3.1.se.conv_expand.weight', 'backbone.0.body.blocks.3.1.se.conv_expand.bias', 'backbone.0.body.blocks.3.1.conv_pwl.weight', 'backbone.0.body.blocks.3.1.bn3.weight', 'backbone.0.body.blocks.3.1.bn3.bias', 'backbone.0.body.blocks.3.1.bn3.running_mean', 'backbone.0.body.blocks.3.1.bn3.running_var', 'backbone.0.body.blocks.3.2.conv_pw.weight', 'backbone.0.body.blocks.3.2.bn1.weight', 'backbone.0.body.blocks.3.2.bn1.bias', 'backbone.0.body.blocks.3.2.bn1.running_mean', 'backbone.0.body.blocks.3.2.bn1.running_var', 'backbone.0.body.blocks.3.2.conv_dw.weight', 'backbone.0.body.blocks.3.2.bn2.weight', 'backbone.0.body.blocks.3.2.bn2.bias', 'backbone.0.body.blocks.3.2.bn2.running_mean', 'backbone.0.body.blocks.3.2.bn2.running_var', 'backbone.0.body.blocks.3.2.se.conv_reduce.weight', 'backbone.0.body.blocks.3.2.se.conv_reduce.bias', 'backbone.0.body.blocks.3.2.se.conv_expand.weight', 'backbone.0.body.blocks.3.2.se.conv_expand.bias', 'backbone.0.body.blocks.3.2.conv_pwl.weight', 'backbone.0.body.blocks.3.2.bn3.weight', 'backbone.0.body.blocks.3.2.bn3.bias', 'backbone.0.body.blocks.3.2.bn3.running_mean', 'backbone.0.body.blocks.3.2.bn3.running_var', 'backbone.0.body.blocks.3.3.conv_pw.weight', 'backbone.0.body.blocks.3.3.bn1.weight', 'backbone.0.body.blocks.3.3.bn1.bias', 'backbone.0.body.blocks.3.3.bn1.running_mean', 'backbone.0.body.blocks.3.3.bn1.running_var', 'backbone.0.body.blocks.3.3.conv_dw.weight', 'backbone.0.body.blocks.3.3.bn2.weight', 'backbone.0.body.blocks.3.3.bn2.bias', 'backbone.0.body.blocks.3.3.bn2.running_mean', 'backbone.0.body.blocks.3.3.bn2.running_var', 'backbone.0.body.blocks.3.3.se.conv_reduce.weight', 'backbone.0.body.blocks.3.3.se.conv_reduce.bias', 'backbone.0.body.blocks.3.3.se.conv_expand.weight', 'backbone.0.body.blocks.3.3.se.conv_expand.bias', 'backbone.0.body.blocks.3.3.conv_pwl.weight', 'backbone.0.body.blocks.3.3.bn3.weight', 'backbone.0.body.blocks.3.3.bn3.bias', 'backbone.0.body.blocks.3.3.bn3.running_mean', 'backbone.0.body.blocks.3.3.bn3.running_var', 'backbone.0.body.blocks.3.4.conv_pw.weight', 'backbone.0.body.blocks.3.4.bn1.weight', 'backbone.0.body.blocks.3.4.bn1.bias', 'backbone.0.body.blocks.3.4.bn1.running_mean', 'backbone.0.body.blocks.3.4.bn1.running_var', 'backbone.0.body.blocks.3.4.conv_dw.weight', 'backbone.0.body.blocks.3.4.bn2.weight', 'backbone.0.body.blocks.3.4.bn2.bias', 'backbone.0.body.blocks.3.4.bn2.running_mean', 'backbone.0.body.blocks.3.4.bn2.running_var', 'backbone.0.body.blocks.3.4.se.conv_reduce.weight', 'backbone.0.body.blocks.3.4.se.conv_reduce.bias', 'backbone.0.body.blocks.3.4.se.conv_expand.weight', 'backbone.0.body.blocks.3.4.se.conv_expand.bias', 'backbone.0.body.blocks.3.4.conv_pwl.weight', 'backbone.0.body.blocks.3.4.bn3.weight', 'backbone.0.body.blocks.3.4.bn3.bias', 'backbone.0.body.blocks.3.4.bn3.running_mean', 'backbone.0.body.blocks.3.4.bn3.running_var', 'backbone.0.body.blocks.3.5.conv_pw.weight', 'backbone.0.body.blocks.3.5.bn1.weight', 'backbone.0.body.blocks.3.5.bn1.bias', 'backbone.0.body.blocks.3.5.bn1.running_mean', 'backbone.0.body.blocks.3.5.bn1.running_var', 'backbone.0.body.blocks.3.5.conv_dw.weight', 'backbone.0.body.blocks.3.5.bn2.weight', 'backbone.0.body.blocks.3.5.bn2.bias', 'backbone.0.body.blocks.3.5.bn2.running_mean', 'backbone.0.body.blocks.3.5.bn2.running_var', 'backbone.0.body.blocks.3.5.se.conv_reduce.weight', 'backbone.0.body.blocks.3.5.se.conv_reduce.bias', 'backbone.0.body.blocks.3.5.se.conv_expand.weight', 'backbone.0.body.blocks.3.5.se.conv_expand.bias', 'backbone.0.body.blocks.3.5.conv_pwl.weight', 'backbone.0.body.blocks.3.5.bn3.weight', 'backbone.0.body.blocks.3.5.bn3.bias', 'backbone.0.body.blocks.3.5.bn3.running_mean', 'backbone.0.body.blocks.3.5.bn3.running_var', 'backbone.0.body.blocks.3.6.conv_pw.weight', 'backbone.0.body.blocks.3.6.bn1.weight', 'backbone.0.body.blocks.3.6.bn1.bias', 'backbone.0.body.blocks.3.6.bn1.running_mean', 'backbone.0.body.blocks.3.6.bn1.running_var', 'backbone.0.body.blocks.3.6.conv_dw.weight', 'backbone.0.body.blocks.3.6.bn2.weight', 'backbone.0.body.blocks.3.6.bn2.bias', 'backbone.0.body.blocks.3.6.bn2.running_mean', 'backbone.0.body.blocks.3.6.bn2.running_var', 'backbone.0.body.blocks.3.6.se.conv_reduce.weight', 'backbone.0.body.blocks.3.6.se.conv_reduce.bias', 'backbone.0.body.blocks.3.6.se.conv_expand.weight', 'backbone.0.body.blocks.3.6.se.conv_expand.bias', 'backbone.0.body.blocks.3.6.conv_pwl.weight', 'backbone.0.body.blocks.3.6.bn3.weight', 'backbone.0.body.blocks.3.6.bn3.bias', 'backbone.0.body.blocks.3.6.bn3.running_mean', 'backbone.0.body.blocks.3.6.bn3.running_var', 'backbone.0.body.blocks.4.0.conv_pw.weight', 'backbone.0.body.blocks.4.0.bn1.weight', 'backbone.0.body.blocks.4.0.bn1.bias', 'backbone.0.body.blocks.4.0.bn1.running_mean', 'backbone.0.body.blocks.4.0.bn1.running_var', 'backbone.0.body.blocks.4.0.conv_dw.weight', 'backbone.0.body.blocks.4.0.bn2.weight', 'backbone.0.body.blocks.4.0.bn2.bias', 'backbone.0.body.blocks.4.0.bn2.running_mean', 'backbone.0.body.blocks.4.0.bn2.running_var', 'backbone.0.body.blocks.4.0.se.conv_reduce.weight', 'backbone.0.body.blocks.4.0.se.conv_reduce.bias', 'backbone.0.body.blocks.4.0.se.conv_expand.weight', 'backbone.0.body.blocks.4.0.se.conv_expand.bias', 'backbone.0.body.blocks.4.0.conv_pwl.weight', 'backbone.0.body.blocks.4.0.bn3.weight', 'backbone.0.body.blocks.4.0.bn3.bias', 'backbone.0.body.blocks.4.0.bn3.running_mean', 'backbone.0.body.blocks.4.0.bn3.running_var', 'backbone.0.body.blocks.4.1.conv_pw.weight', 'backbone.0.body.blocks.4.1.bn1.weight', 'backbone.0.body.blocks.4.1.bn1.bias', 'backbone.0.body.blocks.4.1.bn1.running_mean', 'backbone.0.body.blocks.4.1.bn1.running_var', 'backbone.0.body.blocks.4.1.conv_dw.weight', 'backbone.0.body.blocks.4.1.bn2.weight', 'backbone.0.body.blocks.4.1.bn2.bias', 'backbone.0.body.blocks.4.1.bn2.running_mean', 'backbone.0.body.blocks.4.1.bn2.running_var', 'backbone.0.body.blocks.4.1.se.conv_reduce.weight', 'backbone.0.body.blocks.4.1.se.conv_reduce.bias', 'backbone.0.body.blocks.4.1.se.conv_expand.weight', 'backbone.0.body.blocks.4.1.se.conv_expand.bias', 'backbone.0.body.blocks.4.1.conv_pwl.weight', 'backbone.0.body.blocks.4.1.bn3.weight', 'backbone.0.body.blocks.4.1.bn3.bias', 'backbone.0.body.blocks.4.1.bn3.running_mean', 'backbone.0.body.blocks.4.1.bn3.running_var', 'backbone.0.body.blocks.4.2.conv_pw.weight', 'backbone.0.body.blocks.4.2.bn1.weight', 'backbone.0.body.blocks.4.2.bn1.bias', 'backbone.0.body.blocks.4.2.bn1.running_mean', 'backbone.0.body.blocks.4.2.bn1.running_var', 'backbone.0.body.blocks.4.2.conv_dw.weight', 'backbone.0.body.blocks.4.2.bn2.weight', 'backbone.0.body.blocks.4.2.bn2.bias', 'backbone.0.body.blocks.4.2.bn2.running_mean', 'backbone.0.body.blocks.4.2.bn2.running_var', 'backbone.0.body.blocks.4.2.se.conv_reduce.weight', 'backbone.0.body.blocks.4.2.se.conv_reduce.bias', 'backbone.0.body.blocks.4.2.se.conv_expand.weight', 'backbone.0.body.blocks.4.2.se.conv_expand.bias', 'backbone.0.body.blocks.4.2.conv_pwl.weight', 'backbone.0.body.blocks.4.2.bn3.weight', 'backbone.0.body.blocks.4.2.bn3.bias', 'backbone.0.body.blocks.4.2.bn3.running_mean', 'backbone.0.body.blocks.4.2.bn3.running_var', 'backbone.0.body.blocks.4.3.conv_pw.weight', 'backbone.0.body.blocks.4.3.bn1.weight', 'backbone.0.body.blocks.4.3.bn1.bias', 'backbone.0.body.blocks.4.3.bn1.running_mean', 'backbone.0.body.blocks.4.3.bn1.running_var', 'backbone.0.body.blocks.4.3.conv_dw.weight', 'backbone.0.body.blocks.4.3.bn2.weight', 'backbone.0.body.blocks.4.3.bn2.bias', 'backbone.0.body.blocks.4.3.bn2.running_mean', 'backbone.0.body.blocks.4.3.bn2.running_var', 'backbone.0.body.blocks.4.3.se.conv_reduce.weight', 'backbone.0.body.blocks.4.3.se.conv_reduce.bias', 'backbone.0.body.blocks.4.3.se.conv_expand.weight', 'backbone.0.body.blocks.4.3.se.conv_expand.bias', 'backbone.0.body.blocks.4.3.conv_pwl.weight', 'backbone.0.body.blocks.4.3.bn3.weight', 'backbone.0.body.blocks.4.3.bn3.bias', 'backbone.0.body.blocks.4.3.bn3.running_mean', 'backbone.0.body.blocks.4.3.bn3.running_var', 'backbone.0.body.blocks.4.4.conv_pw.weight', 'backbone.0.body.blocks.4.4.bn1.weight', 'backbone.0.body.blocks.4.4.bn1.bias', 'backbone.0.body.blocks.4.4.bn1.running_mean', 'backbone.0.body.blocks.4.4.bn1.running_var', 'backbone.0.body.blocks.4.4.conv_dw.weight', 'backbone.0.body.blocks.4.4.bn2.weight', 'backbone.0.body.blocks.4.4.bn2.bias', 'backbone.0.body.blocks.4.4.bn2.running_mean', 'backbone.0.body.blocks.4.4.bn2.running_var', 'backbone.0.body.blocks.4.4.se.conv_reduce.weight', 'backbone.0.body.blocks.4.4.se.conv_reduce.bias', 'backbone.0.body.blocks.4.4.se.conv_expand.weight', 'backbone.0.body.blocks.4.4.se.conv_expand.bias', 'backbone.0.body.blocks.4.4.conv_pwl.weight', 'backbone.0.body.blocks.4.4.bn3.weight', 'backbone.0.body.blocks.4.4.bn3.bias', 'backbone.0.body.blocks.4.4.bn3.running_mean', 'backbone.0.body.blocks.4.4.bn3.running_var', 'backbone.0.body.blocks.4.5.conv_pw.weight', 'backbone.0.body.blocks.4.5.bn1.weight', 'backbone.0.body.blocks.4.5.bn1.bias', 'backbone.0.body.blocks.4.5.bn1.running_mean', 'backbone.0.body.blocks.4.5.bn1.running_var', 'backbone.0.body.blocks.4.5.conv_dw.weight', 'backbone.0.body.blocks.4.5.bn2.weight', 'backbone.0.body.blocks.4.5.bn2.bias', 'backbone.0.body.blocks.4.5.bn2.running_mean', 'backbone.0.body.blocks.4.5.bn2.running_var', 'backbone.0.body.blocks.4.5.se.conv_reduce.weight', 'backbone.0.body.blocks.4.5.se.conv_reduce.bias', 'backbone.0.body.blocks.4.5.se.conv_expand.weight', 'backbone.0.body.blocks.4.5.se.conv_expand.bias', 'backbone.0.body.blocks.4.5.conv_pwl.weight', 'backbone.0.body.blocks.4.5.bn3.weight', 'backbone.0.body.blocks.4.5.bn3.bias', 'backbone.0.body.blocks.4.5.bn3.running_mean', 'backbone.0.body.blocks.4.5.bn3.running_var', 'backbone.0.body.blocks.4.6.conv_pw.weight', 'backbone.0.body.blocks.4.6.bn1.weight', 'backbone.0.body.blocks.4.6.bn1.bias', 'backbone.0.body.blocks.4.6.bn1.running_mean', 'backbone.0.body.blocks.4.6.bn1.running_var', 'backbone.0.body.blocks.4.6.conv_dw.weight', 'backbone.0.body.blocks.4.6.bn2.weight', 'backbone.0.body.blocks.4.6.bn2.bias', 'backbone.0.body.blocks.4.6.bn2.running_mean', 'backbone.0.body.blocks.4.6.bn2.running_var', 'backbone.0.body.blocks.4.6.se.conv_reduce.weight', 'backbone.0.body.blocks.4.6.se.conv_reduce.bias', 'backbone.0.body.blocks.4.6.se.conv_expand.weight', 'backbone.0.body.blocks.4.6.se.conv_expand.bias', 'backbone.0.body.blocks.4.6.conv_pwl.weight', 'backbone.0.body.blocks.4.6.bn3.weight', 'backbone.0.body.blocks.4.6.bn3.bias', 'backbone.0.body.blocks.4.6.bn3.running_mean', 'backbone.0.body.blocks.4.6.bn3.running_var', 'backbone.0.body.blocks.5.0.conv_pw.weight', 'backbone.0.body.blocks.5.0.bn1.weight', 'backbone.0.body.blocks.5.0.bn1.bias', 'backbone.0.body.blocks.5.0.bn1.running_mean', 'backbone.0.body.blocks.5.0.bn1.running_var', 'backbone.0.body.blocks.5.0.conv_dw.weight', 'backbone.0.body.blocks.5.0.bn2.weight', 'backbone.0.body.blocks.5.0.bn2.bias', 'backbone.0.body.blocks.5.0.bn2.running_mean', 'backbone.0.body.blocks.5.0.bn2.running_var', 'backbone.0.body.blocks.5.0.se.conv_reduce.weight', 'backbone.0.body.blocks.5.0.se.conv_reduce.bias', 'backbone.0.body.blocks.5.0.se.conv_expand.weight', 'backbone.0.body.blocks.5.0.se.conv_expand.bias', 'backbone.0.body.blocks.5.0.conv_pwl.weight', 'backbone.0.body.blocks.5.0.bn3.weight', 'backbone.0.body.blocks.5.0.bn3.bias', 'backbone.0.body.blocks.5.0.bn3.running_mean', 'backbone.0.body.blocks.5.0.bn3.running_var', 'backbone.0.body.blocks.5.1.conv_pw.weight', 'backbone.0.body.blocks.5.1.bn1.weight', 'backbone.0.body.blocks.5.1.bn1.bias', 'backbone.0.body.blocks.5.1.bn1.running_mean', 'backbone.0.body.blocks.5.1.bn1.running_var', 'backbone.0.body.blocks.5.1.conv_dw.weight', 'backbone.0.body.blocks.5.1.bn2.weight', 'backbone.0.body.blocks.5.1.bn2.bias', 'backbone.0.body.blocks.5.1.bn2.running_mean', 'backbone.0.body.blocks.5.1.bn2.running_var', 'backbone.0.body.blocks.5.1.se.conv_reduce.weight', 'backbone.0.body.blocks.5.1.se.conv_reduce.bias', 'backbone.0.body.blocks.5.1.se.conv_expand.weight', 'backbone.0.body.blocks.5.1.se.conv_expand.bias', 'backbone.0.body.blocks.5.1.conv_pwl.weight', 'backbone.0.body.blocks.5.1.bn3.weight', 'backbone.0.body.blocks.5.1.bn3.bias', 'backbone.0.body.blocks.5.1.bn3.running_mean', 'backbone.0.body.blocks.5.1.bn3.running_var', 'backbone.0.body.blocks.5.2.conv_pw.weight', 'backbone.0.body.blocks.5.2.bn1.weight', 'backbone.0.body.blocks.5.2.bn1.bias', 'backbone.0.body.blocks.5.2.bn1.running_mean', 'backbone.0.body.blocks.5.2.bn1.running_var', 'backbone.0.body.blocks.5.2.conv_dw.weight', 'backbone.0.body.blocks.5.2.bn2.weight', 'backbone.0.body.blocks.5.2.bn2.bias', 'backbone.0.body.blocks.5.2.bn2.running_mean', 'backbone.0.body.blocks.5.2.bn2.running_var', 'backbone.0.body.blocks.5.2.se.conv_reduce.weight', 'backbone.0.body.blocks.5.2.se.conv_reduce.bias', 'backbone.0.body.blocks.5.2.se.conv_expand.weight', 'backbone.0.body.blocks.5.2.se.conv_expand.bias', 'backbone.0.body.blocks.5.2.conv_pwl.weight', 'backbone.0.body.blocks.5.2.bn3.weight', 'backbone.0.body.blocks.5.2.bn3.bias', 'backbone.0.body.blocks.5.2.bn3.running_mean', 'backbone.0.body.blocks.5.2.bn3.running_var', 'backbone.0.body.blocks.5.3.conv_pw.weight', 'backbone.0.body.blocks.5.3.bn1.weight', 'backbone.0.body.blocks.5.3.bn1.bias', 'backbone.0.body.blocks.5.3.bn1.running_mean', 'backbone.0.body.blocks.5.3.bn1.running_var', 'backbone.0.body.blocks.5.3.conv_dw.weight', 'backbone.0.body.blocks.5.3.bn2.weight', 'backbone.0.body.blocks.5.3.bn2.bias', 'backbone.0.body.blocks.5.3.bn2.running_mean', 'backbone.0.body.blocks.5.3.bn2.running_var', 'backbone.0.body.blocks.5.3.se.conv_reduce.weight', 'backbone.0.body.blocks.5.3.se.conv_reduce.bias', 'backbone.0.body.blocks.5.3.se.conv_expand.weight', 'backbone.0.body.blocks.5.3.se.conv_expand.bias', 'backbone.0.body.blocks.5.3.conv_pwl.weight', 'backbone.0.body.blocks.5.3.bn3.weight', 'backbone.0.body.blocks.5.3.bn3.bias', 'backbone.0.body.blocks.5.3.bn3.running_mean', 'backbone.0.body.blocks.5.3.bn3.running_var', 'backbone.0.body.blocks.5.4.conv_pw.weight', 'backbone.0.body.blocks.5.4.bn1.weight', 'backbone.0.body.blocks.5.4.bn1.bias', 'backbone.0.body.blocks.5.4.bn1.running_mean', 'backbone.0.body.blocks.5.4.bn1.running_var', 'backbone.0.body.blocks.5.4.conv_dw.weight', 'backbone.0.body.blocks.5.4.bn2.weight', 'backbone.0.body.blocks.5.4.bn2.bias', 'backbone.0.body.blocks.5.4.bn2.running_mean', 'backbone.0.body.blocks.5.4.bn2.running_var', 'backbone.0.body.blocks.5.4.se.conv_reduce.weight', 'backbone.0.body.blocks.5.4.se.conv_reduce.bias', 'backbone.0.body.blocks.5.4.se.conv_expand.weight', 'backbone.0.body.blocks.5.4.se.conv_expand.bias', 'backbone.0.body.blocks.5.4.conv_pwl.weight', 'backbone.0.body.blocks.5.4.bn3.weight', 'backbone.0.body.blocks.5.4.bn3.bias', 'backbone.0.body.blocks.5.4.bn3.running_mean', 'backbone.0.body.blocks.5.4.bn3.running_var', 'backbone.0.body.blocks.5.5.conv_pw.weight', 'backbone.0.body.blocks.5.5.bn1.weight', 'backbone.0.body.blocks.5.5.bn1.bias', 'backbone.0.body.blocks.5.5.bn1.running_mean', 'backbone.0.body.blocks.5.5.bn1.running_var', 'backbone.0.body.blocks.5.5.conv_dw.weight', 'backbone.0.body.blocks.5.5.bn2.weight', 'backbone.0.body.blocks.5.5.bn2.bias', 'backbone.0.body.blocks.5.5.bn2.running_mean', 'backbone.0.body.blocks.5.5.bn2.running_var', 'backbone.0.body.blocks.5.5.se.conv_reduce.weight', 'backbone.0.body.blocks.5.5.se.conv_reduce.bias', 'backbone.0.body.blocks.5.5.se.conv_expand.weight', 'backbone.0.body.blocks.5.5.se.conv_expand.bias', 'backbone.0.body.blocks.5.5.conv_pwl.weight', 'backbone.0.body.blocks.5.5.bn3.weight', 'backbone.0.body.blocks.5.5.bn3.bias', 'backbone.0.body.blocks.5.5.bn3.running_mean', 'backbone.0.body.blocks.5.5.bn3.running_var', 'backbone.0.body.blocks.5.6.conv_pw.weight', 'backbone.0.body.blocks.5.6.bn1.weight', 'backbone.0.body.blocks.5.6.bn1.bias', 'backbone.0.body.blocks.5.6.bn1.running_mean', 'backbone.0.body.blocks.5.6.bn1.running_var', 'backbone.0.body.blocks.5.6.conv_dw.weight', 'backbone.0.body.blocks.5.6.bn2.weight', 'backbone.0.body.blocks.5.6.bn2.bias', 'backbone.0.body.blocks.5.6.bn2.running_mean', 'backbone.0.body.blocks.5.6.bn2.running_var', 'backbone.0.body.blocks.5.6.se.conv_reduce.weight', 'backbone.0.body.blocks.5.6.se.conv_reduce.bias', 'backbone.0.body.blocks.5.6.se.conv_expand.weight', 'backbone.0.body.blocks.5.6.se.conv_expand.bias', 'backbone.0.body.blocks.5.6.conv_pwl.weight', 'backbone.0.body.blocks.5.6.bn3.weight', 'backbone.0.body.blocks.5.6.bn3.bias', 'backbone.0.body.blocks.5.6.bn3.running_mean', 'backbone.0.body.blocks.5.6.bn3.running_var', 'backbone.0.body.blocks.5.7.conv_pw.weight', 'backbone.0.body.blocks.5.7.bn1.weight', 'backbone.0.body.blocks.5.7.bn1.bias', 'backbone.0.body.blocks.5.7.bn1.running_mean', 'backbone.0.body.blocks.5.7.bn1.running_var', 'backbone.0.body.blocks.5.7.conv_dw.weight', 'backbone.0.body.blocks.5.7.bn2.weight', 'backbone.0.body.blocks.5.7.bn2.bias', 'backbone.0.body.blocks.5.7.bn2.running_mean', 'backbone.0.body.blocks.5.7.bn2.running_var', 'backbone.0.body.blocks.5.7.se.conv_reduce.weight', 'backbone.0.body.blocks.5.7.se.conv_reduce.bias', 'backbone.0.body.blocks.5.7.se.conv_expand.weight', 'backbone.0.body.blocks.5.7.se.conv_expand.bias', 'backbone.0.body.blocks.5.7.conv_pwl.weight', 'backbone.0.body.blocks.5.7.bn3.weight', 'backbone.0.body.blocks.5.7.bn3.bias', 'backbone.0.body.blocks.5.7.bn3.running_mean', 'backbone.0.body.blocks.5.7.bn3.running_var', 'backbone.0.body.blocks.5.8.conv_pw.weight', 'backbone.0.body.blocks.5.8.bn1.weight', 'backbone.0.body.blocks.5.8.bn1.bias', 'backbone.0.body.blocks.5.8.bn1.running_mean', 'backbone.0.body.blocks.5.8.bn1.running_var', 'backbone.0.body.blocks.5.8.conv_dw.weight', 'backbone.0.body.blocks.5.8.bn2.weight', 'backbone.0.body.blocks.5.8.bn2.bias', 'backbone.0.body.blocks.5.8.bn2.running_mean', 'backbone.0.body.blocks.5.8.bn2.running_var', 'backbone.0.body.blocks.5.8.se.conv_reduce.weight', 'backbone.0.body.blocks.5.8.se.conv_reduce.bias', 'backbone.0.body.blocks.5.8.se.conv_expand.weight', 'backbone.0.body.blocks.5.8.se.conv_expand.bias', 'backbone.0.body.blocks.5.8.conv_pwl.weight', 'backbone.0.body.blocks.5.8.bn3.weight', 'backbone.0.body.blocks.5.8.bn3.bias', 'backbone.0.body.blocks.5.8.bn3.running_mean', 'backbone.0.body.blocks.5.8.bn3.running_var', 'backbone.0.body.blocks.6.0.conv_pw.weight', 'backbone.0.body.blocks.6.0.bn1.weight', 'backbone.0.body.blocks.6.0.bn1.bias', 'backbone.0.body.blocks.6.0.bn1.running_mean', 'backbone.0.body.blocks.6.0.bn1.running_var', 'backbone.0.body.blocks.6.0.conv_dw.weight', 'backbone.0.body.blocks.6.0.bn2.weight', 'backbone.0.body.blocks.6.0.bn2.bias', 'backbone.0.body.blocks.6.0.bn2.running_mean', 'backbone.0.body.blocks.6.0.bn2.running_var', 'backbone.0.body.blocks.6.0.se.conv_reduce.weight', 'backbone.0.body.blocks.6.0.se.conv_reduce.bias', 'backbone.0.body.blocks.6.0.se.conv_expand.weight', 'backbone.0.body.blocks.6.0.se.conv_expand.bias', 'backbone.0.body.blocks.6.0.conv_pwl.weight', 'backbone.0.body.blocks.6.0.bn3.weight', 'backbone.0.body.blocks.6.0.bn3.bias', 'backbone.0.body.blocks.6.0.bn3.running_mean', 'backbone.0.body.blocks.6.0.bn3.running_var', 'backbone.0.body.blocks.6.1.conv_pw.weight', 'backbone.0.body.blocks.6.1.bn1.weight', 'backbone.0.body.blocks.6.1.bn1.bias', 'backbone.0.body.blocks.6.1.bn1.running_mean', 'backbone.0.body.blocks.6.1.bn1.running_var', 'backbone.0.body.blocks.6.1.conv_dw.weight', 'backbone.0.body.blocks.6.1.bn2.weight', 'backbone.0.body.blocks.6.1.bn2.bias', 'backbone.0.body.blocks.6.1.bn2.running_mean', 'backbone.0.body.blocks.6.1.bn2.running_var', 'backbone.0.body.blocks.6.1.se.conv_reduce.weight', 'backbone.0.body.blocks.6.1.se.conv_reduce.bias', 'backbone.0.body.blocks.6.1.se.conv_expand.weight', 'backbone.0.body.blocks.6.1.se.conv_expand.bias', 'backbone.0.body.blocks.6.1.conv_pwl.weight', 'backbone.0.body.blocks.6.1.bn3.weight', 'backbone.0.body.blocks.6.1.bn3.bias', 'backbone.0.body.blocks.6.1.bn3.running_mean', 'backbone.0.body.blocks.6.1.bn3.running_var', 'backbone.0.body.blocks.6.2.conv_pw.weight', 'backbone.0.body.blocks.6.2.bn1.weight', 'backbone.0.body.blocks.6.2.bn1.bias', 'backbone.0.body.blocks.6.2.bn1.running_mean', 'backbone.0.body.blocks.6.2.bn1.running_var', 'backbone.0.body.blocks.6.2.conv_dw.weight', 'backbone.0.body.blocks.6.2.bn2.weight', 'backbone.0.body.blocks.6.2.bn2.bias', 'backbone.0.body.blocks.6.2.bn2.running_mean', 'backbone.0.body.blocks.6.2.bn2.running_var', 'backbone.0.body.blocks.6.2.se.conv_reduce.weight', 'backbone.0.body.blocks.6.2.se.conv_reduce.bias', 'backbone.0.body.blocks.6.2.se.conv_expand.weight', 'backbone.0.body.blocks.6.2.se.conv_expand.bias', 'backbone.0.body.blocks.6.2.conv_pwl.weight', 'backbone.0.body.blocks.6.2.bn3.weight', 'backbone.0.body.blocks.6.2.bn3.bias', 'backbone.0.body.blocks.6.2.bn3.running_mean', 'backbone.0.body.blocks.6.2.bn3.running_var', 'answer_type_head.weight', 'answer_type_head.bias', 'answer_rel_head.weight', 'answer_rel_head.bias', 'answer_obj_head.weight', 'answer_obj_head.bias', 'answer_global_head.weight', 'answer_global_head.bias', 'answer_attr_head.weight', 'answer_attr_head.bias', 'answer_cat_head.weight', 'answer_cat_head.bias'])

state_dict["model"]["transformer.text_encoder.embeddings.position_ids"]

tensor([[  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,
          14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,
          28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,  40,  41,
          42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,  53,  54,  55,
          56,  57,  58,  59,  60,  61,  62,  63,  64,  65,  66,  67,  68,  69,
          70,  71,  72,  73,  74,  75,  76,  77,  78,  79,  80,  81,  82,  83,
          84,  85,  86,  87,  88,  89,  90,  91,  92,  93,  94,  95,  96,  97,
          98,  99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
         112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
         126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139,
         140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153,
         154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167,
         168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,
         182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195,
         196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209,
         210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,
         224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,
         238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251,
         252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265,
         266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279,
         280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293,
         294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307,
         308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321,
         322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335,
         336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349,
         350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,
         364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377,
         378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391,
         392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405,
         406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419,
         420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433,
         434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447,
         448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461,
         462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475,
         476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489,
         490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503,
         504, 505, 506, 507, 508, 509, 510, 511, 512, 513]])

学習済みパラメータ側には、学習パラメータとして「position_ids」の定義がありました。

他のパラメータの値を見ると、

state_dict["model"]["transformer.text_encoder.encoder.layer.0.attention.self.query.weight"]

tensor([[ 0.0024, -0.0432, -0.1008,  ...,  0.1183,  0.0301, -0.0913],
        [ 0.0030,  0.2174,  0.0536,  ...,  0.0234,  0.0858,  0.1278],
        [ 0.0735,  0.0854,  0.0009,  ..., -0.0129, -0.0278,  0.0802],
        ...,
        [-0.1637,  0.0062, -0.0330,  ...,  0.0058,  0.0643, -0.1156],
        [-0.2991,  0.0544,  0.0946,  ..., -0.0035, -0.1324,  0.0509],
        [-0.0649, -0.1165,  0.1456,  ..., -0.2730, -0.0320, -0.1308]])

他のパラメータには、データが読み込まれている模様。

変数の型を見ると、

type(model_qa.transformer.text_encoder.embeddings.position_ids)

torch.Tensor

type(model_qa.transformer.text_encoder.encoder.layer[0].attention.self.query.weight)

torch.nn.parameter.Parameter

「position_ids」は、Parameter型ではなくtensor型なので、読み込みエラーが発生していた模様。

このままモデルに学習済みパラメータを読み込むと、

model_qa.load_state_dict(state_dict["model"])

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

<ipython-input-19-5c885fe1a66b> in <cell line: 0>()
----> 1 model_qa.load_state_dict(state_dict["model"])


/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict, assign)
   2579 
   2580         if len(error_msgs) > 0:
-> 2581             raise RuntimeError(
   2582                 "Error(s) in loading state_dict for {}:\n\t{}".format(
   2583                     self.__class__.__name__, "\n\t".join(error_msgs)


RuntimeError: Error(s) in loading state_dict for MDETR:
	Unexpected key(s) in state_dict: "transformer.text_encoder.embeddings.position_ids".

前回同様に「position_ids」のエラーが発生。
モデル側では、「position_ids」には(学習パラメータではない)「tensor」型で既に値が入っている所に、学習済みパラメータ側では、学習パラメータとして「position_ids」の定義があり、学習パラメータとして「position_ids」を読み込もうとしたことに起因して、エラーが発生している模様。

Debug: 「strict=False」でモデル読込

モデル定義と、学習済みパラメータのデータとの間で、整合が取れないもの(position_ids)があるので、それを無視する「strict=False」でモデルを読み込むと、

import torch
torch.__version__

'2.6.0+cu124'

state_dict = torch.hub.load_state_dict_from_url("https://zenodo.org/record/4721981/files/gqa_EB5_checkpoint.pth", map_location='cpu')

model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=False, return_postprocessor=False)

Using cache found in /root/.cache/torch/hub/ashkamath_mdetr_main
WARNING:timm.models._builder:Unexpected keys (bn2.bias, bn2.num_batches_tracked, bn2.running_mean, bn2.running_var, bn2.weight, classifier.bias, classifier.weight, conv_head.weight) found while loading pretrained weights. This may be expected if model is being adapted.
Some weights of RobertaModel were not initialized from the model checkpoint at roberta-base and are newly initialized: ['pooler.dense.bias', 'pooler.dense.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

model_qa.load_state_dict(state_dict["model"], strict=False)

_IncompatibleKeys(missing_keys=[], unexpected_keys=['transformer.text_encoder.embeddings.position_ids'])

エラーなし、読み込みは成功。

読込が成功したモデルを用いて、続きの推論処理を続行

# Torch Hubからモデル読込
# model_qa = torch.hub.load('ashkamath/mdetr:main', 'mdetr_efficientnetB5_gqa', pretrained=True, return_postprocessor=False)
model_qa = model_qa.cuda()
model_qa.eval();

model_qa

MDETR(
  (transformer): Transformer(
    (encoder): TransformerEncoder(
      (layers): ModuleList(
        (0-5): 6 x TransformerEncoderLayer(
          (self_attn): MultiheadAttention(
            (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
          )
          (linear1): Linear(in_features=256, out_features=2048, bias=True)
          (dropout): Dropout(p=0.1, inplace=False)
          (linear2): Linear(in_features=2048, out_features=256, bias=True)
          (norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
          (norm2): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
          (dropout1): Dropout(p=0.1, inplace=False)
          (dropout2): Dropout(p=0.1, inplace=False)
        )
      )
    )
    (decoder): TransformerDecoder(
      (layers): ModuleList(
        (0-5): 6 x TransformerDecoderLayer(
          (self_attn): MultiheadAttention(
            (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
          )
          (cross_attn_image): MultiheadAttention(
            (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)
          )
          (linear1): Linear(in_features=256, out_features=2048, bias=True)
          (dropout): Dropout(p=0.1, inplace=False)
          (linear2): Linear(in_features=2048, out_features=256, bias=True)
          (norm1): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
          (norm3): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
          (norm4): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
          (dropout1): Dropout(p=0.1, inplace=False)
          (dropout3): Dropout(p=0.1, inplace=False)
          (dropout4): Dropout(p=0.1, inplace=False)
        )
      )
      (norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)
    )
    (text_encoder): RobertaModel(
      (embeddings): RobertaEmbeddings(
        (word_embeddings): Embedding(50265, 768, padding_idx=1)
        (position_embeddings): Embedding(514, 768, padding_idx=1)
        (token_type_embeddings): Embedding(1, 768)
        (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
        (dropout): Dropout(p=0.1, inplace=False)
      )
      (encoder): RobertaEncoder(
        (layer): ModuleList(
          (0-11): 12 x RobertaLayer(
            (attention): RobertaAttention(
              (self): RobertaSdpaSelfAttention(
                (query): Linear(in_features=768, out_features=768, bias=True)
                (key): Linear(in_features=768, out_features=768, bias=True)
                (value): Linear(in_features=768, out_features=768, bias=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
              (output): RobertaSelfOutput(
                (dense): Linear(in_features=768, out_features=768, bias=True)
                (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
                (dropout): Dropout(p=0.1, inplace=False)
              )
            )
            (intermediate): RobertaIntermediate(
              (dense): Linear(in_features=768, out_features=3072, bias=True)
              (intermediate_act_fn): GELUActivation()
            )
            (output): RobertaOutput(
              (dense): Linear(in_features=3072, out_features=768, bias=True)
              (LayerNorm): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
              (dropout): Dropout(p=0.1, inplace=False)
            )
          )
        )
      )
      (pooler): RobertaPooler(
        (dense): Linear(in_features=768, out_features=768, bias=True)
        (activation): Tanh()
      )
    )
    (resizer): FeatureResizer(
      (fc): Linear(in_features=768, out_features=256, bias=True)
      (layer_norm): LayerNorm((256,), eps=1e-12, elementwise_affine=True)
      (dropout): Dropout(p=0.1, inplace=False)
    )
  )
  (class_embed): Linear(in_features=256, out_features=256, bias=True)
  (bbox_embed): MLP(
    (layers): ModuleList(
      (0-1): 2 x Linear(in_features=256, out_features=256, bias=True)
      (2): Linear(in_features=256, out_features=4, bias=True)
    )
  )
  (query_embed): Embedding(100, 256)
  (qa_embed): Embedding(6, 256)
  (input_proj): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1))
  (backbone): Joiner(
    (0): TimmBackbone(
      (body): EfficientNetFeatures(
        (conv_stem): Conv2dSame(3, 48, kernel_size=(3, 3), stride=(2, 2), bias=False)
        (bn1): FrozenBatchNorm2d()
        (blocks): Sequential(
          (0): Sequential(
            (0): DepthwiseSeparableConv(
              (conv_dw): Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)
              (bn1): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(12, 48, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pw): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn2): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): DepthwiseSeparableConv(
              (conv_dw): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)
              (bn1): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(24, 6, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(6, 24, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pw): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn2): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): DepthwiseSeparableConv(
              (conv_dw): Conv2d(24, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=24, bias=False)
              (bn1): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(24, 6, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(6, 24, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pw): Conv2d(24, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn2): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
          (1): Sequential(
            (0): InvertedResidual(
              (conv_pw): Conv2d(24, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2dSame(144, 144, kernel_size=(3, 3), stride=(2, 2), groups=144, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(144, 6, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(6, 144, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(144, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): InvertedResidual(
              (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): InvertedResidual(
              (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (3): InvertedResidual(
              (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (4): InvertedResidual(
              (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(240, 40, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
          (2): Sequential(
            (0): InvertedResidual(
              (conv_pw): Conv2d(40, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2dSame(240, 240, kernel_size=(5, 5), stride=(2, 2), groups=240, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(240, 10, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(10, 240, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(240, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): InvertedResidual(
              (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): InvertedResidual(
              (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (3): InvertedResidual(
              (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (4): InvertedResidual(
              (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(384, 384, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=384, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(384, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
          (3): Sequential(
            (0): InvertedResidual(
              (conv_pw): Conv2d(64, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2dSame(384, 384, kernel_size=(3, 3), stride=(2, 2), groups=384, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(384, 16, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(16, 384, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (3): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (4): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (5): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (6): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
          (4): Sequential(
            (0): InvertedResidual(
              (conv_pw): Conv2d(128, 768, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(768, 768, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=768, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(768, 32, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(32, 768, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(768, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (3): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (4): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (5): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (6): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1056, 1056, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 176, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
          (5): Sequential(
            (0): InvertedResidual(
              (conv_pw): Conv2d(176, 1056, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2dSame(1056, 1056, kernel_size=(5, 5), stride=(2, 2), groups=1056, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1056, 44, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(44, 1056, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1056, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (3): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (4): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (5): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (6): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (7): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (8): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 304, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
          (6): Sequential(
            (0): InvertedResidual(
              (conv_pw): Conv2d(304, 1824, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(1824, 1824, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=1824, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(1824, 76, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(76, 1824, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(1824, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (1): InvertedResidual(
              (conv_pw): Conv2d(512, 3072, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(3072, 3072, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=3072, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(3072, 128, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(128, 3072, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(3072, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
            (2): InvertedResidual(
              (conv_pw): Conv2d(512, 3072, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn1): FrozenBatchNorm2d()
              (conv_dw): Conv2d(3072, 3072, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=3072, bias=False)
              (bn2): FrozenBatchNorm2d()
              (aa): Identity()
              (se): SqueezeExcite(
                (conv_reduce): Conv2d(3072, 128, kernel_size=(1, 1), stride=(1, 1))
                (act1): SiLU(inplace=True)
                (conv_expand): Conv2d(128, 3072, kernel_size=(1, 1), stride=(1, 1))
                (gate): Sigmoid()
              )
              (conv_pwl): Conv2d(3072, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
              (bn3): FrozenBatchNorm2d()
              (drop_path): Identity()
            )
          )
        )
      )
    )
    (1): PositionEmbeddingSine()
  )
  (answer_type_head): Linear(in_features=256, out_features=5, bias=True)
  (answer_rel_head): Linear(in_features=256, out_features=1594, bias=True)
  (answer_obj_head): Linear(in_features=256, out_features=3, bias=True)
  (answer_global_head): Linear(in_features=256, out_features=111, bias=True)
  (answer_attr_head): Linear(in_features=256, out_features=403, bias=True)
  (answer_cat_head): Linear(in_features=256, out_features=678, bias=True)
)

def plot_inference_qa(im, caption):
    img = transform(im).unsqueeze(0).cuda()

    # モデル伝搬
    memory_cache = model_qa(img, [caption], encode_and_save=True)
    outputs = model_qa(img, [caption], encode_and_save=False, memory_cache=memory_cache)

    probas = 1 - outputs['pred_logits'].softmax(-1)[0, :, -1].cpu()
    keep = (probas > 0.7).cpu()

    bboxes_scaled = rescale_bboxes(outputs['pred_boxes'].cpu()[0, keep], im.size)

    # 各ボックスで予測されるテキストスパンを抽出
    positive_tokens = (outputs["pred_logits"].cpu()[0, keep].softmax(-1) > 0.1).nonzero().tolist()
    predicted_spans = defaultdict(str)
    for tok in positive_tokens:
        item, pos = tok
        if pos < 255:
            # Encoderで処理したテキスト情報から、テキストスパンを取得
            span = memory_cache["tokenized"].token_to_chars(0, pos)
            # 入力文字(caption)から対応する文字を取得
            predicted_spans [item] += " " + caption[span.start:span.end]

    labels = [predicted_spans [k] for k in sorted(list(predicted_spans .keys()))]
    plot_results(im, probas[keep], bboxes_scaled, labels)

    # 質問タイプの分類
    type_conf, type_pred = outputs["pred_answer_type"].softmax(-1).max(-1)
    ans_type = type_pred.item()
    types = ["obj", "attr", "rel", "global", "cat"]

    # 質問タイプから一番マッチする回答をIDからマッピングして取得
    ans_conf, ans = outputs[f"pred_answer_{types[ans_type]}"][0].softmax(-1).max(-1)
    answer = id2answerbytype[f"answer_{types[ans_type]}"][ans.item()]
    print(f"Predicted answer: {answer}\t confidence={round(100 * type_conf.item() * ans_conf.item(), 2)}")

下記で推論を実行

url = "https://s3.us-east-1.amazonaws.com/images.cocodataset.org/val2017/000000076547.jpg"
im3 = Image.open(requests.get(url, stream=True).raw)
plot_inference_qa(im3, "What is on the table?")

Predicted answer: no	 confidence=nan

推論した結果、「answer」は「no」、「confidence」はnan値となり、処理は正しく行われない。

入力データの「img」、出力の「memory_cache」と「outputs」の中身を見ると、

img = transform(im3).unsqueeze(0).cuda()
caption = "What is on the table?"

# モデル伝搬
memory_cache = model_qa(img, [caption], encode_and_save=True)
outputs = model_qa(img, [caption], encode_and_save=False, memory_cache=memory_cache)

img

tensor([[[[-1.7754, -1.7412, -1.6727,  ...,  0.3309,  0.1939,  0.0912],
          [-1.7412, -1.7069, -1.6727,  ...,  0.4851,  0.3823,  0.3138],
          [-1.6727, -1.6727, -1.6727,  ...,  0.7248,  0.6734,  0.6392],
          ...,
          [-1.3302, -1.3987, -1.5014,  ..., -0.7993, -0.7822, -0.7650],
          [-1.4500, -1.4672, -1.5014,  ..., -0.7822, -0.7993, -0.7993],
          [-1.5185, -1.5185, -1.5014,  ..., -0.7822, -0.7993, -0.8164]],

         [[-1.6856, -1.6331, -1.5455,  ...,  0.4853,  0.4153,  0.3627],
          [-1.6681, -1.6331, -1.5630,  ...,  0.6604,  0.6254,  0.6078],
          [-1.6506, -1.6155, -1.5805,  ...,  0.9230,  0.9580,  0.9930],
          ...,
          [-1.1954, -1.2304, -1.3004,  ..., -0.3200, -0.2850, -0.2500],
          [-1.3004, -1.3004, -1.2829,  ..., -0.3025, -0.3200, -0.3200],
          [-1.3704, -1.3354, -1.2829,  ..., -0.3025, -0.3375, -0.3725]],

         [[-1.4559, -1.4036, -1.3339,  ...,  0.7751,  0.7751,  0.7751],
          [-1.4384, -1.4036, -1.3339,  ...,  0.9668,  0.9494,  0.9145],
          [-1.4036, -1.3861, -1.3513,  ...,  1.2631,  1.1934,  1.1411],
          ...,
          [-0.7587, -0.7936, -0.8633,  ..., -0.6715, -0.5844, -0.5321],
          [-0.9156, -0.8981, -0.8807,  ..., -0.6367, -0.6367, -0.6367],
          [-1.0201, -0.9678, -0.8807,  ..., -0.6193, -0.6715, -0.7064]]]],
       device='cuda:0')

memory_cache

{'text_memory_resized': tensor([[[ 0.0463,  0.0675,  0.1477,  ...,  0.0164,  0.2151,  0.0931]],
 
         [[-0.5830,  0.0047,  0.8694,  ..., -0.5799, -0.6114, -0.4202]],
 
         [[-0.5820, -0.0295,  0.8522,  ..., -0.6351, -0.6301, -0.4519]],
 
         ...,
 
         [[-0.7565,  0.5123, -0.3289,  ..., -0.4070,  0.3803, -0.2708]],
 
         [[ 0.0463,  0.0675,  0.1477,  ...,  0.0164,  0.2151,  0.0931]],
 
         [[ 0.0463,  0.0675,  0.1477,  ...,  0.0164,  0.2151,  0.0931]]],
        device='cuda:0'),
 'text_memory': tensor([[[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         ...,
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]]], device='cuda:0'),
 'img_memory': tensor([[[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         ...,
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]],
 
         [[nan, nan, nan,  ..., nan, nan, nan]]], device='cuda:0'),
 'text_pooled_op': None,
 'img_pooled_op': None,
 'mask': tensor([[False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False, False, False,
          False, False, False, False, False, False, False, False]],
        device='cuda:0'),
 'text_attention_mask': tensor([[False, False, False, False, False, False, False, False]],
        device='cuda:0'),
 'pos_embed': tensor([[[2.4869e-01, 9.6858e-01, 2.1593e-01,  ..., 1.0000e+00,
           2.1340e-05, 1.0000e+00]],
 
         [[2.4869e-01, 9.6858e-01, 2.1593e-01,  ..., 1.0000e+00,
           4.2681e-05, 1.0000e+00]],
 
         [[2.4869e-01, 9.6858e-01, 2.1593e-01,  ..., 1.0000e+00,
           6.4021e-05, 1.0000e+00]],
 
         ...,
 
         [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00]],
 
         [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00]],
 
         [[0.0000e+00, 0.0000e+00, 0.0000e+00,  ..., 0.0000e+00,
           0.0000e+00, 0.0000e+00]]], device='cuda:0'),
 'query_embed': tensor([[[ 0.0491,  1.1184, -0.2196,  ...,  0.4614, -0.0693,  0.6637]],
 
         [[-0.6772, -0.7437, -0.5867,  ...,  0.1200,  1.8001,  0.5810]],
 
         [[ 1.9782,  0.2545, -0.6935,  ...,  0.1258, -1.6969, -0.6355]],
 
         ...,
 
         [[ 0.4497,  0.5344,  0.3115,  ..., -0.6557,  0.1694, -0.1874]],
 
         [[-2.3338, -0.9938, -0.3027,  ...,  0.2182,  0.9463,  0.0895]],
 
         [[-1.5761, -0.5586, -0.4865,  ..., -1.8347, -1.2846,  0.0172]]],
        device='cuda:0'),
 'tokenized': {'input_ids': tensor([[   0, 2264,   16,   15,    5, 2103,  116,    2]], device='cuda:0'), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]], device='cuda:0')}}

outputs

{'pred_answer_type': tensor([[nan, nan, nan, nan, nan]], device='cuda:0'),
 'pred_answer_obj': tensor([[nan, nan, nan]], device='cuda:0'),
 'pred_answer_rel': tensor([[nan, nan, nan,  ..., nan, nan, nan]], device='cuda:0'),
 'pred_answer_attr': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]],
        device='cuda:0'),
 'pred_answer_cat': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan]], device='cuda:0'),
 'pred_answer_global': tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
          nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]],
        device='cuda:0'),
 'pred_logits': tensor([[[nan, nan, nan,  ..., nan, nan, nan],
          [nan, nan, nan,  ..., nan, nan, nan],
          [nan, nan, nan,  ..., nan, nan, nan],
          ...,
          [nan, nan, nan,  ..., nan, nan, nan],
          [nan, nan, nan,  ..., nan, nan, nan],
          [nan, nan, nan,  ..., nan, nan, nan]]], device='cuda:0'),
 'pred_boxes': tensor([[[nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan],
          [nan, nan, nan, nan]]], device='cuda:0')}

nan値がかなり多い結果となっています。
「memory_cache」の時点で、既にnan値が入っているので、画像・言語から中間ベクトルを算出する始めのEncoderの所で、うまく動作していない模様。

推論がうまく動作しない

諦める。