Error
MacOSで使えていた pgvector がなぜか旧に使えなくなった。
select * from langchain_pg_embedding limit 1;
ERROR: could not access file "$libdir/vector": No such file or directory
原因
brew install pgvectorでインストールしてもなぜかvector.controlなどの設定がうまく出来ていなかった。
(brew installの問題なのか自分が何か変な操作をしてしまったのかは不明)
解決方法
もしも postgresql@15 にpgvectorをインストールしているなら
-
pg_configのパスの確認 (e.g./opt/homebrew/bin/pg_config) -
pg_configを指定してmake install(ref: https://github.com/pgvector/pgvector#installation)
調査
pg_config --bindir
/opt/homebrew/Cellar/libpq/15.3_1/bin
libpqを使うとpg_config の場所が変わってしまうのか、一旦libpgをuninstall
brew uninstall libpq
Uninstalling /opt/homebrew/Cellar/libpq/15.3_1... (2,369 files, 28.5MB)
まだ使えない (ところどころpostgresql@15とpostgresql@14がでてくるのは、2つのバージョンを試してたので)
psql postgres
psql (15.5 (Homebrew))
Type "help" for help.
postgres=# create extension vector;
ERROR: extension "vector" is not available
DETAIL: Could not open extension control file "/opt/homebrew/opt/postgresql@15/share/postgresql@15/extension/vector.control": No such file or directory.
HINT: The extension must first be installed on the system where PostgreSQL is running.
postgres=#
pg_configは/opt/homebrew/Cellar/postgresql@14/14.10_1/bin brewでインストールしたpostgresql以下に変わった
pg_config --bindir
/opt/homebrew/Cellar/postgresql@14/14.10_1/bin
create extension vectorを試してみる→まだ参照されているところが違う
test=# create extension vector;
ERROR: could not open extension control file "/opt/homebrew/share/postgresql@14/extension/vector.control": No such file or directory
参照されているパス/opt/homebrew/share/postgresql@14 とpg_configのパス /opt/homebrew/Cellar/postgresql@14/14.10_1が違うので両方のextensionを確かめてみる→ 両方vectorにヒットするものはなし
ls /opt/homebrew/share/postgresql@14/extension | grep -c vector
0
ls /opt/homebrew/Cellar/postgresql@14/14.10_1/share/postgresql@14/extension | grep -c vector
0
pg_config の確認
which pg_config
/opt/homebrew/bin/pg_config
ls -l /opt/homebrew/bin/pg_config
lrwxr-xr-x 1 m.naka admin 45 Jan 14 11:28 /opt/homebrew/bin/pg_config -> ../Cellar/postgresql@14/14.10_1/bin/pg_config
could not open extension control file "/opt/homebrew/share/postgresql@14/extension/vector.control": No such file or directory 上ででていたエラーからも分かる通り使いたい brewでinstallしたpostgresql@14 は参照していそうではある
ls -l /opt/homebrew/share/postgresql@14
lrwxr-xr-x 1 m.naka admin 51 Jan 14 11:28 /opt/homebrew/share/postgresql@14 -> ../Cellar/postgresql@14/14.10_1/share/postgresql@14
pgvectorでvector.controlを見てみると、 https://github.com/pgvector/pgvector/blob/master/vector.control にあり
comment = 'vector data type and ivfflat and hnsw access methods'
default_version = '0.5.1'
module_pathname = '$libdir/vector'
relocatable = true
vectorを指定するファイルのよう。brew install pgvectorでこのファイルが /opt/homebrew/share/postgresql@14/extension/vector.control ここに置かれていないのが問題のよう。
brew install pgvectorのインストール先に問題がありそう
brewのインストールをやめて make installでインストールしてみる
echo $PG_CONFIG
/opt/homebrew/bin/pg_config
cd /tmp
git clone --branch v0.5.1 https://github.com/pgvector/pgvector.git
cd pgvector
make
make
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/hnsw.o src/hnsw.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/hnswbuild.o src/hnswbuild.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/hnswinsert.o src/hnswinsert.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/hnswscan.o src/hnswscan.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/hnswutils.o src/hnswutils.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/hnswvacuum.o src/hnswvacuum.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfbuild.o src/ivfbuild.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfflat.o src/ivfflat.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfinsert.o src/ivfinsert.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfkmeans.o src/ivfkmeans.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfscan.o src/ivfscan.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfutils.o src/ivfutils.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/ivfvacuum.o src/ivfvacuum.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -I. -I./ -I/opt/homebrew/include/postgresql@14/server -I/opt/homebrew/include/postgresql@14/internal -I/opt/homebrew/Cellar/icu4c/73.2/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/readline/include -I/opt/homebrew/Cellar/lz4/1.9.4/include -c -o src/vector.o src/vector.c
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -O2 -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math -bundle -o vector.so src/hnsw.o src/hnswbuild.o src/hnswinsert.o src/hnswscan.o src/hnswutils.o src/hnswvacuum.o src/ivfbuild.o src/ivfflat.o src/ivfinsert.o src/ivfkmeans.o src/ivfscan.o src/ivfutils.o src/ivfvacuum.o src/vector.o -L/opt/homebrew/lib/postgresql@14 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/opt/homebrew/opt/openssl@3/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/Cellar/lz4/1.9.4/lib -Wl,-dead_strip_dylibs -bundle_loader /opt/homebrew/Cellar/postgresql@14/14.10_1/bin/postgres
cp sql/vector.sql sql/vector--0.5.1.sql
make installを実行
/bin/sh /opt/homebrew/lib/postgresql@14/pgxs/src/makefiles/../../config/install-sh -c -d '/opt/homebrew/lib/postgresql@14'
/bin/sh /opt/homebrew/lib/postgresql@14/pgxs/src/makefiles/../../config/install-sh -c -d '/opt/homebrew/share/postgresql@14/extension'
/bin/sh /opt/homebrew/lib/postgresql@14/pgxs/src/makefiles/../../config/install-sh -c -d '/opt/homebrew/share/postgresql@14/extension'
/usr/bin/install -c -m 755 vector.so '/opt/homebrew/lib/postgresql@14/vector.so'
/usr/bin/install -c -m 644 .//vector.control '/opt/homebrew/share/postgresql@14/extension/'
/usr/bin/install -c -m 644 .//sql/vector--0.1.0--0.1.1.sql .//sql/vector--0.1.1--0.1.3.sql .//sql/vector--0.1.3--0.1.4.sql .//sql/vector--0.1.4--0.1.5.sql .//sql/vector--0.1.5--0.1.6.sql .//sql/vector--0.1.6--0.1.7.sql .//sql/vector--0.1.7--0.1.8.sql .//sql/vector--0.1.8--0.2.0.sql .//sql/vector--0.2.0--0.2.1.sql .//sql/vector--0.2.1--0.2.2.sql .//sql/vector--0.2.2--0.2.3.sql .//sql/vector--0.2.3--0.2.4.sql .//sql/vector--0.2.4--0.2.5.sql .//sql/vector--0.2.5--0.2.6.sql .//sql/vector--0.2.6--0.2.7.sql .//sql/vector--0.2.7--0.3.0.sql .//sql/vector--0.3.0--0.3.1.sql .//sql/vector--0.3.1--0.3.2.sql .//sql/vector--0.3.2--0.4.0.sql .//sql/vector--0.4.0--0.4.1.sql .//sql/vector--0.4.1--0.4.2.sql .//sql/vector--0.4.2--0.4.3.sql .//sql/vector--0.4.3--0.4.4.sql .//sql/vector--0.4.4--0.5.0.sql .//sql/vector--0.5.0--0.5.1.sql .//sql/vector--0.5.1.sql '/opt/homebrew/share/postgresql@14/extension/'
/bin/sh /opt/homebrew/lib/postgresql@14/pgxs/src/makefiles/../../config/install-sh -c -d '/opt/homebrew/include/postgresql@14/server/extension/vector/'
/usr/bin/install -c -m 644 .//src/vector.h '/opt/homebrew/include/postgresql@14/server/extension/vector/'
vector.control がセットされた 
ls /opt/homebrew/share/postgresql@14/extension/vector.control
/opt/homebrew/share/postgresql@14/extension/vector.control
select 1 from langchain_pg_embedding limit 1;
?column?
----------
1
(1 row)
Error再来 (2024/04/22) (まだ未解決)
同じ現象が起こったのでもう一度試すと pgvectorレポを持ってきて 0.6.2で makeを実行すると以下のエラー
make
Makefile:48: /opt/homebrew/opt/libpq/lib/postgresql/pgxs/src/makefiles/pgxs.mk: No such file or directory
make: *** No rule to make target `/opt/homebrew/opt/libpq/lib/postgresql/pgxs/src/makefiles/pgxs.mk'. Stop.
brew reinstall postgresql@15
echo 'export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc
brew services start postgresql@15
make
export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"
make
Makefile:48: /opt/homebrew/opt/libpq/lib/postgresql/pgxs/src/makefiles/pgxs.mk: No such file or directory
make: *** No rule to make target `/opt/homebrew/opt/libpq/lib/postgresql/pgxs/src/makefiles/pgxs.mk'. Stop.
brew reinstall libpq
echo 'export PATH="/opt/homebrew/opt/libpq/bin:$PATH"' >> ~/.zshrc
まだだめ
export LDFLAGS="-L/opt/homebrew/opt/libpq/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libpq/include"