概要
写真中の長方形（被写体）を正面向きに補正します。
以下、末尾プログラムのスクリーンショットです。
手法
次の図は透視投影された長方形とします。
内側の四角形は立体感を出すための模様です。全てテキトーに描いたので正確な投影図ではありません。
頂点 $p_{n=(1,2,3,4)}$ の画面上の座標を $ p_n \left( s_n , t_n \right) $ とします。
対角線の交点 $q$ の座標
\begin{eqnarray}
q & = & \left( s_q , t_q \right) \\
s_q & = & \frac{
\left( s_1 t_4 - s_4 t_1 \right) \left( s_2 - s_3 \right) -
\left( s_2 t_3 - s_3 t_2 \right) \left( s_1 - s_4 \right)
}{
\left( s_1 - s_4 \right) \left( t_2 - t_3 \right) -
\left( s_2 - s_3 \right) \left( t_1 - t_4 \right)
} \\
t_q & = & \frac{
\left( s_1 t_4 - s_4 t_1 \right) \left( t_2 - t_3 \right) -
\left( s_2 t_3 - s_3 t_2 \right) \left( t_1 - t_4 \right)
}{
\left( s_1 - s_4 \right) \left( t_2 - t_3 \right) -
\left( s_2 - s_3 \right) \left( t_1 - t_4 \right)
} \\
\end{eqnarray}
$p_n$ に投影面の奥行き座標 $z_s$ を追加して
\begin{eqnarray}
p_n & = & \left( s_n , t_n , z_s \right) \\
\end{eqnarray}
とし、$p_n$ を $m_n$ 倍（各点別）したものが三次元空間上の $P_n$
P_n = m_n\ p_n
になります。線分 $P_1 P_2$ と $P_3 P_4$ は平行で同じ長さだから、Mathematica に解いてもらうと
Mathematica
In[1]:= Module[{
p1 = {s1, t1, zs},
p2 = {s2, t2, zs},
p3 = {s3, t3, zs},
p4 = {s4, t4, zs},
P1, P2, P3, P4
},
P1 = m1 p1;
P2 = m2 p2;
P3 = m3 p3;
P4 = m4 p4;
FullSimplify[Solve[P2 - P1 == P4 - P3, {m1, m2, m3, m4}]]
]
Out[1]= {{
m2 -> (m1 (s4 (-t1 + t3) + s3 (t1 - t4) + s1 (-t3 + t4)))/(s4 (-t2 + t3) + s3 (t2 - t4) + s2 (-t3 + t4)),
m3 -> (m1 (s4 (t1 - t2) + s1 (t2 - t4) + s2 (-t1 + t4)))/(s4 (-t2 + t3) + s3 (t2 - t4) + s2 (-t3 + t4)),
m4 -> (m1 (s3 (t1 - t2) + s1 (t2 - t3) + s2 (-t1 + t3)))/(s4 (-t2 + t3) + s3 (t2 - t4) + s2 (-t3 + t4))
}}
が得られます。しかし、この解き方は $m_1$,$m_2$ と $m_4$,$m_3$ （対角）が入れ替わった場合も成立してしまいます。2通りしかないので、うまくいく方を選択します。
$z_s$ は出てこないので任意でよいことがわかりますが、透視投影なので $z_s\gt 0$ です。指定する $p_n$ 次第では $P_n$ による形状が平行四辺形（入力形状）になりますが、長方形（出力形状）と見做します。
$m_1$ は任意の値を指定可能なので
\begin{eqnarray}
m_d & = & \frac{ 1 }{ s_2 (t_4 - t_3) + s_3 (t_2 - t_4) + s_4 (t_3 - t_2) } \\
m_1 & = & 1 \\
m_2 & = & m_d\ m_1 (+ s_3 (t_1 - t_4) + s_4 (t_3 - t_1) + s_1 (t_4 - t_3)) \\
m_3 & = & m_d\ m_1 (- s_4 (t_2 - t_1) - s_1 (t_4 - t_2) - s_2 (t_1 - t_4)) \\
m_4 & = & m_d\ m_1 (- s_1 (t_3 - t_2) - s_2 (t_1 - t_3) - s_3 (t_2 - t_1)) \\
\end{eqnarray}
としますが、これではうまくいかず $m_1$,$m_2$ と $m_4$,$m_3$ が入れ替わっていました。そこで、
\begin{eqnarray}
m_1 \to m_4 \\
m_2 \to m_3 \\
m_3 \to m_2 \\
m_4 \to m_1 \\
\end{eqnarray}
に置き換えます。
出力先の長方形の縦横比は（横 : 縦 = $P_1 P_2 : P_1 P_3$）とします。$P_n$ に対応する出力先長方形の投影面座標を $g_n (x_n, y_n, z_s)$ とすると、対応する空間座標 $G_n$ を
G_n = \frac{1}{ m_n } g_n
として透視投影することで立体感を打ち消す形にできます。
プログラム
ソースコード HTML+JavaScript (長いので折りたたみ)
<!DOCTYPE html>
<html lang="ja">
<head>
<meta charset="utf-8">
<title>画像中の四角形を立体補正</title>
<style>
canvas {
border: solid 1px black;
}
table {
border: solid 1px; black;
border-collaspe: collaspe;
border-spacing: 0;
}
th, td {
border: solid 1px gray;
padding: 4px;
}
.editor {
position: absolute;
top: 4px; left: 4px;
z-index: 1;
}
.preview {
position: absolute;
top: 4px; left: 4px;
z-index: 2;
}
.output {
position: absolute;
top: 0px; left: 0px;
z-index: 2;
}
.noevent {
position: absolute;
top: 0px; left: 0px;
z-index: 3;
border-width: 0;
padding: 0px;
margin: 0px;
background-color: rgb(0 0 0 / .5);
}
.rectimage {
position: absolute;
top: 0px; left: 0px;
z-index: 4;
border: solid 8px white;
padding: 0px;
background-color: black;
}
.help {
position: absolute;
top: 16px; left: 16px;
z-index: 5;
background-color: white;
}
</style>
</head>
<body>
<canvas id="tagImageCanvas" class="editor" width="512" height="512"></canvas>
<canvas id="tagPreviewCanvas" class="preview" hidden></canvas>
<canvas id="tagRectangleCanvas" class="output" hidden></canvas>
<canvas id="tagTextureCanvas" hidden></canvas>
<p class="noevent"><canvas id="tagNoEvent" hidden></canvas></p>
<img id="tagRectangleImage" class="rectimage" hidden>
<table id="tagHelp" class="help" hidden>
<tr><th>操作</th><th>内容</th></tr>
<tr><td>ドラッグ</td><td>画像の移動</td></tr>
<tr><td>頂点をドラッグ</td><td>頂点の移動</td></tr>
<tr><td>[SHIFT]+ドラッグ</td><td>四角形の移動</td></tr>
<tr><td>ホイール</td><td>画像の拡大・縮小</td></tr>
<tr><td>[SHIFT]+ホイール</td><td>四角形の拡大・縮小</td></tr>
<tr><td>キー：[+] または [;]</td><td>画像の拡大</td></tr>
<tr><td>キー：[-]</td><td>画像の縮小</td></tr>
<tr><td>キー：[=] または [0]</td><td>画像を等倍で表示</td></tr>
<tr><td>キー：[r]</td><td>画像をドロップしたときの状態に戻す</td></tr>
<tr><td>キー：[v]</td><td>逆変換画像の表示/非表示</td></tr>
<tr><td>キー：[h] または [?]</td><td>操作説明を表示</td></tr>
<tr><td colspan="2">
逆変換画像は元画像と同じサイズです。<br>
画像の保存は右クリックのメニューから行ってください。<br>
ブラウザによっては画像の保存ができません。
</td></tr>
</table>
<script>
function addVec(a, b) { return a.map((v, i) => v + b[i]); }
function subVec(a, b) { return a.map((v, i) => v - b[i]); }
function dotVec(a, b) { return a.reduce((a, v, i) => a + v * b[i], 0); }
function mulVec(v, m) { return v.map((v) => v * m); }
function divVec(v, m) { return v.map((v) => v / m); }
function vecLen(v) { return Math.sqrt(dotVec(v, v)); }
function vecLen2(s, e) { return vecLen2(subVec(e, s)); }
function unitVec(v) { return divVec(v, vecLen(v)); }
function unitVec2(s, e) { return unitVec(subVec(e, s)); }
function zoomVec(p, c, s) { return addVec(mulVec(subVec(p, c), s), c); }
/*
* 四角形の奥行きを算出
* 入力： 4点の平面座標
* 出力： 奥行き
*
* m1 -- m2
* | |
* m3 -- m4
*/
function getQuadrangleDepth(quadrangle) {
const [[s1, t1], [s2, t2], [s3, t3], [s4, t4]] = quadrangle;
const d = s2 * t4 - s2 * t3 + s3 * t2 - s3 * t4 + s4 * t3 - s4 * t2;
const m1 = (s1 * t2 - s1 * t3 + s2 * t3 - s2 * t1 + s3 * t1 - s3 * t2) / d;
const m2 = (s4 * t1 - s4 * t2 + s1 * t2 - s1 * t4 + s2 * t4 - s2 * t1) / d;
const m3 = (s3 * t1 - s3 * t4 + s4 * t3 - s4 * t1 + s1 * t4 - s1 * t3) / d;
return [m1, m2, m3, 1.0];
}
/*
* Drag & Drop 用クラス
*/
class DropImage {
// ドロップされたファイル(イベント)から image 型だけを選択する.
static fromEvent(event) {
return [...event.dataTransfer.files]
.filter((f) => f.type.match('^image/'));
}
// イメージ ファイルの読み込み (Promiseを返す)
static load(blobs) {
const files = [...blobs].map((b) => ({blob: b}));
return new Promise((resolve, reject) => {
function next(files, index) {
const drop = files[index];
drop.image = new Image();
drop.image.onload = (() => (++index < files.length)
? next(files, index) : resolve(files));
drop.image.onerror = reject;
drop.image.src = drop.src = URL.createObjectURL(drop.blob);
/* 画像が不要になったら URL.revokeObjectURL で解放すること */
}
(files.length > 0) ? next(files, 0) : resolve(files);
});
}
}
/*
* 選択領域編集
*/
class QuadrangleSelectEditor {
// コンストラクタ.
constructor(canvas) {
this.canvas = canvas;
this.context = this.canvas.getContext('2d');
this.image = null;
this.ondraw = null;
this.pointerMode = null;
this.pointerCapture = -1;
this.pointerDown = null;
this.pointerData = [0, 0];
this.mousePosition = null;
this.reset();
}
// 選択状況リセット.
reset() {
this.image_scale = null;
this.image_pos = null;
this.quadrangle = null;
this.quadrangleIndex = null;
this.quadrangleSelect = null;
this.quadrangleEdge = null;
this.quadrangleEdgeSelect = null;
const image = this.image;
if (image) {
const [iw, ih] = [image.width, image.height];
const [cx, cy] = [iw / 2, ih / 2];
const [dx, dy] = [cx / 2, cy / 2];
this.quadrangle = [
[cx - dx, cy - dy],
[cx + dx, cy - dy],
[cx - dx, cy + dy],
[cx + dx, cy + dy],
];
}
this.updateImagePosition();
}
// 拡大率の補正.
adjustScale(scale) {
const image = this.image;
if (!image)
return scale;
const ctx = this.context;
const cw = ctx.canvas.clientWidth;
const ch = ctx.canvas.clientHeight;
const [iw, ih] = [image.width, image.height];
const [sw, sh] = [iw / cw, ih / ch];
const slmin = Math.floor(Math.log2(1 / Math.max(sw, sh)));
const smin = Math.pow(2, slmin);
return Math.min(Math.max(smin / 4, scale ?? smin), 8.0);
}
// 画像表示位置の更新.
updateImagePosition() {
const ctx = this.context;
const cw = ctx.canvas.clientWidth;
const ch = ctx.canvas.clientHeight;
const csz = [cw, ch];
this.canvas_size = csz;
const image = this.image;
if (!image)
return;
const [iw, ih] = [image.width, image.height];
const [sw, sh] = [iw / cw, ih / ch];
const scale = this.adjustScale(this.image_scale);
const gv = mulVec([iw, ih], scale);
this.image_scale = scale;
const [px, py] = this.image_pos ?? divVec(subVec(csz, gv), 2);
function fix(pos, dlen, glen) {
const sdlt = dlen - (glen * scale);
return ((sdlt < 0)
? Math.max(sdlt, Math.min(0, pos))
: Math.max(0, Math.min(pos, sdlt)));
}
this.image_pos = [fix(px, cw, iw), fix(py, ch, ih)];
}
// 画像を設定.
setImage(image) {
this.image = image;
this.reset();
}
// 画像位置を設定.
setImagePos(pos) {
this.image_pos = pos;
this.updateImagePosition();
}
// タグのサイズを変更.
setTagSize(width, height) {
this.canvas.width = Math.max(320, width);
this.canvas.height = Math.max(240, height);
this.updateImagePosition();
}
// 画像座標に変換.
toImage(p) {
return divVec(subVec(p, this.image_pos), this.image_scale);
}
// 表示座標に変換.
toCanvas(p) {
return addVec(mulVec(p, this.image_scale), this.image_pos);
}
// 対角線の交点を取得.
getIOD() {
const quadrangle = this.quadrangle;
if (!quadrangle)
return null;
const [[s1, t1], [s2, t2], [s3, t3], [s4, t4]] = quadrangle;
const qd = (s1 * t2 - s1 * t3
+ s2 * t4 - s2 * t1
+ s3 * t1 - s3 * t4
+ s4 * t3 - s4 * t2);
const qx = (s1 * s2 * t4 - s1 * s2 * t3
+ s2 * s4 * t3 - s2 * s4 * t1
+ s3 * s1 * t2 - s3 * s1 * t4
+ s4 * s3 * t1 - s4 * s3 * t2) / qd;
const qy = (s1 * t2 * t4 - s1 * t3 * t4
+ s2 * t3 * t4 - s2 * t1 * t3
+ s3 * t2 * t1 - s3 * t2 * t4
+ s4 * t3 * t1 - s4 * t2 * t1) / qd;
return [qx, qy];
}
// 有効な形状か?
isValidQ() {
const iod = this.getIOD();
if (!iod) return false;
const check = ((v1, v2) => {
const s = this.quadrangle[v1];
const t = this.quadrangle[v2];
return dotVec(subVec(t, s), subVec(iod, s)) > 0;
});
return (check(0, 3) && check(3, 0) &&
check(1, 2) && check(2, 1));
}
// 選択状況を描画.
draw() {
const ctx = this.context;
const [cw, ch] = this.canvas_size;
ctx.fillStyle = 'gray';
ctx.fillRect(0, 0, cw, ch);
this.drawImage();
}
// 画像を描画.
drawImage() {
const image = this.image;
if (!image)
return this.drawDropInfo();
const scale = this.image_scale;
const [iw, ih] = [image.width, image.height];
const [dx, dy] = this.image_pos;
const [dw, dh] = mulVec([iw, ih], scale);
const ctx = this.context;
ctx.imageSmoothingEnabled = true;
ctx.imageSmoothingQuality = 'high';
ctx.drawImage(image, 0, 0, iw, ih, dx, dy, dw, dh);
this.drawQuadrangle();
}
drawDropInfo() {
const ctx = this.context;
const getPos = ((msg) => {
const text = ctx.measureText(msg);
const [cw, ch] = this.canvas_size;
return [
(cw - text.width) / 2,
(ch - text.hangingBaseline) / 2,
];
});
const putText = ((msg, x, y) => {
ctx.strokeStyle = 'black';
ctx.strokeText(msg, x, y);
ctx.fillStyle = 'white';
ctx.fillText(msg, x, y);
});
ctx.textAlign = 'left';
ctx.textBaseline = 'top';
ctx.direction = 'ltr';
ctx.font = '24px serif';
const message = 'ここに画像ファイルをドロップしてください';
const [mx, my] = getPos(message);
putText(message, mx, my);
ctx.font = '16px serif';
const help = '[h]キーで操作説明を表示';
const [hx, hy] = getPos(help);
putText(help, hx, hy + 36);
}
// 選択領域を描画.
drawQuadrangle() {
const quad = this.quadrangle;
if (!quad)
return;
const ipos = this.image_pos;
const scale = this.image_scale;
const qpos = quad.map((p) => addVec(mulVec(p, scale), ipos));
const ctx = this.context;
ctx.lineWidth = 2;
ctx.strokeStyle = this.isValidQ()
? 'rgb(0 255 0 / 0.75)'
: 'rgb(255 0 0 / 0.75)';
ctx.beginPath();
ctx.moveTo(qpos[0][0], qpos[0][1]);
[2, 3, 1].forEach((i) => ctx.lineTo(qpos[i][0], qpos[i][1]));
ctx.closePath();
ctx.stroke();
ctx.fillStyle = ctx.strokeStyle;
for (const pos of qpos) {
ctx.beginPath();
ctx.arc(pos[0], pos[1], 3, 0, Math.PI * 2);
ctx.fill();
}
if (false) {
ctx.lineWidth = 1;
ctx.strokeStyle = 'rgb(255 0 / 0.25)';
ctx.beginPath();
ctx.moveTo(qpos[0][0], qpos[0][1]);
ctx.lineTo(qpos[3][0], qpos[3][1]);
ctx.stroke();
ctx.beginPath();
ctx.moveTo(qpos[1][0], qpos[1][1]);
ctx.lineTo(qpos[2][0], qpos[2][1]);
ctx.stroke();
}
const [qi, rad] = this.quadrangleSelect ?? [null, 0];
if (qi != null) {
ctx.lineWidth = 2;
ctx.strokeStyle = (this.quadrangleIndex
? 'rgb(255 255 255 / 0.8)'
: 'rgb(255 255 255 / 0.4)');
ctx.beginPath();
ctx.arc(qpos[qi][0], qpos[qi][1], rad, 0, Math.PI * 2);
ctx.stroke();
}
const ei = this.quadrangleEdgeSelect;
if (ei != null) {
const [e1, e2] = ei;
ctx.lineWidth = 2;
ctx.strokeStyle = (this.quadrangleEdge
? 'rgb(255 255 255 / 0.8)'
: 'rgb(255 255 255 / 0.4)');
ctx.beginPath();
ctx.moveTo(qpos[e1][0], qpos[e1][1]);
ctx.lineTo(qpos[e2][0], qpos[e2][1]);
ctx.stroke();
}
if (this.ondraw)
this.ondraw(this.quadrangle);
}
/*
* ユーザー操作
*/
onUserEvent(event) {
this.pointerData = getEventOffset(event);
const image = this.image;
if (!image)
return;
let update = false;
let pointer = false;
switch (event.type) {
case 'pointermove':
this.searchShape();
pointer = true;
update = true;
break;
case 'pointerdown':
if (event.buttons != 1)
break;
this.setPointerDown(event);
this.setPointerCapture(event.pointerId);
if (event.shiftKey)
this.pointerMode = 'quadrangle';
else if (this.selectShape())
; // NO-OP
else
this.pointerMode = 'scroll';
pointer = true;
break;
case 'pointerup':
this.setPointerDown();
this.setPointerCapture(-1);
if (this.quadrangleIndex != null) {
this.quadrangleIndex = null;
update = true;
} else if (this.quadrangleEdge != null) {
this.quadrangleEdge = null;
update = true;
}
this.pointerMode = null;
break;
case 'wheel':
{
const zoom = Math.pow(2, - event.deltaY / 128);
const mpos = getEventOffset(event);
if (event.shiftKey)
this.setZoomQuadrange(zoom, mpos);
else
this.setZoomImage(zoom, mpos);
}
break;
case 'keydown':
switch (event.key) {
case '+':
case ';':
this.setZoomImage(2.0);
break;
case '-':
this.setZoomImage(0.5);
break;
case '=':
case '0':
this.setZoomImage(1 / this.image_scale);
break;
case 'r':
this.reset();
update = true;
break;
}
break;
}
if (pointer)
this.movePointer(event);
if (update)
this.draw();
}
setPointerDown(event) {
this.pointerDown = event ? this.pointerData : null;
}
setPointerCapture(id) {
const tag = this.canvas;
const pid = this.pointerCapture;
this.pointerCapture = id;
if (id >= 0)
tag.setPointerCapture(id)
else if (pid >= 0)
tag.releasePointerCapture(pid)
}
searchVertex(pointer, rad) {
const mp = pointer ?? this.pointerData;
const rad1 = rad ?? 6;
const rad2 = rad1 * rad1;
const qsel = this.quadrangle.map((p, i) => {
const dv = subVec(this.toCanvas(p), mp);
return dotVec(dv, dv) < rad2 ? i : null;
}).filter((i) => (i != null));
this.quadrangleSelect = null;
if (qsel.length > 0) {
this.quadrangleSelect = [qsel[0], rad1];
return true;
}
return false;
}
selectVertex() {
this.quadrangleIndex = null;
if (this.searchVertex(this.pointerDown)) {
this.quadrangleIndex = this.quadrangleSelect;
this.pointerMode = 'pointer';
return true;
}
return false;
}
searchEdge(pointer, distance) {
const elen = distance ?? 6;
const quad = this.quadrangle.map((p) => this.toCanvas(p));
const mp = pointer ?? this.pointerData;
const qsel = [[0, 1], [1, 3], [3, 2], [2, 0]].map((p, i) => {
const [sp, ep] = [quad[p[0]], quad[p[1]]];
const [sev, smv] = [subVec(ep, sp), subVec(mp, sp)];
const [sel, sml] = [vecLen(sev), vecLen(smv)];
const [sen, smn] = [divVec(sev, sel), divVec(smv, sml)];
const scos = dotVec(sen, smn);
if (scos < 0) return null;
const [esv, emv] = [subVec(sp, ep), subVec(mp, ep)];
const [esl, eml] = [vecLen(esv), vecLen(emv)];
const [esn, emn] = [divVec(esv, esl), divVec(emv, eml)];
const ecos = dotVec(esn, emn);
if (ecos < 0) return null;
const dv = subVec(mp, addVec(sp, mulVec(sen, sml * scos)));
return vecLen(dv) < elen ? p : null;
}).filter((i) => (i != null));
this.quadrangleEdgeSelect = null;
if (qsel.length > 0) {
this.quadrangleEdgeSelect = qsel[0];
return true;
}
return false;
}
selectEdge() {
this.quadrangleEdge = null;
if (this.searchEdge(this.pointerDown)) {
this.quadrangleEdge = this.quadrangleEdgeSelect;
this.pointerMode = 'pointer';
return true;
}
return false;
}
searchShape() {
const sv = this.searchVertex();
const se = this.searchEdge();
return sv || se;
}
selectShape() {
const sv = this.selectVertex();
const se = this.selectEdge();
return sv || se;
}
movePointer(event) {
const ipos = this.image_pos;
const ppos = this.mousePosition;
const apos = ppos ?? [0, 0];
const cpos = getEventOffset(event);
const pmov = subVec(cpos, apos);
const dmov = addVec(ipos, pmov);
this.mousePosition = cpos;
switch (this.pointerMode) {
case 'scroll':
if (!ppos) {
this.pointerMode = null;
break;
}
this.setImagePos(dmov);
this.draw();
break;
case 'quadrangle':
if (!event.shiftKey || !ppos) {
this.pointerMode = null;
} else {
const m = this.toImage(dmov);
this.quadrangle = this.quadrangle.map((v) => addVec(v, m));
this.draw();
}
break;
case 'pointer':
if (this.quadrangleIndex != null) {
const ipos = this.toImage(cpos);
this.quadrangle[this.quadrangleIndex[0]] = ipos;
this.draw();
} else if (this.quadrangleEdge != null) {
const imov = this.toImage(dmov);
this.quadrangleEdge.forEach((e) => {
this.quadrangle[e] = addVec(this.quadrangle[e], imov);
});
this.draw();
}
break;
}
}
setZoomImage(scale, position) {
if (!this.image_scale || !this.image_pos)
return;
const cpos = position ?? [
this.canvas.width / 2,
this.canvas.height / 2,
];
const pscale = this.image_scale;
const nscale = this.adjustScale(pscale * scale);
const dscale = nscale / pscale;
this.image_scale = nscale;
this.setImagePos(zoomVec(this.image_pos, cpos, dscale));
this.draw();
}
setZoomQuadrange(scale, position) {
if (!this.quadrangle || !this.image_pos)
return;
this.quadrangle = this.quadrangle.map((q) =>
this.toImage(zoomVec(this.toCanvas(q), position, scale)));
this.draw();
}
}
/*
* WebGL: 逆透視矩形描画用
*/
const splitSize = 64;
const triangleStripSize = splitSize * (splitSize + 1) * 2;
function interpVertex1(u, v, s) {
return interpVertex2(u, v, 1.0 - s, s);
}
function interpVertex2(u, v, s, t) {
return [
u[0] * s + v[0] * t,
u[1] * s + v[1] * t,
u[2] * s + v[2] * t,
u[3] * s + v[3] * t,
u[4] * s + v[4] * t,
];
}
function splitVertex(u, v) {
const r = Array(splitSize + 1);
for (let i = 0; i <= splitSize; i++)
r[i] = interpVertex1(u, v, i / splitSize);
return r;
}
function splitQuadrangle(quadrangle) {
const c1 = splitVertex(quadrangle[0], quadrangle[2]);
const c2 = splitVertex(quadrangle[1], quadrangle[3]);
const r = Array(splitSize + 1);
for (let i = 0; i <= splitSize; i++)
r[i] = splitVertex(c1[i], c2[i]);
return r;
}
function createTriangleStrip(quadrangle) {
const vertex = splitQuadrangle(quadrangle);
const r = Array(triangleStripSize);
for (let i = 0, p = 0; i < splitSize; i++) {
const l1 = vertex[i + 0];
const l2 = vertex[i + 1];
if ((i & 1) == 0)
for (let j = 0; j <= splitSize; j++) {
r[p++] = l1[j];
r[p++] = l2[j];
}
else
for (let j = splitSize; j >= 0; j--) {
r[p++] = l1[j];
r[p++] = l2[j];
}
}
return r;
}
class RenderRectangle {
constructor(canvas) {
this.canvas = canvas;
this.image = null;
this.valid = true;
const gl = canvas.getContext('webgl');
if (!gl) {
this.valid = false;
alert('WebGL を初期化出来ませんでした。このブラウザでは表示できない項目があります。');
return;
}
this.context = gl;
if (!this.initShader())
this.destroy();
this.vertexBuffer = gl.createBuffer();
this.textureCoord = gl.createBuffer();
}
destroy() {
if (!this.valid)
return;
this.valid = false;
const gl = this.context;
const shader = this.shader_program;
if (shader)
gl.deleteProgram(shader);
this.deleteBuffer(this.vertexBuffer);
this.deleteBuffer(this.textureCoord);
this.shader_info = null;
this.shader_program = null;
this.shader_attrib = null;
this.shader_uniform = null;
this.texture = null;
this.texImage = null;
this.texImageData = null;
this.vertexBuffer = null;
this.textureCoord = null;
}
deleteBuffer(buffer) {
if (buffer)
this.context.deleteBuffer(buffer);
return this;
}
loadShader(type, source) {
const gl = this.context;
const shader = gl.createShader(type);
gl.shaderSource(shader, source);
gl.compileShader(shader);
if (gl.getShaderParameter(shader, gl.COMPILE_STATUS))
return shader;
console.log(gl.getShaderInfoLog(shader));
gl.deleteShader(shader);
return null;
}
initShader() {
const gl = this.context;
const shader = gl.createProgram();
const vertex = this.loadShader(gl.VERTEX_SHADER, [
'uniform mat4 uMatrix;',
'attribute vec4 aPosition;',
'attribute vec2 aTexture;',
'varying highp vec2 vTexture;',
'',
'void main() {',
' gl_Position = uMatrix * aPosition;',
' vTexture = aTexture;',
'}',
].join('\n'));
const fragment = this.loadShader(gl.FRAGMENT_SHADER, [
'uniform sampler2D uSampler;',
'varying highp vec2 vTexture;',
'',
'void main() {',
' gl_FragColor = texture2D(uSampler, vTexture);',
'}',
].join('\n'));
this.shader_program = shader;
if (!shader || !vertex || !fragment)
return false;
gl.attachShader(shader, vertex);
gl.attachShader(shader, fragment);
gl.linkProgram(shader);
if (!gl.getProgramParameter(shader, gl.LINK_STATUS))
return false;
this.shader_attrib = {
position: gl.getAttribLocation(shader, 'aPosition'),
texture: gl.getAttribLocation(shader, 'aTexture'),
}
this.shader_uniform = {
matrix: gl.getUniformLocation(shader, 'uMatrix'),
sampler: gl.getUniformLocation(shader, 'uSampler'),
}
return true;
}
clearTexture() {
if (this.texture) {
this.context.deleteTexture(this.texture);
this.texture = null;
this.texImage = null;
this.texImageData = null;
}
}
setTexture(image) {
this.clearTexture();
this.texture = this.context.createTexture();
this.texImage = image;
this.texImageData = new Uint8Array(image.data);
}
drawRectangle(quadrangle) {
const depth = getQuadrangleDepth(quadrangle);
depth.push(depth.reduce((a, v) => a + v, 0) / 4);
const depmin = Math.min(...depth);
const idep = depth.map((v) => v / depmin);
const zmax = Math.max(...idep);
const gl = this.context;
const width = gl.drawingBufferWidth;
const height = gl.drawingBufferHeight;
const aspect = width / Math.max(1, height);
gl.viewport(0, 0, width, height);
gl.clearDepth(1.0);
gl.enable(gl.DEPTH_TEST);
gl.depthFunc(gl.LEQUAL);
gl.clear(gl.DEPTH_BUFFER_BIT);
gl.clearColor(0.5, 0.5, 0.5, 1.0);
gl.clear(gl.COLOR_BUFFER_BIT);
gl.useProgram(this.shader_program);
{
const znear = 0.5;
const zfar = zmax * 1.5;
const zoom = 1.0;
const zd = zfar - znear;
const m00 = zoom / aspect;
const m11 = zoom;
const m22 = (zfar + znear) / zd;
const m23 = - (zfar * znear * 2.0) / zd;
gl.uniformMatrix4fv(
this.shader_uniform.matrix,
false, new Float32Array([
m00, 0.0, 0.0, 0.0,
0.0, m11, 0.0, 0.0,
0.0, 0.0, m22, 1.0,
0.0, 0.0, m23, 0.0,
]));
}
{
const rpos = [0, 1, 2, 3].map((d, i) => {
const p = quadrangle[i];
return [p[0], p[1], depth[i]];
});
const [rp1, rp2, rp3, rp4] = rpos;
const iw = vecLen(rp1.map((v, i) => rp2[i] - v));
const ih = vecLen(rp1.map((v, i) => rp3[i] - v));
const [cw, ch] = [width, height];
const [rw, rh] = [iw / cw, ih / ch];
const rs = Math.max(rw, rh);
const [sw, sh] = [rw / rs * aspect, rh / rs];
const [dx, dy] = [sw, sh];
const image = this.texImage;
const [[s0, t0], [s1, t1], [s2, t2], [s3, t3]] = quadrangle;
const [tw, th] = [image.width, image.height];
const vStrip = createTriangleStrip([
[- dx * idep[0], + dy * idep[0], idep[0], s0 / tw, t0 / th],
[+ dx * idep[1], + dy * idep[1], idep[1], s1 / tw, t1 / th],
[- dx * idep[2], - dy * idep[2], idep[2], s2 / tw, t2 / th],
[+ dx * idep[3], - dy * idep[3], idep[3], s3 / tw, t3 / th],
]);
const vertex = new Float32Array(vStrip.map((v) => [v[0], v[1], v[2], 1.0]).flat());
const texture = new Float32Array(vStrip.map((v) => [v[3], v[4]]).flat());
gl.bindBuffer(gl.ARRAY_BUFFER, this.vertexBuffer);
gl.bufferData(gl.ARRAY_BUFFER, vertex, gl.STATIC_DRAW);
gl.vertexAttribPointer(this.shader_attrib.position, 4, gl.FLOAT, false, 0, 0);
gl.enableVertexAttribArray(this.shader_attrib.position);
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, this.texture);
gl.uniform1i(this.shader_uniform.sampler, 0);
gl.texImage2D(gl.TEXTURE_2D, 0,
gl.RGBA, tw, th, 0,
gl.RGBA, gl.UNSIGNED_BYTE, this.texImageData);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.bindBuffer(gl.ARRAY_BUFFER, this.textureCoord);
gl.bufferData(gl.ARRAY_BUFFER, texture, gl.STATIC_DRAW);
gl.vertexAttribPointer(this.shader_attrib.texture, 2, gl.FLOAT, false, 0, 0);
gl.enableVertexAttribArray(this.shader_attrib.texture);
}
gl.disable(gl.CULL_FACE);
gl.drawArrays(gl.TRIANGLE_STRIP, 0, triangleStripSize);
gl.finish();
gl.flush();
}
readPixels() {
const gl = this.context;
const width = gl.drawingBufferWidth;
const height = gl.drawingBufferHeight;
const pixels = new Uint8Array(width * height * 4)
gl.readPixels(0, 0, width, height, gl.RGBA, gl.UNSIGNED_BYTE, pixels);
return pixels;
}
}
/*
*
*/
class PreviewCanvas {
constructor(canvas) {
this.canvas = canvas;
this.context = canvas.getContext('2d');
}
}
/*
*
*/
var clickTimestamp = Date.now();
var previewImage;
var previewCanvas;
var rectangleCanvas;
var quadrangleCanvas;
function setFocus() {
window.focus();
}
function getWindowInnerSize() {
return [window.innerWidth, window.innerHeight];
}
function getWindowSize() {
return [window.innerWidth - 20, window.innerHeight - 20];
}
function setTexture(image) {
const [iw, ih] = [image.width, image.height];
tagRectangleCanvas.width = iw;
tagRectangleCanvas.height = ih;
tagTextureCanvas.width = iw;
tagTextureCanvas.height = ih;
const ctx = tagTextureCanvas.getContext('2d');
ctx.imageSmoothingEnabled = true;
ctx.imageSmoothingQuality = "high";
ctx.drawImage(image, 0, 0);
rectangleCanvas.setTexture(
ctx.getImageData(0, 0, iw, ih, { colorSpace: "srgb" }));
}
function updateRectImage() {
const ic = tagImageCanvas;
const pc = tagPreviewCanvas;
const tc = tagTextureCanvas;
const qc = quadrangleCanvas;
const qi = qc.image;
if (!qi || !qc.mousePosition) {
pc.hidden = true;
return;
}
const [mx, my] = qc.mousePosition;
const [cw, ch] = [ic.width, ic.height];
const [cx, cy] = [cw / 2, ch / 2];
const [dw, dh] = [cw / 4, ch / 4];
const caspect = cw / Math.max(1, ch);
const [tw, th] = [tc.width, tc.height];
const taspect = tw / Math.max(1, th);
let rw, rh;
if (taspect > caspect) {
rw = Math.trunc(cw / 2.5);
rh = Math.trunc(rw / taspect);
} else {
rh = Math.trunc(ch / 2.5);
rw = Math.trunc(rh * taspect);
}
const rx = (mx > cx) ? 16 : (cw - rw - 8);
const ry = (my > cy) ? 16 : (ch - rh - 8);
pc.style.left = `${Math.trunc(rx)}px`;
pc.style.top = `${Math.trunc(ry)}px`;
pc.width = rw;
pc.height = rh;
const pi = previewImage;
if (pi) {
const ctx = previewCanvas.context;
ctx.drawImage(pi[0], 0, 0, pi[1], pi[2], 0, 0, rw, rh);
}
pc.hidden = false;
}
function updatePreview() {
const rc = rectangleCanvas;
const iw = rc.context.drawingBufferWidth;
const ih = rc.context.drawingBufferHeight;
createImageBitmap(rc.canvas).then(
(res) => previewImage = [res, iw, ih]);
}
function updateTagSize() {
const [winw, winh] = getWindowSize();
quadrangleCanvas.setTagSize(winw, winh);
quadrangleCanvas.draw();
updateRectImage();
if (!tagRectangleImage.hidden)
setViewImagePosition();
}
function setViewImagePosition() {
const itag = tagRectangleImage;
const canvas = tagRectangleCanvas;
const [ww, wh] = getWindowSize();
const [cw, ch] = [canvas.width, canvas.height];
const i2w = Math.min(ww / cw, wh / ch) * 0.9;
const [iw, ih] = [cw * i2w + 16, ch * i2w + 16];
const [ix, iy] = [(ww - iw) / 2, (wh - ih) / 2];
itag.style.left = `${Math.trunc(ix)}px`;
itag.style.top = `${Math.trunc(iy)}px`;
itag.width = iw;
itag.height = ih;
const [iww, iwh] = getWindowInnerSize();
tagNoEvent.style.left = '0px;'
tagNoEvent.style.top = '0px;'
tagNoEvent.width = iww;
tagNoEvent.height = iwh;
}
function viewImage() {
if (!previewImage)
return;
const itag = tagRectangleImage;
if (!itag.hidden)
return;
setViewImagePosition();
itag.src = tagRectangleCanvas.toDataURL();
itag.hidden = false;
tagNoEvent.hidden = false;
enableScroll();
}
function hideImage() {
tagRectangleImage.hidden = true;
tagNoEvent.hidden = true;
disableScroll();
clickTimestamp = Date.now();
}
function hideHelp() {
tagHelp.hidden = true;
}
function getEventOffset(event) {
return [event.offsetX, event.offsetY];
}
function clearEventDefault(event) {
event.preventDefault();
event.stopPropagation();
}
function dropEvent(event) {
clearEventDefault(event);
const images = DropImage.fromEvent(event);
if (images.length == 1)
DropImage.load(images).then(
(files) => {
const image = files[0].image;
setTexture(image);
quadrangleCanvas.setImage(image);
quadrangleCanvas.draw();
setFocus();
},
console.error
);
}
function pointerEvent(event) {
if (event.type == 'pointerdown') {
hideImage();
hideHelp();
}
updateRectImage();
}
function keydownEvent(event) {
let help = false;
if (event.type == 'keydown') {
switch (event.key) {
case '?':
case 'h':
help = tagHelp.hidden;
break;
case 'v':
if (tagRectangleImage.hidden)
viewImage();
else
hideImage();
break;
default:
hideImage();
break;
}
tagHelp.hidden = !help;
}
}
function keyupEvent(event) {
}
function wheelEvent(event) {
updateRectImage();
}
/*
*
*/
var scrollingFlag = true;
function preventDefault(event) {
event.preventDefault();
}
function preventDefault4Key(event) {
if ([37, 38, 39, 40].includes(event.keyCode)) {
preventDefault(event);
return false;
}
}
function disableScroll() {
if (!scrollingFlag)
return;
scrollingFlag = false;
window.addEventListener('DOMMouseScroll', preventDefault, false);
window.addEventListener('keydown', preventDefault4Key, false);
window.addEventListener('touchmove', preventDefault, { passive: false });
window.addEventListener('wheel', preventDefault, { passive: false });
}
function enableScroll() {
if (scrollingFlag)
return;
scrollingFlag = true;
window.removeEventListener('DOMMouseScroll', preventDefault, false);
window.removeEventListener('keydown', preventDefault4Key, false);
window.removeEventListener('touchmove', preventDefault, { passive: false });
window.removeEventListener('wheel', preventDefault, { passive: false });
}
/*
*
*/
window.onload = function() {
previewCanvas = new PreviewCanvas(tagPreviewCanvas);
rectangleCanvas = new RenderRectangle(tagRectangleCanvas);
quadrangleCanvas = new QuadrangleSelectEditor(tagImageCanvas);
quadrangleCanvas.ondraw = ((q) => {
rectangleCanvas.drawRectangle(q);
updatePreview();
});
const tag = document.body;
tag.addEventListener('dragover', clearEventDefault);
tag.addEventListener('drop', dropEvent);
const userEvent = ((event) => {
if (tagRectangleImage.hidden &&
((Date.now() - clickTimestamp) > 100)) {
quadrangleCanvas.onUserEvent(event);
}
});
tagImageCanvas.addEventListener('pointerdown', userEvent);
tagImageCanvas.addEventListener('pointerdown', pointerEvent);
tagImageCanvas.addEventListener('pointerup', userEvent);
tagImageCanvas.addEventListener('pointerup', pointerEvent);
tagImageCanvas.addEventListener('pointermove', userEvent);
tagImageCanvas.addEventListener('pointermove', pointerEvent);
tagRectangleImage.addEventListener('pointerdown', (event) => {
clearEventDefault(event);
hideHelp();
});
window.addEventListener('pointerdown', pointerEvent);
window.addEventListener('keydown', userEvent);
window.addEventListener('keydown', keydownEvent);
window.addEventListener('keyup', keyupEvent);
window.addEventListener('wheel', userEvent);
window.addEventListener('wheel', wheelEvent);
window.onresize = updateTagSize;
updateTagSize();
disableScroll();
setFocus();
}
</script>
</body>
</html>
See the Pen 画像中の四角形を立体補正 by Ikiuo (@ikiuo) on CodePen.