デジタルオフィスというお題で、それぞれのリモートワークを仮想的なオフィス空間に表示することに取り組んでいます。
ざっとですが以下のようなイメージです。
今後、複数のリモート会議のブースが並ぶ大規模会場が出てくるかと思います。そういった複数のリモート会議も外から覗いたように見られれば、どのブースに入ろうか、というのがわかりやすいかな、という感じです。（多めの参加者をブレークアウトルームに分けるオンライン飲み会とかにも活用できないかな、と思ってみたりしています。）
この中で、人の姿勢推定データを活用し、VR上の人型オブジェクトに姿勢の再現を試していますので、その暫定内容を共有させていただきます。
（中途半端な内容ですが、次の段階へのステップアップを目標にするため、または、何かの参考になれば、と思い記事にしてみます）
現状では、以下のgifアニメーションイメージのような感じで、カメラに写っている人の姿勢（一人）を、VR上のオブジェクトが再現するようになりました。(残念ながらまだまだ方式や実装は不十分な状態で改良が必要です)
（VRの人型オブジェクトは2つありますが、角度が変えてあるだけで、同じ人の姿勢を再現させています。）
※ AWS EC2（GPU付）インスタンスにて動作確認
前提として、以下がありますので、ご注意ください。
- 動作確認のブラウザは、Google Chromeを使いました。
- 2021年8～9月頃に調べたり動作確認した内容です。
- Google Mediapipe Holisticのバージョンは「0.4.1628005088」を指定して使用しています。（世界座標:results.Xが取得できる版を指定しています。現在の最新版ではパラメータ変更等があったためかresults.Xが存在しなかったため(ただ、新たにresults.eaがありましたが詳細がわかっていません)。）
- サンプルのコード内では2か所（scriptエレメントのjsファイルURI、Holisticインスタンス生成時）でバージョンを指定しています。
以下の各章に、現状までの試行錯誤の流れをつづっていますので、イメージだけでもざっと流して見てもらえれば幸いです。
（経験者の方や知っている方から見ると、当たり前の話かもしれませんが、入門者の私は色々と躓いてしまいました。）
1. A-Frameで人型オブジェクトを作成
まず、人の姿勢を再現させるためのモデルを作成しました。
簡単で軽いモデルで良いと思い、A-Frameのa-boxを複数つなげただけで構成させました。
関節の曲げ伸ばしができるように、関節オブジェクトを入れて、腕等を動かしてみたり、少し動かしながら作り上げました。
以下のようなコード（一部抜粋）で親子関係を作って関節をつなげていきました。
次の章で補足していますが、右肩等の関節オブジェクトを2つ(upperとlower)作成しています。
関節1つだけだと（私の頭の理解力不足で）多段階の回転が計算できず、姿勢を再現できなかったので、2つ用意してみました。
const scale = 0.1;
const body_width = 2 * scale;
const body_height = 4 * scale;
const body_depth = 1 * scale;
const body_color = "#777";
const body = document.createElement("a-box");
body.setAttribute("position", "0.0, 0.0, -2.0");
body.setAttribute("rotation", "0.0, 0.0, 0.0");
body.setAttribute("width", String(body_width));
body.setAttribute("height", String(body_height));
body.setAttribute("depth", String(body_depth));
body.setAttribute("color", body_color);
body.setAttribute("id", "body_" + id);
const lower_neck_width = body_depth / 2;
const lower_neck_height = body_depth / 5;
const lower_neck_depth = body_depth / 2;
const lower_neck_color = "#555";
const lower_neck = document.createElement("a-box");
lower_neck.setAttribute("position", "0.0, " + String(body_height / 2 + lower_neck_height / 2) + ", 0.0");
lower_neck.setAttribute("rotation", "0.0, 0.0, 0.0");
lower_neck.setAttribute("width", String(lower_neck_width));
lower_neck.setAttribute("height", String(lower_neck_height));
lower_neck.setAttribute("depth", String(lower_neck_depth));
lower_neck.setAttribute("color", lower_neck_color);
lower_neck.setAttribute("id", "lower_neck_" + id);
body.appendChild(lower_neck);
const upper_neck_width = body_depth / 2;
const upper_neck_height = body_depth / 5;
const upper_neck_depth = body_depth / 2;
const upper_neck_color = "#555";
const upper_neck = document.createElement("a-box");
upper_neck.setAttribute("position", "0.0, 0.0, 0.0");
upper_neck.setAttribute("rotation", "0.0, 0.0, 0.0");
upper_neck.setAttribute("width", String(upper_neck_width));
upper_neck.setAttribute("height", String(upper_neck_height));
upper_neck.setAttribute("depth", String(upper_neck_depth));
upper_neck.setAttribute("color", upper_neck_color);
upper_neck.setAttribute("id", "upper_neck_" + id);
lower_neck.appendChild(upper_neck);
const head_width = body_depth;
const head_height = body_depth;
const head_depth = body_depth;
const head_color = "#AAA";
const head = document.createElement("a-box");
head.setAttribute("position", "0.0, " + String(lower_neck_height / 2 + head_height / 2) + ", 0.0");
head.setAttribute("rotation", "0.0, 0.0, 0.0");
head.setAttribute("width", String(head_width));
head.setAttribute("height", String(head_height));
head.setAttribute("depth", String(head_depth));
head.setAttribute("color", head_color);
head.setAttribute("id", "head_" + id);
upper_neck.appendChild(head);
...(略)
2. 姿勢の再現を試行
Google Mediapipe Holisticを使うと姿勢推定データ（Poseの全身とHandsの手とFaceMeshの顔の3点セット）が取得できます。
そのデータを使って各関節の座標データからベクトルを算出し、オイラー回転の角度を算出しました。
（ベクトルデータからの再現なので2軸の回転までとなり、関節の方向合わせまでです。手の甲が自分を向いているかどうか等の回転は未対応です。）
最初は、手が違う方向を向いていました。
オイラーの回転では、XYZ軸の回転の順番が違うとおかしくなるので直しつつやってみましたが、やっぱりうまく再現できませんでした（思ってたんと違う感じです。。。）
思い返してみると、取得していたデータは、カメラ投影座標のような、実世界の座標とは違うことに気づきました。
もう一度Mediapipeのページを見直したりインターネット検索で他の方の記事等を拝見していると、Mediapipe Poseが更新されて世界座標が取得できるようになっていました。
実世界の座標データに変えてやってみたのですが、肩等の1回目の回転はなんとなくできていそうですが、次の肘等の2回目の回転がうまくいきません。
親子関係で肩と肘をつなげているので、肩を回転すると肘の座標系（XYZ軸）が回転して変わってしまっているようです。なので、そこを考慮しないといけないようです。
オイラーの回転を1回実施するだけで、とても苦労をしたのですが、それを2段階となると、私には難しすぎて諦めました。
何かよい回避方法はないかと思って試したのが、関節にもう一つダミー関節を追加して、親側の回転の逆回転をさせることです。
これを行うことで上記の親側の肩の回転により肘の座標系が回転していたのを元に戻します。
それから肘の回転を試してみると、なんとか姿勢を再現できているような状態までいきました。
3. ひとまずここまでが限界かも
入門者としては、ここまでが限界と一旦区切りました。
これまでに結構時間がかかってしまったこともあり、ひとまず現状のありもので形にしてみました。
まだまだ再現できていない姿勢部分もありますが、手と足が大まかに動けばよしとしました。
4. 全体コードサンプル
コードが長く、また実験的な処理もあり、あまり人様に見せれるような感じではないですが、何かの参考までに全体のコードサンプルを載せておきます。
姿勢推定「Mediapipe」＋VR「A-Frame」連携コードサンプル：ソース
（この行をクリックするとソースコードが表示されます）
<html>
<head>
<title>Mediapipe A-Frame</title>
<!--
latest
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/holistic/holistic.js" crossorigin="anonymous"></script>
-->
<!--
majar version specify, but holistic can not results.X(world coordinate) now(2021/10/06)
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils@0.3/camera_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils@0.6/control_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils@0.3/drawing_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/holistic@ 0.4/holistic.js" crossorigin="anonymous"></script>
-->
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils@0.3.1632432234/camera_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils@0.6.1629159505/control_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils@0.3.1620248257/drawing_utils.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/holistic@ 0.4.1628005088/holistic.js" crossorigin="anonymous"></script>
<!-- include holistic extention and depend start -->
<script src="https://cdn.jsdelivr.net/gh/nicolaspanel/numjs@0.15.1/dist/numjs.min.js"></script>
<!--
<script src="holisticExtention.js"></script>
-->
<!-- include holistic extention and depend end -->
</head>
<body>
<label>
<span>facingMode(user(front camera), environment(rear camera), other)</span>
<input id="camera_option_facingMode" type="text" value="user" onchange="onChangeConfig()" />
</label>
<br>
<!-- need modify camera_utils.js(use deviceId) -->
<label>
<span>deviceId(not implement)</span>
<input id="camera_option_deviceId" type="text" value="" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>camera width</span>
<input id="camera_option_width" type="number" value="640" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>camera height</span>
<input id="camera_option_height" type="number" value="360" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>modelComplexity (long side)</span>
<select id="holistic_option_modelComplexity" onchange="onChangeConfig()">
<option value="0">Lite(0)</option>
<option value="1" selected="selected">Full(1)</option>
<option value="2">Heavy(2)</option>
</select>
</label>
<br>
<label>
<span>upperBodyOnly</span>
<input id="holistic_option_upperBodyOnly" type="checkbox" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>smoothLandmarks</span>
<input id="holistic_option_smoothLandmarks" type="checkbox" onchange="onChangeConfig()" checked="checked" />
</label>
<br>
<label>
<span>minDetectionConfidence</span>
<input id="holistic_option_minDetectionConfidence" type="number" step="0.1" value="0.5" min="0.0" max="1.0" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>minTrackingConfidence</span>
<input id="holistic_option_minTrackingConfidence" type="number" step="0.1" value="0.5" min="0.0" max="1.0" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>proc interval_msec(ms)</span>
<input id="proc_interval_msec" type="number" value="100" min="0" onchange="onChangeConfig()" />
</label>
<br>
<label>
<span>visibility threshold</span>
<input id="visibility_threshold" type="number" step="0.1" value="0.7" min="-1.0" max="1.0" onchange="onChangeConfig()" />
</label>
<br>
<br>
<p>debug info:<textarea id="debug_info" rows="7" cols="60" readonly=TRUE></textarea></p>
<div class="container">
<video class="input_video" autoplay playsinline></video>
<canvas class="output_canvas" width="640px" height="360px"></canvas>
</div>
<p>
<button onclick="stop()">stop</button>
<button onclick="start()" id="start">start</button>
</p>
<p>
<button onclick="stopMP()" id="stopMP">stop Mediapipe</button>
<button onclick="startMP()" id="startMP">start Mediapipe</button>
</p>
<script>
var debug_message_array = [];
var debug_message_max_size = 50;
var debug_message = "";
function add_debug_message(message) {
var now = new Date();
debug_message_array.push(now.toISOString() + " " + message);
if (debug_message_array.length > debug_message_max_size) {
debug_message_array.shift();
}
var all_debug_message = "";
for (var one_debug_message_index in debug_message_array) {
var one_debug_message = debug_message_array[one_debug_message_index];
all_debug_message += one_debug_message + "\r\n";
}
document.getElementById("debug_info").value = all_debug_message;
}
</script>
<script>
const canvasElement = document.getElementsByClassName('output_canvas')[0];
const camera_option_facingMode_element = document.getElementById('camera_option_facingMode');
const camera_option_deviceId_element = document.getElementById('camera_option_deviceId');
const camera_option_width_element = document.getElementById('camera_option_width');
const camera_option_height_element = document.getElementById('camera_option_height');
const holistic_option_upperBodyOnly_element = document.getElementById('holistic_option_upperBodyOnly');
const holistic_option_smoothLandmarks_element = document.getElementById('holistic_option_smoothLandmarks');
const holistic_option_modelComplexity_element = document.getElementById('holistic_option_modelComplexity');
const holistic_option_minDetectionConfidence_element = document.getElementById('holistic_option_minDetectionConfidence');
const holistic_option_minTrackingConfidence_element = document.getElementById('holistic_option_minTrackingConfidence');
const procIntervalMSec_element = document.getElementById('proc_interval_msec');
const visibility_threshold_element = document.getElementById('visibility_threshold');
var facing_mode = "user";
var device_id = "";
var image_width = 640;
var image_height = 360;
var is_upperBodyOnly = false;
var is_smoothLandmarks = true;
var modelComplexity = 1;
var minDetectionConfidence = 0.5;
var minTrackingConfidence = 0.5;
var procIntervalMSec = 100;
var visibility_threshold = 0.7
function getOption() {
facing_mode = camera_option_facingMode_element.value;
device_id = camera_option_deviceId_element.value;
image_width = parseInt(camera_option_width_element.value);
image_height = parseInt(camera_option_height_element.value);
if (holistic_option_upperBodyOnly_element.checked == true) {
is_upperBodyOnly = true;
} else {
is_upperBodyOnly = false;
}
if (holistic_option_smoothLandmarks_element.checked == true) {
is_smoothLandmarks = true;
} else {
is_smoothLandmarks = false;
}
modelComplexity = holistic_option_modelComplexity_element.selectedIndex;
minDetectionConfidence = parseFloat(holistic_option_minDetectionConfidence_element.value);
minTrackingConfidence = parseFloat(holistic_option_minTrackingConfidence_element.value);
procIntervalMSec = parseInt(procIntervalMSec_element.value);
visibility_threshold = parseFloat(visibility_threshold_element.value);
}
function setOption() {
camera_option_facingMode_element.value = facing_mode;
camera_option_deviceId_element.value = device_id;
camera_option_width_element.value = image_width;
camera_option_height_element.value = image_height;
holistic_option_upperBodyOnly_element.checked = is_upperBodyOnly;
holistic_option_smoothLandmarks_element.checked = is_smoothLandmarks;
holistic_option_modelComplexity_element.selectedIndex = modelComplexity;
holistic_option_minDetectionConfidence_element.value = String(minDetectionConfidence);
holistic_option_minTrackingConfidence_element.value = String(minTrackingConfidence);
procIntervalMSec_element.value = String(procIntervalMSec);
visibility_threshold_element.value = String(visibility_threshold);
canvasElement.setAttribute("width", String(image_width) + "px");
canvasElement.setAttribute("height", String(image_height) + "px");
}
function onChangeConfig() {
getOption();
setOption();
}
</script>
<style>
#scene, a-scene {
width: 100%;
height: 400px;
}
</style>
<script src="https://aframe.io/releases/1.2.0/aframe.min.js"></script>
<!-- -->
<A-scene embedded id="a-frame-vr2">
<A-entity id="camera_parent" position="0.0 0.0 0.0" rotation="0.0 0.0 0.0">
<A-camera id="camera" cursor="rayOrigin: mouse;" position="0.0 0.0 0.0" rotation="0.0 0.0 0.0" reverse-mouse-drag ></A-camera>
</A-entity>
</A-scene>
<!-- -->
<table>
<tr>
<th>move x y</th>
<th> </th>
<th>move x z</th>
<th> </th>
<th>rotation x y</th>
</tr>
<tr>
<td>
<!-- camera x y移動ボタン -->
<table>
<tr>
<td><input id="camera_move_l2_u2" type="button" onClick="camera_move(-2.0, -2.0, 0.0)" value="┏"></td>
<td><input id="camera_move_l1_u2" type="button" onClick="camera_move(-1.0, -2.0, 0.0)" value="↑"></td>
<td><input id="camera_move_u2" type="button" onClick="camera_move( 0.0, -2.0, 0.0)" value="↑"></td>
<td><input id="camera_move_r1_u2" type="button" onClick="camera_move( 1.0, -2.0, 0.0)" value="↑"></td>
<td><input id="camera_move_r2_u2" type="button" onClick="camera_move( 2.0, -2.0, 0.0)" value="┓"></td>
</tr>
<tr>
<td><input id="camera_move_l2_u1" type="button" onClick="camera_move(-2.0, -1.0, 0.0)" value="←"></td>
<td><input id="camera_move_l1_u1" type="button" onClick="camera_move(-1.0, -1.0, 0.0)" value="┌"></td>
<td><input id="camera_move_u1" type="button" onClick="camera_move( 0.0, -1.0, 0.0)" value="↑"></td>
<td><input id="camera_move_r1_u1" type="button" onClick="camera_move( 1.0, -1.0, 0.0)" value="┐"></td>
<td><input id="camera_move_r2_u1" type="button" onClick="camera_move( 2.0, -1.0, 0.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_move_l2" type="button" onClick="camera_move(-2.0, 0.0, 0.0)" value="←"></td>
<td><input id="camera_move_l1" type="button" onClick="camera_move(-1.0, 0.0, 0.0)" value="←"></td>
<td><input id="camera_move_0" type="button" onClick="camera_move_0()" value=" "></td>
<td><input id="camera_move_r1" type="button" onClick="camera_move( 1.0, 0.0, 0.0)" value="→"></td>
<td><input id="camera_move_r2" type="button" onClick="camera_move( 2.0, 0.0, 0.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_move_l2_d1" type="button" onClick="camera_move(-2.0, 1.0, 0.0)" value="←"></td>
<td><input id="camera_move_l1_d1" type="button" onClick="camera_move(-1.0, 1.0, 0.0)" value="└"></td>
<td><input id="camera_move_d1" type="button" onClick="camera_move( 0.0, 1.0, 0.0)" value="↓"></td>
<td><input id="camera_move_r1_d1" type="button" onClick="camera_move( 1.0, 1.0, 0.0)" value="┘"></td>
<td><input id="camera_move_r2_d1" type="button" onClick="camera_move( 2.0, 1.0, 0.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_move_l2_d2" type="button" onClick="camera_move(-2.0, 2.0, 0.0)" value="┗"></td>
<td><input id="camera_move_l1_d2" type="button" onClick="camera_move(-1.0, 2.0, 0.0)" value="↓"></td>
<td><input id="camera_move_d2" type="button" onClick="camera_move( 0.0, 2.0, 0.0)" value="↓"></td>
<td><input id="camera_move_r1_d2" type="button" onClick="camera_move( 1.0, 2.0, 0.0)" value="↓"></td>
<td><input id="camera_move_r2_d2" type="button" onClick="camera_move( 2.0, 2.0, 0.0)" value="┛"></td>
</tr>
</table>
</td>
<td> </td>
<td>
<!-- camera x z移動ボタン -->
<table>
<tr>
<td><input id="camera_move_l2_f2" type="button" onClick="camera_move(-2.0, 0.0, 2.0)" value="┏"></td>
<td><input id="camera_move_l1_f2" type="button" onClick="camera_move(-1.0, 0.0, 2.0)" value="↑"></td>
<td><input id="camera_move_f2" type="button" onClick="camera_move( 0.0, 0.0, 2.0)" value="↑"></td>
<td><input id="camera_move_r1_f2" type="button" onClick="camera_move( 1.0, 0.0, 2.0)" value="↑"></td>
<td><input id="camera_move_r2_f2" type="button" onClick="camera_move( 2.0, 0.0, 2.0)" value="┓"></td>
</tr>
<tr>
<td><input id="camera_move_l2_f1" type="button" onClick="camera_move(-2.0, 0.0, 1.0)" value="←"></td>
<td><input id="camera_move_l1_f1" type="button" onClick="camera_move(-1.0, 0.0, 1.0)" value="┌"></td>
<td><input id="camera_move_f1" type="button" onClick="camera_move( 0.0, 0.0, 1.0)" value="↑"></td>
<td><input id="camera_move_r1_f1" type="button" onClick="camera_move( 1.0, 0.0, 1.0)" value="┐"></td>
<td><input id="camera_move_r2_f1" type="button" onClick="camera_move( 2.0, 0.0, 1.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_move_l2" type="button" onClick="camera_move(-2.0, 0.0, 0.0)" value="←"></td>
<td><input id="camera_move_l1" type="button" onClick="camera_move(-1.0, 0.0, 0.0)" value="←"></td>
<td><input id="camera_move_0" type="button" onClick="camera_move_0()" value=" "></td>
<td><input id="camera_move_r1" type="button" onClick="camera_move( 1.0, 0.0, 0.0)" value="→"></td>
<td><input id="camera_move_r2" type="button" onClick="camera_move( 2.0, 0.0, 0.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_move_l2_b1" type="button" onClick="camera_move(-2.0, 0.0, -1.0)" value="←"></td>
<td><input id="camera_move_l1_b1" type="button" onClick="camera_move(-1.0, 0.0, -1.0)" value="└"></td>
<td><input id="camera_move_b1" type="button" onClick="camera_move( 0.0, 0.0, -1.0)" value="↓"></td>
<td><input id="camera_move_r1_b1" type="button" onClick="camera_move( 1.0, 0.0, -1.0)" value="┘"></td>
<td><input id="camera_move_r2_b1" type="button" onClick="camera_move( 2.0, 0.0, -1.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_move_l2_b2" type="button" onClick="camera_move(-2.0, 0.0, -2.0)" value="┗"></td>
<td><input id="camera_move_l1_b2" type="button" onClick="camera_move(-1.0, 0.0, -2.0)" value="↓"></td>
<td><input id="camera_move_b2" type="button" onClick="camera_move( 0.0, 0.0, -2.0)" value="↓"></td>
<td><input id="camera_move_r1_b2" type="button" onClick="camera_move( 1.0, 0.0, -2.0)" value="↓"></td>
<td><input id="camera_move_r2_b2" type="button" onClick="camera_move( 2.0, 0.0, -2.0)" value="┛"></td>
</tr>
</table>
</td>
<td> </td>
<td>
<!-- camera方向ボタン -->
<table>
<tr>
<td><input id="camera_rotation_l2_u2" type="button" onClick="camera_rotation(-2.0, 0.0, -2.0)" value="┏"></td>
<td><input id="camera_rotation_l1_u2" type="button" onClick="camera_rotation(-1.0, 0.0, -2.0)" value="↑"></td>
<td><input id="camera_rotation_u2" type="button" onClick="camera_rotation( 0.0, 0.0, -2.0)" value="↑"></td>
<td><input id="camera_rotation_l1_u2" type="button" onClick="camera_rotation( 1.0, 0.0, -2.0)" value="↑"></td>
<td><input id="camera_rotation_l2_u2" type="button" onClick="camera_rotation( 2.0, 0.0, -2.0)" value="┓"></td>
</tr>
<tr>
<td><input id="camera_rotation_l2_u1" type="button" onClick="camera_rotation(-2.0, 0.0, -1.0)" value="←"></td>
<td><input id="camera_rotation_l1_u1" type="button" onClick="camera_rotation(-1.0, 0.0, -1.0)" value="┌"></td>
<td><input id="camera_rotation_u1" type="button" onClick="camera_rotation( 0.0, 0.0, -1.0)" value="↑"></td>
<td><input id="camera_rotation_r1_u1" type="button" onClick="camera_rotation( 1.0, 0.0, -1.0)" value="┐"></td>
<td><input id="camera_rotation_r2_u1" type="button" onClick="camera_rotation( 2.0, 0.0, -1.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_rotation_l2" type="button" onClick="camera_rotation(-2.0, 0.0, 0.0)" value="←"></td>
<td><input id="camera_rotation_l1" type="button" onClick="camera_rotation(-1.0, 0.0, 0.0)" value="←"></td>
<td><input id="camera_rotation_0" type="button" onClick="camera_rotation_0()" value=" "></td>
<td><input id="camera_rotation_r1" type="button" onClick="camera_rotation( 1.0, 0.0, 0.0)" value="→"></td>
<td><input id="camera_rotation_r2" type="button" onClick="camera_rotation( 2.0, 0.0, 0.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_rotation_l2_d1" type="button" onClick="camera_rotation(-2.0, 0.0, 1.0)" value="←"></td>
<td><input id="camera_rotation_l1_d1" type="button" onClick="camera_rotation(-1.0, 0.0, 1.0)" value="└"></td>
<td><input id="camera_rotation_d1" type="button" onClick="camera_rotation( 0.0, 0.0, 1.0)" value="↓"></td>
<td><input id="camera_rotation_r1_d1" type="button" onClick="camera_rotation( 1.0, 0.0, 1.0)" value="┘"></td>
<td><input id="camera_rotation_r2_d1" type="button" onClick="camera_rotation( 2.0, 0.0, 1.0)" value="→"></td>
</tr>
<tr>
<td><input id="camera_rotation_l2_d2" type="button" onClick="camera_rotation(-2.0, 0.0, 2.0)" value="┗"></td>
<td><input id="camera_rotation_l1_d2" type="button" onClick="camera_rotation(-1.0, 0.0, 2.0)" value="↓"></td>
<td><input id="camera_rotation_d2" type="button" onClick="camera_rotation( 0.0, 0.0, 2.0)" value="↓"></td>
<td><input id="camera_rotation_l1_d2" type="button" onClick="camera_rotation( 1.0, 0.0, 2.0)" value="↓"></td>
<td><input id="camera_rotation_l2_d2" type="button" onClick="camera_rotation( 2.0, 0.0, 2.0)" value="┛"></td>
</tr>
</table>
</td>
</tr>
</table>
<script>
var debug_message_array = [];
var debug_message_max_size = 50;
var debug_message = "";
function add_debug_message(message) {
var now = new Date();
debug_message_array.push(now.toISOString() + " " + message);
if (debug_message_array.length > debug_message_max_size) {
debug_message_array.shift();
}
var all_debug_message = "";
for (var one_debug_message_index in debug_message_array) {
var one_debug_message = debug_message_array[one_debug_message_index];
all_debug_message += one_debug_message + "\r\n";
}
document.getElementById("debug_info").value = all_debug_message;
}
// camera
const camera_element = document.getElementById("camera");
var position = null;
var rotation = null;
if (camera_element != null) {
position = camera_element.getAttribute("position");
rotation = camera_element.getAttribute("rotation");
}
// camera move
var move_level = 1.0;
var move_ratio = 1.0;
var move_reverse_left_right = 1.0;
var move_reverse_up_down = -1.0;
var move_reverse_front_back = -1.0;
function camera_move(move_ratio_x, move_ratio_y, move_ratio_z) {
position.x += 1 * move_ratio_x * move_ratio * move_level * move_reverse_left_right;
position.y += 1 * move_ratio_y * move_ratio * move_level * move_reverse_up_down;
position.z += 1 * move_ratio_z * move_ratio * move_level * move_reverse_front_back;
}
function camera_move_0() {
position.x = 0.0;
position.y = 0.0;
position.z = 0.0;
}
// camera rotation
var rotation_level = 1.0;
var rotation_ratio = 10.0;
var rotation_reverse_left_right = -1.0;
var rotation_reverse_up_down = -1.0;
var rotation_reverse_front_back = -1.0;
function camera_rotation(rotation_ratio_x, rotation_ratio_y, rotation_ratio_z) {
camera_element.components["look-controls"].pitchObject.rotation.x += 1 * rotation_ratio_z * rotation_ratio * rotation_level * rotation_reverse_left_right * (Math.PI / 180);
camera_element.components["look-controls"].yawObject.rotation.y += 1 * rotation_ratio_x * rotation_ratio * rotation_level * rotation_reverse_up_down * (Math.PI / 180);
camera_element.components["look-controls"].pitchObject.rotation.z += 1 * rotation_ratio_y * rotation_ratio * rotation_level * rotation_reverse_front_back * (Math.PI / 180);
}
function camera_rotation_0() {
camera_element.components["look-controls"].pitchObject.rotation.x = 0.0;
camera_element.components["look-controls"].yawObject.rotation.y = 0.0;
camera_element.components["look-controls"].pitchObject.rotation.z = 0.0;
}
function add_avatar(id) {
const parent_element = document.getElementById("a-frame-vr2");
if (parent_element == null) {
return;
}
const scale = 0.1;
const body_width = 2 * scale;
const body_height = 4 * scale;
const body_depth = 1 * scale;
const body_color = "#777";
const body = document.createElement("a-box");
body.setAttribute("position", "0.0, 0.0, -2.0");
body.setAttribute("rotation", "0.0, 0.0, 0.0");
body.setAttribute("width", String(body_width));
body.setAttribute("height", String(body_height));
body.setAttribute("depth", String(body_depth));
body.setAttribute("color", body_color);
body.setAttribute("id", "body_" + id);
const lower_neck_width = body_depth / 2;
const lower_neck_height = body_depth / 5;
const lower_neck_depth = body_depth / 2;
const lower_neck_color = "#555";
const lower_neck = document.createElement("a-box");
lower_neck.setAttribute("position", "0.0, " + String(body_height / 2 + lower_neck_height / 2) + ", 0.0");
lower_neck.setAttribute("rotation", "0.0, 0.0, 0.0");
lower_neck.setAttribute("width", String(lower_neck_width));
lower_neck.setAttribute("height", String(lower_neck_height));
lower_neck.setAttribute("depth", String(lower_neck_depth));
lower_neck.setAttribute("color", lower_neck_color);
lower_neck.setAttribute("id", "lower_neck_" + id);
body.appendChild(lower_neck);
const upper_neck_width = body_depth / 2;
const upper_neck_height = body_depth / 5;
const upper_neck_depth = body_depth / 2;
const upper_neck_color = "#555";
const upper_neck = document.createElement("a-box");
upper_neck.setAttribute("position", "0.0, 0.0, 0.0");
upper_neck.setAttribute("rotation", "0.0, 0.0, 0.0");
upper_neck.setAttribute("width", String(upper_neck_width));
upper_neck.setAttribute("height", String(upper_neck_height));
upper_neck.setAttribute("depth", String(upper_neck_depth));
upper_neck.setAttribute("color", upper_neck_color);
upper_neck.setAttribute("id", "upper_neck_" + id);
lower_neck.appendChild(upper_neck);
const head_width = body_depth;
const head_height = body_depth;
const head_depth = body_depth;
const head_color = "#AAA";
const head = document.createElement("a-box");
head.setAttribute("position", "0.0, " + String(lower_neck_height / 2 + head_height / 2) + ", 0.0");
head.setAttribute("rotation", "0.0, 0.0, 0.0");
head.setAttribute("width", String(head_width));
head.setAttribute("height", String(head_height));
head.setAttribute("depth", String(head_depth));
head.setAttribute("color", head_color);
head.setAttribute("id", "head_" + id);
upper_neck.appendChild(head);
const nose_width = head_depth / 5;
const nose_height = head_depth / 5;
const nose_depth = head_depth / 5;
const nose_color = "#555";
const nose = document.createElement("a-box");
nose.setAttribute("position", "0.0, 0.0, " + String(- head_depth / 2 - nose_depth / 2));
nose.setAttribute("rotation", "90.0, 0.0, 0.0");
nose.setAttribute("width", String(nose_width));
nose.setAttribute("height", String(nose_height));
nose.setAttribute("depth", String(nose_depth));
nose.setAttribute("color", nose_color);
nose.setAttribute("id", "nose_" + id);
head.appendChild(nose);
const left_upper_shoulder_width = body_depth / 2 * 0.9;
const left_upper_shoulder_height = body_height / 4 / 2 * 0.9;
const left_upper_shoulder_depth = body_depth / 2 * 0.9;
const left_upper_shoulder_color = "#555";
const left_upper_shoulder = document.createElement("a-box");
left_upper_shoulder.setAttribute("position", String(- body_width / 2 - left_upper_shoulder_width / 2) + ", " + String(- left_upper_shoulder_height + body_height / 2) + ", 0.0");
left_upper_shoulder.setAttribute("rotation", "0.0, 0.0, 0.0");
left_upper_shoulder.setAttribute("width", String(left_upper_shoulder_width));
left_upper_shoulder.setAttribute("height", String(left_upper_shoulder_height));
left_upper_shoulder.setAttribute("depth", String(left_upper_shoulder_depth));
left_upper_shoulder.setAttribute("color", left_upper_shoulder_color);
left_upper_shoulder.setAttribute("id", "left_upper_shoulder_" + id);
body.appendChild(left_upper_shoulder);
const left_lower_shoulder_width = body_depth / 2 * 0.9;
const left_lower_shoulder_height = body_height / 4 / 2 * 0.9;
const left_lower_shoulder_depth = body_depth / 2 * 0.9;
const left_lower_shoulder_color = "#555";
const left_lower_shoulder = document.createElement("a-box");
left_lower_shoulder.setAttribute("position", "0.0, 0.0, 0.0");
left_lower_shoulder.setAttribute("rotation", "0.0, 0.0, 0.0");
left_lower_shoulder.setAttribute("width", String(left_lower_shoulder_width));
left_lower_shoulder.setAttribute("height", String(left_lower_shoulder_height));
left_lower_shoulder.setAttribute("depth", String(left_lower_shoulder_depth));
left_lower_shoulder.setAttribute("color", left_lower_shoulder_color);
left_lower_shoulder.setAttribute("id", "left_lower_shoulder_" + id);
left_upper_shoulder.appendChild(left_lower_shoulder);
const left_upper_arm_width = body_depth / 2 * 0.9;
const left_upper_arm_height = body_height / 2 * 0.9;
const left_upper_arm_depth = body_depth / 2 * 0.9;
const left_upper_arm_color = "#AAA";
const left_upper_arm = document.createElement("a-box");
left_upper_arm.setAttribute("position", String(- left_upper_arm_width / 2) + ", " + String(- left_upper_arm_height / 2) + ", 0.0");
left_upper_arm.setAttribute("rotation", "0.0, 0.0, 0.0");
left_upper_arm.setAttribute("width", String(left_upper_arm_width));
left_upper_arm.setAttribute("height", String(left_upper_arm_height));
left_upper_arm.setAttribute("depth", String(left_upper_arm_depth));
left_upper_arm.setAttribute("color", left_upper_arm_color);
left_upper_arm.setAttribute("id", "left_upper_arm_" + id);
left_lower_shoulder.appendChild(left_upper_arm);
const left_upper_elbow_width = body_depth / 2 * 0.8;
const left_upper_elbow_height = body_height / 4 / 2 * 0.8;
const left_upper_elbow_depth = body_depth / 2 * 0.8;
const left_upper_elbow_color = "#555";
const left_upper_elbow = document.createElement("a-box");
left_upper_elbow.setAttribute("position", "0.0, " + String(- left_upper_elbow_height / 2 - left_upper_arm_height / 2) + ", 0.0");
left_upper_elbow.setAttribute("rotation", "0.0, 0.0, 0.0");
left_upper_elbow.setAttribute("width", String(left_upper_elbow_width));
left_upper_elbow.setAttribute("height", String(left_upper_elbow_height));
left_upper_elbow.setAttribute("depth", String(left_upper_elbow_depth));
left_upper_elbow.setAttribute("color", left_upper_elbow_color);
left_upper_elbow.setAttribute("id", "left_upper_elbow_" + id);
left_upper_arm.appendChild(left_upper_elbow);
const left_lower_elbow_width = body_depth / 2 * 0.8;
const left_lower_elbow_height = body_height / 4 / 2 * 0.8;
const left_lower_elbow_depth = body_depth / 2 * 0.8;
const left_lower_elbow_color = "#555";
const left_lower_elbow = document.createElement("a-box");
left_lower_elbow.setAttribute("position", "0.0, 0.0, 0.0");
left_lower_elbow.setAttribute("rotation", "0.0, 0.0, 0.0");
left_lower_elbow.setAttribute("width", String(left_lower_elbow_width));
left_lower_elbow.setAttribute("height", String(left_lower_elbow_height));
left_lower_elbow.setAttribute("depth", String(left_lower_elbow_depth));
left_lower_elbow.setAttribute("color", left_lower_elbow_color);
left_lower_elbow.setAttribute("id", "left_lower_elbow_" + id);
left_upper_elbow.appendChild(left_lower_elbow);
const left_lower_arm_width = body_depth / 2 * 0.8;
const left_lower_arm_height = body_height / 2 * 0.8;
const left_lower_arm_depth = body_depth / 2 * 0.8;
const left_lower_arm_color = "#AAA";
const left_lower_arm = document.createElement("a-box");
left_lower_arm.setAttribute("position", "0.0, " + String(- left_lower_arm_height / 2) + ", 0.0");
left_lower_arm.setAttribute("rotation", "0.0, 0.0, 0.0");
left_lower_arm.setAttribute("width", String(left_lower_arm_width));
left_lower_arm.setAttribute("height", String(left_lower_arm_height));
left_lower_arm.setAttribute("depth", String(left_lower_arm_depth));
left_lower_arm.setAttribute("color", left_lower_arm_color);
left_lower_arm.setAttribute("id", "left_lower_arm_" + id);
left_lower_elbow.appendChild(left_lower_arm);
const right_upper_shoulder_width = body_depth / 2 * 0.9;
const right_upper_shoulder_height = body_height / 4 / 2 * 0.9;
const right_upper_shoulder_depth = body_depth / 2 * 0.9;
const right_upper_shoulder_color = "#555";
const right_upper_shoulder = document.createElement("a-box");
right_upper_shoulder.setAttribute("position", String( body_width / 2 + right_upper_shoulder_width / 2) + ", " + String(- right_upper_shoulder_height + body_height / 2) + ", 0.0");
right_upper_shoulder.setAttribute("rotation", "0.0, 0.0, 0.0");
right_upper_shoulder.setAttribute("width", String(right_upper_shoulder_width));
right_upper_shoulder.setAttribute("height", String(right_upper_shoulder_height));
right_upper_shoulder.setAttribute("depth", String(right_upper_shoulder_depth));
right_upper_shoulder.setAttribute("color", right_upper_shoulder_color);
right_upper_shoulder.setAttribute("id", "right_upper_shoulder_" + id);
body.appendChild(right_upper_shoulder);
const right_lower_shoulder_width = body_depth / 2 * 0.9;
const right_lower_shoulder_height = body_height / 4 / 2 * 0.9;
const right_lower_shoulder_depth = body_depth / 2 * 0.9;
const right_lower_shoulder_color = "#555";
const right_lower_shoulder = document.createElement("a-box");
right_lower_shoulder.setAttribute("position", "0.0, 0.0, 0.0");
right_lower_shoulder.setAttribute("rotation", "0.0, 0.0, 0.0");
right_lower_shoulder.setAttribute("width", String(right_lower_shoulder_width));
right_lower_shoulder.setAttribute("height", String(right_lower_shoulder_height));
right_lower_shoulder.setAttribute("depth", String(right_lower_shoulder_depth));
right_lower_shoulder.setAttribute("color", right_lower_shoulder_color);
right_lower_shoulder.setAttribute("id", "right_lower_shoulder_" + id);
right_upper_shoulder.appendChild(right_lower_shoulder);
const right_upper_arm_width = body_depth / 2 * 0.9;
const right_upper_arm_height = body_height / 2 * 0.9;
const right_upper_arm_depth = body_depth / 2 * 0.9;
const right_upper_arm_color = "#AAA";
const right_upper_arm = document.createElement("a-box");
right_upper_arm.setAttribute("position", String(+ right_upper_arm_width / 2) + ", " + String(- right_upper_arm_height / 2) + ", 0.0");
right_upper_arm.setAttribute("rotation", "0.0, 0.0, 0.0");
right_upper_arm.setAttribute("width", String(right_upper_arm_width));
right_upper_arm.setAttribute("height", String(right_upper_arm_height));
right_upper_arm.setAttribute("depth", String(right_upper_arm_depth));
right_upper_arm.setAttribute("color", right_upper_arm_color);
right_upper_arm.setAttribute("id", "right_upper_arm_" + id);
right_lower_shoulder.appendChild(right_upper_arm);
const right_upper_elbow_width = body_depth / 2 * 0.8;
const right_upper_elbow_height = body_height / 4 / 2 * 0.8;
const right_upper_elbow_depth = body_depth / 2 * 0.8;
const right_upper_elbow_color = "#555";
const right_upper_elbow = document.createElement("a-box");
right_upper_elbow.setAttribute("position", "0.0, " + String(- right_upper_elbow_height / 2 - right_upper_arm_height / 2) + ", 0.0");
right_upper_elbow.setAttribute("rotation", "0.0, 0.0, 0.0");
right_upper_elbow.setAttribute("width", String(right_upper_elbow_width));
right_upper_elbow.setAttribute("height", String(right_upper_elbow_height));
right_upper_elbow.setAttribute("depth", String(right_upper_elbow_depth));
right_upper_elbow.setAttribute("color", right_upper_elbow_color);
right_upper_elbow.setAttribute("id", "right_upper_elbow_" + id);
right_upper_arm.appendChild(right_upper_elbow);
const right_lower_elbow_width = body_depth / 2 * 0.8;
const right_lower_elbow_height = body_height / 4 / 2 * 0.8;
const right_lower_elbow_depth = body_depth / 2 * 0.8;
const right_lower_elbow_color = "#555";
const right_lower_elbow = document.createElement("a-box");
right_lower_elbow.setAttribute("position", "0.0, 0.0, 0.0");
right_lower_elbow.setAttribute("rotation", "0.0, 0.0, 0.0");
right_lower_elbow.setAttribute("width", String(right_lower_elbow_width));
right_lower_elbow.setAttribute("height", String(right_lower_elbow_height));
right_lower_elbow.setAttribute("depth", String(right_lower_elbow_depth));
right_lower_elbow.setAttribute("color", right_lower_elbow_color);
right_lower_elbow.setAttribute("id", "right_lower_elbow_" + id);
right_upper_elbow.appendChild(right_lower_elbow);
const right_lower_arm_width = body_depth / 2 * 0.8;
const right_lower_arm_height = body_height / 2 * 0.8;
const right_lower_arm_depth = body_depth / 2 * 0.8;
const right_lower_arm_color = "#AAA";
const right_lower_arm = document.createElement("a-box");
right_lower_arm.setAttribute("position", "0.0, " + String(- right_lower_arm_height / 2) + ", 0.0");
right_lower_arm.setAttribute("rotation", "0.0, 0.0, 0.0");
right_lower_arm.setAttribute("width", String(right_lower_arm_width));
right_lower_arm.setAttribute("height", String(right_lower_arm_height));
right_lower_arm.setAttribute("depth", String(right_lower_arm_depth));
right_lower_arm.setAttribute("color", right_lower_arm_color);
right_lower_arm.setAttribute("id", "right_lower_arm_" + id);
right_lower_elbow.appendChild(right_lower_arm);
const left_upper_base_foot_width = body_depth * 0.9;
const left_upper_base_foot_height = body_height / 4 * 0.9;
const left_upper_base_foot_depth = body_depth * 0.9;
const left_upper_base_foot_color = "#555";
const left_upper_base_foot = document.createElement("a-box");
left_upper_base_foot.setAttribute("position", String(- body_width / 4) + ", " + String(- body_height / 2 - left_upper_base_foot_height / 2) + ", 0.0");
left_upper_base_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
left_upper_base_foot.setAttribute("width", String(left_upper_base_foot_width));
left_upper_base_foot.setAttribute("height", String(left_upper_base_foot_height));
left_upper_base_foot.setAttribute("depth", String(left_upper_base_foot_depth));
left_upper_base_foot.setAttribute("color", left_upper_base_foot_color);
left_upper_base_foot.setAttribute("id", "left_upper_base_foot_" + id);
body.appendChild(left_upper_base_foot);
const left_lower_base_foot_width = body_depth * 0.9;
const left_lower_base_foot_height = body_height / 4 * 0.9;
const left_lower_base_foot_depth = body_depth * 0.9;
const left_lower_base_foot_color = "#555";
const left_lower_base_foot = document.createElement("a-box");
left_lower_base_foot.setAttribute("position", "0.0, 0.0, 0.0");
left_lower_base_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
left_lower_base_foot.setAttribute("width", String(left_lower_base_foot_width));
left_lower_base_foot.setAttribute("height", String(left_lower_base_foot_height));
left_lower_base_foot.setAttribute("depth", String(left_lower_base_foot_depth));
left_lower_base_foot.setAttribute("color", left_lower_base_foot_color);
left_lower_base_foot.setAttribute("id", "left_lower_base_foot_" + id);
left_upper_base_foot.appendChild(left_lower_base_foot);
const left_upper_foot_width = body_depth * 0.9;
const left_upper_foot_height = body_height / 4 / 2 * 5 * 0.9;
const left_upper_foot_depth = body_depth * 0.9;
const left_upper_foot_color = "#AAA";
const left_upper_foot = document.createElement("a-box");
left_upper_foot.setAttribute("position", "0.0, " + String(- left_upper_foot_height / 2) + ", 0.0");
left_upper_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
left_upper_foot.setAttribute("width", String(left_upper_foot_width));
left_upper_foot.setAttribute("height", String(left_upper_foot_height));
left_upper_foot.setAttribute("depth", String(left_upper_foot_depth));
left_upper_foot.setAttribute("color", left_upper_foot_color);
left_upper_foot.setAttribute("id", "left_upper_foot_" + id);
left_lower_base_foot.appendChild(left_upper_foot);
const left_upper_knee_width = body_depth * 0.9;
const left_upper_knee_height = body_height / 4 * 0.9;
const left_upper_knee_depth = body_depth * 0.9;
const left_upper_knee_color = "#555";
const left_upper_knee = document.createElement("a-box");
left_upper_knee.setAttribute("position", "0.0, " + String(- left_upper_foot_height / 2 - left_upper_knee_height / 2) + ", 0.0");
left_upper_knee.setAttribute("rotation", "0.0, 0.0, 0.0");
left_upper_knee.setAttribute("width", String(left_upper_knee_width));
left_upper_knee.setAttribute("height", String(left_upper_knee_height));
left_upper_knee.setAttribute("depth", String(left_upper_knee_depth));
left_upper_knee.setAttribute("color", left_upper_knee_color);
left_upper_knee.setAttribute("id", "left_upper_knee_" + id);
left_upper_foot.appendChild(left_upper_knee);
const left_lower_knee_width = body_depth * 0.9;
const left_lower_knee_height = body_height / 4 * 0.9;
const left_lower_knee_depth = body_depth * 0.9;
const left_lower_knee_color = "#555";
const left_lower_knee = document.createElement("a-box");
left_lower_knee.setAttribute("position", "0.0, 0.0, 0.0");
left_lower_knee.setAttribute("rotation", "0.0, 0.0, 0.0");
left_lower_knee.setAttribute("width", String(left_lower_knee_width));
left_lower_knee.setAttribute("height", String(left_lower_knee_height));
left_lower_knee.setAttribute("depth", String(left_lower_knee_depth));
left_lower_knee.setAttribute("color", left_lower_knee_color);
left_lower_knee.setAttribute("id", "left_lower_knee_" + id);
left_upper_knee.appendChild(left_lower_knee);
const left_lower_foot_width = body_depth * 0.8;
const left_lower_foot_height = body_height / 4 / 2 * 5 * 0.8;
const left_lower_foot_depth = body_depth * 0.8;
const left_lower_foot_color = "#AAA";
const left_lower_foot = document.createElement("a-box");
left_lower_foot.setAttribute("position", "0.0, " + String(- left_lower_foot_height / 2 - left_lower_knee_height / 2) + ", 0.0");
left_lower_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
left_lower_foot.setAttribute("width", String(left_lower_foot_width));
left_lower_foot.setAttribute("height", String(left_lower_foot_height));
left_lower_foot.setAttribute("depth", String(left_lower_foot_depth));
left_lower_foot.setAttribute("color", left_lower_foot_color);
left_lower_foot.setAttribute("id", "left_lower_foot_" + id);
left_lower_knee.appendChild(left_lower_foot);
const right_upper_base_foot_width = body_depth * 0.9;
const right_upper_base_foot_height = body_height / 4 * 0.9;
const right_upper_base_foot_depth = body_depth * 0.9;
const right_upper_base_foot_color = "#555";
const right_upper_base_foot = document.createElement("a-box");
right_upper_base_foot.setAttribute("position", String( body_width / 4) + ", " + String(- body_height / 2 - right_upper_base_foot_height / 2) + ", 0.0");
right_upper_base_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
right_upper_base_foot.setAttribute("width", String(right_upper_base_foot_width));
right_upper_base_foot.setAttribute("height", String(right_upper_base_foot_height));
right_upper_base_foot.setAttribute("depth", String(right_upper_base_foot_depth));
right_upper_base_foot.setAttribute("color", right_upper_base_foot_color);
right_upper_base_foot.setAttribute("id", "right_upper_base_foot_" + id);
body.appendChild(right_upper_base_foot);
const right_lower_base_foot_width = body_depth * 0.9;
const right_lower_base_foot_height = body_height / 4 * 0.9;
const right_lower_base_foot_depth = body_depth * 0.9;
const right_lower_base_foot_color = "#555";
const right_lower_base_foot = document.createElement("a-box");
right_lower_base_foot.setAttribute("position", "0.0, 0.0, 0.0");
right_lower_base_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
right_lower_base_foot.setAttribute("width", String(right_lower_base_foot_width));
right_lower_base_foot.setAttribute("height", String(right_lower_base_foot_height));
right_lower_base_foot.setAttribute("depth", String(right_lower_base_foot_depth));
right_lower_base_foot.setAttribute("color", right_lower_base_foot_color);
right_lower_base_foot.setAttribute("id", "right_lower_base_foot_" + id);
right_upper_base_foot.appendChild(right_lower_base_foot);
const right_upper_foot_width = body_depth * 0.9;
const right_upper_foot_height = body_height / 4 / 2 * 5 * 0.9;
const right_upper_foot_depth = body_depth * 0.9;
const right_upper_foot_color = "#AAA";
const right_upper_foot = document.createElement("a-box");
right_upper_foot.setAttribute("position", "0.0, " + String(- right_upper_foot_height / 2) + ", 0.0");
right_upper_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
right_upper_foot.setAttribute("width", String(right_upper_foot_width));
right_upper_foot.setAttribute("height", String(right_upper_foot_height));
right_upper_foot.setAttribute("depth", String(right_upper_foot_depth));
right_upper_foot.setAttribute("color", right_upper_foot_color);
right_upper_foot.setAttribute("id", "right_upper_foot_" + id);
right_lower_base_foot.appendChild(right_upper_foot);
const right_upper_knee_width = body_depth * 0.9;
const right_upper_knee_height = body_height / 4 * 0.9;
const right_upper_knee_depth = body_depth * 0.9;
const right_upper_knee_color = "#555";
const right_upper_knee = document.createElement("a-box");
right_upper_knee.setAttribute("position", "0.0, " + String(- right_upper_foot_height / 2 - right_upper_knee_height / 2) + ", 0.0");
right_upper_knee.setAttribute("rotation", "0.0, 0.0, 0.0");
right_upper_knee.setAttribute("width", String(right_upper_knee_width));
right_upper_knee.setAttribute("height", String(right_upper_knee_height));
right_upper_knee.setAttribute("depth", String(right_upper_knee_depth));
right_upper_knee.setAttribute("color", right_upper_knee_color);
right_upper_knee.setAttribute("id", "right_upper_knee_" + id);
right_upper_foot.appendChild(right_upper_knee);
const right_lower_knee_width = body_depth * 0.9;
const right_lower_knee_height = body_height / 4 * 0.9;
const right_lower_knee_depth = body_depth * 0.9;
const right_lower_knee_color = "#555";
const right_lower_knee = document.createElement("a-box");
right_lower_knee.setAttribute("position", "0.0, 0.0, 0.0");
right_lower_knee.setAttribute("rotation", "0.0, 0.0, 0.0");
right_lower_knee.setAttribute("width", String(right_lower_knee_width));
right_lower_knee.setAttribute("height", String(right_lower_knee_height));
right_lower_knee.setAttribute("depth", String(right_lower_knee_depth));
right_lower_knee.setAttribute("color", right_lower_knee_color);
right_lower_knee.setAttribute("id", "right_lower_knee_" + id);
right_upper_knee.appendChild(right_lower_knee);
const right_lower_foot_width = body_depth * 0.8;
const right_lower_foot_height = body_height / 4 / 2 * 5 * 0.8;
const right_lower_foot_depth = body_depth * 0.8;
const right_lower_foot_color = "#AAA";
const right_lower_foot = document.createElement("a-box");
right_lower_foot.setAttribute("position", "0.0, " + String(- right_lower_foot_height / 2 - right_lower_knee_height / 2) + ", 0.0");
right_lower_foot.setAttribute("rotation", "0.0, 0.0, 0.0");
right_lower_foot.setAttribute("width", String(right_lower_foot_width));
right_lower_foot.setAttribute("height", String(right_lower_foot_height));
right_lower_foot.setAttribute("depth", String(right_lower_foot_depth));
right_lower_foot.setAttribute("color", right_lower_foot_color);
right_lower_foot.setAttribute("id", "right_lower_foot_" + id);
right_lower_knee.appendChild(right_lower_foot);
parent_element.appendChild(body);
}
function update_avatar(id, move, pose) {
var body = document.getElementById("body_" + id);
if (body == null) {
console.log("not found avatar: body_" + id);
return;
}
var upper_neck = document.getElementById("upper_neck_" + id);
var lower_neck = document.getElementById("lower_neck_" + id);
var head = document.getElementById("head_" + id);
var left_upper_shoulder = document.getElementById("left_upper_shoulder_" + id);
var left_lower_shoulder = document.getElementById("left_lower_shoulder_" + id);
var left_upper_arm = document.getElementById("left_upper_arm_" + id);
var left_upper_elbow = document.getElementById("left_upper_elbow_" + id);
var left_lower_elbow = document.getElementById("left_lower_elbow_" + id);
var left_lower_arm = document.getElementById("left_lower_arm_" + id);
var right_upper_shoulder = document.getElementById("right_upper_shoulder_" + id);
var right_lower_shoulder = document.getElementById("right_lower_shoulder_" + id);
var right_upper_arm = document.getElementById("right_upper_arm_" + id);
var right_upper_elbow = document.getElementById("right_upper_elbow_" + id);
var right_lower_elbow = document.getElementById("right_lower_elbow_" + id);
var right_lower_arm = document.getElementById("right_lower_arm_" + id);
var left_upper_base_foot = document.getElementById("left_upper_base_foot_" + id);
var left_lower_base_foot = document.getElementById("left_lower_base_foot_" + id);
var left_upper_foot = document.getElementById("left_upper_foot_" + id);
var left_upper_knee = document.getElementById("left_upper_knee_" + id);
var left_lower_knee = document.getElementById("left_lower_knee_" + id);
var left_lower_foot = document.getElementById("left_lower_foot_" + id);
var right_upper_base_foot = document.getElementById("right_upper_base_foot_" + id);
var right_lower_base_foot = document.getElementById("right_lower_base_foot_" + id);
var right_upper_foot = document.getElementById("right_upper_foot_" + id);
var right_upper_knee = document.getElementById("right_upper_knee_" + id);
var right_lower_knee = document.getElementById("right_lower_knee_" + id);
var right_lower_foot = document.getElementById("right_lower_foot_" + id);
// move
// TODO
if (move != null) {
update_move(body, move.x, move.y, move.z);
}
if (pose == null || pose.relative_coordinates == null || pose.relative_coordinates.pose == null) {
return;
}
const relative_coordinates = pose.relative_coordinates;
const rel_pose = relative_coordinates.pose.relative;
const rel_pose_parent = rel_pose.parent;
const pose_lm = pose.pose.landmarks;
// rotation
// TODO
var orient = "YXZ";
var orient_reverse = "ZXY";
update_rotation(body, rel_pose_parent[POSE_NO.CENTER_HIP].angle_y, orient);
update_rotation(lower_neck, get_reverse_rotation(rel_pose_parent[POSE_NO.CENTER_HIP].angle_y), orient_reverse);
// 上向けなので、x軸を反転させる
// x軸: 上下回転 = x角度がz軸：左右回転
// z軸: 左右回転 = z角度がx軸：上下回転
// TODO y軸を回転しなければ顔の回転が再現できない。
var rev_x_face = {x: rel_pose_parent[POSE_NO.CENTER_EYE].angle_y.x, y: rel_pose_parent[POSE_NO.CENTER_EYE].angle_y.y, z: rel_pose_parent[POSE_NO.CENTER_EYE].angle_y.z};
if (rev_x_face.x < 0) {
rev_x_face.x += 180;
} else {
rev_x_face.x -= 180;
}
// z軸は傾けておく
rev_x_face.z = rev_x_face.z * -1 - 30;
update_rotation(upper_neck, rev_x_face, orient);
// ここで顔をY軸回転しておく
// 計算:zとxでカメラに対する顔の横回転の角度を計算する
var parent_eye = rel_pose_parent[POSE_NO.LEFT_EYE];
var line_x = {x: parent_eye.x, y: 0, z: 0};
var line_xz = {x: parent_eye.x, y: 0, z: parent_eye.z};
var face_y = holisticExtention.calc_angle(line_x, line_xz);
if (parent_eye.z < 0) {
face_y *= -1;
}
update_rotation(head, {x: 0, y: face_y, z: 0}, orient);
if (pose_lm[POSE_NO.LEFT_ELBOW].visibility >= visibility_threshold) {
update_rotation(left_upper_shoulder, get_reverse_rotation(rel_pose_parent[POSE_NO.CENTER_HIP].angle_y), orient_reverse);
update_rotation(left_lower_shoulder, rel_pose_parent[POSE_NO.LEFT_ELBOW].angle_y, orient);
if (pose_lm[POSE_NO.LEFT_WRIST].visibility >= visibility_threshold) {
update_rotation(left_upper_elbow, get_reverse_rotation(rel_pose_parent[POSE_NO.LEFT_ELBOW].angle_y), orient_reverse);
update_rotation(left_lower_elbow, rel_pose_parent[POSE_NO.LEFT_WRIST].angle_y, orient);
}
}
if (pose_lm[POSE_NO.RIGHT_ELBOW].visibility >= visibility_threshold) {
update_rotation(right_upper_shoulder, get_reverse_rotation(rel_pose_parent[POSE_NO.CENTER_HIP].angle_y), orient_reverse);
update_rotation(right_lower_shoulder, rel_pose_parent[POSE_NO.RIGHT_ELBOW].angle_y, orient);
if (pose_lm[POSE_NO.RIGHT_WRIST].visibility >= visibility_threshold) {
update_rotation(right_upper_elbow, get_reverse_rotation(rel_pose_parent[POSE_NO.RIGHT_ELBOW].angle_y), orient_reverse);
update_rotation(right_lower_elbow, rel_pose_parent[POSE_NO.RIGHT_WRIST].angle_y, orient);
}
}
if (pose_lm[POSE_NO.LEFT_KNEE].visibility >= visibility_threshold) {
update_rotation(left_upper_base_foot, get_reverse_rotation(rel_pose_parent[POSE_NO.CENTER_HIP].angle_y), orient_reverse);
update_rotation(left_lower_base_foot, rel_pose_parent[POSE_NO.LEFT_KNEE].angle_y, orient);
if (pose_lm[POSE_NO.LEFT_ANKLE].visibility >= visibility_threshold) {
update_rotation(left_upper_knee, get_reverse_rotation(rel_pose_parent[POSE_NO.LEFT_KNEE].angle_y), orient_reverse);
update_rotation(left_lower_knee, rel_pose_parent[POSE_NO.LEFT_ANKLE].angle_y, orient);
}
}
if (pose_lm[POSE_NO.RIGHT_KNEE].visibility >= visibility_threshold) {
update_rotation(right_upper_base_foot, get_reverse_rotation(rel_pose_parent[POSE_NO.CENTER_HIP].angle_y), orient_reverse);
update_rotation(right_lower_base_foot, rel_pose_parent[POSE_NO.RIGHT_KNEE].angle_y, orient);
if (pose_lm[POSE_NO.RIGHT_ANKLE].visibility >= visibility_threshold) {
update_rotation(right_upper_knee, get_reverse_rotation(rel_pose_parent[POSE_NO.RIGHT_KNEE].angle_y), orient_reverse);
update_rotation(right_lower_knee, rel_pose_parent[POSE_NO.RIGHT_ANKLE].angle_y, orient);
}
}
}
function update_move(obj, x, y, z) {
obj.object3D.position.set(x, y, z);
}
function update_rotation_left(obj, parent_info, orient) {
var left_parent_info = get_reverse_rotation_x(parent_info);
update_rotation(obj, left_parent_info, orient);
}
function update_rotation(obj, parent_info, orient) {
// setAttribute("rotation", "");は性能面で非推奨:
// https://aframe.io/docs/1.2.0/components/rotation.html#updating-rotation
obj.object3D.rotation.set(- THREE.Math.degToRad(parent_info.z), THREE.Math.degToRad(parent_info.y), THREE.Math.degToRad(- parent_info.x), orient);
}
function get_reverse_rotation(parent_info) {
return {"x":- parent_info.x, "y": - parent_info.y, "z": - parent_info.z};
}
function get_reverse_rotation_x(parent_info) {
return {"x": - parent_info.x, "y": parent_info.y, "z": parent_info.z};
}
function add_rotation_vector(id, color, size) {
// line
const line_width = size * 400;
const line_height = size;
const line_depth = size;
const line_color = color;
const line = document.createElement("a-box");
line.setAttribute("position", "0.0, 0,0, 0.0");
line.setAttribute("rotation", "0.0, 0.0, 0.0");
line.setAttribute("width", String(line_width));
line.setAttribute("height", String(line_height));
line.setAttribute("depth", String(line_depth));
line.setAttribute("color", line_color);
line.setAttribute("id", "line_" + id);
// top
const top_width = size * 20;
const top_color = color;
const top = document.createElement("a-octahedron");
top.setAttribute("position", String(line_width / 2) + ", 0,0, 0.0");
top.setAttribute("rotation", "0.0, 0.0, 0.0");
top.setAttribute("radius", String(top_width));
top.setAttribute("color", top_color);
top.setAttribute("id", "top_" + id);
line.appendChild(top);
return line;
}
function add_xyz_vector(obj, id, size, front_color, back_color) {
x = add_rotation_vector("x_" + id, "#" + front_color + back_color + back_color, size);
x.setAttribute("rotation", "0.0, 0.0, 0.0");
obj.appendChild(x);
y = add_rotation_vector("y_" + id, "#" + back_color + front_color + back_color, size);
y.setAttribute("rotation", "0.0, 0.0, 90.0");
obj.appendChild(y);
z = add_rotation_vector("z_" + id, "#" + back_color + back_color + front_color, size);
z.setAttribute("rotation", "0.0, 90.0, 0.0");
obj.appendChild(z);
}
// TODO develop code testing start
var id = "1";
add_avatar(id);
//if (true) {
if (false) {
// debug tool：姿勢データ投入とアバター姿勢再現
var pose_str = `{"todo": "copy paste data this var(pose_str)"}`;
var pose = JSON.parse(pose_str);
var move = null;
update_avatar(id, move, pose);
}
if (false) {
// debug tool：確認・チューニング用
var id_2 = "1_2";
add_avatar(id_2);
var a_camera = document.getElementById("camera");
a_camera.components["look-controls"].yawObject.rotation.y = 3.14;
var move_2 = {x:2, y:0, z:-2};
update_avatar(id_2, move_2, null);
var body_2 = document.getElementById("body_" + id_2);
body_2.object3D.rotation.y = THREE.Math.degToRad(135);
var body = document.getElementById("body_" + id);
body.object3D.rotation.y = THREE.Math.degToRad(-45);
var upper_neck = document.getElementById("upper_neck_" + id);
var lower_neck = document.getElementById("lower_neck_" + id);
var head = document.getElementById("head_" + id);
var left_upper_shoulder = document.getElementById("left_upper_shoulder_" + id);
var left_lower_shoulder = document.getElementById("left_lower_shoulder_" + id);
var upper = upper_neck;
var lower = lower_neck;
var size = 0.005
add_xyz_vector(upper, "up", size, "FF", "00");
add_xyz_vector(lower, "lo", size, "FF", "88");
head.setAttribute("height", "0.7");
var zero = {x: 0, y: 0, z: 0};
var orient = "YXZ";
var orient_reverse = "ZXY";
//update_rotation(upper_neck, zero, orient);
//update_rotation(upper_neck, {x: 0, y: 0, z: 0}, orient);
}
// TODO develop code testing end
// mediapipe
// holisticExtention.js start_all_in ****************************************************
class POSE_NO {
static NOSE = 0;
static LEFT_EYE_INNER = 1;
static LEFT_EYE = 2;
static LEFT_EYE_OUTER = 3;
static RIGHT_EYE_INNER = 4;
static RIGHT_EYE = 5;
static RIGHT_EYE_OUTER = 6;
static LEFT_EAR = 7;
static RIGHT_EAR = 8;
static MOUTH_LEFT = 9;
static MOUTH_RIGHT = 10;
static LEFT_SHOULDER = 11;
static RIGHT_SHOULDER = 12;
static LEFT_ELBOW = 13;
static RIGHT_ELBOW = 14;
static LEFT_WRIST = 15;
static RIGHT_WRIST = 16;
static LEFT_PINKY = 17;
static RIGHT_PINKY = 18;
static LEFT_INDEX = 19;
static RIGHT_INDEX = 20;
static LEFT_THUMB = 21;
static RIGHT_THUMB = 22;
static LEFT_HIP = 23;
static RIGHT_HIP = 24;
static LEFT_KNEE = 25;
static RIGHT_KNEE = 26;
static LEFT_ANKLE = 27;
static RIGHT_ANKLE = 28;
static LEFT_HEEL = 29;
static RIGHT_HEEL = 30;
static LEFT_FOOT_INDEX = 31;
static RIGHT_FOOT_INDEX = 32;
// added mark
static CENTER_BODY = 33;
static CENTER_SHOULDER = 34;
static CENTER_HIP = 35;
static LEFT_BODY = 36;
static RIGHT_BODY = 37;
static FRONT_BODY = 38;
static FRONT_SHOULDER = 39;
static FRONT_HIP = 40;
static CENTER_LEFT_FINGER = 41;
static CENTER_RIGHT_FINGER = 42;
static CENTER_EYE = 43;
static CENTER_EAR = 44;
static CENTER_MOUTH = 45;
static CENTER_NOSE = 46;
}
const left_hand_pose_point_no = {'elbow': POSE_NO.LEFT_ELBOW, 'wrist': POSE_NO.LEFT_WRIST, 'pinky': POSE_NO.LEFT_PINKY, 'index': POSE_NO.LEFT_INDEX, 'thumb': POSE_NO.LEFT_THUMB, 'center_finger': POSE_NO.CENTER_LEFT_FINGER};
const right_hand_pose_point_no = {'elbow': POSE_NO.RIGHT_ELBOW, 'wrist': POSE_NO.RIGHT_WRIST, 'pinky': POSE_NO.RIGHT_PINKY, 'index': POSE_NO.RIGHT_INDEX, 'thumb': POSE_NO.RIGHT_THUMB, 'center_finger': POSE_NO.CENTER_RIGHT_FINGER};
class HAND_NO {
static WRIST = 0;
static THUMB_CMC = 1;
static THUMB_MCP = 2;
static THUMB_IP = 3;
static THUMB_TIP = 4;
static INDEX_FINGER_MCP = 5;
static INDEX_FINGER_PIP = 6;
static INDEX_FINGER_DIP = 7;
static INDEX_FINGER_TIP = 8;
static MIDDLE_FINGER_MCP = 9;
static MIDDLE_FINGER_PIP = 10;
static MIDDLE_FINGER_DIP = 11;
static MIDDLE_FINGER_TIP = 12;
static RING_FINGER_MCP = 13;
static RING_FINGER_PIP = 14;
static RING_FINGER_DIP = 15;
static RING_FINGER_TIP = 16;
static PINKY_MCP = 17;
static PINKY_PIP = 18;
static PINKY_DIP = 19;
static PINKY_TIP = 20;
}
var pose_vector_map = [];
for (index in POSE_NO) {
pose_vector_map.push(-1);
}
pose_vector_map[POSE_NO.NOSE] = POSE_NO.CENTER_NOSE;
pose_vector_map[POSE_NO.LEFT_EYE_INNER] = POSE_NO.CENTER_EYE;
pose_vector_map[POSE_NO.LEFT_EYE] = POSE_NO.CENTER_EYE;
pose_vector_map[POSE_NO.LEFT_EYE_OUTER] = POSE_NO.CENTER_EYE;
pose_vector_map[POSE_NO.RIGHT_EYE_INNER] = POSE_NO.CENTER_EYE;
pose_vector_map[POSE_NO.RIGHT_EYE] = POSE_NO.CENTER_EYE;
pose_vector_map[POSE_NO.RIGHT_EYE_OUTER] = POSE_NO.CENTER_EYE;
pose_vector_map[POSE_NO.LEFT_EAR] = POSE_NO.CENTER_EAR;
pose_vector_map[POSE_NO.RIGHT_EAR] = POSE_NO.CENTER_EAR;
pose_vector_map[POSE_NO.MOUTH_LEFT] = POSE_NO.CENTER_MOUTH;
pose_vector_map[POSE_NO.MOUTH_RIGHT] = POSE_NO.CENTER_MOUTH;
pose_vector_map[POSE_NO.LEFT_SHOULDER] = POSE_NO.CENTER_SHOULDER;
pose_vector_map[POSE_NO.RIGHT_SHOULDER] = POSE_NO.CENTER_SHOULDER;
pose_vector_map[POSE_NO.LEFT_ELBOW] = POSE_NO.LEFT_SHOULDER;
pose_vector_map[POSE_NO.RIGHT_ELBOW] = POSE_NO.RIGHT_SHOULDER;
pose_vector_map[POSE_NO.LEFT_WRIST] = POSE_NO.LEFT_ELBOW;
pose_vector_map[POSE_NO.RIGHT_WRIST] = POSE_NO.RIGHT_ELBOW;
pose_vector_map[POSE_NO.LEFT_PINKY] = POSE_NO.LEFT_WRIST;
pose_vector_map[POSE_NO.RIGHT_PINKY] = POSE_NO.RIGHT_WRIST;
pose_vector_map[POSE_NO.LEFT_INDEX] = POSE_NO.LEFT_WRIST;
pose_vector_map[POSE_NO.RIGHT_INDEX] = POSE_NO.RIGHT_WRIST;
pose_vector_map[POSE_NO.LEFT_THUMB] = POSE_NO.LEFT_WRIST;
pose_vector_map[POSE_NO.RIGHT_THUMB] = POSE_NO.RIGHT_WRIST;
pose_vector_map[POSE_NO.LEFT_HIP] = POSE_NO.CENTER_HIP;
pose_vector_map[POSE_NO.RIGHT_HIP] = POSE_NO.CENTER_HIP;
pose_vector_map[POSE_NO.LEFT_KNEE] = POSE_NO.LEFT_HIP;
pose_vector_map[POSE_NO.RIGHT_KNEE] = POSE_NO.RIGHT_HIP;
pose_vector_map[POSE_NO.LEFT_ANKLE] = POSE_NO.LEFT_KNEE;
pose_vector_map[POSE_NO.RIGHT_ANKLE] = POSE_NO.RIGHT_KNEE;
pose_vector_map[POSE_NO.LEFT_HEEL] = POSE_NO.LEFT_ANKLE;
pose_vector_map[POSE_NO.RIGHT_HEEL] = POSE_NO.RIGHT_ANKLE;
pose_vector_map[POSE_NO.LEFT_FOOT_INDEX] = POSE_NO.LEFT_ANKLE;
pose_vector_map[POSE_NO.RIGHT_FOOT_INDEX] = POSE_NO.RIGHT_ANKLE;
pose_vector_map[POSE_NO.CENTER_BODY] = POSE_NO.CENTER_BODY;
pose_vector_map[POSE_NO.CENTER_SHOULDER] = POSE_NO.CENTER_BODY;
pose_vector_map[POSE_NO.CENTER_HIP] = POSE_NO.CENTER_BODY;
pose_vector_map[POSE_NO.LEFT_BODY] = POSE_NO.CENTER_BODY;
pose_vector_map[POSE_NO.RIGHT_BODY] = POSE_NO.CENTER_BODY;
pose_vector_map[POSE_NO.FRONT_BODY] = POSE_NO.CENTER_BODY;
pose_vector_map[POSE_NO.FRONT_SHOULDER] = POSE_NO.CENTER_SHOULDER;
pose_vector_map[POSE_NO.FRONT_HIP] = POSE_NO.CENTER_HIP;
pose_vector_map[POSE_NO.CENTER_LEFT_FINGER] = POSE_NO.LEFT_WRIST;
pose_vector_map[POSE_NO.CENTER_RIGHT_FINGER] = POSE_NO.RIGHT_WRIST;
pose_vector_map[POSE_NO.CENTER_EYE] = POSE_NO.CENTER_NOSE;
pose_vector_map[POSE_NO.CENTER_EAR] = POSE_NO.CENTER_NOSE;
pose_vector_map[POSE_NO.CENTER_MOUTH] = POSE_NO.CENTER_NOSE;
pose_vector_map[POSE_NO.CENTER_NOSE] = POSE_NO.CENTER_NOSE;
var pose_angle_vector_map = [];
for (index in POSE_NO) {
pose_angle_vector_map.push(-1);
}
const POSE_HEAD_BASE_LIN_Y_INDEX = POSE_NO.CENTER_EYE;
const POSE_BODY_BASE_LIN_Y_INDEX = POSE_NO.CENTER_SHOULDER;
pose_angle_vector_map[POSE_NO.NOSE] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_EYE_INNER] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_EYE] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_EYE_OUTER] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_EYE_INNER] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_EYE] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_EYE_OUTER] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_EAR] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_EAR] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.MOUTH_LEFT] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.MOUTH_RIGHT] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_SHOULDER] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_SHOULDER] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_ELBOW] = POSE_NO.LEFT_SHOULDER;
pose_angle_vector_map[POSE_NO.RIGHT_ELBOW] = POSE_NO.RIGHT_SHOULDER;
pose_angle_vector_map[POSE_NO.LEFT_WRIST] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_WRIST] = POSE_NO.RIGHT_ELBOW;
pose_angle_vector_map[POSE_NO.LEFT_PINKY] = POSE_NO.LEFT_WRIST;
pose_angle_vector_map[POSE_NO.RIGHT_PINKY] = POSE_NO.RIGHT_WRIST;
pose_angle_vector_map[POSE_NO.LEFT_INDEX] = POSE_NO.LEFT_WRIST;
pose_angle_vector_map[POSE_NO.RIGHT_INDEX] = POSE_NO.RIGHT_WRIST;
pose_angle_vector_map[POSE_NO.LEFT_THUMB] = POSE_NO.LEFT_WRIST;
pose_angle_vector_map[POSE_NO.RIGHT_THUMB] = POSE_NO.RIGHT_WRIST;
pose_angle_vector_map[POSE_NO.LEFT_HIP] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_HIP] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_KNEE] = POSE_NO.LEFT_HIP;
pose_angle_vector_map[POSE_NO.RIGHT_KNEE] = POSE_NO.RIGHT_HIP;
pose_angle_vector_map[POSE_NO.LEFT_ANKLE] = POSE_NO.LEFT_KNEE;
pose_angle_vector_map[POSE_NO.RIGHT_ANKLE] = POSE_NO.RIGHT_KNEE;
pose_angle_vector_map[POSE_NO.LEFT_HEEL] = POSE_NO.LEFT_ANKLE;
pose_angle_vector_map[POSE_NO.RIGHT_HEEL] = POSE_NO.RIGHT_ANKLE;
pose_angle_vector_map[POSE_NO.LEFT_FOOT_INDEX] = POSE_NO.LEFT_ANKLE;
pose_angle_vector_map[POSE_NO.RIGHT_FOOT_INDEX] = POSE_NO.RIGHT_ANKLE;
pose_angle_vector_map[POSE_NO.CENTER_BODY] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.CENTER_SHOULDER] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.CENTER_HIP] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.LEFT_BODY] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.RIGHT_BODY] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.FRONT_BODY] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.FRONT_SHOULDER] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.FRONT_HIP] = POSE_BODY_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.CENTER_LEFT_FINGER] = POSE_NO.LEFT_WRIST;
pose_angle_vector_map[POSE_NO.CENTER_RIGHT_FINGER] = POSE_NO.RIGHT_WRIST;
pose_angle_vector_map[POSE_NO.CENTER_EYE] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.CENTER_EAR] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.CENTER_MOUTH] = POSE_HEAD_BASE_LIN_Y_INDEX;
pose_angle_vector_map[POSE_NO.CENTER_NOSE] = POSE_HEAD_BASE_LIN_Y_INDEX;
var pose_angle_map = [];
for (index in POSE_NO) {
pose_angle_map.push({});
}
pose_angle_map[POSE_NO.NOSE] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.LEFT_EYE_INNER] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.LEFT_EYE] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.LEFT_EYE_OUTER] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.RIGHT_EYE_INNER] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.RIGHT_EYE] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.RIGHT_EYE_OUTER] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.LEFT_EAR] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.RIGHT_EAR] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.MOUTH_LEFT] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.MOUTH_RIGHT] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.LEFT_SHOULDER] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_SHOULDER] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.LEFT_ELBOW] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_ELBOW] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.LEFT_WRIST] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_WRIST] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.LEFT_PINKY] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_PINKY] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.LEFT_INDEX] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_INDEX] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.LEFT_THUMB] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_THUMB] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.LEFT_HIP] = {'side': POSE_NO.LEFT_BODY, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.RIGHT_HIP] = {'side': POSE_NO.RIGHT_BODY, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.LEFT_KNEE] = {'side': POSE_NO.LEFT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.RIGHT_KNEE] = {'side': POSE_NO.RIGHT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.LEFT_ANKLE] = {'side': POSE_NO.LEFT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.RIGHT_ANKLE] = {'side': POSE_NO.RIGHT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.LEFT_HEEL] = {'side': POSE_NO.LEFT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.RIGHT_HEEL] = {'side': POSE_NO.RIGHT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.LEFT_FOOT_INDEX] = {'side': POSE_NO.LEFT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_FOOT_INDEX] = {'side': POSE_NO.RIGHT_HIP, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.CENTER_BODY] = {'side': POSE_NO.CENTER_BODY, 'front': POSE_NO.FRONT_BODY, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.CENTER_SHOULDER] = {'side': POSE_NO.CENTER_BODY, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.CENTER_HIP] = {'side': POSE_NO.CENTER_BODY, 'front': POSE_NO.FRONT_HIP, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.LEFT_BODY] = {'side': POSE_NO.LEFT_BODY, 'front': POSE_NO.FRONT_BODY, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.RIGHT_BODY] = {'side': POSE_NO.RIGHT_BODY, 'front': POSE_NO.FRONT_BODY, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.FRONT_BODY] = {'side': POSE_NO.CENTER_BODY, 'front': POSE_NO.FRONT_BODY, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.FRONT_SHOULDER] = {'side': POSE_NO.FRONT_BODY, 'front': POSE_NO.FRONT_BODY, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.FRONT_HIP] = {'side': POSE_NO.FRONT_BODY, 'front': POSE_NO.FRONT_BODY, 'up': POSE_NO.CENTER_HIP};
pose_angle_map[POSE_NO.CENTER_LEFT_FINGER] = {'side': POSE_NO.LEFT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.CENTER_RIGHT_FINGER] = {'side': POSE_NO.RIGHT_SHOULDER, 'front': POSE_NO.FRONT_SHOULDER, 'up': POSE_NO.CENTER_SHOULDER};
pose_angle_map[POSE_NO.CENTER_EYE] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.CENTER_EAR] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.CENTER_MOUTH] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
pose_angle_map[POSE_NO.CENTER_NOSE] = {'side': POSE_NO.LEFT_EYE, 'front': POSE_NO.NOSE, 'up': POSE_NO.CENTER_EYE};
var HAND_CONNECTIONS = [];
HAND_CONNECTIONS.push([HAND_NO.WRIST, HAND_NO.THUMB_CMC]);
HAND_CONNECTIONS.push([HAND_NO.THUMB_CMC, HAND_NO.THUMB_MCP]);
HAND_CONNECTIONS.push([HAND_NO.THUMB_MCP, HAND_NO.THUMB_IP]);
HAND_CONNECTIONS.push([HAND_NO.THUMB_IP, HAND_NO.THUMB_TIP]);
HAND_CONNECTIONS.push([HAND_NO.WRIST, HAND_NO.INDEX_FINGER_MCP]);
HAND_CONNECTIONS.push([HAND_NO.INDEX_FINGER_MCP, HAND_NO.INDEX_FINGER_PIP]);
HAND_CONNECTIONS.push([HAND_NO.INDEX_FINGER_PIP, HAND_NO.INDEX_FINGER_DIP]);
HAND_CONNECTIONS.push([HAND_NO.INDEX_FINGER_DIP, HAND_NO.INDEX_FINGER_TIP]);
HAND_CONNECTIONS.push([HAND_NO.INDEX_FINGER_MCP, HAND_NO.MIDDLE_FINGER_MCP]);
HAND_CONNECTIONS.push([HAND_NO.MIDDLE_FINGER_MCP, HAND_NO.MIDDLE_FINGER_PIP]);
HAND_CONNECTIONS.push([HAND_NO.MIDDLE_FINGER_PIP, HAND_NO.MIDDLE_FINGER_DIP]);
HAND_CONNECTIONS.push([HAND_NO.MIDDLE_FINGER_DIP, HAND_NO.MIDDLE_FINGER_TIP]);
HAND_CONNECTIONS.push([HAND_NO.MIDDLE_FINGER_MCP, HAND_NO.RING_FINGER_MCP]);
HAND_CONNECTIONS.push([HAND_NO.RING_FINGER_MCP, HAND_NO.RING_FINGER_PIP]);
HAND_CONNECTIONS.push([HAND_NO.RING_FINGER_PIP, HAND_NO.RING_FINGER_DIP]);
HAND_CONNECTIONS.push([HAND_NO.RING_FINGER_DIP, HAND_NO.RING_FINGER_TIP]);
HAND_CONNECTIONS.push([HAND_NO.RING_FINGER_MCP, HAND_NO.PINKY_MCP]);
HAND_CONNECTIONS.push([HAND_NO.WRIST, HAND_NO.PINKY_MCP]);
HAND_CONNECTIONS.push([HAND_NO.PINKY_MCP, HAND_NO.PINKY_PIP]);
HAND_CONNECTIONS.push([HAND_NO.PINKY_PIP, HAND_NO.PINKY_DIP]);
HAND_CONNECTIONS.push([HAND_NO.PINKY_DIP, HAND_NO.PINKY_TIP]);
var hand_vector_map = [];
for (index in HAND_NO) {
hand_vector_map.push(-1);
}
for (hand_pair_index in HAND_CONNECTIONS) {
var hand_pair = HAND_CONNECTIONS[hand_pair_index];
hand_vector_map[hand_pair[1]] = hand_pair[0];
}
hand_vector_map[HAND_NO.WRIST] = HAND_NO.WRIST;
// over write
hand_vector_map[HAND_NO.MIDDLE_FINGER_MCP] = HAND_NO.WRIST;
hand_vector_map[HAND_NO.RING_FINGER_MCP] = HAND_NO.WRIST;
hand_vector_map[HAND_NO.PINKY_MCP] = HAND_NO.WRIST;
var hand_angle_map = [];
for (index in HAND_NO) {
hand_angle_map.push({});
}
// TODO
//hand_angle_map[HAND_NO.NOSE] = {'side': HAND_NO.NOSE, 'front': POSE_NO.NOSE, 'up': POSE_NO.NOSE};
var hand_mcp_vector_map = [];
for (index in HAND_NO) {
hand_mcp_vector_map.push(-1);
}
hand_mcp_vector_map[HAND_NO.WRIST] = HAND_NO.WRIST;
hand_mcp_vector_map[HAND_NO.THUMB_CMC] = HAND_NO.THUMB_CMC;
hand_mcp_vector_map[HAND_NO.THUMB_MCP] = HAND_NO.THUMB_CMC;
hand_mcp_vector_map[HAND_NO.THUMB_IP] = HAND_NO.THUMB_CMC;
hand_mcp_vector_map[HAND_NO.THUMB_TIP] = HAND_NO.THUMB_CMC;
hand_mcp_vector_map[HAND_NO.INDEX_FINGER_MCP] = HAND_NO.INDEX_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.INDEX_FINGER_PIP] = HAND_NO.INDEX_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.INDEX_FINGER_DIP] = HAND_NO.INDEX_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.INDEX_FINGER_TIP] = HAND_NO.INDEX_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.MIDDLE_FINGER_MCP] = HAND_NO.MIDDLE_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.MIDDLE_FINGER_PIP] = HAND_NO.MIDDLE_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.MIDDLE_FINGER_DIP] = HAND_NO.MIDDLE_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.MIDDLE_FINGER_TIP] = HAND_NO.MIDDLE_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.RING_FINGER_MCP] = HAND_NO.RING_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.RING_FINGER_PIP] = HAND_NO.RING_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.RING_FINGER_DIP] = HAND_NO.RING_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.RING_FINGER_TIP] = HAND_NO.RING_FINGER_MCP;
hand_mcp_vector_map[HAND_NO.PINKY_MCP] = HAND_NO.PINKY_MCP;
hand_mcp_vector_map[HAND_NO.PINKY_PIP] = HAND_NO.PINKY_MCP;
hand_mcp_vector_map[HAND_NO.PINKY_DIP] = HAND_NO.PINKY_MCP;
hand_mcp_vector_map[HAND_NO.PINKY_TIP] = HAND_NO.PINKY_MCP;
class holisticExtention {
static image_z = 600;
static extendHolisticResult(holistic, image_width, image_height, image_z) {
var response = this.create_holistic_result(holistic, image_width, image_height, image_z);
response.relative_coordinates = this.calc_holistic_relative_coordinates(response);
return response;
}
static create_holistic_result(holistic, image_width, image_height, image_z) {
var response = {};
if (holistic.X != null) {
response['pose'] = this.create_pose_result_world(holistic.X);
} else {
response['pose'] = this.create_pose_result(holistic.poseLandmarks, image_width, image_height, image_z);
}
response['left_hand'] = this.create_hand_result(holistic.leftHandLandmarks, image_width, image_height, image_z);
response['right_hand'] = this.create_hand_result(holistic.rightHandLandmarks, image_width, image_height, image_z);
response['face_mesh'] = this.create_face_mesh_result(holistic.faceLandmarks, image_width, image_height, image_z);
return response;
}
static create_pose_result(pose_landmarks, image_width, image_height, image_z) {
var response = {};
if (pose_landmarks == null) {
//console.log("no detected pose landmark.");
return response;
}
var index = 0;
response['landmarks'] = [];
const float_rank = 3;
for (var data_point_index in pose_landmarks) {
var data_point = pose_landmarks[data_point_index];
var landmark = {};
landmark['x'] = mathUtile.round(data_point.x * image_width, float_rank);
landmark['y'] = mathUtile.round(data_point.y * image_height, float_rank);
landmark['z'] = mathUtile.round(data_point.z * image_z, float_rank);
landmark['visibility'] = mathUtile.round(data_point.visibility, float_rank);
response['landmarks'].push(landmark);
//console.log("no: %d, (x, y, z): (%f, %f, %f), visibility: %f" % (index, data_point.x * image_width, data_point.y * image_height, data_point.z, data_point.visibility));
index += 1;
}
return response;
}
static create_pose_result_world(pose_world_landmarks) {
var response = {};
if (pose_world_landmarks == null) {
//console.log("no detected pose landmark.");
return response;
}
var index = 0;
response['landmarks'] = [];
const float_rank = 6;
for (var data_point_index in pose_world_landmarks) {
var data_point = pose_world_landmarks[data_point_index];
var landmark = {};
landmark['x'] = mathUtile.round(data_point.x, float_rank);
landmark['y'] = mathUtile.round(data_point.y, float_rank);
landmark['z'] = mathUtile.round(data_point.z, float_rank);
landmark['visibility'] = mathUtile.round(data_point.visibility, float_rank);
response['landmarks'].push(landmark);
//console.log("no: %d, (x, y, z): (%f, %f, %f), visibility: %f" % (index, data_point.x * image_width, data_point.y * image_height, data_point.z, data_point.visibility));
index += 1;
}
return response;
}
static create_hand_result(hand_landmarks, image_width, image_height, image_z) {
var response = {}
if (hand_landmarks == null) {
//print("no detected hand landmark.");
return response;
}
var index = 0;
response['landmarks'] = []
const float_rank = 3;
for (var data_point_index in hand_landmarks) {
var data_point = hand_landmarks[data_point_index];
var landmark = {};
landmark['x'] = mathUtile.round(data_point.x * image_width, float_rank);
landmark['y'] = mathUtile.round(data_point.y * image_height, float_rank);
landmark['z'] = mathUtile.round(data_point.z * image_z, float_rank);
response['landmarks'].push(landmark);
//print("no: %d, (x, y, z): (%f, %f, %f)" % (index, data_point.x * image_width, data_point.y * image_height, data_point.z));
index += 1;
}
return response;
}
static create_face_mesh_result(face_landmarks, image_width, image_height, image_z) {
var response = {};
if (face_landmarks == null) {
//print("no detected face landmark.");
return response;
}
var index = 0;
response['landmarks'] = [];
const float_rank = 3;
for (var data_point_index in face_landmarks) {
var data_point = face_landmarks[data_point_index];
var landmark = {};
landmark['x'] = mathUtile.round(data_point.x * image_width, float_rank);
landmark['y'] = mathUtile.round(data_point.y * image_height, float_rank);
landmark['z'] = mathUtile.round(data_point.z * image_z, float_rank);
response['landmarks'].push(landmark);
//print("no: %d, (x, y, z): (%f, %f, %f)" % (index, data_point.x * image_width, data_point.y * image_height, data_point.z));
index += 1;
}
return response;
}
static calc_holistic_relative_coordinates(holistic) {
this.add_holistic_mark(holistic);
var results = {};
var pose = holistic['pose'];
if (pose.landmarks != null) {
results['pose'] = this.calc_pose_relative_coordinates(pose);
}
var left_hand_pose = holistic['left_hand'];
if (left_hand_pose.landmarks != null) {
results['left_hand'] = this.calc_hand_relative_coordinates(results['pose'], left_hand_pose);
}
var right_hand_pose = holistic['right_hand'];
if (right_hand_pose.landmarks != null) {
results['right_hand'] = this.calc_hand_relative_coordinates(results['pose'], right_hand_pose);
}
return results;
}
static add_holistic_mark(holistic) {
var pose = holistic['pose'];
if (pose.landmarks != null) {
this.add_pose_mark(pose);
}
}
static add_pose_mark(pose) {
// added pose mark
// center
// body
var pose_center_points = [];
pose_center_points.push(pose['landmarks'][POSE_NO.LEFT_SHOULDER]);
pose_center_points.push(pose['landmarks'][POSE_NO.RIGHT_SHOULDER]);
pose_center_points.push(pose['landmarks'][POSE_NO.LEFT_HIP]);
pose_center_points.push(pose['landmarks'][POSE_NO.RIGHT_HIP]);
var body_center = this.calc_center(pose_center_points);
pose['landmarks'].push(body_center); // CENTER_BODY
// shoulder
var pose_shoulder_center_points = [];
pose_shoulder_center_points.push(pose['landmarks'][POSE_NO.LEFT_SHOULDER]);
pose_shoulder_center_points.push(pose['landmarks'][POSE_NO.RIGHT_SHOULDER]);
var shoulder_center = this.calc_center(pose_shoulder_center_points);
pose['landmarks'].push(shoulder_center); // CENTER_SHOULDER
// hip
var pose_hip_center_points = [];
pose_hip_center_points.push(pose['landmarks'][POSE_NO.LEFT_HIP]);
pose_hip_center_points.push(pose['landmarks'][POSE_NO.RIGHT_HIP]);
var hip_center = this.calc_center(pose_hip_center_points);
pose['landmarks'].push(hip_center); // CENTER_HIP
// left
var pose_left_shoulder_hip_points = [];
pose_left_shoulder_hip_points.push(pose['landmarks'][POSE_NO.LEFT_SHOULDER]);
pose_left_shoulder_hip_points.push(pose['landmarks'][POSE_NO.LEFT_HIP]);
var body_left = this.calc_center(pose_left_shoulder_hip_points);
pose['landmarks'].push(body_left); // LEFT_BODY
// right
var pose_right_shoulder_hip_points = [];
pose_right_shoulder_hip_points.push(pose['landmarks'][POSE_NO.RIGHT_SHOULDER]);
pose_right_shoulder_hip_points.push(pose['landmarks'][POSE_NO.RIGHT_HIP]);
var body_right = this.calc_center(pose_right_shoulder_hip_points);
pose['landmarks'].push(body_right); // RIGHT_BODY
// front
// body
var center_body_list = this.get_vector_list(pose['landmarks'][POSE_NO.CENTER_BODY]);
var center_body_vector = this.calc_vector_from_index(pose, pose_vector_map, POSE_NO.CENTER_BODY);
var center_body_vector_list = this.get_vector_list(center_body_vector);
var left_body_vector = this.calc_vector_from_index(pose, pose_vector_map, POSE_NO.LEFT_BODY);
var left_body_vector_list = this.get_vector_list(left_body_vector);
var ajust_vec = coordinateTransformationLib.line_turn_ajust_x(left_body_vector_list);
var front_ajust_array = coordinateTransformationLib.y_turn(90);
var front_ajust_vector = nj.dot(ajust_vec, front_ajust_array);
var front_body_vector = coordinateTransformationLib.turn_vec_x(front_ajust_vector, left_body_vector_list);
var front_body_list = front_body_vector.add(center_body_list);
var front_body = {"x": front_body_list.get(0), "y": front_body_list.get(1), "z": front_body_list.get(2)};
pose['landmarks'].push(front_body); // FRONT_BODY
//print("center_body_list: ", center_body_list);
var left_body_list = this.get_vector_list(pose['landmarks'][POSE_NO.LEFT_BODY]);
//print("left_body_list: ", left_body_list);
//print("center_body_vector_list: ", center_body_vector_list);
//print("left_body_vector_list: ", left_body_vector_list);
//print("ajust_vec: ", ajust_vec);
//print("front_ajust_array: ", front_ajust_array);
//print("front_ajust_vector: ", front_ajust_vector);
//print("front_body_vector: ", front_body_vector);
//print("front_body_list: ", front_body_list);
// shoulder
var center_shoulder_list = this.get_vector_list(pose['landmarks'][POSE_NO.CENTER_SHOULDER]);
var front_shoulder_list = front_body_vector.add(center_shoulder_list);
var front_shoulder = {"x": front_shoulder_list.get(0), "y": front_shoulder_list.get(1), "z": front_shoulder_list.get(2)};
pose['landmarks'].push(front_shoulder); // FRONT_SHOULDER
// hip
var center_hip_list = this.get_vector_list(pose['landmarks'][POSE_NO.CENTER_HIP]);
var front_hip_list = front_body_vector.add(center_hip_list);
var front_hip = {"x": front_hip_list.get(0), "y": front_hip_list.get(1), "z": front_hip_list.get(2)};
pose['landmarks'].push(front_hip); // FRONT_HIP
// finger center
// left
var left_finger_center_points = [];
left_finger_center_points.push(pose['landmarks'][POSE_NO.LEFT_PINKY]);
left_finger_center_points.push(pose['landmarks'][POSE_NO.LEFT_THUMB]);
var left_finger_center = this.calc_center(left_finger_center_points);
pose['landmarks'].push(left_finger_center); // CENTER_LEFT_FINGER
// right
var right_finger_center_points = [];
right_finger_center_points.push(pose['landmarks'][POSE_NO.RIGHT_PINKY]);
right_finger_center_points.push(pose['landmarks'][POSE_NO.RIGHT_THUMB]);
var right_finger_center = this.calc_center(right_finger_center_points);
pose['landmarks'].push(right_finger_center); // CENTER_RIGHT_FINGER
// head
// eye
var eye_center_points = [];
eye_center_points.push(pose['landmarks'][POSE_NO.LEFT_EYE]);
eye_center_points.push(pose['landmarks'][POSE_NO.RIGHT_EYE]);
var eye_center = this.calc_center(eye_center_points);
pose['landmarks'].push(eye_center); // CENTER_EYE
var ear_center_points = [];
ear_center_points.push(pose['landmarks'][POSE_NO.LEFT_EAR]);
ear_center_points.push(pose['landmarks'][POSE_NO.RIGHT_EAR]);
var ear_center = this.calc_center(ear_center_points);
pose['landmarks'].push(ear_center); // CENTER_EAR
var mouth_center_points = [];
mouth_center_points.push(pose['landmarks'][POSE_NO.MOUTH_LEFT]);
mouth_center_points.push(pose['landmarks'][POSE_NO.MOUTH_RIGHT]);
var mouth_center = this.calc_center(mouth_center_points);
pose['landmarks'].push(mouth_center); // CENTER_MOUTH
// TODO 要修正：ひとまず、目と口のそれぞれの真ん中を鼻の中心とする
var nose_center_points = [];
nose_center_points.push(pose['landmarks'][POSE_NO.CENTER_EYE]);
nose_center_points.push(pose['landmarks'][POSE_NO.CENTER_MOUTH]);
var nose_center = this.calc_center(nose_center_points);
pose['landmarks'].push(nose_center); // CENTER_NOSE
}
static calc_holistic_relative_coordinates(holistic) {
this.add_holistic_mark(holistic);
var results = {};
var pose = holistic['pose'];
if (pose.landmarks != null) {
results['pose'] = this.calc_pose_relative_coordinates(pose);
}
var left_hand_pose = holistic['left_hand'];
if (left_hand_pose.landmarks != null) {
results['left_hand'] = this.calc_hand_relative_coordinates(results['pose'], left_hand_pose);
}
var right_hand_pose = holistic['right_hand'];
if (right_hand_pose.landmarks != null) {
results['right_hand'] = this.calc_hand_relative_coordinates(results['pose'], right_hand_pose);
}
return results;
}
static calc_pose_relative_coordinates(pose) {
var results = {};
// 関節等の付け根に対する相対的な位置データを算出
// 中心位置を両肩(SHOULDER)と両尻(HIP)の間とする
// vector
// parent
var parent = [];
var index = 0;
for (index in pose_vector_map) {
var vector = this.calc_vector_from_index(pose, pose_vector_map, index);
parent.push(vector);
}
// ajust coordinates(turn center line)
// TODO
// https://qiita.com/MENDY/items/eeb0a4b067e596fd98dd
// https://keisan.casio.jp/exec/system/1536110745
// angle
for (index in pose_vector_map) {
var pair_index = pose_vector_map[index];
if (pair_index == -1) {
parent[index]['angle'] = {};
} else {
var r_vector_pair = this.calc_vector_reverse(parent[pair_index]);
parent[index]['angle'] = this.calc_angle(r_vector_pair, parent[index]);
}
}
// 最初に頭と体のY軸の角度を算出
var center_head_index = POSE_NO.NOSE;
parent[center_head_index]['angle_y'] = coordinateTransformationLib.get_turn_angle_y_0(holisticExtention.get_vector_list(parent[center_head_index]));
var center_body_index = POSE_NO.CENTER_SHOULDER;
parent[center_body_index]['angle_y'] = coordinateTransformationLib.get_turn_angle_y_0(holisticExtention.get_vector_list(parent[center_body_index]));
for (index in parent) {
if (index == center_head_index) {
continue;
}
if (index == center_body_index) {
continue;
}
// TODO delete test code
var is_use_no_parent = true;
//is_use_no_parent = false;
if (is_use_no_parent == true) {
// TODO ひとまず差分なし版
parent[index]['angle_y'] = coordinateTransformationLib.get_turn_angle_y_0(holisticExtention.get_vector_list(parent[index]));
} else if (true) {
// TODO center bodyとの差分で計算中、ただ、これでいいのか不明。
var pair_index = pose_angle_vector_map[index];
if (pair_index == -1) {
parent[index]['angle_y'] = coordinateTransformationLib.get_turn_angle_y_0(holisticExtention.get_vector_list(parent[index]));
} else {
parent[index]['angle_y'] = coordinateTransformationLib.get_turn_angle_y(holisticExtention.get_vector_list(parent[index]), holisticExtention.get_vector_list(parent[pair_index]));
}
}
}
results['relative'] = {};
results['relative']['parent'] = parent;
// center body
var center_body = [];
for (index in pose_vector_map) {
center_body.push(this.calc_vector(pose['landmarks'][POSE_NO.CENTER_BODY], pose['landmarks'][index]));
}
// angle
var key = null;
for (index in pose_angle_map) {
var angle = {};
var pair_index_dict = pose_angle_map[index];
for (key in pair_index_dict) {
var pair_index = pair_index_dict[key];
angle[key] = this.calc_angle(parent[pair_index], parent[index]);
}
center_body[index]['angle'] = angle;
}
results['relative']['center_body'] = center_body;
return results;
}
static calc_hand_relative_coordinates(body_relative_coordinates, hand_pose) {
var results = {};
// 関節等の付け根に対する相対的な位置データを算出
results['relative'] = {};
// vector
// parent
var parent = [];
var index = 0;
for (index in hand_vector_map) {
var vector = this.calc_vector_from_index(hand_pose, hand_vector_map, index);
parent.push(vector);
}
// ajust coordinates(turn center line)
// TODO
// https://qiita.com/MENDY/items/eeb0a4b067e596fd98dd
// https://keisan.casio.jp/exec/system/1536110745
// angle
for (index in hand_vector_map) {
var pair_index = hand_vector_map[index];
if (pair_index == -1) {
parent[index]['angle'] = {};
} else {
var r_vector_pair = this.calc_vector_reverse(parent[pair_index]);
parent[index]['angle'] = this.calc_angle(r_vector_pair, parent[index]);
}
}
results['relative']['parent'] = parent;
// xyz line
// x: MIDDLE_FINGER_MCP -> :TODO
// y: PINKY_MCP -> INDEX_FINGER_MCP
// z: WRIST -> MIDDLE_FINGER_MCP
var base_line = {};
base_line['x'] = this.calc_vector(hand_pose['landmarks'][HAND_NO.WRIST], hand_pose['landmarks'][HAND_NO.MIDDLE_FINGER_MCP]);
base_line['y'] = this.calc_vector(hand_pose['landmarks'][HAND_NO.PINKY_MCP], hand_pose['landmarks'][HAND_NO.INDEX_FINGER_MCP]);
var x_vector_list = this.get_vector_list(base_line['x']);
var ajust_vec = coordinateTransformationLib.line_turn_ajust_x(x_vector_list);
var z_ajust_array = coordinateTransformationLib.y_turn(90);
var z_ajust_vector = nj.dot(ajust_vec, z_ajust_array);
var z_vector = coordinateTransformationLib.turn_vec_x(z_ajust_vector, x_vector_list);
base_line['z'] = {"x": z_vector[0], "y": z_vector[1], "z": z_vector[2]};
//var middle_finger_mcp_list = this.get_vector_list(hand_pose['landmarks'][HAND_NO.MIDDLE_FINGER_MCP]);
//var z_list = z_vector + middle_finger_mcp_list;
//var z_landmark = {"x": z_list[0], "y": z_list[1], "z": z_list[2]};
// bodyとの軸の角度を算出
var body_parent = body_relative_coordinates["relative"]["parent"];
var body_base_line = {"x": body_parent[POSE_NO.LEFT_BODY], "y": body_parent[POSE_NO.CENTER_SHOULDER], "z": body_parent[POSE_NO.FRONT_BODY]};
var key = null;
var body_key = null;
for (key in base_line) {
var angle = {};
var base_line_one = base_line[key];
for (body_key in body_base_line) {
var body_base_line_one = body_base_line[body_key];
angle[body_key] = this.calc_angle(body_base_line_one, base_line_one);
}
base_line_one['angle'] = angle;
}
results['relative']['base_line'] = base_line;
// vector
// mcp parent
var parent_mcp = [];
for (index in hand_mcp_vector_map) {
var vector = this.calc_vector_from_index(hand_pose, hand_mcp_vector_map, index);
parent_mcp.push(vector);
}
// angle
for (index in hand_mcp_vector_map) {
var pair_index = hand_mcp_vector_map[index];
if (pair_index == -1) {
parent_mcp[index]['angle'] = {};
} else {
var r_vector_pair = this.calc_vector_reverse(parent[pair_index]);
parent_mcp[index]['angle'] = this.calc_angle(r_vector_pair, parent_mcp[index]);
}
}
results['relative']['parent_mcp'] = parent_mcp;
// center hand
// parent mcpの関節から指先までの角度を算出する。base_lineとの比較
var center_hand = JSON.parse(JSON.stringify(parent_mcp));
// angle
for (index in hand_angle_map) {
var angle = {};
var hand_vec = center_hand[index];
for (key in base_line) {
if (key == "angle") {
continue;
}
var base_line_one = base_line[key];
angle[key] = this.calc_angle(base_line_one, hand_vec);
}
hand_vec['angle'] = angle;
}
results['relative']['center_hand'] = center_hand;
return results;
}
static calc_vector_from_index(pose, pose_vector_map, index) {
var pair_index = pose_vector_map[index];
var result = null;
if (pair_index == -1) {
result = {};
} else {
//print("pose['landmarks'] len: ", len(pose['landmarks']), "index: ", index, ", pair_index: ", pair_index);
result = this.calc_vector(pose['landmarks'][pair_index], pose['landmarks'][index]);
}
return result;
}
static calc_vector(point_1, point_2) {
var vector = {};
for (var point_key in point_1) {
if ('visibility' == point_key) {
continue;
}
vector[point_key] = point_2[point_key] - point_1[point_key];
}
return vector;
}
static calc_center(points) {
var sum = {'x': 0, 'y': 0, 'z': 0};
var coordinate_key = null;
for (var point_index in points) {
var point = points[point_index];
for (coordinate_key in sum) {
if (point[coordinate_key] == null) {
continue;
}
sum[coordinate_key] += point[coordinate_key];
}
}
for (coordinate_key in sum) {
sum[coordinate_key] /= points.length;
}
return sum;
}
static calc_angle(vector_1, vector_2) {
var point_1 = this.get_vector_list(vector_1);
var point_2 = this.get_vector_list(vector_2);
var angle = coordinateTransformationLib.get_subtended_angle(point_1, point_2);
return angle;
}
static calc_vector_reverse(vector) {
var reverse = {};
for (var key in vector) {
if (typeof(vector[key]) == 'number') {
reverse[key] = vector[key] * -1;
}
}
return reverse;
}
static get_vector_list(vector) {
var result = nj.array([vector['x'], vector['y'], vector['z']]);
return result;
}
// TODO delete get_vector_array, no use, old function
static get_vector_array(vector) {
var result = [];
result.push(vector['x']);
result.push(vector['y']);
result.push(vector['z']);
return result;
}
}
class coordinateTransformationLib {
static x_line = nj.array([1.0, 0.0, 0.0]);
static y_line = nj.array([0.0, 1.0, 0.0]);
static z_line = nj.array([0.0, 0.0, 1.0]);
static xyz_line = nj.array([1.0, 1.0, 1.0]);
static x_turn(angle) {
var angle_radians = mathUtile.radians(angle);
return nj.array([[1.0, 0.0, 0.0],
[0.0, Math.cos(angle_radians), Math.sin(angle_radians)],
[0.0, -Math.sin(angle_radians), Math.cos(angle_radians)]]);
}
static y_turn(angle) {
var angle_radians = mathUtile.radians(angle);
return nj.array([[Math.cos(angle_radians), 0.0, -Math.sin(angle_radians)],
[0.0, 1.0, 0.0],
[Math.sin(angle_radians), 0.0, Math.cos(angle_radians)]]);
}
static z_turn(angle) {
var angle_radians = mathUtile.radians(angle);
return nj.array([[Math.cos(angle_radians), Math.sin(angle_radians), 0.0],
[-Math.sin(angle_radians), Math.cos(angle_radians), 0.0],
[0.0, 0.0, 1.0]]);
}
static get_subtended_angle(p1, p2) {
var inner = nj.dot(p1, p2).get(0);
var length1 = this.get_vector_length(p1);
var length2 = this.get_vector_length(p2);
var radian = Math.acos(inner / (length1 * length2));
var angle = mathUtile.rad2deg(radian);
if (Number.isNaN(angle) == true) {
angle = -360.0; // TODO verify test ★★★
}
return angle;
}
static get_vector_length(vec) {
var length = 0;
if (vec.size == 2) {
length = Math.sqrt(vec.get(0) * vec.get(0) + vec.get(1) * vec.get(1));
} else if (vec.size == 3) {
var length_2 = Math.sqrt(vec.get(0) * vec.get(0) + vec.get(1) * vec.get(1));
length = Math.sqrt(length_2 * length_2 + vec.get(2) * vec.get(2));
}
return length;
}
static LINE_ID_X = 0;
static LINE_ID_Y = 1;
static LINE_ID_Z = 2;
static get_line_consider(consider_vec, target_line, turn_line) {
var target_angle_vec = nj.array([0.0, 0.0, 0.0]);
var ajust_line = null;
if (target_line == this.LINE_ID_X) {
ajust_line = this.x_line;
target_angle_vec.set(0, consider_vec.get(0));
var turn_api = null;
if (turn_line == this.LINE_ID_X) {
// TODO invalid ?
} else if (turn_line == this.LINE_ID_Y) {
target_angle_vec.set(2, consider_vec.get(2));
turn_api = this.y_turn;
} else if (turn_line == this.LINE_ID_Z) {
target_angle_vec.set(1, consider_vec.get(1));
turn_api = this.z_turn;
}
} else if (target_line == this.LINE_ID_Y) {
ajust_line = this.y_line;
target_angle_vec.set(1, consider_vec.get(1));
if (turn_line == this.LINE_ID_X) {
target_angle_vec.set(2, consider_vec.get(2));
turn_api = this.x_turn;
} else if (turn_line == this.LINE_ID_Y) {
// TODO invalid ?
} else if (turn_line == this.LINE_ID_Z) {
target_angle_vec.set(0, consider_vec.get(0));
turn_api = this.z_turn;
}
} else if (target_line == this.LINE_ID_Z) {
ajust_line = this.z_line;
target_angle_vec.set(2, consider_vec.get(2));
if (turn_line == this.LINE_ID_X) {
target_angle_vec.set(1, consider_vec.get(1));
turn_api = this.x_turn;
} else if (turn_line == this.LINE_ID_Y) {
target_angle_vec.set(0, consider_vec.get(0));
turn_api = this.y_turn;
} else if (turn_line == this.LINE_ID_Z) {
// TODO invalid ?
}
}
var ajust_angle = this.get_subtended_angle(target_angle_vec, ajust_line);
return {angle: ajust_angle, api: turn_api};
}
static get_line_consider_and_to(consider_vec, target_line, turn_line) {
var target_angle_vec = nj.array([0.0, 0.0, 0.0]);
var and_to = null;
var ajust_line = null;
if (target_line == this.LINE_ID_X) {
ajust_line = this.x_line;
target_angle_vec.set(0, consider_vec.get(0));
var turn_api = null;
if (turn_line == this.LINE_ID_X) {
// TODO invalid ?
} else if (turn_line == this.LINE_ID_Y) {
target_angle_vec.set(2, consider_vec.get(2));
turn_api = this.y_turn;
} else if (turn_line == this.LINE_ID_Z) {
target_angle_vec.set(1, consider_vec.get(1));
turn_api = this.z_turn;
}
} else if (target_line == this.LINE_ID_Y) {
ajust_line = this.y_line;
target_angle_vec.set(1, consider_vec.get(1));
if (turn_line == this.LINE_ID_X) {
target_angle_vec.set(2, consider_vec.get(2));
turn_api = this.x_turn;
// TODO 暫定処置
and_to = consider_vec.get(2);
} else if (turn_line == this.LINE_ID_Y) {
// TODO invalid ?
} else if (turn_line == this.LINE_ID_Z) {
target_angle_vec.set(0, consider_vec.get(0));
turn_api = this.z_turn;
// TODO 暫定処置
and_to = consider_vec.get(0);
}
} else if (target_line == this.LINE_ID_Z) {
ajust_line = this.z_line;
target_angle_vec.set(2, consider_vec.get(2));
if (turn_line == this.LINE_ID_X) {
target_angle_vec.set(1, consider_vec.get(1));
turn_api = this.x_turn;
} else if (turn_line == this.LINE_ID_Y) {
target_angle_vec.set(0, consider_vec.get(0));
turn_api = this.y_turn;
} else if (turn_line == this.LINE_ID_Z) {
// TODO invalid ?
}
}
var ajust_angle = this.get_subtended_angle(target_angle_vec, ajust_line);
if (and_to < 0) {
ajust_angle *= -1;
}
return {angle: ajust_angle, api: turn_api};
}
static line_turn_ajust(input_vec, consider_vec, target_line, turn_line, is_reverse=false) {
var line_consider = this.get_line_consider(consider_vec, target_line, turn_line);
var ajust_angle = line_consider.angle;
var turn_api = line_consider.api;
if (is_reverse == true) {
ajust_angle *= -1;
}
var ajust_array = turn_api(ajust_angle);
var result_vec = nj.dot(input_vec, ajust_array);
return result_vec;
}
static line_turn_ajust_x(input_vec) {
//print("input_vec: ", input_vec);
var y_line_ajust = this.line_turn_ajust(input_vec, input_vec, this.LINE_ID_X, this.LINE_ID_Z, false);
//print("y_line_ajust: ", y_line_ajust);
var zy_line_ajust = this.line_turn_ajust(y_line_ajust, y_line_ajust, this.LINE_ID_X, this.LINE_ID_Y, true);
//print("zy_line_ajust: ", zy_line_ajust);
return zy_line_ajust;
}
// TODO test
static line_turn_ajust_y(input_vec) {
var x_line_ajust = this.line_turn_ajust(input_vec, input_vec, this.LINE_ID_Y, this.LINE_ID_Z, true);
var zx_line_ajust = this.line_turn_ajust(x_line_ajust, x_line_ajust, this.LINE_ID_Y, this.LINE_ID_X, true);
return zx_line_ajust;
}
// TODO test
static line_turn_ajust_z(input_vec) {
var y_line_ajust = this.line_turn_ajust(input_vec, input_vec, LINE_ID_Z, LINE_ID_X, true);
var xy_line_ajust = this.line_turn_ajust(y_line_ajust, y_line_ajust, LINE_ID_Z, LINE_ID_Y, true);
return xy_line_ajust;
}
static turn_vec_x(input_vec, consider_vec) {
var y_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_X, this.LINE_ID_Z);
var y_ajust_angle = y_line_consider.angle;
var y_turn_api = y_line_consider.api;
var y_ajust_array = y_turn_api(-y_ajust_angle);
var y_ajust_vec = nj.dot(input_vec, y_ajust_array);
//print("y_ajust_vec: ", y_ajust_vec);
var zy_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_X, this.LINE_ID_Y);
var zy_ajust_angle = zy_line_consider.angle;
var zy_turn_api = zy_line_consider.api;
var zy_ajust_array = zy_turn_api(zy_ajust_angle);
var zy_ajust_vec = nj.dot(y_ajust_vec, zy_ajust_array);
//print("zy_ajust_vec: ", zy_ajust_vec);
return zy_ajust_vec;
}
static turn_vec_y(input_vec, consider_vec) {
var x_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_Y, this.LINE_ID_Z);
var x_ajust_angle = x_line_consider.angle;
var x_turn_api = x_line_consider.api;
var x_ajust_array = x_turn_api(x_ajust_angle);
var x_ajust_vec = nj.dot(input_vec, x_ajust_array);
//print("x_ajust_vec: ", x_ajust_vec);
var zx_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_Y, this.LINE_ID_X);
var zx_ajust_angle = zx_line_consider.angle;
var zx_turn_api = zx_line_consider.api;
var zx_ajust_array = zx_turn_api(zx_ajust_angle);
var zx_ajust_vec = nj.dot(x_ajust_vec, zx_ajust_array);
//print("zx_ajust_vec: ", zx_ajust_vec);
return zx_ajust_vec;
}
static get_turn_angle_y_0(input_vec) {
var x_line_consider = this.get_line_consider_and_to(input_vec, this.LINE_ID_Y, this.LINE_ID_Z);
var x_ajust_angle = x_line_consider.angle;
var x_turn_api = x_line_consider.api;
var x_ajust_array = x_turn_api(x_ajust_angle);
var x_ajust_vec = nj.dot(input_vec, x_ajust_array);
//print("x_ajust_vec: ", x_ajust_vec);
var zx_line_consider = this.get_line_consider_and_to(x_ajust_vec, this.LINE_ID_Y, this.LINE_ID_X);
var zx_ajust_angle = zx_line_consider.angle;
var zx_turn_api = zx_line_consider.api;
var zx_ajust_array = zx_turn_api(zx_ajust_angle);
var zx_ajust_vec = nj.dot(x_ajust_vec, zx_ajust_array);
//print("zx_ajust_vec: ", zx_ajust_vec);
return {"x": x_ajust_angle, "y": 0.0, "z": zx_ajust_angle};
}
static get_turn_angle_y(input_vec, consider_vec) {
var x_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_Y, this.LINE_ID_Z);
var x_ajust_angle = x_line_consider.angle;
var x_turn_api = x_line_consider.api;
var x_ajust_array = x_turn_api(x_ajust_angle);
var x_ajust_vec = nj.dot(input_vec, x_ajust_array);
//print("x_ajust_vec: ", x_ajust_vec);
var zx_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_Y, this.LINE_ID_X);
var zx_ajust_angle = zx_line_consider.angle;
var zx_turn_api = zx_line_consider.api;
var zx_ajust_array = zx_turn_api(zx_ajust_angle);
var zx_ajust_vec = nj.dot(x_ajust_vec, zx_ajust_array);
//print("zx_ajust_vec: ", zx_ajust_vec);
return {"x": x_ajust_angle, "y": 0.0, "z": zx_ajust_angle};;
}
static turn_vec_z(input_vec, consider_vec) {
var x_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_Z, this.LINE_ID_Y);
var x_ajust_angle = x_line_consider.angle;
var x_turn_api = x_line_consider.api;
var x_ajust_array = x_turn_api(x_ajust_angle);
var x_ajust_vec = nj.dot(input_vec, x_ajust_array);
//print("x_ajust_vec: ", x_ajust_vec);
var yx_line_consider = this.get_line_consider(consider_vec, this.LINE_ID_Z, this.LINE_ID_X);
var yx_ajust_angle = yx_line_consider.angle;
var yx_turn_api = yx_line_consider.api;
var yx_ajust_array = yx_turn_api(yx_ajust_angle);
var yx_ajust_vec = nj.dot(x_ajust_vec, yx_ajust_array);
//print("yx_ajust_vec: ", yx_ajust_vec);
return yx_ajust_vec;
}
}
class mathUtile {
static radians(angle) {
return angle * Math.PI / 180;
}
static rad2deg(radian) {
return radian / Math.PI * 180;
}
static round(number, base) {
var multi = 10 ** base;
var round_number = Math.round(number * multi) / multi;
return round_number;
}
}
// holisticExtention.js end_all_in ******************************************************
function call_holisticExtention(holistic, image_width, image_height) {
add_debug_message("call_holisticExtention() start");
const image_z = holisticExtention.image_z;
var results = holisticExtention.extendHolisticResult(holistic, image_width, image_height, image_z);
add_debug_message("call_holisticExtention() end");
return results;
}
//const videoElement = document.getElementsByClassName('input_video')[0] as HTMLVideoElement;
const videoElement = document.getElementsByClassName('input_video')[0];
var options = {};
options.selfieMode = true;
videoElement.classList.toggle("selfie", options.selfieMode);
//const canvasElement = document.getElementsByClassName('output_canvas')[0];
const canvasCtx = canvasElement.getContext('2d');
function onResults(results) {
canvasCtx.save();
canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
canvasCtx.drawImage(
results.image, 0, 0, canvasElement.width, canvasElement.height);
drawConnectors(canvasCtx, results.poseLandmarks, POSE_CONNECTIONS,
{color: '#00FF00', lineWidth: 4});
drawLandmarks(canvasCtx, results.poseLandmarks,
{color: '#FF0000', lineWidth: 2});
drawConnectors(canvasCtx, results.faceLandmarks, FACEMESH_TESSELATION,
{color: '#C0C0C070', lineWidth: 1});
drawConnectors(canvasCtx, results.leftHandLandmarks, HAND_CONNECTIONS,
{color: '#CC0000', lineWidth: 5});
drawLandmarks(canvasCtx, results.leftHandLandmarks,
{color: '#00FF00', lineWidth: 2});
drawConnectors(canvasCtx, results.rightHandLandmarks, HAND_CONNECTIONS,
{color: '#00CC00', lineWidth: 5});
drawLandmarks(canvasCtx, results.rightHandLandmarks,
{color: '#FF0000', lineWidth: 2});
canvasCtx.restore();
var holistic_ext = call_holisticExtention(results, image_width, image_height);
console.log(holistic_ext);
return holistic_ext;
}
function proc_analysisdata(data) {
var startAnalysisTime = new Date();
judge_pose_info = onResults(data);
var data_time_start = judge_pose_info['z_perf_01__data_time_msec'];
var data_time_end = startAnalysisTime.getTime();
debug_message = "tat frame all proc: " + String(data_time_end - data_time_start);
console.log(debug_message);
add_debug_message(debug_message);
//judge_pose_info_text = JSON.stringify(data);
//console.log("judge_pose_info_text: " + judge_pose_info_text);
update_avatar(id, move, judge_pose_info);
update_avatar(id_2, move, judge_pose_info);
}
const holistic = new Holistic({locateFile: (file) => {
//return `https://cdn.jsdelivr.net/npm/@mediapipe/holistic/${file}`; // latest
return `https://cdn.jsdelivr.net/npm/@mediapipe/holistic@ 0.4.1628005088/${file}`;
}});
holistic.setOptions({
upperBodyOnly: is_upperBodyOnly, // delete. from code pen
modelComplexity: modelComplexity, // add. from code pen ['Lite', 'Full', 'Heavy']
smoothLandmarks: is_smoothLandmarks,
minDetectionConfidence: minDetectionConfidence,
minTrackingConfidence: minTrackingConfidence
});
//holistic.onResults(onResults);
holistic.onResults(proc_analysisdata);
var isStartedMP = false;
function onResultiOS(data) {
proc_analysisdata(data);
if (isStartedMP == true) {
setTimeout(do_MP, procIntervalMSec);
}
}
holistic.onResults(onResultiOS);
var isStarted = false;
function start() {
if (isStarted == true) {
add_debug_message("already started.");
return;
}
add_debug_message("camera start called.");
var constraints = { audio: false, video: { facingMode: facing_mode, width: image_width, height: image_height } };
add_debug_message("input video setting size(x, y): " + String(image_width) + ", " + String(image_height));
navigator.mediaDevices.getUserMedia(constraints)
.then(function(mediaStream) {
var video = document.querySelector('video');
video.srcObject = mediaStream;
var settings = mediaStream.getVideoTracks()[0].getSettings();
image_width = settings.width;
image_height = settings.height;
add_debug_message("result video setting size(x, y): " + String(image_width) + ", " + String(image_height));
add_debug_message("cameraStart() end");
startMP();
if (false) {
video.onloadedmetadata = function(e) {
video.play();
};
}
})
.catch(function(err) { console.log(err.name + ": " + err.message); });
isStarted = true;
}
function stop() {
if (isStarted == false) {
add_debug_message("already stopped.");
return;
}
stopMP();
var video = document.querySelector('video');
var mediaStream = video.srcObject;
var tracks = mediaStream.getTracks();
tracks.forEach(function(track) {
track.stop();
});
video.srcObject.null;
add_debug_message("camera stop called.");
isStarted = false;
}
function startMP() {
if (isStartedMP == true) {
add_debug_message("mediapipe already started.");
return;
}
add_debug_message("mediapipe start.");
setTimeout(do_MP, procIntervalMSec);
isStartedMP = true;
}
function do_MP() {
add_debug_message("do mediapipe.");
holistic.send({image: videoElement});
}
function stopMP() {
if (isStartedMP == false) {
add_debug_message("mediapipe already stoped.");
return;
}
add_debug_message("mediapipe stop.");
isStartedMP = false;
}
start();
</script>
</body>
</html>
今更ですが、JavaScript 3Dライブラリのthree.jsに座標を扱う上でのよいクラス（Vector3、Quaternion）や関数が提供されているので、こちらを活用するとコードも短くできそうでした。
5. さいごに
以前は、エッジ側のカメラ映像をクラウドに送ってクラウド側で姿勢推定していましたが、
手元にあったiPhone 7でもMediapipe HolisticのJS版(JavaScript)のWeb demoが動くようになっていたので今回のようなエッジ側での分析をやってみました。
（ただ、iPhone 7では、以下のメッセージが表示されましたので、まだ十分にサポートはされていない状態と思われます ）
- Mediapipe Pose
This demo, running on Chrome Mobile iOS/iOS, is not well supported at this time, expect some flakiness while we improve our code.
- Mediapipe Holistic
This demo, running on Chrome Mobile iOS/iOS, is not well supported at this time, continue at your own risk.
また、AWS EC2（GPU付）インスタンスだとFPSはそこそこでましたが、iPhone 7だと、FPSが1～2程度と滑らかではなく、かくついてしまいます。
ただ、低FPSでも用途次第では使えるケースがあるのではと思っています。（デバイスのスペック依存なので、高スペックの端末だと関係ないかもしれませんが）
現状の自分でやってみた範囲は、まだまだ中途半端なので、この先はもう少し自己学習して進めれるようになりたいと思います。
うまくいったらまた共有させてください。
a1. おまけ１：クラウド連携
AWS Kinesis Data Streamsを使って、複数のユーザの姿勢をVR上に一緒に表示することを試しました。
画像データは、画像サイズにより大きくなりやすいですが、姿勢データは必要なデータのみに絞りやすく軽量になるので、よさそうです。
姿勢データをそのまま数百キロのサイズをやり取りしていたら、Kinesisの通信帯域を超えてエラーとなってしまいましたので、今後最適化する予定です。
エッジ側とビューワ側両方ともAWS SDK for JavaScriptを使ってブラウザで動作させました。
a2. おまけ２：VRアバター技術
今回、最初にどう進めていくかの段階で調査が足りず、浅い調査範囲で思いつくままに進めてしまったのですが、
後で社内の近場の方が調べられた情報をもらい、もう一度調べなおしてみると、標準というか既存技術が色々と把握できてきました。
アバターはVRMというのが主流で、モーションキャプチャを使って連携もできるようですね。
VTuberが流行っていったのが数年前ですからそりゃそうですよね。。。（しくじりました、ただ自分でやってみていい勉強になりました）
次はVRM連携できるようにしようと考えています。
参考までに、調査や簡単に動かしたりした内容をイメージで残しておきます。
- つくよみちゃん フリー素材キャラクター
- ■企画・デザイン：夢前黎様 https://tyc.rei-yumesaki.net
最初に3Dアバターを見つけたのは「つくよみちゃん」でした、ありがたいことに商用利用可能（条件付き）として公開されており、情報共有の方針が素晴らしいと感じました。
あと、VRoid Hubも、3Dアバターを共有できるプラットフォームとして大変ありがたい存在だと思いました。（サンプル等、二次配布も可能なのでWebアプリとしては活用しやすそうです）
自分で作成するのは大変ですので、ありがたく有効に利用させてもらおうと思います。
今回の実装では、関節の回転のオイラー角に苦しんだのですが（まだ私の頭では理解できず苦しんでいますが）、他の回転方式のクォータニオンも勉強していくことが良い解決方法かと思えています。
ベクトルから回転が行えそうなので、姿勢推定の座標データからの連携はしやすい感触があります。
a3. おまけ３：クォータニオンを勉強
上記の章でこれからクォータニオンを勉強すると記載しましたが、9月末から夜な夜な勉強したところ進展しましたので、おまけでその内容を添えておきます。
以下の記事を何度も読みなおして、コードを組んでみたところ、何とかクォータニオンを使って、VRMのアバターの腕や足を狙ったベクトル方向に向けることができました。本当にこれらの記事のおかげです。（他にも色々な記事を参照させていただきました。ありがとうございました。）
各関節の向きに対し、任意な方向のXYZベクトルを用意して試したところ、以下のようなバレーのアタック前のような姿勢になりました。（VRoid HubのサンプルモデルAさんを活用させていただきました）
試したコードは以下のようなになりました。（A-Frame-vrmを活用して検証）
現状、骨の向きだけ再現できた感じです。手の甲を相手側や自分側に向けるといったロール回転はまだ未対応です。
できればまた夜な夜な勉強してみます。
Mediapipeと結合したコードも、また段階的に共有させてもらいたいと思います。
VRMをクォータニオンで姿勢変更を試したソースの抜粋
（この行をクリックするとソースコードが表示されます）
...(略)
// avatarは遅れてセットされるので、後で読み込むようにしている
var vrm_a = document.getElementById("vrm"); // a-entity elementを取得
var a_obj = vrm_a.components.vrm;
var a_anim = vrm_a.components["vrm-anim"];
var a_ava = a_obj.avatar;
if (a_ava != null) {
var a_bones = a_ava.bones;
}
function loaded() {
console.log('loaded');
a_obj = vrm_a.components.vrm;
a_anim = vrm_a.components["vrm-anim"];
load_param();
}
window.onload = loaded;
function load_param() {
a_ava = a_obj.avatar;
if (a_ava != null) {
a_bones = a_ava.bones;
console.log('set parameter done.');
set_first_pose();
} else {
console.log('reload parameter call set.');
setTimeout(load_param, 1000);
}
}
function set_first_pose() {
//set_pose_1();
set_pose_test_l_leg_1();
set_pose_test_r_leg_1();
set_pose_test_l_arm_1();
set_pose_test_r_arm_1();
set_pose_test_head_1();
//set_pose_test_chest_1();
//set_pose_test_body_1();
//set_move_test_body_1();
}
// 共通関数
function turn_obj(obj_q, reverse_q, base_vec, to_vec) {
var cross = new THREE.Vector3();
cross.crossVectors(base_vec, to_vec);
var cross_1 = cross.normalize();
var to_vec_1 = to_vec.normalize();
var angle_r = base_vec.angleTo(to_vec);
//var angle = angle_r * 180 / Math.PI;
var q_cross = new THREE.Quaternion();
q_cross.setFromAxisAngle(cross_1, angle_r);
var q = null;
if (reverse_q != null) {
q = new THREE.Quaternion();
q.multiply(reverse_q);
q.multiply(q_cross);
} else {
q = q_cross;
}
obj_q.set(q.x, q.y, q.z, q.w);
}
function role_obj(obj_q, turn_q, cross, local_q) {
// TODO
}
// 共通ベクトル
const base_down = new THREE.Vector3(0, -1, 0);
const base_up = new THREE.Vector3(0, 1, 0);
const base_left = new THREE.Vector3(-1, 0, 0);
const base_right = new THREE.Vector3(1, 0, 0);
const base_front = new THREE.Vector3(0, 0, -1);
const base_back = new THREE.Vector3(0, 0, 1);
const base_left_down = new THREE.Vector3(-1, -1, 0);
const base_right_down = new THREE.Vector3(1, -1, 0);
// 以下の各関数で、左右の手や足等の各関節の曲げ伸ばしを検証
function set_pose_test_l_arm_1() {
var turn_2 = new THREE.Vector3(0, 1, -1);
var turn_3 = new THREE.Vector3(0, 1, 0);
var turn_4 = new THREE.Vector3(0, 1, 0);
var turn_5 = new THREE.Vector3(0, 1, 0);
//turn_2 = base_left;
//turn_3 = base_left;
//turn_4 = base_left;
//turn_5 = base_left;
// TODO role
//var role_2 = TODO;
var r_q_2 = null;
turn_obj(a_bones.leftUpperArm.quaternion, r_q_2, base_left, turn_2);
var r_q_3 = new THREE.Quaternion();
r_q_3.multiply(a_bones.leftUpperArm.quaternion.clone().conjugate());
turn_obj(a_bones.leftLowerArm.quaternion, r_q_3, base_left, turn_3);
var r_q_4 = new THREE.Quaternion();
r_q_4.multiply(a_bones.leftLowerArm.quaternion.clone().conjugate());
r_q_4.multiply(a_bones.leftUpperArm.quaternion.clone().conjugate());
turn_obj(a_bones.leftHand.quaternion, r_q_4, base_left, turn_4);
var r_q_5 = new THREE.Quaternion();
r_q_5.multiply(a_bones.leftHand.quaternion.clone().conjugate());
r_q_5.multiply(a_bones.leftLowerArm.quaternion.clone().conjugate());
r_q_5.multiply(a_bones.leftUpperArm.quaternion.clone().conjugate());
turn_obj(a_bones.leftIndexProximal.quaternion, r_q_5, base_left, turn_5);
var r_q_6 = new THREE.Quaternion();
// TODO other finger
}
function set_pose_test_r_arm_1() {
var turn_2 = new THREE.Vector3(1, 1, 1);
var turn_3 = new THREE.Vector3(0, 1, -1);
var turn_4 = new THREE.Vector3(0, 1, 0);
var turn_5 = new THREE.Vector3(0, 1, 0);
turn_1 = base_right;
//turn_2 = base_right;
//turn_3 = base_right;
//turn_4 = base_right;
//turn_5 = base_right;
// TODO role
//var role_2 = TODO;
var r_q_2 = null;
turn_obj(a_bones.rightUpperArm.quaternion, r_q_2, base_right, turn_2);
var r_q_3 = new THREE.Quaternion();
r_q_3.multiply(a_bones.rightUpperArm.quaternion.clone().conjugate());
turn_obj(a_bones.rightLowerArm.quaternion, r_q_3, base_right, turn_3);
var r_q_4 = new THREE.Quaternion();
r_q_4.multiply(a_bones.rightLowerArm.quaternion.clone().conjugate());
r_q_4.multiply(a_bones.rightUpperArm.quaternion.clone().conjugate());
turn_obj(a_bones.rightHand.quaternion, r_q_4, base_right, turn_4);
var r_q_5 = new THREE.Quaternion();
r_q_5.multiply(a_bones.rightHand.quaternion.clone().conjugate());
r_q_5.multiply(a_bones.rightLowerArm.quaternion.clone().conjugate());
r_q_5.multiply(a_bones.rightUpperArm.quaternion.clone().conjugate());
turn_obj(a_bones.rightIndexProximal.quaternion, r_q_5, base_right, turn_5);
var r_q_6 = new THREE.Quaternion();
// TODO other finger
}
function set_pose_test_l_leg_1() {
var turn_1 = new THREE.Vector3(-0.2, -1, -1);
var turn_2 = new THREE.Vector3(-1, 0, 1);
var turn_3 = new THREE.Vector3(0, -1, 0);
var turn_4 = new THREE.Vector3(0, -1, 0);
//turn_1 = base_down;
//turn_2 = base_down;
//turn_3 = base_front;
//turn_4 = base_front;
// TODO role
//var role_2 = TODO;
var r_q_1 = null;
turn_obj(a_bones.leftUpperLeg.quaternion, r_q_1, base_down, turn_1);
var r_q_2 = new THREE.Quaternion();
r_q_2.multiply(a_bones.leftUpperLeg.quaternion.clone().conjugate());
turn_obj(a_bones.leftLowerLeg.quaternion, r_q_2, base_down, turn_2);
var r_q_3 = new THREE.Quaternion();
r_q_3.multiply(a_bones.leftLowerLeg.quaternion.clone().conjugate());
r_q_3.multiply(a_bones.leftUpperLeg.quaternion.clone().conjugate());
turn_obj(a_bones.leftFoot.quaternion, r_q_3, base_front, turn_3);
var r_q_4 = new THREE.Quaternion();
r_q_4.multiply(a_bones.leftFoot.quaternion.clone().conjugate());
r_q_4.multiply(a_bones.leftLowerLeg.quaternion.clone().conjugate());
r_q_4.multiply(a_bones.leftUpperLeg.quaternion.clone().conjugate());
turn_obj(a_bones.leftToes.quaternion, r_q_4, base_front, turn_4);
}
function set_pose_test_r_leg_1() {
var turn_1 = new THREE.Vector3(0.2, -1, -1);
var turn_2 = new THREE.Vector3(1, 0, 1);
var turn_3 = new THREE.Vector3(0, -1, 0);
var turn_4 = new THREE.Vector3(0, -1, 0);
//turn_1 = base_down;
//turn_2 = base_down;
//turn_3 = base_front;
//turn_4 = base_front;
// TODO role
//var role_2 = TODO;
var r_q_1 = null;
turn_obj(a_bones.rightUpperLeg.quaternion, r_q_1, base_down, turn_1);
var r_q_2 = new THREE.Quaternion();
r_q_2.multiply(a_bones.rightUpperLeg.quaternion.clone().conjugate());
turn_obj(a_bones.rightLowerLeg.quaternion, r_q_2, base_down, turn_2);
var r_q_3 = new THREE.Quaternion();
r_q_3.multiply(a_bones.rightLowerLeg.quaternion.clone().conjugate());
r_q_3.multiply(a_bones.rightUpperLeg.quaternion.clone().conjugate());
turn_obj(a_bones.rightFoot.quaternion, r_q_3, base_front, turn_3);
var r_q_4 = new THREE.Quaternion();
r_q_4.multiply(a_bones.rightFoot.quaternion.clone().conjugate());
r_q_4.multiply(a_bones.rightLowerLeg.quaternion.clone().conjugate());
r_q_4.multiply(a_bones.rightUpperLeg.quaternion.clone().conjugate());
turn_obj(a_bones.rightToes.quaternion, r_q_4, base_front, turn_4);
}
function set_pose_test_head_1() {
var turn_1 = new THREE.Vector3(0, 1, -0.3);
var turn_2 = new THREE.Vector3(0.2, 1, 0);
//turn_1 = base_up;
//turn_2 = base_up;
// TODO role
//var role_2 = TODO;
if (false) {
var r_q_1 = null;
turn_obj(a_bones.neck.quaternion, r_q_1, base_up, turn_1);
var r_q_2 = new THREE.Quaternion();
r_q_2.multiply(a_bones.neck.quaternion.clone().conjugate());
turn_obj(a_bones.head.quaternion, r_q_2, base_up, turn_2);
} else {
var r_q_2 = null;
turn_obj(a_bones.head.quaternion, r_q_2, base_up, turn_2);
var r_r_2 = null;
}
// TODO eye
}
function set_pose_test_chest_1() {
var turn_1 = new THREE.Vector3(1, 0, 0); // spine回転しない？
var turn_2 = new THREE.Vector3(1, 1, 0);
//turn_1 = base_up;
//turn_2 = base_up;
// TODO role
//var role_2 = TODO;
if (false) {
// spine and chest
var r_q_1 = null;
turn_obj(a_bones.spine.quaternion, r_q_1, base_up, turn_1);
var r_q_2 = new THREE.Quaternion();
r_q_2.multiply(a_bones.spine.quaternion.clone().conjugate());
turn_obj(a_bones.chest.quaternion, r_q_2, base_up, turn_2);
} else {
// chest only
var r_q_2 = null;
turn_obj(a_bones.chest.quaternion, r_q_2, base_up, turn_2);
}
}
function set_pose_test_body_1() {
var turn_1 = new THREE.Vector3(0, 1, 0); // 体全体を回転する場合
//turn_1 = base_up;
var r_q_1 = null;
turn_obj(a_bones.hips.quaternion, r_q_1, base_up, turn_1);
}
function set_move_test_body_1() {
var x = 0.1;
var y = 0.0;
var z = 0.1;
a_bones.hips.position.x += x;
a_bones.hips.position.y += y;
a_bones.hips.position.z += z;
}