LoginSignup
0
2

More than 1 year has passed since last update.

@teramoh

WebサーバーでWebRTCコネクション

Last updated at Posted at 2022-09-13

ここで知れる情報

WebRTCのコネクションをVPSなどの、websocketを利用ではなくて、
さくらレンタルWebサーバーで実現した際に、発生したトラブルと解決の話。
主にJS部分でdatachannelがopenされない事象を解決しました
※Webサーバー側はDBにデータを保存、データの受け渡しができたらなんでもいいと思うので、特段解説はいれない

作成したアプリ

iOSで簡単なインターフォンアプリ
(SFSafariてタブやツールバーが見えないように位置調整したアプリ)
あとはHTML５でJavascriptとWebサーバーでデータのやりとりしてWebRTCコネクション

  • ローカルネットワーク内でコネクション
  • 接続は1:1です
  • 相互で映像と音声の交換
  • テキストデータのやりとりで、メッセーシングの代替

WebRTCがコネクションされるまでの流れ(失敗篇)

User1とUser2のコネクションをする想定

  • User1
  1. 映像と音声用のストリームを取得
  2. RTCPeerConnectionを作成
  3. 映像と音声用のストリームをRTCPeerConnectionに登録
  4. RTCPeerConnectionにcreateDataChannelでchannelオブジェクト作成
  5. WebRTCのコネクションcreateOfferで作成
  6. SDPをsetLocalDescriptionで登録
  7. User1SDPをサーバーに登録
  • User2
  1. 映像と音声用のストリームを取得
  2. サーバーにUser1SDPが登録されているか確認
  3. User1SDPを確認したら、RTCPeerConnectionを作成
  4. ondatachannelトリガーを設定
  5. User1SDPをsetRemoteDescriptionで登録
  6. WebRTCのコネクションcreateAnswerで作成
  7. SDPをsetLocalDescriptionで登録
  8. User2SDPをサーバーに登録
  • User1
  1. サーバーにUser2SDPが登録されているか確認
  2. User2SDPを確認したら、User2SDPをsetRemoteDescriptionで登録
  3. 自動的にUser2とネゴシエーションし、コネクションを確立する

  • コネクション確立後、DataChannelも開通する予定だったがしない

  • User2

  1. ondatachannelがトリガーされる
  2. トリガーされた際に、channelオブジェクトがあるので変数に格納する
  • 開通失敗

失敗のまえおき

第一にWebrtcのことをあまり理解せずにサンプルのスクリプトを、こねくりまわしてなんとか動作したのを整理して利用した際に問題多発だった

なにが失敗だったか（どんな問題が発生したか）

動画と音声のストリームは、受け渡せても、datachannelが全然開通しなかった(一番解決に時間を要した)

未だに、なぜ解決したかはよくわってないですが、下記をちゃんとすることでopenされました

ICEに関して、まったく処理をほどこしてなかった、(整理前はなぜ動いていたんだと思うほどに)

  1. まずcreateOfferのタイミングを間違えると適切なICE情報を取得できなかった
    createOfferはonnegotiationneededで実行すれば解決

  2. remoteのSDPの登録のタイミングの後にremoteのIce情報を登録する必要がある

promiseとイベントトリガー（onイベント）での時系列を把握できていなかった

ログ出力をして、時系列を把握、onイベントがかなり重要だった

WebRTCがコネクションされるまでの流れ(成功篇)

  • User1
  1. 映像と音声用のストリームを取得
  2. Promise.thenでRTCPeerConnectionを作成
  3. onnegotiationneededにcreateOffer実行文を登録
  4. onicecandidateにcandidateのデータとSDPをWebサーバーに追加実行文を登録
  5. RTCPeerConnectionにcreateDataChannelでchannelオブジェクト作成
  6. RTCPeerConnectionの準備が整うとonnegotiationneededイベントが呼ばれcreateOfferを実行
  7. createOffer.thenのpromiseでsetLocalDescriptionを実行
  8. createOffer後、onicecandidateイベントが何度か呼ばれる
  9. candidateがある場合配列に格納しておく、
  10. candidateがない場合にUser1SDPとcandidateの配列をサーバーに登録
  • User2
  1. 映像と音声用のストリームを取得
  2. サーバーにUser1SDPとcandidateの配列が登録されているか確認
  3. User1SDPを確認したら、RTCPeerConnectionを作成
  4. ondatachannelトリガーを設定
  5. onicecandidateにcandidateのデータとSDPをWebサーバーに追加実行文を登録
  6. User1SDPをsetRemoteDescriptionで登録
  7. Promise.thenでUser1のcandidateの配列をforEachを使って、addIceCandidateで登録
  8. WebRTCのコネクションcreateAnswerで作成
  9. SDPをsetLocalDescriptionで登録
  10. createAnswer後、onicecandidateイベントが何度か呼ばれる
  11. candidateがある場合配列に格納しておく、
  12. candidateがない場合にUser2SDPとcandidateの配列をサーバーに登録
  • User1
  1. サーバーにUser2SDPとcandidateの配列が登録されているか確認
  2. User2SDPを確認したら、User2SDPをsetRemoteDescriptionで登録
  3. Promise.thenでUser２のcandidateの配列をforEachを使って、addIceCandidateで登録
  4. 自動的にUser2とネゴシエーションし、コネクションを確立する

  • コネクション確立後、DataChannelも開通する、ondatachannelがトリガーされて

  • データチャンネルもopenされる

  • User2

  1. ondatachannelがトリガーされる
  2. トリガーされた際に、channelオブジェクトがあるので変数に格納する
  • 開通完了

##　まとめ
仕組みを理解しないまま、開発するなということでした。

まだ色々問題はあります
candidateがない場合でプログラム実行しているのが
あまり良くないと思いますが、何回チェックしても、onicecandidateの最後の実行がない場合だったので
そこで動作させています。

だいぶ前にWebRTC触った時は、sdpだけ交換すればよかったような気がしたんですが、、、
時代が変わったのか、気のせいなのか、、、Webって進化のスピードがすごい

社内用にインターフォンアプリを作ってと急に言われたので、
とりあえず動けばいいかと思ったのが、運の尽きでした。
個人的に、あまり使用シーンがないけど、datachennelのエラーについて
調べても、問題はあっても、解決の情報がなかったので
同じ問題に頭を抱えているひとがいれば、お役にたつといいなぁ

細かい仕様の変化をおいかけるより、WebRTCはメジャーなサービスのAPIを使うのが一番ですね

ソースコード

bootstrap5とjQuery3を使用しています

User1.html
<div class="container min-vh-100">
  <div class="d-flex align-items-center justify-content-center min-vh-100">
    <div id="test" class="d-none">
      <input id="sendtype" type="text">
      <input id="sendtext" type="text">
      <input type="button" onclick="click_message()" value="submit">
      <p id="message">
      </p>
      <video id="localVideo" muted autoplay playsinline></video>
      <video id="remoteVideo" autoplay playsinline></video>
      <audio id="call_audio" src="/call.mp3" loop controls="1" style="width: 160px; height: 40px; border: 1px solid black;"></audio>
    </div>

    <div id="Disconnect" class="d-none">
    <span class="text-white">Reconnecting</span>
    </div>

    <div id="Start">
      <div class="display-table-cell align-middle vh-100 vw-100 text-center">
        <btn onclick="clickStart();" class="normal-btn btntouch text-white text-decoration-none mx-auto">
          START
        </btn>
      </div>
    </div>

    <div id="Connecting" class="d-none">
      <span class="text-white">Connecting</span>
    </div>


    <div id="Waiting" class="d-none">
      <div class="display-table-cell align-middle vh-100 vw-100 text-center">
        <h3 class="text-white text-decoration-none mx-auto">T601株式会社</h3>
        <br /><br /><br /><br />
        <btn onclick="clickCalling();" class="normal-btn btntouch text-white text-decoration-none mx-auto">
          呼び出し
        </btn>
      </div>
    </div>

    <div id="Calling" class="d-none">
      <btn class="pulse-btn text-white text-decoration-none mx-auto">
        Calling...
      </btn>
      <br />
      <br />
      <btn onclick="clickCancel();" class="cancel-btn text-white text-decoration-none mx-auto">
        Cancel
      </btn>
    </div>

    <div id="Talking" class="d-none">
      <btn class="pulse-btn text-white text-decoration-none mx-auto">
        Talking...
      </btn>
      <br />
      <br />
      <btn onclick="clickCancel();" class="cancel-btn text-white text-decoration-none mx-auto">
        Cancel
      </btn>
    </div>

    <div id="Finished" class="d-none">
      <btn class="finish-btn text-white text-decoration-none mx-auto">
        Finished
      </btn>
    </div>

  </div>
</div>

<script type="text/javascript">
  // --- prefix -----
  navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
  RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection || window.mozRTCPeerConnection;
  RTCSessionDescription = window.RTCSessionDescription || window.webkitRTCSessionDescription || window.mozRTCSessionDescription;

  let phase = null;
  let LocalSdp = null;
  let RemoteSdp = null;

  let DataLocalIces = [];
  let DataRemoteIces = [];

  let peerConnection = null;
  let dataChannel = null;

  let isStartConnect = false;
  let isCheckOpenDataChannel = false;

  const room_id = <?= $Room->id ?>;
  const room_role = '<?= $role ?>';
  const room_name = '<?= $Room->name ?>';

  const callAudio = document.getElementById('call_audio');
  const remoteVideo = document.getElementById("remoteVideo");
  const localVideo = document.getElementById("localVideo");

  let localStream = null;
  let remoteStream = null;

  let isFirstStart = true;

  //DBとのやりとり用
  var AjaxMethod = {
    getRoomData: function(data) {
      return ($.ajax("getRoomData用Url", {
        type: 'post',
        data: data,
        dataType: 'json',
        cache: false,
        timeout: 10000,
      }));
    },
    addSdp: function(data) {
      return ($.ajax("addSdp用Url", {
        type: 'post',
        data: data,
        dataType: 'json',
        cache: false,
        timeout: 10000,
      }));
    },
    reset: function(data) {
      return ($.ajax("reset用Url", {
        type: 'post',
        data: data,
        dataType: 'json',
        cache: false,
        timeout: 10000,
      }));
    },
  };

  //表示の切り替え
  function SetStage(id) {
    $('#Disconnect').addClass("d-none");
    $('#Start').addClass("d-none");
    $('#Connecting').addClass("d-none");
    $('#Waiting').addClass("d-none");
    $('#Touch').addClass("d-none");
    $('#Calling').addClass("d-none");
    $('#Talking').addClass("d-none");
    $('#Finished').addClass("d-none");
    $('#' + id).removeClass("d-none");
  }

  function StartMedia() {
    navigator.mediaDevices.getUserMedia({
        video: {
          facingMode: 'user',
          frameRate: 30,
        },
        audio: {
          sampleRate: 32000,
          sampleSize: 16,
          volume: 1.0,
          echoCancellation: true,
          autoGainControl: true,
          noiseSuppression: true
        }
      }).then((stream) => {
        console.log(stream);
        localStream = stream;
        localVideo.srcObject = stream;
        localVideo.onloadedmetadata = (e) => {
          localVideo.play();
          sendOffer();
        };
        remoteVideo.onloadedmetadata = (e) => {
          remoteVideo.play();
        };
      })
      .catch(err => {
        console.log(JSON.stringify(err));
      });

  }

  //コネクション作成からsendSdp登録までの一連の処理
  function sendOffer() {
    window.localConnection = peerConnection = prepareNewConnection();
    dataChannel = peer.createDataChannel("channel");
    dataChannel = prepareDataChannel(dataChannel);

  };

  //コネクション作成
  function prepareNewConnection() {
    const pc_config = {
      "iceServers": []
    };
    peer = new RTCPeerConnection(pc_config);

    if ('ontrack' in peer) {
      console.log('-- peer.ontrack');
      peer.ontrack = event => {
        let stream = event.streams[0];
        if (stream) {
          remoteStream = stream;
          remoteVideo.srcObject = remoteStream;
          remoteVideo.onloadedmetadata = (e) => {
            remoteVideo.play();
          };
        }
        console.log('-- peer.ontrack(): track kind=' + event.track.kind);
        if (event.streams.length > 0 && stream) {

          console.log('got multi-stream, but play only 1 stream');
          let track = event.track;
          if (track.kind === 'video') {
            console.log('ontrack video');
            remoteStream.addTrack(track, stream);

          } else if (track.kind === 'audio') {
            console.log('ontrack audio');
            remoteStream.addTrack(track, stream);
          }
        }

        stream.onaddtrack = evt => {
          console.log('stream addtrack');
        };
        stream.onremovetrack = evt => {
          console.log('stream removetrack');
        };
      };

      peer.onaddstream = event => {
        remoteStream = event.stream;
        remoteVideo.srcObject = remoteStream;
        // -- log only --
        console.log('-- peer.onaddstream(), but do nothing');
      }
    } else {
      peer.onaddstream = event => {
        remoteStream = event.stream;
        remoteVideo.srcObject = remoteStream;
        // -- log only --
        console.log('else-- peer.onaddstream(), but do nothing');
      }
    }

    if (localStream) {
      console.log('Adding local stream...');
      if ('addTrack' in peer) {
        console.log('use addTrack()');
        let tracks = localStream.getTracks();
        for (let track of tracks) {
          let sender = peer.addTrack(track, localStream);
        }
      } else {
        console.log('use addStream()');
        peer.addStream(localStream);
      }
    } else {
      console.log('no local stream, but continue.');
    }

    peer.ondatachannel = function(evt) {
      console.log('ondatachannel',evt);
      isCheckOpenDataChannel = false;
      if (dataChannel) {
        console.log('dataChannel ALREAY EXIST');
      } else {

      }

      console.log(evt);
      dataChannel = prepareDataChannel(evt.channel);
      isStartConnect = true;
      SetStage("Waiting");
    }

    // --- on get local ICE candidate
    peer.onicecandidate = function(evt) {
      if (evt.candidate) {
        console.log('ice event', evt.candidate);
        DataLocalIces.push(evt.candidate);
      } else {
        console.log('empty ice event', evt.candidate);
        console.log('settime3000');
        addSendSdp();
      }
    };

    peer.onnegotiationneeded = evt => {
      console.log(evt);
      console.log('onnegotiationneeded');

      const options = {};
      peerConnection.createOffer(options)
        .then(function(sessionDescription) {
          return peerConnection.setLocalDescription(sessionDescription);
        }).then(function() {
          LocalSdp = peerConnection.localDescription.sdp;

        }).catch(function(err) {
          console.log(err);
        });
    };

    peer.onconnectionstatechange = evt => {
      console.log(evt);
    };

    return peer;
  };

  //データチャンネルを扱えるようにする
  function prepareDataChannel(dc) {

    dc.onmessage = evt => {
      console.log(evt);
      const msg = evt.data;
      const obj = JSON.parse(msg);
      if (obj.type === 'text') {
        console.log('text Message over DataChannel:', obj.text);
      }
      if (obj.type === 'Connecting') {
        Connecting();
        console.log('Waiting');
      }
      if (obj.type === 'Waiting') {
        Waiting();
        console.log('Waiting');
      }
      if (obj.type === 'Calling') {
        Calling();
        console.log('Calling');
      }
      if (obj.type === 'Talking') {
        Talking();
        console.log('Talking');
      }
      if (obj.type === 'Cancel') {
        Cancel();
        console.log('Cancel');
      }
    };

    dc.onopen = evt => {
      console.log('DataChannel OPEN:',evt);
    };
    dc.onclose = evt => {
      console.log('DataChannel CLOSE:',evt);
    };
    dc.onerror = evt => {
      console.log('DataChannel ERROR:', evt);
    };

    return dc;
  };

  //DBにsendSdp登録
  function addSendSdp() {
    console.log("addSendSdp");
    let data = {
      id: room_id,
      role: room_role,
      sdp: LocalSdp,
      ice: JSON.stringify(DataLocalIces),
    }

    console.log(data.ice);
    $.when(
        AjaxMethod.addSdp(data),
      )
      .done(function(ret) {
        // すべて成功した時の処理
        if (ret.status == "fail") {
          console.log('fail ', ret);
          setTimeout(function() {
            addSendSdp();
          }, 2000);
        }

        if (ret.status == 'success') {
          checkAnswer();
        }
      })
      .fail(function(xhr) {
        // エラーがあった時
        console.log('error ', xhr);
      });
  };

  //DBにrecvSdpが登録されたか確認（ループ）
  function checkAnswer() {
    var data = {
      name: room_name
    };

    $.when(
        AjaxMethod.getRoomData(data)
      )
      .done(function(ret) {
        console.log(ret);
        if (ret.status == 'success') {
          if (ret.room.sendsdp == "") {
            Reset();
          } else {
            if (ret.room.recvsdp == "") {
              setTimeout(function() {
                checkAnswer();
              }, 2000);
            } else {
              RemoteSdp = ret.room.recvsdp;
              DataRemoteIces = JSON.parse(ret.room.recvice);
              onSdpText();
            }
          }
        }
      })
      .fail(function(xhr) {
        // エラーがあった時

      });
  };

  //接続が確認できたら、DBリセット
  function resetsdp() {
    var data = {
      id: room_id
    };

    $.when(
        AjaxMethod.reset(data)
      )
      .done(function(ret) {
        if (ret.status == 'fail') {
          setTimeout(function() {
            resetsdp();
          }, 500);
        }

        if (ret.status == 'success') {

        }

      })
      .fail(function(xhr) {
        // エラーがあった時
        setTimeout(function() {
          resetsdp();
        }, 500);
      });
  };

  //RemoteSdp(recvSdp)があった際の処理
  function onSdpText() {

    let text = RemoteSdp;
    text = _trimTailDoubleLF(text); // for Safar TP --> Chrome
    let answer = new RTCSessionDescription({
      type: 'answer',
      sdp: text,
    });
    peerConnection.setRemoteDescription(answer).then(evt => {
      isCheckOpenDataChannel = true;
      if (DataRemoteIces || DataRemoteIces.length > 0) {
        DataRemoteIces.forEach(e => {
          peerConnection.addIceCandidate(e);
        });
      }
      resetsdp();
      console.log('setRemoteDescription(answer) succsess in promise');
      setTimeout(function() {
        CheckOpenDataChannel();
      }, 20000);
    }).catch(function(err) {
      console.log('setRemoteDescription(answer) ERROR: ', err);
    });



  };

  //１秒ごとに確認
  function update() {
    checkConnection();
  }

  //切断検知　update関数
  function checkConnection() {
    if (isStartConnect) {
      if (!peerConnection || peerConnection.connectionState != 'connected') {
        isStartConnect = false;
        SetStage('Disconnect');
        stopCallAudio();
        sendOffer();
      }
    }
  }

  //接続チェック　時間が経過して接続されていなければ、最初から
  function CheckOpenDataChannel() {
    if (isCheckOpenDataChannel == true) {
      Reset();
      isCheckOpenDataChannel = false;
    }
  }

  //ライブラリ
  function _trimTailDoubleLF(str) {
    let trimed = str.trim();
    return trimed + String.fromCharCode(13, 10);
  }

  function sendMessage(type = null, msg = null) {

    obj = {
      type: type,
      text: msg
    }
    const str = JSON.stringify(obj);
    dataChannel.send(str);
    return true;
  }

  //テスト用
  function click_message() {
    let t = document.getElementById('sendtype').value;
    let v = document.getElementById('sendtext').value;
    console.log(v);
    sendMessage(t, v);
  }

  //インターフォンアプリ
  function Reset() {

    phase = null;
    LocalSdp = null;
    RemoteSdp = null;

    DataLocalIces = [];
    DataRemoteIces = [];

    peerConnection = null;
    dataChannel = null;

    isStartConnect = false;
    isCheckOpenDataChannel = false;

    localStream = null;
    remoteStream = null;

    localVideo.muted = true;
    remoteVideo.muted = true;
    resetsdp();
    if (isFirstStart) {
      SetStage("Start");
    } else {
      clickStart();
    }
  }

  function Start() {
    Connecting();
    localVideo.muted = true;
    remoteVideo.muted = true;
    sendOffer();

  }

  function Connecting() {
    stopCallAudio();

    localVideo.muted = true;
    remoteVideo.muted = true;
    SetStage("Connecting");
  }

  function Waiting() {
    stopCallAudio();
    localVideo.muted = true;
    remoteVideo.muted = true;
    SetStage("Waiting");
  }

  function Calling() {
    playCallAudio();

    localVideo.muted = true;
    remoteVideo.muted = true;
    SetStage("Calling");
  }

  function Talking() {
    stopCallAudio();
    localVideo.muted = true;
    remoteVideo.muted = false;
    SetStage("Talking");
  }

  function Cancel() {
    stopCallAudio();
    localVideo.muted = true;
    remoteVideo.muted = true;
    SetStage("Finished");
    setTimeout(() => {
      SetStage("Waiting");
    }, 3000);
  }


  //クリック操作
  function clickStart() {
    isFirstStart = false;
    StartMedia();
    setInterval(update, 1000);
    $('#Start').addClass("d-none");
    playCallAudio();
    stopCallAudio();
    console.log("end_clickStart");
  }

  function clickWaiting() {
    sendMessage("Waiting", "send");
    Waiting();
  }

  function clickCalling() {
    sendMessage("Calling", "send");
    Calling();
  }

  function clickTalking() {
    sendMessage("Talking", "send");
    Talking();
  }

  function clickCancel() {
    sendMessage("Cancel", "send");
    Cancel();
  }

  //呼び出し音操作
  function playCallAudio() {
    callAudio.play();
    callAudio.volume = 0.01;
  }

  function stopCallAudio() {
    callAudio.pause();
  }

  //初期動作
  document.addEventListener('DOMContentLoaded', function() {
    Reset();
  });
</script>
User2.html
<video id="remoteVideo" autoplay playsinline class="d-none" style="position: absolute;
 	top:0;
 	width: 100%;
 	height: 100%;
 	background: rgba(0,0,0,1);
 	z-index: 0;"></video>
<div class="container min-vh-100" style="z-index: 1000;">
  <div id="CenterContent" class="d-flex align-items-center justify-content-center min-vh-100">
    <div id="test" class="d-none">
      <input id="sendtype" type="text">
      <input id="sendtext" type="text">
      <input type="button" onclick="click_message()" value="submit">
      <p id="message">
      </p>
      <video id="localVideo" muted autoplay playsinline></video>

      <audio id="call_audio" src="/call.mp3" loop controls="1" style="width: 160px; height: 40px; border: 1px solid black;"></audio>
    </div>

    <div id="Disconnect" class="d-none">
    <span class="text-white">Reconnecting</span>
    </div>

    <div id="Start">
      <div class="display-table-cell align-middle vh-100 vw-100 text-center">
        <btn onclick="clickStart();" class="normal-btn btntouch text-white text-decoration-none mx-auto">
          START
        </btn>
      </div>
    </div>

    <div id="Connecting" class="d-none">
      <span class="text-white">Connecting</span>
    </div>

    <div id="Waiting" class="d-none">
      <span class="text-white">待機中</span>
    </div>

    <div id="Finished" class="d-none">
      <btn class="finish-btn text-white text-decoration-none mx-auto">
        Finished
      </btn>
    </div>

  </div>
  <div id="BottomContent" class="d-flex align-items-end justify-content-center min-vh-100">
    <div id="Calling" class="d-none align-bottom">
      <span class="d-inline-block">
        <btn onclick="clickCancel();" class="cancel-btn text-white text-decoration-none">
          Cancel
        </btn>
      </span>
      <span class="d-inline-block" width="200px" height="200px">
        <br /> <br />
      </span>
      <span class="d-inline-block">
        <btn onclick="clickTalking();" class="normal-btn btntouch text-white text-decoration-none" width="50px">
          Talk
        </btn>

      </span>
      <br />
    </div>

    <div id="Talking" class="d-none align-bottom">
      <span class="d-inline-block">
        <btn onclick="clickCancel();" class="cancel-btn text-white text-decoration-none">
          Cancel
        </btn>
      </span>
      <span class="d-inline-block" height="200px">
        <br /> <br />
      </span>
      <span class="d-inline-block">
        <btn class="pulse-small-btn text-white text-decoration-none">
          Talking...
        </btn>
      </span>

    </div>
  </div>
</div>
<script type="text/javascript">
  // --- prefix -----
  navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
  RTCPeerConnection = window.RTCPeerConnection || window.webkitRTCPeerConnection || window.mozRTCPeerConnection;
  RTCSessionDescription = window.RTCSessionDescription || window.webkitRTCSessionDescription || window.mozRTCSessionDescription;

  let phase = null;
  let DataLocalSdp = null;
  let DataRemoteSdp = null;

  let DataLocalIces = [];
  let DataRemoteIces = [];

  let peerConnection = null;
  let dataChannel = null;

  let isStartConnect = false;
  let isCheckOpenDataChannel = false;

  const room_id = <?= $Room->id ?>;
  const room_role = '<?= $role ?>';
  const room_name = '<?= $Room->name ?>';

  const callAudio = document.getElementById('call_audio');
  const remoteVideo = document.getElementById("remoteVideo");
  const localVideo = document.getElementById("localVideo");

  let localStream = null;
  let remoteStream = null;

  let isFirstStart = true;

  //DBとのやりとり用
  var AjaxMethod = {
    getRoomData: function(data) {
      return ($.ajax("<?= $this->Url->build(['controller' => $this->request->getParam('controller'), 'action' => 'getRoomData']); ?>", {
        type: 'post',
        data: data,
        dataType: 'json',
        cache: false,
        timeout: 10000,
      }));
    },
    addSdp: function(data) {
      return ($.ajax("<?= $this->Url->build(['controller' => $this->request->getParam('controller'), 'action' => 'addSdp']); ?>", {
        type: 'post',
        data: data,
        dataType: 'json',
        cache: false,
        timeout: 10000,
      }));
    },
    reset: function(data) {
      return ($.ajax("<?= $this->Url->build(['controller' => $this->request->getParam('controller'), 'action' => 'reset']); ?>", {
        type: 'post',
        data: data,
        dataType: 'json',
        cache: false,
        timeout: 10000,
      }));
    },
  };

  //表示の切り替え
  function SetStage(id) {
    $('#Disconnect').addClass("d-none");
    $('#Start').addClass("d-none");
    $('#Connecting').addClass("d-none");
    $('#Waiting').addClass("d-none");
    $('#Touch').addClass("d-none");
    $('#Calling').addClass("d-none");
    $('#Talking').addClass("d-none");
    $('#Finished').addClass("d-none");
    $('#' + id).removeClass("d-none");


    if (id == "Calling" || id == "Talking") {
      $('#CenterContent').addClass("d-none");
      $('#BottomContent').removeClass("d-none");
    } else {
      $('#CenterContent').removeClass("d-none");
      $('#BottomContent').addClass("d-none");
    }
  }

  function StartMedia() {
    navigator.mediaDevices.getUserMedia({
        video: {
          facingMode: 'user',
          frameRate: 30,
        },
        audio: {
          sampleRate: 32000,
          sampleSize: 16,
          volume: 1.0,
          echoCancellation: true,
          autoGainControl: true,
          noiseSuppression: true
        }
      }).then((stream) => {
        console.log(stream);
        localStream = stream;
        localVideo.srcObject = stream;
        localVideo.onloadedmetadata = (e) => {
          localVideo.play();
          checkOffer();
        };
        remoteVideo.onloadedmetadata = (e) => {
          remoteVideo.play();
        };
      })
      .catch(err => {
        console.log(JSON.stringify(err));
      });

  }

  //RemoteSdp(sendSdp)があるか確認
  function checkOffer() {

    var data = {
      name: room_name
    };

    $.when(
        AjaxMethod.getRoomData(data)
      )
      .done(function(ret) {
        console.log(ret);
        if (ret.status == 'success') {
          if (ret.room.sendsdp == "") {
            setTimeout(function() {
              checkOffer();
            }, 2000);

          } else {
            DataRemoteSdp = ret.room.sendsdp;
            DataRemoteIces = JSON.parse(ret.room.sendice);

            console.log("DataRemoteIces",DataRemoteIces);
            onSdpText();
          }
        }
      })
      .fail(function(xhr) {
        // エラーがあった時

      });
  };

  //RemoteSdp(sendSdp)があった際の処理
  function onSdpText() {
    window.localConnection = peerConnection = prepareNewConnection();
    dataChannel = peer.createDataChannel("channel");
    dataChannel = prepareDataChannel(dataChannel);

    let text = DataRemoteSdp;
    text = _trimTailDoubleLF(text); // for Safar TP --> Chrome
    let offer = new RTCSessionDescription({
      type: 'offer',
      sdp: text,
    });

    peerConnection.setRemoteDescription(offer).then(function() {
      console.log('setRemoteDescription(offer) succsess in promise2');
      if (DataRemoteIces || DataRemoteIces.length > 0) {
        DataRemoteIces.forEach(e => {
          peerConnection.addIceCandidate(e);
        });
      }
      console.log("setTimeout3000");
      makeAnswer();
    }).catch(function(err) {
      console.error('setRemoteDescription(offer) ERROR: ', err);
    });

  };

  //recvSdp作成
  function makeAnswer() {
    console.log("makeAnswer");
    let options = {};
    peerConnection.createAnswer(options)
      .then(function(sessionDescription) {
        console.log('createAnswer() succsess in promise');
        return peerConnection.setLocalDescription(sessionDescription);
      }).then(function() {
        console.log('setLocalDescription() succsess in promise');
        DataLocalSdp = peerConnection.localDescription.sdp;
      }).catch(function(err) {
        console.error(err);
      });
  };

    //接続が確認できたら、DBリセット
    function resetsdp() {
    var data = {
      id: room_id
    };

    $.when(
        AjaxMethod.reset(data)
      )
      .done(function(ret) {
        if (ret.status == 'fail') {
          setTimeout(function() {
            resetsdp();
          }, 500);
        }

        if (ret.status == 'success') {

        }

      })
      .fail(function(xhr) {
        // エラーがあった時
        setTimeout(function() {
          resetsdp();
        }, 500);
      });
  };

  //DBにrecvSdp登録
  function addRecvSdp() {
    console.log("addRecvSdp");
    let data = {
      id: room_id,
      role: room_role,
      sdp: DataLocalSdp,
      ice: JSON.stringify(DataLocalIces),
    }

    $.when(
        AjaxMethod.addSdp(data),
      )
      .done(function(ret) {
        console.log(ret);
        // すべて成功した時の処理
        if (ret.status == "fail") {
          console.log('fail ', ret);
          setTimeout(function() {
            addRecvSdp();
          }, 2000);
        }
        if (ret.status == 'success') {
          isCheckOpenDataChannel = true;
          setTimeout(function() {
            CheckOpenDataChannel();
          }, 20000);
        }
      })
      .fail(function(xhr) {
        // エラーがあった時
        console.log('error ', xhr);
      });
  };

  //コネクション作成
  function prepareNewConnection() {
    const pc_config = {
      "iceServers": []
    };
    peer = new RTCPeerConnection(pc_config);

    if ('ontrack' in peer) {
      console.log('-- peer.ontrack');
      peer.ontrack = event => {
        let stream = event.streams[0];
        if (stream) {
          remoteStream = stream;
          remoteVideo.srcObject = remoteStream;
          remoteVideo.onloadedmetadata = (e) => {
            remoteVideo.play();
          };
        }
        console.log('-- peer.ontrack(): track kind=' + event.track.kind);
        if (event.streams.length > 0 && stream) {

          console.log('got multi-stream, but play only 1 stream');
          let track = event.track;
          if (track.kind === 'video') {
            console.log('ontrack video');
            remoteStream.addTrack(track, stream);

          } else if (track.kind === 'audio') {
            console.log('ontrack audio');
            remoteStream.addTrack(track, stream);
          }
        }

        stream.onaddtrack = evt => {
          console.log('stream addtrack');
        };
        stream.onremovetrack = evt => {
          console.log('stream removetrack');
        };
      };

      peer.onaddstream = event => {
        remoteStream = event.stream;
        remoteVideo.srcObject = remoteStream;
        // -- log only --
        console.log('-- peer.onaddstream(), but do nothing');
      }
    } else {
      peer.onaddstream = event => {
        remoteStream = event.stream;
        remoteVideo.srcObject = remoteStream;
        // -- log only --
        console.log('else-- peer.onaddstream(), but do nothing');
      }
    }

    if (localStream) {
      console.log('Adding local stream...');
      if ('addTrack' in peer) {
        console.log('use addTrack()');
        let tracks = localStream.getTracks();
        for (let track of tracks) {
          let sender = peer.addTrack(track, localStream);
        }
      } else {
        console.log('use addStream()');
        peer.addStream(localStream);
      }
    } else {
      console.log('no local stream, but continue.');
    }

    peer.ondatachannel = function(evt) {
      console.log('ondatachannel');
      isCheckOpenDataChannel = false;
      if (dataChannel) {
        console.log('dataChannel ALREAY EXIST');
      } else {

      }

      dataChannel = prepareDataChannel(evt.channel);
      isStartConnect = true;
      SetStage("Waiting");
    }

    // --- on get local ICE candidate
    peer.onicecandidate = function(evt) {
      if (evt.candidate) {
        console.log('ice event', evt.candidate);
        DataLocalIces.push(evt.candidate);
      } else {
        console.log('empty ice event', evt.candidate);
        addRecvSdp();
      }
    };

    peer.onnegotiationneeded = evt => {
      console.log(evt);
      console.log('onnegotiationneeded');
    };

    peer.onconnectionstatechange = evt => {
      console.log(evt);
    };

    return peer;
  };

  //データチャンネルを扱えるようにする
  function prepareDataChannel(dc) {

    dc.onmessage = evt => {
      console.log(evt);
      const msg = evt.data;
      const obj = JSON.parse(msg);
      if (obj.type === 'text') {
        console.log('text Message over DataChannel:', obj.text);
      }
      if (obj.type === 'Connecting') {
        Connecting();
        console.log('Waiting');
      }
      if (obj.type === 'Waiting') {
        Waiting();
        console.log('Waiting');
      }
      if (obj.type === 'Calling') {
        Calling();
        console.log('Calling');
      }
      if (obj.type === 'Talking') {
        Talking();
        console.log('Talking');
      }
      if (obj.type === 'Cancel') {
        Cancel();
        console.log('Cancel');
      }
    };

    dc.onopen = evt => {
      console.log('DataChannel OPEN:',evt);
    };
    dc.onclose = evt => {
      console.log('DataChannel CLOSE:',evt);
    };
    dc.onerror = evt => {
      console.log('DataChannel ERROR:', evt);
    };

    return dc;
  };

  //１秒ごとに確認
  function update() {
    checkConnection();
  }

  //切断検知　update関数
  function checkConnection() {
    if (isStartConnect) {
      if (!peerConnection || peerConnection.connectionState != 'connected') {
        isStartConnect = false;
        SetStage('Disconnect');
        $(remoteVideo).addClass('d-none');
        stopCallAudio();
        checkOffer();
      }
    }
  }

  //接続チェック　時間が経過して接続されていなければ、最初から
  function CheckOpenDataChannel() {
    if (isCheckOpenDataChannel == true) {
      Reset();
      isCheckOpenDataChannel = false;
    }
  }

  //ライブラリ
  function _trimTailDoubleLF(str) {
    let trimed = str.trim();
    return trimed + String.fromCharCode(13, 10);
  };

  function sendMessage(type = null, msg = null) {

    obj = {
      type: type,
      text: msg
    }
    const str = JSON.stringify(obj);
    dataChannel.send(str);
    return true;
  };

  //テスト用
  function click_message() {
    let t = document.getElementById('sendtype').value;
    let v = document.getElementById('sendtext').value;
    console.log(v);
    sendMessage(t, v);
  }

  //インターフォンアプリ
  function Reset() {
    $(remoteVideo).addClass('d-none');
    phase = null;
    LocalSdp = null;
    RemoteSdp = null;

    DataLocalIces = [];
    DataRemoteIces = [];

    peerConnection = null;
    dataChannel = null;

    isStartConnect = false;
    isCheckOpenDataChannel = false;

    localStream = null;
    remoteStream = null;

    localVideo.muted = true;
    remoteVideo.muted = true;

    resetsdp();
    if (isFirstStart) {
      SetStage("Start");
    } else {
      clickStart();
    }
  }

  function Start() {
    Connecting();
    localVideo.muted = true;
    remoteVideo.muted = true;
    checkOffer();
  }

  function Connecting() {
    stopCallAudio();

    localVideo.muted = true;
    remoteVideo.muted = true;
    SetStage("Connecting");
  }

  function Waiting() {
    stopCallAudio();

    localVideo.muted = true;
    remoteVideo.muted = true;
    SetStage("Waiting");
  }

  function Calling() {
    playCallAudio();

    localVideo.muted = true;
    remoteVideo.muted = true;

    $(remoteVideo).removeClass('d-none');
    SetStage("Calling");
  }

  function Talking() {
    stopCallAudio();
    $(remoteVideo).removeClass('d-none');
    localVideo.muted = true;
    remoteVideo.muted = false;
    SetStage("Talking");
  }

  function Cancel() {
    stopCallAudio();
    $(remoteVideo).addClass('d-none');
    localVideo.muted = true;
    remoteVideo.muted = true;

    SetStage("Finished");
    setTimeout(() => {
      SetStage("Waiting");
    }, 3000);
  }

  //クリック操作
  function clickStart() {
    isFirstStart = false;
    StartMedia();
    setInterval(update, 1000);
    $('#Start').addClass("d-none");
    playCallAudio();
    stopCallAudio();
    console.log("end_clickStart");
  }

  function clickWaiting() {
    sendMessage("Waiting", "send");
    Waiting();
  }

  function clickCalling() {
    sendMessage("Calling", "send");
    Calling();
  }

  function clickTalking() {
    sendMessage("Talking", "send");
    Talking();
  }

  function clickCancel() {
    sendMessage("Cancel", "send");
    Cancel();
  }

  //呼び出し音操作
  function playCallAudio() {
    callAudio.play();
    callAudio.volume = 0.01;
  }

  function stopCallAudio() {
    callAudio.pause();
  }

  //初期動作
  document.addEventListener('DOMContentLoaded', function() {
    Reset();
  });
</script>
0
2
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
0
2