Qiita Teams that are logged in
You are not logged in to any team

Log in to Qiita Team
OrganizationAdvent CalendarQiitadon (β)
Qiita JobsQiita ZineQiita Blog
Help us understand the problem. What is going on with this article?

A tool offers P2P communication to transfer files and connect pipes between PCs in different NATs

More than 1 year has passed since last update.

I am Ryo Kanbayashi (ryo_grid).

This post introduces new software I developed.


What is this?

  • a tool transfer files and connect files between PCs in different NATs, etc.
  • using the data channel functionality of WebRTC
  • with UDP hole punching, my tool climbs over NAT
  • the P2P communication path is SCTP over DTLS over UDP (it is a reliable communication path and is also an encrypted communication path)

Architecture & Implementation

  • basically the same as WebRTC
  • for getting information for NAT traversal (UDP hole punching), WebRTC client uses the STUN server (ICE mechanism)
  • STUN server which Google is offering for testing is used
  • For implementing WebRTC, a server which is called signaling server is needed to mediate machines (two machines want to do p2p communication) in addition to STUN server
  • so, I am offering a singaling server which can be shared by multiple users by reference to Piping Server
  • my open signaling server's address is wss://ryogrid.net:11985/ . it uses WebSocket protcol over SSL
    • The communication between the user and the signaling server is encrypted
  • implementation language is Python. The part of the data channel construction of WebRTC uses a library called aiortc

Requirement to use

  • OS has been confirmed to work on Windows, Mac, Linux
  • Python runtime version 3.7+ is needed. if not installed, please install with considerigin using pyenv etc.

Attention (1)

  • UDP hole punching may not be possible depending on NAT specifications, functions, settings and network configuration, so it's not always possible to establish p2p commucation path
  • but about 80% works well I think
  • signaling server is basically no more than a bridge of information to set up a communication path, but if I tired, I can see the contents

Installationn (pip module)

$ pip install onatlib

if you install with pip module, description of my server and tool excetion below are replaced to ...

  • python p2p_com_local_server.py [arguments] -> p2p_com_local_serv [arguments]
  • python pync.py [arguments] -> pync [arguments]
  • python shareable_ws_signailing_serv.py [arguments] -> shareable_ws_signaling_serv [arguments]

Installation (from github)

clone repository
$ git clone -b first-release-0624 https://github.com/ryogrid/Over-NAT-Lib.git
install required pip modues
$ cd Over-NAT-Lib
$ pip install -r requirements.txt

Using(preparation part)

First, move to the tools directory of the repository you cloned

$ cd Over-Nat-Lib/tools
  • Signaling Server match by receiving identifier which is shared between two machines (persons)
  • the identifier should be alphanumeric character string and length of it should be longer than 9 chalacters
  • it must not able to be guess by other persons for security
  • Therefore, First, please decide identifier sting and share between persons who want to p2p communicate
  • due to my implementation reason, in order to identify which of the two, each person select "bob" or "tom"
  • there is no particular reason about selected names (tom and bob)

Attention (2)

  • on Windows, please use the command prompt. In the case of Power Shell, the handling of pipes and stdout is special, and it does not work well with my tools
  • from the characteristics of the established communication path, the local server on the sending side buffering 1 MB of data and do not send data to the remote until buffered size reaches 1MB (If the EOF is detected even if 1 MB is not reached, the buffered data will be sent regardless of the size) for communication performance (bandwidth)

Using (channel establishment)

  • current directory is in the tools directory
  • executing my tool (p2P_com_local_server.py) like below (left of this post) spawns p2P_com_local_server.py process with diffent parameters. as a result, 3 processes starts.
  • If the parent is terminated with Ctrl-C, the other two processes are also terminated, but if the parent dies for some reason and can not perform termination processing, the two child processes remain as they are. So, if that cese occurs, please kill these with your own....

first, establish a communication channel. The program which is executed here is like a server program that runs on the local host.
the established communication path is designed to be maintained by this server program and to be used by other programs.

for example, the identifier is "d5ocew761d" on below.
Note that below command execution redirecting stderr to a file, but this is because I output a not little amount of debugging output to stderr. it is not necessary.
if you want to see the detailed debug output for establishing a communication path (output by the library which is used internally), it is interesting to add the -v option.

Tom side

$ python p2p_com_local_server.py --signaling share-websocket --signaling-host ryogrid.net --signaling-port 11985 --name tom --secure-signaling d5ocew761d 2> hogehoge.txt

Bob side

$ python p2p_com_local_server.py --signaling share-websocket --signaling-host ryogrid.net --signaling-port 11985 --name bob --secure-signaling d5ocew761d 2> hogehoge.txt

execution order can be reversed.

communication path will be established in as little as 3 minutes after both start up.
You can confirm that it is established if the following is outputted.

sender_proc: datachannel established
recv_proc: datachannel established

if NAT traversal fails, output like below are shown, but if you wait for 10 minutes, you can think that the traversal fails.

hole punching to remote machine failed.

if communication channel is established, in each of the host, the write port of the communication channel
opens at 10100. read port at 10200.

If you want to try it on a single PC, please use --slide-stream-ports option at second execution, which changes write port to 10101 and read port to 10201.

If you want to quit, please type Ctrl-c in the terminal which my local server was executed.

Using (pipe transfer)

  • following assumes that the communication channel is established.
  • pync.py which is like Netcat(nc) tool is used
  • this is a example that uses pipe stdin and redirect stdout to remote machine
  • current directory is tools directory

The sender

send contens of sometexts.txt to remote.

$ cat sometexts.txt | python pync.py -c -p 10100

on windows, you can use type command instead of cat command

The receiver

count the number of received data lines.

$ python pync.py -r -p 10200 | wc -l

Note that the order of execution does not matter.

Transfer end

  • transfer is completed, pync.py ends
  • the size of the receive buffer at the socket (write port) of the local server is not small. so, if the data to be transferred is about several KB in size, the sending side pync.py may finish sending (to the local server) and exit. though you receiver side pync.py is connected yet
  • even if data transfer finished, established communication channel is not disconnected. so, for example, you can do same transfer like above multi times

Using (file transfer)

  • following assumes that the communication channel is established.
  • current directory is tools directory
  • this is a feature that file transfer without the receiving side person approve
  • transferred files are basically written to the current directory at the time of execution of p2p_com_local_server.py on the receiving side, but it is NOT if the file name is specified with malicious intent (check mechanism is not implemented yet...)
  • the difference with pipe transfer example already explained is that pync.py is given the -f option to specify the file name at the write destination

File sender side

file name is somefile.tar.gz. Here, the same name as the file which is cated & piped.
but it does not matter if another file name is specified, and the filename is used of file transferred data is written.

$ cat somefile.tar.gz | python pync.py -f somefile.tar.gz -c -p 10100

receiver does not need to do anything.
pync.py exits when the transfer is complete.

Library for using established transport

Setup own signaling server

  • if you can't trust me, you can set up your own signaling server.
  • current directory is tools directory

With encryption

  • place certificate files to tools directory
    • privkey.pem and fullchain.pem

start server.

$ python python shareable_ws_signaling_serv.py --secure --port <port to listen> &> sigserv_log.txt

server exit can be with typing Ctrl-c in the terminal which my local server was executed.
the local server (p2p_com_local_server.py) started for data transfer as described above can be used with the --secure-signaling option.

With no encryption

  • current directory is tools directory

start server.

$ python python shareable_ws_signaling_serv.py --port <port to listen> &> sigserv_log.txt

server exit can be with typing Ctrl-c in the terminal which my local server was executed.
the local server (p2p_com_local_server.py) started for data transfer as described above can not be used with the --secure-signaling option.

Binary (execution file format) distribution (Windows platform only)

I created Binary distribution of tools of Over-NAT-Lib.
These works without python environment.

Have Fun!

Help us understand the problem. What is going on with this article?
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
svengali,kikker,はてブおせっかい,ひらめいったー,顔面偏差値スカウター, UZOMUZO, Over-NAT-Lib 等の作者.機械学習でシステムトレードしたりも.元の専門はHPC (修士).機械学習,分散システム, Maker的なもの,コミュニケーションTech, 投資などに関心.Scrum-Upプロジェクト参画中. ryo.contact[at]gmail.com


No comments
Sign up for free and join this conversation.
Sign Up
If you already have a Qiita account Login
Help us understand the problem. What is going on with this article?