I am Ryo Kanbayashi (ryo_grid).

This post introduces new software I developed.

Over-NAT-Lib

What is this?

a tool transfer files and connect pipe between PCs in different NATs, etc
- It is a feeling inspired by Piping Server
using the data channel functionality of WebRTC
with UDP hole punching, my tool climbs over NAT
the P2P communication path is SCTP over DTLS over UDP (it is a reliable communication path and is also an encrypted communication path)

Architecture & Implementation

basically the same as WebRTC
for getting information for NAT traversal (UDP hole punching), WebRTC client uses the STUN server (ICE mechanism)
STUN server which Google is offering for testing is used
For WebRTC working, a server which is called signaling server is needed to mediate machines (two machines want to do p2p communication) in addition to STUN server
so, I am offering a singaling server which can be shared by multiple users
my signaling server's address is wss://ryogrid.net:11985/ . it uses WebSocket protcol over SSL
- The communication between the user and the signaling server is encrypted
implementation language is Python. The part of the data channel construction of WebRTC uses a library called aiortc

Requirement to use

OS has been confirmed to work on Windows, Mac, Linux
Python runtime version 3.7+ is needed. if not installed, please install with considering using pyenv etc.

Attention (1)

UDP hole punching may not be possible depending on NAT specifications, functions, settings and network configuration, so it's not always possible to establish p2p commucation path
but about 80% works well I think
signaling server is basically no more than a bridge of information to set up a communication path, but if I
tried, I can see the contents of information for communication path establishment (info like key for encryption is not included!)
- signaling server does not relay much secret information. so it's meaningless if I look at
  it
- I can only say that it is an extreme theory, please believe me :)
- About information of me is as follows
  - http://ryogrid.net/profile/index_en.html
  - https://www.linkedin.com/in/ryo-kanbayashi-3a78266a/

Installationn (pip module)

$ pip install onatlib

if you install with pip module, description of my server and tool exection below are replaced to ...

python p2p_com_local_server.py [arguments] -> p2p_com_local_serv [arguments]
python pync.py [arguments] -> pync [arguments]
python shareable_ws_signailing_serv.py [arguments] -> shareable_ws_signaling_serv [arguments]

Installation (from github)

clone repository
$ git clone -b first-release-0624 https://github.com/ryogrid/Over-NAT-Lib.git
install required pip modues
$ cd Over-NAT-Lib
$ pip install -r requirements.txt

Using（preparation part）

First, move to the tools directory of the repository you cloned

$ cd Over-Nat-Lib/tools

Signaling Server match users by receiving identifier which is shared between two machines (persons)
the identifier should be alphanumeric character string and length of it should be longer than 9 characters
it must not able to be guess by other persons for security
Therefore, First, please decide identifier string and share between persons who want to p2p communicate
due to my implementation reason, in order to identify which of the two, each person select "bob" or "tom"
there is no particular reason about selected names (tom and bob)

Attention (2)

on Windows, please use the command prompt. In the case of Power Shell, the handling of pipes and stdout is special, and it does not work well with my tools
from the characteristics of the established communication path, the local server on the sending side buffering 1 MB of data and do not send data to the remote until buffered size reaches 1MB (If the EOF is detected even if 1 MB is not reached, the buffered data will be sent regardless of the size) for communication performance(=throughput)

Using (channel establishment)

current directory is in the tools directory
executing my tool (p2P_com_local_server.py) like below (left of this post) spawns p2P_com_local_server.py process with diffent parameters. as a result, 3 processes starts.
If the parent is terminated with Ctrl-C, the other two processes are also terminated, but if the parent dies for some reason and can not perform termination processing, the two child processes remain as they are. So, if that case occurs, please kill these with your own....

first, establish a communication channel. The program which is executed here is like a server program that runs on the local host.
the established communication path is designed to be maintained by this server program and to be used by other programs.

for example, the identifier is "d5ocew761d" on below.
Note that below command execution redirecting stderr to a file, but this is because I output a not little amount of debugging output to stderr. it is not necessary.
if you want to see the detailed debug output for establishing a communication path (output by the library which is used internally), it is interesting to add the -v option.

Tom side

$ python p2p_com_local_server.py --signaling share-websocket --signaling-host ryogrid.net --signaling-port 11985 --name tom --secure-signaling d5ocew761d 2> hogehoge.txt

Bob side

$ python p2p_com_local_server.py --signaling share-websocket --signaling-host ryogrid.net --signaling-port 11985 --name bob --secure-signaling d5ocew761d 2> hogehoge.txt

execution order can be reversed.

communication path will be established in as little as 3 minutes after both start up.
You can confirm that it is established if the following is outputted.

sender_proc: datachannel established
recv_proc: datachannel established

if NAT traversal fails, output like below are shown, but if you wait for 10 minutes, you can think that the traversal fails.

hole punching to remote machine failed.

if communication channel is established, in each of the host, the write port of the communication channel
opens at 10100. read port at 10200.

If you want to try it on a single PC, please use --slide-stream-ports option at second execution, which changes write port to 10101 and read port to 10201.

If you want to quit, please type Ctrl-c in the terminal which my local server was executed.

Using (pipe transfer)

following assumes that the communication channel has been established via pipe.
pync.py which is like Netcat(nc) tool is used
this is a example that uses pipe stdin and redirect stdout to remote machine
current directory is tools directory

The sender

send contens of sometexts.txt to remote.

$ cat sometexts.txt | python pync.py -c -p 10100

on windows, you can use type command instead of cat command

The receiver

count the number of received data lines.

$ python pync.py -r -p 10200 | wc -l

Note that the order of execution does not matter.

Transfer end

transfer is completed, pync.py ends
the size of the receive buffer at the socket (write port) of the local server is not small. so, if the data to be transferred is about several KB in size, the sending side pync.py may finish sending (to the local server) and exit. though your receiver side pync.py is connected yet
even if data transfer finished, established communication channel is not disconnected. so, for example, you can do same transfer like above multi times

Using (direct file transfer)

following assumes that the already communication channel has been established.
current directory is tools directory
this is a feature that file transfer without the receiving side person approve
transferred files are basically written to the current directory at the time of execution of p2p_com_local_server.py on the receiving side, but does NOT if the file name is specified with malicious intent (check mechanism is not implemented yet...)
the difference with pipe transfer example already explained is that pync.py is given the -f option to specify the file name at the write destination

File sender side

file name is somefile.tar.gz. Here, the same name as the file which is cated & piped.
but it does not matter if different file name is specified as argument of pync.py, the filename specified to pync.py is used of file cration for write transferred data.

$ cat somefile.tar.gz | python pync.py -f somefile.tar.gz -c -p 10100

receiver does not need to do anything.
pync.py exits when the transfer is complete.

Library for using established transport

I wrote simple library
it enable you to access local server's transport transparently
receiver side can use data receive callback
library returns pure python socket object
please use below example code as a refecence (simple echo server and client)
- https://github.com/ryogrid/Over-NAT-Lib/blob/f042801b29b6865d875d35417cda9031d7afcee5/example/echo_test.py

Setup own signaling server

if you can't trust me, you can set up your own signaling server.
current directory is tools directory

With encryption

place certificate files to tools directory
- privkey.pem and fullchain.pem

start server.

$ python python shareable_ws_signaling_serv.py --secure --port <port to listen> &> sigserv_log.txt

server exit can be with typing Ctrl-c in the terminal which my local server was executed.
the local server (p2p_com_local_server.py) for data transfer as described above can be executed with the --secure-signaling option.

With no encryption

current directory is tools directory

start server.

$ python python shareable_ws_signaling_serv.py --port <port to listen> &> sigserv_log.txt

server exit can be with typing Ctrl-c in the terminal which my local server was executed.
the local server (p2p_com_local_server.py) for data transfer as described above can not be executed with the --secure-signaling option.

Binary (execution file format) distribution (Windows platform only)

I created Binary distribution of tools of Over-NAT-Lib.
These works without python environment.

Have Fun!