From 6a24da74ef8b4ee33860cde752a3d885e2c8f226 Mon Sep 17 00:00:00 2001 From: Akilan Date: Tue, 3 Jun 2025 15:22:05 +0100 Subject: [PATCH] removed index.org~ from main directory --- index.org~ | 1017 ---------------------------------------------------- 1 file changed, 1017 deletions(-) delete mode 100644 index.org~ diff --git a/index.org~ b/index.org~ deleted file mode 100644 index 545089f..0000000 --- a/index.org~ +++ /dev/null @@ -1,1017 +0,0 @@ -#+SETUPFILE: https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup - -#+HTML_HEAD: -#+HTML_HEAD: - -#+attr_html: :width 300px :align center -[[./artwork/p2prc-logos/Colored-On-Light-Image.png]] - -* Guide through video -*** The video below shows the setup and usage of P2PRC. -#+attr_html: :class video -[[https://www.youtube.com/watch?v=OMwCpedu5cs][https://i3.ytimg.com/vi/OMwCpedu5cs/maxresdefault.jpg]] - -* Introduction -:PROPERTIES: -:CUSTOM_ID: chapter1-introduction -:END: -This project focuses on creating a framework for running heavy computational tasks that a regular -computer cannot handle easily. These tasks may include graphically demanding video games, rendering -3D animations, and performing complex protein folding simulations. The major focus of this project -is not on financial incentives but rather on building a robust and efficient peer-to-peer (P2P) -network to decentralise task execution and increase the computational bandwidth available for -such tasks. - -The P2PRC framework serves as a foundation for decentralised rendering and computation, -providing insights into how tasks can be distributed efficiently across a network of peers. -Leveraging the P2PRC approach, this project extends its capabilities to handle a -wider range of computationally intensive tasks. - -** Motivation -:PROPERTIES: -:CUSTOM_ID: motivation -:END: -Many of the users rely on our PC / Laptop or servers that belong to a -server farm to run heavy tasks and with the demand of high creativity -requires higher computing power. Buying a powerful computer every few -years to run a bunch of heavy tasks which are not executed as frequently -to reap the benefits can be inefficient utilization of hardware. On the -other end, renting servers to run these heavy tasks can be really -useful. Ethically speaking this is leading to monopolisation of -computing power similar to what is happening in the web server area. By -using peer to peer principles it is possible to remove the -monopolisation factor and increase the bandwidth between the client and -server. - -* Installation -:PROPERTIES: -:CUSTOM_ID: installation -:END: - -Over here we will cover the basic steps to get the server and client -side running. - -** Latest release installation -:PROPERTIES: -:CUSTOM_ID: latest-release-install -:END: -https://github.com/Akilan1999/p2p-rendering-computation/releases - - -** Manual Installation from Github master branch -:PROPERTIES: -:CUSTOM_ID: install-from-github-master-branch -:END: -*** Install Go lang -:PROPERTIES: -:CUSTOM_ID: install-go-lang -:END: -The entire the implementation of this project is done using Go lang. -Thus, we need go lang to compile to code to a binary file. -[[https://golang.org/doc/install][Instructions to install Go lang]] - -*** Install Docker -:PROPERTIES: -:CUSTOM_ID: install-docker -:END: -In this project the choice of virtualisation is Docker due to it's wide -usage in the developer community. In the server module we use the Docker -Go API to create and interact with the containers. - -[[https://docs.docker.com/get-docker/][Instructions to install docker]] - -[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions -to install docker GPU]] - -#+begin_example -// Do ensure that the docker command does not need sudo to run -sudo chmod 666 /var/run/docker.sock -#+end_example - -*** Build Project and install project -:PROPERTIES: -:CUSTOM_ID: build-project-and-install-project -:END: -To set up the internal dependencies and build the entire go code into a -single binary - -#+begin_example -make -#+end_example - -**** For Windows -:PROPERTIES: -:CUSTOM_ID: for-windows -:END: -To set up P2PRC on Windows, simply run this batch file. *Make sure you -are not in admin mode when running this.* - -#+begin_example -.\install.bat -#+end_example - -*** Add appropriate paths to =.bashrc= -:PROPERTIES: -:CUSTOM_ID: add-appropriate-paths-to-.bashrc -:END: -#+begin_example -export P2PRC=//p2p-rendering-computation -export PATH=//p2p-rendering-computation:${PATH} -#+end_example - -*** Test if binary works -:PROPERTIES: -:CUSTOM_ID: test-if-binary-works -:END: -#+begin_example -p2prc --help -#+end_example - -**** Output: -:PROPERTIES: -:CUSTOM_ID: output -:END: -#+begin_example -NAME: - p2p-rendering-computation - p2p cli application to create and access VMs in other servers - -USAGE: - p2prc [global options] command [command options] [arguments...] - -VERSION: - 2.0.0 - -COMMANDS: - help, h Shows a list of commands or help for one command - -GLOBAL OPTIONS: - --Server, -s Starts server (default: false) [$SERVER] - --UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST] - --ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS] - --AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER] - --ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES] - --CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM] - --ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME] - --BaseImage value, --bi value Specifying the docker base image to template the dockerfile [$CONTAINER_NAME] - --RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM] - --ID value, --id value Docker Container ID [$ID] - --Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS] - --GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU] - --Specification value, --specs value Specs of the server node [$SPECS] - --SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG] - --NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE] - --ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN] - --TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS] - --ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN] - --CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP] - --Group value, --group value group flag with argument group ID [$GROUP] - --Groups, --groups View all groups (default: false) [$GROUPS] - --RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP] - --RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP] - --MAPPort value, --mp value Maps port for a specific port provided as the parameter [$MAPPORT] - --DomainName value, --dn value While mapping ports allows to set a domain name to create a mapping in the proxy server [$DOMAINNAME] - --Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE] - --ModuleName value, --mod value New go project module name [$MODULENAME] - --PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN] - --RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN] - --AddMetaData value, --amd value Adds metadata about the current node in the p2p network which is then propagated through the network [$ADDMETADATA] - --help, -h show help (default: false) - --version, -v print the version (default: false) -#+end_example - --------------- - - -* Nix Flake - -Nix is a growing ecosystem that allows flexibility on how you develop, build and package software and configurations. It brings all programming languages (and all other tooling) to an equal footing, despite deep design differences. More importantly, integrates all the "packaging" into the context of a "pure" function. - - -P2PRC aims to become a utility that can be used in various flexible manners and having Nix support is a good alternative to accomplish this goal. - - -Nix Flake is a format, within the Nix ecosystem, intentionally designed to encourage a standard in packaging distribution. The current packaging assumes that you have "nix flake" installed because it is currently an experimental feature of Nix. - - -** P2PRC core Go language repo - -In case you want to develop, build or integrate using nix, you just need to run either "nix develop" or "nix run" from the command line locally in a cloned git repository or by running "nix run github:akilan1999/p2p-rendering-computation -- --help" - - -P2PRC library also is ready to be imported into other nix flakes. To accomplish that please make sure to override the target nixpkgs environment in the following manner; - -#+begin_example -pkgs = import nixpkgs { - inherit system; - overlays = [ - p2prc-flake.overlays.default - ]; -}; -#+end_example - - -This will make the p2prc executable available in the environment of any application you use. - - -* P2PRC Haskell library - - -The project is structured to provide language bindings to any programming language. The first one being supported in this manner is the Haskell programming language. It provides a bootstrapping script for a new Cabal project with p2prc binary available in the environment and, more relevantly, the Haskell library bindings available in the virtual environment cabal environment. - -#+begin_example -nix run git+https://github.com/akilan1999/p2p-rendering-computation#initHaskellProject -- -#+end_example - - -This will generate a new haskell project setup to automatically work with the p2prc development and running environment. - - -Once completed, you should go into the project directory and copy the nix flake template, necessary to define the project's environment. - -#+begin_example -nix flake init -t github:akilan1999/p2p-rendering-computation#haskell -#+end_example - -The previous command sets up the flake environment and its dependencies. It will look like the following - -#+begin_example -{ - description = "Start of Haskell P2PRC flake"; - - inputs = - { - nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable"; - - flake-util.url = "github:numtide/flake-utils"; - - p2prc-flake.url = "github:akilan1999/p2p-rendering-computation"; - }; - - outputs = { nixpkgs, p2prc-flake, flake-utils, ... }: - (flake-utils.lib.eachDefaultSystem (system: - let - - pkgs = import nixpkgs { - inherit system; - overlays = [ - p2prc-flake.overlays.default - p2prc-flake.overlays.bindings - ]; - }; - - in { - - packages.default = pkgs.haskellPackages.callPackage ./cabal.nix { }; - - devShells.default = pkgs.haskellPackages.shellFor { - - packages = p: [ - (p.callPackage ./cabal.nix { }) - ]; - - buildInputs = with pkgs; [ - p2prc-flake.packages.${system}.default - ghc - cabal2nix - cabal-install - ]; - - shellHook = '' - cabal2nix . > ./cabal.nix - ''; - }; - } - )); -} -#+end_example - -The template uses Cabal2Nix which manages the Haskell virtual environment automatically. It applies the system overlays, sets up the shell environment for the project (updating the Cabal2Nix configuration) and packages the main executable. - - -* Using basic commands -:PROPERTIES: -:CUSTOM_ID: using-basic-commands -:END: -*** Start as a server -:PROPERTIES: -:CUSTOM_ID: start-as-a-server -:END: - -#+begin_example -p2prc -s -#+end_example - -*** View server Specification -:PROPERTIES: -:CUSTOM_ID: view-server-specification -:END: -#+begin_example -p2prc --specs= -#+end_example - -*** Run container -:PROPERTIES: -:CUSTOM_ID: run-container -:END: -use the =--gpu= if you know the other machine has a gpu. - -#+begin_example -p2prc --touch= -p --gpu -#+end_example - -*** Remove container -:PROPERTIES: -:CUSTOM_ID: remove-container -:END: -The docker id is present in the output where you create a container - -#+begin_example -p2prc --rm= --id= -#+end_example - -*** Adding servers to ip table -:PROPERTIES: -:CUSTOM_ID: adding-servers-to-ip-table -:END: -#+begin_example -p2prc --as= -#+end_example - -*** Update ip table -:PROPERTIES: -:CUSTOM_ID: update-ip-table -:END: -#+begin_example -p2prc --us -#+end_example - -*** List Servers -:PROPERTIES: -:CUSTOM_ID: list-servers -:END: -#+begin_example -p2prc --ls -#+end_example - -*** View Network interfaces -:PROPERTIES: -:CUSTOM_ID: view-network-interfaces -:END: -#+begin_example -p2prc --ni -#+end_example - -*** Viewing Containers created Client side -:PROPERTIES: -:CUSTOM_ID: viewing-containers-created-client-side -:END: -#+begin_example -p2prc --tc -#+end_example - -*** Running plugin -:PROPERTIES: -:CUSTOM_ID: running-plugin -:END: -#+begin_example -p2prc --plugin --id -#+end_example - -*** Create group -:PROPERTIES: -:CUSTOM_ID: create-group -:END: -#+begin_example -p2prc --cgroup -#+end_example - -*** Add container to group -:PROPERTIES: -:CUSTOM_ID: add-container-to-group -:END: -#+begin_example -p2prc --group --id -#+end_example - -*** View groups -:PROPERTIES: -:CUSTOM_ID: view-groups -:END: -#+begin_example -p2prc --groups -#+end_example - -*** View specific group -:PROPERTIES: -:CUSTOM_ID: view-specific-group -:END: -#+begin_example -p2prc --group -#+end_example - -*** Delete container from group -:PROPERTIES: -:CUSTOM_ID: delete-container-from-group -:END: -#+begin_example -p2prc --rmcgroup --group --id -#+end_example - -*** Delete entire group -:PROPERTIES: -:CUSTOM_ID: delete-entire-group -:END: -#+begin_example -p2prc --rmgroup -#+end_example - -*** Pulling plugin from a remote repo -:PROPERTIES: -:CUSTOM_ID: pulling-plugin-from-a-remote-repo -:END: -#+begin_example -p2prc --pp -#+end_example - -*** Deleting plugin from the plugin directory -:PROPERTIES: -:CUSTOM_ID: deleting-plugin-from-the-plugin-directory -:END: -#+begin_example -p2prc --rp -#+end_example - -*** Added custom metadata about the current node -:PROPERTIES: -:CUSTOM_ID: added-custom-metadata-about-the-current-node -:END: -#+begin_example -p2prc --amd "custom metadata" -#+end_example - -*** MapPort and link to domain name -Allows to expose remote ports from a machine in the P2P network. -#+begin_example -p2prc --mp --dn -#+end_example -**** MapPort in remote machine -This is to ensure ports on remote machines on the P2PRC can be easily opened. -#+begin_example -p2prc --mp --dn --ra -#+end_example - --------------- - -* P2P Module Implementation -:PROPERTIES: -:CUSTOM_ID: p2p-module-implementation -:END: -The P2P module is for managing server information within the network. -It maintains and updates the IP table, ensuring accuracy by preventing duplicates and removing -entries for unreachable servers. Furthermore, the module conducts speed tests on the listed servers -to determine upload and download speeds. This valuable information enables users to identify nearby -servers with optimal performance, enhancing their overall network experience. - -#+caption: UML diagram of P2P module -[[file:Docs/images/p2pmoduleArch.png] - -The peer to peer implementation was built from scratch. This is because -other peer to peer libraries were on the implementation of the -Distributed hash table. At the current moment all those heavy features -are not needed because the objective is to search and list all possible -servers available. The limitation being that to be a part of the network -the user has to know at least 1 server. The advantage of building from -scratch makes the module super light and possibility for custom -functions and structs. The sub topics below will mention the -implementations of each functionality in depth. - -** IP Table -:PROPERTIES: -:CUSTOM_ID: ip-table -:END: -The ip table file is a json as the format with a list of servers ip -addresses, latencies, downloads and uploads speeds. The functions -implemented include read file, write file and remove duplicate IP -addresses. The remove duplicate IP address function exists because -sometimes servers IP tables can have the same ip addresses as what the -client has. The path of the IP table json file is received from the -configuration module. - -#+begin_src json -{ - "ip_address": [ - { - "Name": "", - "MachineUsername": "", - "IPV4": "", - "IPV6": "", - "Latency": , - "Download": , - "Upload": , - "ServerPort": "", - "BareMetalSSHPort": "", - "NAT": "", - "EscapeImplementation": "", - "ProxyServer": "", - "UnSafeMode": ", - "PublicKey": "", - "CustomInformation": "" - } - ] -} -#+end_src - -*** Latency -:PROPERTIES: -:CUSTOM_ID: latency -:END: -The latency is measured in milliseconds. The route /server_info is -called from the server and time it takes to provide a json response is -recorded. - -** NAT Traversal -:PROPERTIES: -:CUSTOM_ID: nat-traversal -:END: -P2PRC currently supports TURN for NAT traversal. - -** TURN -:PROPERTIES: -:CUSTOM_ID: turn -:END: -The current TURN implementation used is FRP. The TURN server is also -required when a P2PRC node is acting as a Server. The TURN server is -determined based on the Node with the least amount of latency based on -the Nodes available on the IPTable. Once a TURN server is determined -there are 2 actions performed. The first one is =/FRPPort= to the TURN -server to receive a port which is used to generate the external port -from the TURN server. The flow below describes the workflow. - -*** Client mode -:PROPERTIES: -:CUSTOM_ID: client-mode -:END: -- Call =/FRPPort= - -#+begin_example -http://:/FRPport -#+end_example - -- Call the TURN server in the following manner. The following is a - sample code snippet below. - -#+begin_src go -import ( - "github.com/Akilan1999/p2p-rendering-computation/p2p/frp" -) - -func main() { - serverPort, err := frp.GetFRPServerPort("http://" + + ":" + lowestLatencyIpAddress.ServerPort) - if err != nil { - return nil, err - } - // Create 1 second delay to allow FRP server to start - time.Sleep(1 * time.Second) - // Starts FRP as a client with - proxyPort, err := frp.StartFRPClientForServer(, serverPort, ) - if err != nil { - return nil, err - } -} -#+end_src - -* Language Bindings -:PROPERTIES: -:CUSTOM_ID: language-bindings -:END: -[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]] -refers to wrappers to bridge 2 programming languages. This is used in -P2PRC to extend calling P2PRC functions in other programming languages. -Currently this is done by generating =.so= and =.h= from the Go -compiler. - -** How to build shared object files -:PROPERTIES: -:CUSTOM_ID: how-to-build-shared-object-files -:END: -**** The easier way -:PROPERTIES: -:CUSTOM_ID: the-easier-way -:END: -#+begin_src sh -# Run -make sharedObjects -#+end_src - -**** Or the direct way -:PROPERTIES: -:CUSTOM_ID: or-the-direct-way -:END: -#+begin_src sh -# Run -cd Bindings && go build -buildmode=c-shared -o p2prc.so -#+end_src - -**** If successfully built: -:PROPERTIES: -:CUSTOM_ID: if-successfully-built -:END: -#+begin_src sh -# Enter into the Bindings directory -cd Bindings -# List files -ls -# Find files -p2prc.h p2prc.so -#+end_src - -** Workings under the hood -:PROPERTIES: -:CUSTOM_ID: workings-under-the-hood -:END: -Below are a sample set of commands to open the bindings implementation. - -#+begin_example -# run -cd Bindings/ -# list files -ls -# search for file -Client.go -#+end_example - -*** In Client go -:PROPERTIES: -:CUSTOM_ID: in-client-go -:END: -There a few things to notice which are different from your standard Go -programs: - -**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required. -:PROPERTIES: -:CUSTOM_ID: we-import-c-which-means-cgo-is-required. -:END: -#+begin_src go -import "C" -#+end_src - -**** 2. All functions which are required to be called from other programming languages have comment such as. -:PROPERTIES: -:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as. -:END: -#+begin_src go -//export - -// ------------ Example ---------------- -// The function below allows to externally -// to call the P2PRC function to start containers -// in a specific node in the know list of nodes -// in the p2p network. -// Note: the comment "//export StartContainer". - -//export StartContainer -func StartContainer(IP string) (output *C.char) { - container, err := client.StartContainer(IP, 0, false, "", "") - if err != nil { - return C.CString(err.Error()) - } - return ConvertStructToJSONString(container) - } -#+end_src - -**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure). -:PROPERTIES: -:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure. -:END: -#+begin_src go -// --------- Example ------------ - -//export StartContainer -func StartContainer(IP string) (output *C.char) { - container, err := client.StartContainer(IP, 0, false, "", "") - if err != nil { - return C.CString(err.Error()) - } - return ConvertStructToJSONString(container) -} - -//export ViewPlugin -func ViewPlugin() (output *C.char) { - plugins, err := plugin.DetectPlugins() - if err != nil { - return C.CString(err.Error()) - } - return ConvertStructToJSONString(plugins) -} -#+end_src - -**** It is easy to notice that: -:PROPERTIES: -:CUSTOM_ID: it-is-easy-to-notice-that -:END: -- =ConvertStructToJSONString()=: This is a helper function - that convert a go object to JSON string initially and converts it to - =CString=. -- =(output *C.char)=: This is the return type for most of the functions. - -**** A Pseudo code to refer to the common function implementation shape could be represented as: -:PROPERTIES: -:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as -:END: -#+begin_example -func (output *C.char) { - , := () - if != nil { - return C.CString(.Error()) - } - return ConvertStructToJSONString() -} -#+end_example - -** Current languages supported -:PROPERTIES: -:CUSTOM_ID: current-languages-supported -:END: -*** Python - -**** Build sample python program -:PROPERTIES: -:CUSTOM_ID: build-sample-python-program -:END: -The easier way - -#+begin_src sh -# Run -make python -# Expected output -Output is in the Directory Bindings/python/export/ -# Run -cd Bindings/python/export/ -# list files -ls -# Expected output -SharedObjects/ library.py requirements.txt -#+end_src - -Above shows a generated folder which consists of a folder called -"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files. -=p2prc.py= refers to a sample python script calling P2PRC go functions. -To start an any project to extend P2PRC with python, This generated -folder can copied and created as a new git repo for P2PRC extensions -scripted or used a reference point as proof of concept that P2PRC can be -called from other programming languages. - -*** Haskell -P2PRC officially supports Haskell bindings and will further support -project using Haskell to build orchestrators on top of P2PRC. - -[[https://p2prc.akilan.io/Docs/haskell][Read more...]] - -* Config Implementation -:PROPERTIES: -:CUSTOM_ID: config-implementation -:END: -The configuration module is responsible to store basic information of -absolute paths of files being called in the Go code. In a full-fledged -Cli the configuration file can be found in the directory /etc/ and from -there points to location such as where the IP table file is located. In -the future implementation the config file will have information such as -number of hops and other parameters to tweak and to improve the -effectiveness of the peer to peer network. The configuration module was -implemented using the library Viper. The Viper library automates -features such as searching in default paths to find out if the -configuration file is present. If the configuration file is not present -in the default paths then it auto generates the configuration file. The -configurations file can be in any format. In this project the -configuration file was generated using JSON format. - -#+begin_src json -{ - "MachineName": "pc-74-120.customer.ask4.lan", - "IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json", - "DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/", - "DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/", - "SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin", - "IPV6Address": "", - "PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy", - "TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json", - "ServerPort": "8088", - "GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json", - "FRPServerPort": "True", - "BehindNAT": "True", - "CustomConfig": null -} -#+end_src - -* Abstractions -:PROPERTIES: -:CUSTOM_ID: abstractions -:END: - -The Abstractions package consists of black-boxed functions for P2PRC. - -** Functions -:PROPERTIES: -:CUSTOM_ID: functions -:END: -- =Init()=: Initializes P2PRC with all the needed - configurations. -- =Start()=: Starts p2prc as a server and makes it possible to extend by - adding other routes and functionality to P2PRC. -- =MapPort()=: On the local machine the port you want to export - to world. -- =StartContainer()=: The machine on the p2p network where - you want to spin up a docker container. -- =RemoveContainer(,)=: Terminate container - based on the IP address and container name. -- =GetSpecs()=: Get specs of a machine on the network based - on the IP address. -- =ViewIPTable()=: View the IP table which about nodes in the network. -- =UpdateIPTable()=: Force update IP table to learn about new nodes - faster. - -* NAT Traversal -:PROPERTIES: -:CUSTOM_ID: nat-traversal -:END: -P2PRC currently supports TURN for NAT traversal. - -** TURN -:PROPERTIES: -:CUSTOM_ID: turn -:END: -The current TURN implementation used is FRP. The TURN server is also -required when a P2PRC node is acting as a Server. The TURN server is -determined based on the Node with the least amount of latency based on -the Nodes available on the IPTable. Once a TURN server is determined -there are 2 actions performed. The first one is =/FRPPort= to the TURN -server to receive a port which is used to generate the external port -from the TURN server. The flow below describes the workflow. - -* Client mode -:PROPERTIES: -:CUSTOM_ID: client-mode -:END: -- Call =/FRPPort= - -#+begin_example -http://:/FRPport -#+end_example - -- Call the TURN server in the following manner. The following is a - sample code snippet below. - -#+begin_src go -import ( - "github.com/Akilan1999/p2p-rendering-computation/p2p/frp" -) - -func main() { - serverPort, err := frp.GetFRPServerPort("http://" + + ":" + lowestLatencyIpAddress.ServerPort) - if err != nil { - return nil, err - } - // Create 1 second delay to allow FRP server to start - time.Sleep(1 * time.Second) - // Starts FRP as a client with - proxyPort, err := frp.StartFRPClientForServer(, serverPort, ) - if err != nil { - return nil, err - } -} -#+end_src - - - - -* Blog posts -** Self host within 5 minutes any program - - - Author: [[http://akilan.io/][Akilan Selvacoumar]] - - Date: 28-01-2025 - - - - Video tutorial: - [[https://youtu.be/rN4SiVowg5E][https://i3.ytimg.com/vi/rN4SiVowg5E/maxresdefault.jpg]] - -This is a fun experiment for anyone to try to quickly run a server and -quickly do a map port and domain name mapping in a single command. - -*** 1. Find a program you want to run - Let's try to setup a really easy program (Let's do with Linkwarden - with docker compose :) ). This is under the assumption you have docker - compose installed on your local machine. - -**** Let's run Linkwarden using docker compose and P2PRC - [[https://docs.linkwarden.app/self-hosting/installation][Installation instructions]]: - #+BEGIN_SRC -mkdir linkwarden && cd linkwarden -curl -O https://raw.githubusercontent.com/linkwarden/linkwarden/refs/heads/main/docker-compose.yml -curl -L https://raw.githubusercontent.com/linkwarden/linkwarden/refs/heads/main/.env.sample -o ".env" - #+END_SRC - - Environment configuration - #+BEGIN_SRC -vim .env - -# Change values -NEXTAUTH_URL=https:///api/v1/auth -NEXTAUTH_SECRET=VERY_SENSITIVE_SECRET -POSTGRES_PASSWORD=CUSTOM_POSTGRES_PASSWORD - #+END_SRC - Run linkwarden! - #+BEGIN_SRC -docker compose up - #+END_SRC - - If setup correctly linkwarden should - be running. - Local link: http://localhost:3000 - - Time to setup P2PRC - [[https://p2prc.akilan.io/Docs/#build-project-and-install-project][Installation Instructions]] - - Run p2prc as a background - #+BEGIN_SRC -p2prc -s & - #+END_SRC - - Run map port and domain mapping - #+BEGIN_SRC -p2prc --mp 3000 --dn - #+END_SRC - - Sample response - #+BEGIN_SRC - { - "IPAddress": "217.76.63.222", - "PortNo": "61582", - "EntireAddress": "217.76.63.222:61582" -} - #+END_SRC - - Add DNS entry - #+BEGIN_SRC -A entry 217.76.63.222 - #+END_SRC - - Your done now just head to the DOMAIN NAME you added. - ex: https://linkwarden.akilan.io - - - - - - - - - - -* Ideas for future potential features -Consists of personal ideas for the future of P2PRC. -At moment only has main contributors writing to this. - -** To support heterogenous set of Nodes that cannot run P2PRC -This stems from a personal issue I have when doing research -on [[https://github.com/CTSRD-CHERI/cheribsd][CheriBSD]] kernel. For my research I am using the ARM morello -which is a 128bit ARMv8 processor. At the moment Go programs -can cannot compile and run inside the CPU. This means I cannot -run P2PRC at the moment inside the ARM morello to remotely access -it when it's behind NAT using P2PRC. This would indeed be a common -problem when running against various Architectures that do not -support running P2PRC. As you will see soon this also creates -opportunity space to scale faster to nodes in a local network -and would introduce a new layer fault tolerance within a local -network nodes. - -*** Assumptions: - - I have a Morello board that cannot run P2PRC - - The Morello has a local IP address (ex: 192.168.0.10) - - I have 2 laptops running P2PRC in that local network. - - This means I have 2 ways to access the Morello board: Which is to SSH - into either 2 laptops and then SSH into 192.168.0.10 to gain access - to the board. Wouldn't it be great to automate this whole layer and - as well look into custom tasks into the heterogeneous hardware. -*** Set of interesting possible: - We build a cool set possibilities before and use this to build up the implementation - plan. - - We can use P2PRC access the morello board remotely in a single command. - - We can use the P2PRC protocol to run servers inside the morello board via traversed - node locally which can access that Node. - - Spin servers on node not running P2PRC using the P2PRC standard abstractions. - - Auto-setup P2PRC nodes with just SSH access via potentially a DSL. - - A neat use case for CHERI for instance would be use the architecture to run light - weight hypervisors. -*** Implementation - - To use implementations similar to [[https://linux.die.net/man/1/socat][socat]] to ensure we can bind address of local - nodes to a node running P2PRC and the node running P2PRC can do a local map port. - - We are working on hardening the implementation of the --mp (Map port) to even - map ports to machines which remotely running P2PRC. This means of instance I - can issue a command to the Morello board without the morello board being in - my local network. - - We would want to implement the existing P2PRC public key mechanism as well so that - other nodes can access the Morello board who have permission access. - -#+CAPTION: Implementation idea (To be improved upon) - [[./Docs/images/P2PRCRemoteNodes.png]]