From 5cf1e144aa0d72d73b0583c0d04887c9332fcaa3 Mon Sep 17 00:00:00 2001 From: Akilan Date: Fri, 25 Apr 2025 23:07:24 +0100 Subject: [PATCH] moved docs to root folder --- .github/workflows/pages.yml | 49 -- Docs/style.css | 25 - Docs/index.html => index.html | 441 +++++++++-------- Docs/index.org => index.org | 19 +- index.org~ | 909 ++++++++++++++++++++++++++++++++++ style.css | 51 +- 6 files changed, 1174 insertions(+), 320 deletions(-) delete mode 100644 .github/workflows/pages.yml delete mode 100644 Docs/style.css rename Docs/index.html => index.html (77%) rename Docs/index.org => index.org (98%) create mode 100644 index.org~ diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml deleted file mode 100644 index 341c731..0000000 --- a/.github/workflows/pages.yml +++ /dev/null @@ -1,49 +0,0 @@ -name: Deploy Jekyll with GitHub Pages dependencies preinstalled - -on: - # Runs on pushes targeting the default branch - push: - branches: ["main", "master"] - - # Allows you to run this workflow manually from the Actions tab - workflow_dispatch: - -# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages -permissions: - contents: read - pages: write - id-token: write - -# Allow one concurrent deployment -concurrency: - group: "pages" - cancel-in-progress: true - -jobs: - # Build job - build: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v3 - - name: Setup Pages - uses: actions/configure-pages@v2 - - name: Build with Jekyll - uses: actions/jekyll-build-pages@v1 - with: - source: ./ - destination: ./_site - - name: Upload artifact - uses: actions/upload-pages-artifact@v1 - - # Deployment job - deploy: - environment: - name: github-pages - url: ${{ steps.deployment.outputs.page_url }} - runs-on: ubuntu-latest - needs: build - steps: - - name: Deploy to GitHub Pages - id: deployment - uses: actions/deploy-pages@v1 diff --git a/Docs/style.css b/Docs/style.css deleted file mode 100644 index 8e9554d..0000000 --- a/Docs/style.css +++ /dev/null @@ -1,25 +0,0 @@ -#table-of-contents h2{ - z-index: 200; - background-color: #b96c29; - text-align: center; - padding: 0.809em; - display: block; - color: #fcfcfc; - font-size: 100%; - margin-bottom: 0.809em; -} - -#search-results li { - background-color: #b96c29; - color: white; - margin-bottom: 5px; - padding: 5px; - border-radius: 4px; - cursor: pointer; - font-size: 14px; -} - -h4,h5,h6{ - color: #b96c29; - font-weight:300; -} diff --git a/Docs/index.html b/index.html similarity index 77% rename from Docs/index.html rename to index.html index 56c1702..dce9d62 100644 --- a/Docs/index.html +++ b/index.html @@ -3,7 +3,7 @@ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> - + ‎ @@ -26,113 +26,122 @@

Table of Contents

-
-

Colored-On-Light-Image.png +

+

Colored-On-Light-Image.png

-
-

1. Introduction

-
+
+

1. Guide through video

+
-
-

1.1. Abstract

-
+
+

1.0.1. The video below shows the setup and usage of P2PRC.

+
+ +
+

maxresdefault.jpg +

+
+
+
+ +
+
+

2. Introduction

+

This project focuses on creating a framework for running heavy computational tasks that a regular computer cannot handle easily. These tasks may include graphically demanding video games, rendering @@ -168,9 +191,8 @@ Leveraging the P2PRC approach, this project extends its capabilities to handle a wider range of computationally intensive tasks.

-
-

1.2. Motivation

+

2.1. Motivation

Many of the users rely on our PC / Laptop or servers that belong to a @@ -189,7 +211,7 @@ server.

-

2. Installation

+

3. Installation

Over here we will cover the basic steps to get the server and client @@ -197,7 +219,7 @@ side running.

-

2.1. Latest release install

+

3.1. Latest release install

https://github.com/Akilan1999/p2p-rendering-computation/releases @@ -205,11 +227,11 @@ side running.

-

2.2. Install from Github master branch

+

3.2. Install from Github master branch

-

2.2.1. Install Go lang

+

3.2.1. Install Go lang

The entire the implementation of this project is done using Go lang. @@ -219,7 +241,7 @@ Thus, we need go lang to compile to code to a binary file.

-

2.2.2. Install Docker

+

3.2.2. Install Docker

In this project the choice of virtualization is Docker due to it's wide @@ -236,21 +258,21 @@ Go API to create and interact with the containers. to install docker GPU

-
+
 // Do ensure that the docker command does not need sudo to run
 sudo chmod 666 /var/run/docker.sock
-

2.2.3. Build Project and install project

+

3.2.3. Build Project and install project

To set up the internal dependencies and build the entire go code into a single binary

-
+
 make
@@ -262,7 +284,7 @@ To set up P2PRC on Windows, simply run this batch file. Make sure you are not in admin mode when running this.

-
+
 .\install.bat
@@ -270,25 +292,25 @@ are not in admin mode when running this.
-

2.2.4. Add appropriate paths to .bashrc

+

3.2.4. Add appropriate paths to .bashrc

-
+
 export P2PRC=/<PATH>/p2p-rendering-computation
 export PATH=/<PATH>/p2p-rendering-computation:${PATH}
-

2.2.5. Test if binary works

+

3.2.5. Test if binary works

-
+
 p2prc --help
  1. Output:
    -
    +
     NAME:
    p2p-rendering-computation - p2p cli application to create and access VMs in other servers
 
@@ -344,186 +366,186 @@ GLOBAL OPTIONS:
-

3. Using basic commands

+

4. Using basic commands

-

3.0.1. Start as a server

+

4.0.1. Start as a server

-
+
 p2prc -s
-

3.0.2. View server Specification

+

4.0.2. View server Specification

-
+
 p2prc --specs=<ip address>
-

3.0.3. Run container

+

4.0.3. Run container

use the --gpu if you know the other machine has a gpu.

-
+
 p2prc --touch=<server ip address> -p <number of ports> --gpu
-

3.0.4. Remove container

+

4.0.4. Remove container

The docker id is present in the output where you create a container

-
+
 p2prc --rm=<server ip address> --id=<docker container id>
-

3.0.5. Adding servers to ip table

+

4.0.5. Adding servers to ip table

-
+
 p2prc --as=<server ip address you want to add>
-

3.0.6. Update ip table

+

4.0.6. Update ip table

-
+
 p2prc --us
-

3.0.7. List Servers

+

4.0.7. List Servers

-
+
 p2prc --ls
-

3.0.8. View Network interfaces

+

4.0.8. View Network interfaces

-
+
 p2prc --ni
-

3.0.9. Viewing Containers created Client side

+

4.0.9. Viewing Containers created Client side

-
+
 p2prc --tc
-

3.0.10. Running plugin

+

4.0.10. Running plugin

-
+
 p2prc --plugin <plugin name> --id <container id or group id>
-

3.0.11. Create group

+

4.0.11. Create group

-
+
 p2prc --cgroup
-

3.0.12. Add container to group

+

4.0.12. Add container to group

-
+
 p2prc --group <group id> --id <container id>
-

3.0.13. View groups

+

4.0.13. View groups

-
+
 p2prc --groups
-

3.0.14. View specific group

+

4.0.14. View specific group

-
+
 p2prc --group <group id>
-

3.0.15. Delete container from group

+

4.0.15. Delete container from group

-
+
 p2prc --rmcgroup --group <group id> --id <container id>
-

3.0.16. Delete entire group

+

4.0.16. Delete entire group

-
+
 p2prc --rmgroup <group id>
-

3.0.17. Pulling plugin from a remote repo

+

4.0.17. Pulling plugin from a remote repo

-
+
 p2prc --pp <repo link>
-

3.0.18. Deleting plugin from the plugin directory

+

4.0.18. Deleting plugin from the plugin directory

-
+
 p2prc --rp <plugin name>
-

3.0.19. Added custom metadata about the current node

+

4.0.19. Added custom metadata about the current node

-
+
 p2prc --amd "custom metadata"
-
-

3.0.20. MapPort and link to domain name

-
+
+

4.0.20. MapPort and link to domain name

+

Allows to expose remote ports from a machine in the P2P network.

-
+
 p2prc --mp <port no to map> --dn <domain name to link Mapped port against>
    -
  1. MapPort in remote machine
    -
    +
  2. MapPort in remote machine
    +

    This is to ensure ports on remote machines on the P2PRC can be easily opened.

    -
    +
     p2prc --mp <port no to map> --dn <domain name to link Mapped port against> --ra <remote server address>
 
    
 
@@ -534,7 +556,7 @@ p2prc --mp <port no to map> --dn <domain name to link Mapped port again
-

4. P2P Module Implementation

+

5. P2P Module Implementation

The P2P module is for managing server information within the network. @@ -544,12 +566,9 @@ to determine upload and download speeds. This valuable information enables users servers with optimal performance, enhancing their overall network experience.

- -
-

p2pmoduleArch.png +

+[[p2pmoduleArch.png]

-

Figure 1: UML diagram of P2P module

-

The peer to peer implementation was built from scratch. This is because @@ -564,7 +583,7 @@ implementations of each functionality in depth.

-

4.1. IP Table

+

5.1. IP Table

The ip table file is a json as the format with a list of servers ip @@ -603,7 +622,7 @@ configuration module.

-

4.1.1. Latency

+

5.1.1. Latency

The latency is measured in milliseconds. The route /serverinfo is @@ -614,7 +633,7 @@ recorded.

-

4.2. NAT Traversal

+

5.2. NAT Traversal

P2PRC currently supports TURN for NAT traversal. @@ -622,7 +641,7 @@ P2PRC currently supports TURN for NAT traversal.

-

4.3. TURN

+

5.3. TURN

The current TURN implementation used is FRP. The TURN server is also @@ -635,13 +654,13 @@ from the TURN server. The flow below describes the workflow.

-

4.3.1. Client mode

+

5.3.1. Client mode

  • Call /FRPPort
-
+
 http://<turn server ip>:<server port no>/FRPport
 

 
@@ -675,7 +694,7 @@ func main() {
-

5. Language Bindings

+

6. Language Bindings

Language bindings @@ -686,7 +705,7 @@ compiler.

-

5.1. How to build shared object files

+

6.1. How to build shared object files

    @@ -724,13 +743,13 @@ p2prc.h p2prc.so
-

5.2. Workings under the hood

+

6.2. Workings under the hood

Below are a sample set of commands to open the bindings implementation.

-
+
 # run
 cd Bindings/
 # list files
@@ -740,7 +759,7 @@ Client.go
-

5.2.1. In Client go

+

6.2.1. In Client go

There a few things to notice which are different from your standard Go @@ -818,7 +837,7 @@ that convert a go object to JSON string initially and converts it to

  • A Pseudo code to refer to the common function implementation shape could be represented as:
    -
    +
     func <Function name> (output *C.char) {
       <response>,<error> := <P2PRC function name>(<parameters if needed>)
       if <error> != nil {
@@ -833,12 +852,12 @@ func <Function name> (output *C.char) {
    • -

    5.3. Current languages supported

    +

    6.3. Current languages supported

    -
    -

    5.3.1. Python

    -
    +
    +

    6.3.1. Python

    +
    1. Build sample python program
      @@ -874,9 +893,9 @@ called from other programming languages.
    -
    -

    5.3.2. Haskell

    -
    +
    +

    6.3.2. Haskell

    +

    P2PRC officially supports Haskell bindings and will further support project using Haskell to build orchestrators on top of P2PRC. @@ -890,7 +909,7 @@ project using Haskell to build orchestrators on top of P2PRC.

    -

    6. Config Implementation

    +

    7. Config Implementation

    The configuration module is responsible to store basic information of @@ -929,14 +948,14 @@ configuration file was generated using JSON format.

    -

    7. Abstractions

    +

    8. Abstractions

    The Abstractions package consists of black-boxed functions for P2PRC.

    -

    7.1. Functions

    +

    8.1. Functions

    • Init(<Project name>): Initializes P2PRC with all the needed @@ -959,14 +978,14 @@ faster.
    -

    8. NAT Traversal

    +

    9. NAT Traversal

    P2PRC currently supports TURN for NAT traversal.

    -

    8.1. TURN

    +

    9.1. TURN

    The current TURN implementation used is FRP. The TURN server is also @@ -981,13 +1000,13 @@ from the TURN server. The flow below describes the workflow.

    -

    9. Client mode

    +

    10. Client mode

    • Call /FRPPort
    -
    +
     http://<turn server ip>:<server port no>/FRPport
 
    
 
@@ -1018,13 +1037,13 @@ func main() {
    -
    -

    10. Blog posts

    -
    +
    +

    11. Blog posts

    +
    -
    -

    10.1. Self host within 5 minutes any program

    -
    +
    +

    11.1. Self host within 5 minutes any program

    +
    • Author: Akilan Selvacoumar
    • Date: 28-01-2025
      • @@ -1035,8 +1054,8 @@ func main() {
    • Video tutorial:
    -
    -

    maxresdefault.jpg +

    +

    maxresdefault.jpg

    @@ -1045,9 +1064,9 @@ This is a fun expirement for anyone to try to quickly run a server and quickly do a map port and domain name mapping in a single command.

    -
    -

    10.1.1. 1. Find a program you want to run

    -
    +
    +

    11.1.1. 1. Find a program you want to run

    +

    Let's try to setup a really easy program (Let's do with Linkwarden with docker compose :) ). This is under the assumption you have docker @@ -1055,8 +1074,8 @@ compose installed on your local machine.

      -
    1. Let's run Linkwarden using docker compose and P2PRC
      -
      +
    2. Let's run Linkwarden using docker compose and P2PRC
      +

      Installation instructions:

      @@ -1144,17 +1163,17 @@ ex: https://linkwarden.akilan.io
    -
    -

    11. Ideas for future potencial features

    -
    +
    +

    12. Ideas for future potencial features

    +

    Consists of personal loideas for the future of P2PRC. At moment only has main contributors writiing to this.

    -
    -

    11.1. To support hetrogenous set of Nodes that cannot run P2PRC

    -
    +
    +

    12.1. To support hetrogenous set of Nodes that cannot run P2PRC

    +

    This stems from a personal issue I have when doing research on CheriBSD kernel. For my research I am using the ARM morello @@ -1169,9 +1188,9 @@ and would introduce a new layer fault tolerance within a local network nodes.

    -
    -

    11.1.1. Assumptions:

    -
    +
    +

    12.1.1. Assumptions:

    +
    • I have a Morello board that cannot run P2PRC
    • The Morello has a local IP address (ex: 192.168.0.10)
      • @@ -1183,9 +1202,9 @@ as well look into custom tasks into the hetrogenous hardware.
    -
    -

    11.1.2. Set of interesting possible:

    -
    +
    +

    12.1.2. Set of interesting possible:

    +

    We build a cool set possibilities before and use this to build up the implementation plan. @@ -1201,9 +1220,9 @@ weight hypervisors.

    -
    -

    11.1.3. Implementation

    -
    +
    +

    12.1.3. Implementation

    +
    • To use implementations similar to socat to ensure we can bind address of local nodes to a node running P2PRC and the node running P2PRC can do a local map port.
      • @@ -1216,10 +1235,10 @@ other nodes can access the Morello board who have permission access.
    -
    -

    P2PRCRemoteNodes.png +

    +

    P2PRCRemoteNodes.png

    -

    Figure 2: Implementation idea (To be improved upon)

    +

    Figure 1: Implementation idea (To be improved upon)

    @@ -1228,7 +1247,7 @@ other nodes can access the Morello board who have permission access.

    Author: Akilan

    -

    Created: 2025-04-25 Fri 22:23

    +

    Created: 2025-04-25 Fri 23:06

    Validate

    diff --git a/Docs/index.org b/index.org similarity index 98% rename from Docs/index.org rename to index.org index 9ec1f46..eb6f536 100644 --- a/Docs/index.org +++ b/index.org @@ -4,17 +4,19 @@ #+HTML_HEAD: #+attr_html: :width 300px -[[../artwork/p2prc-logos/Colored-On-Light-Image.png]] +[[./artwork/p2prc-logos/Colored-On-Light-Image.png]] + +* Guide through video +*** The video below shows the setup and usage of P2PRC. +#+attr_html: :class video +[[https://www.youtube.com/watch?v=OMwCpedu5cs][https://i3.ytimg.com/vi/OMwCpedu5cs/maxresdefault.jpg]] + +*** Source code: https://github.com/Akilan1999/p2p-rendering-computation * Introduction :PROPERTIES: :CUSTOM_ID: chapter1-introduction :END: - -** Abstract -:PROPERTIES: -:CUSTOM_ID: abstract -:END: This project focuses on creating a framework for running heavy computational tasks that a regular computer cannot handle easily. These tasks may include graphically demanding video games, rendering 3D animations, and performing complex protein folding simulations. The major focus of this project @@ -365,7 +367,7 @@ to determine upload and download speeds. This valuable information enables users servers with optimal performance, enhancing their overall network experience. #+caption: UML diagram of P2P module -[[file:images/p2pmoduleArch.png]] +[[file:Docs/images/p2pmoduleArch.png] The peer to peer implementation was built from scratch. This is because other peer to peer libraries were on the implementation of the @@ -785,6 +787,7 @@ func main() { - Video tutorial: + #+attr_html: :class video [[https://youtu.be/rN4SiVowg5E][https://i3.ytimg.com/vi/rN4SiVowg5E/maxresdefault.jpg]] This is a fun expirement for anyone to try to quickly run a server and @@ -906,4 +909,4 @@ network nodes. other nodes can access the Morello board who have permission access. #+CAPTION: Implementation idea (To be improved upon) - [[./images/P2PRCRemoteNodes.png]] + [[./Docs/images/P2PRCRemoteNodes.png]] diff --git a/index.org~ b/index.org~ new file mode 100644 index 0000000..687a97e --- /dev/null +++ b/index.org~ @@ -0,0 +1,909 @@ +#+SETUPFILE: https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup + +#+HTML_HEAD: +#+HTML_HEAD: + +#+attr_html: :width 300px :align center +[[./artwork/p2prc-logos/Colored-On-Light-Image.png]] + +* Guide through video +*** The video below shows the setup and usage of P2PRC. +#+attr_html: :class video +[[https://www.youtube.com/watch?v=OMwCpedu5cs][https://i3.ytimg.com/vi/OMwCpedu5cs/maxresdefault.jpg]] + +* Introduction +:PROPERTIES: +:CUSTOM_ID: chapter1-introduction +:END: +This project focuses on creating a framework for running heavy computational tasks that a regular +computer cannot handle easily. These tasks may include graphically demanding video games, rendering +3D animations, and performing complex protein folding simulations. The major focus of this project +is not on financial incentives but rather on building a robust and efficient peer-to-peer (P2P) +network to decentralise task execution and increase the computational bandwidth available for +such tasks. + +The P2PRC framework serves as a foundation for decentralised rendering and computation, +providing insights into how tasks can be distributed efficiently across a network of peers. +Leveraging the P2PRC approach, this project extends its capabilities to handle a +wider range of computationally intensive tasks. + +** Motivation +:PROPERTIES: +:CUSTOM_ID: motivation +:END: +Many of the users rely on our PC / Laptop or servers that belong to a +server farm to run heavy tasks and with the demand of high creativity +requires higher computing power. Buying a powerful computer every few +years to run a bunch of heavy tasks which are not executed as frequently +to reap the benefits can be inefficient utilization of hardware. On the +other end, renting servers to run these heavy tasks can be really +useful. Ethically speaking this is leading to monopolisation of +computing power similar to what is happening in the web server area. By +using peer to peer principles it is possible to remove the +monopolisation factor and increase the bandwidth between the client and +server. + +* Installation +:PROPERTIES: +:CUSTOM_ID: installation +:END: + +Over here we will cover the basic steps to get the server and client +side running. + +** Latest release install +:PROPERTIES: +:CUSTOM_ID: latest-release-install +:END: +https://github.com/Akilan1999/p2p-rendering-computation/releases + +** Install from Github master branch +:PROPERTIES: +:CUSTOM_ID: install-from-github-master-branch +:END: +*** Install Go lang +:PROPERTIES: +:CUSTOM_ID: install-go-lang +:END: +The entire the implementation of this project is done using Go lang. +Thus, we need go lang to compile to code to a binary file. +[[https://golang.org/doc/install][Instructions to install Go lang]] + +*** Install Docker +:PROPERTIES: +:CUSTOM_ID: install-docker +:END: +In this project the choice of virtualization is Docker due to it's wide +usage in the developer community. In the server module we use the Docker +Go API to create and interact with the containers. + +[[https://docs.docker.com/get-docker/][Instructions to install docker]] + +[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions +to install docker GPU]] + +#+begin_example +// Do ensure that the docker command does not need sudo to run +sudo chmod 666 /var/run/docker.sock +#+end_example + +*** Build Project and install project +:PROPERTIES: +:CUSTOM_ID: build-project-and-install-project +:END: +To set up the internal dependencies and build the entire go code into a +single binary + +#+begin_example +make +#+end_example + +**** For Windows +:PROPERTIES: +:CUSTOM_ID: for-windows +:END: +To set up P2PRC on Windows, simply run this batch file. *Make sure you +are not in admin mode when running this.* + +#+begin_example +.\install.bat +#+end_example + +*** Add appropriate paths to =.bashrc= +:PROPERTIES: +:CUSTOM_ID: add-appropriate-paths-to-.bashrc +:END: +#+begin_example +export P2PRC=//p2p-rendering-computation +export PATH=//p2p-rendering-computation:${PATH} +#+end_example + +*** Test if binary works +:PROPERTIES: +:CUSTOM_ID: test-if-binary-works +:END: +#+begin_example +p2prc --help +#+end_example + +**** Output: +:PROPERTIES: +:CUSTOM_ID: output +:END: +#+begin_example +NAME: + p2p-rendering-computation - p2p cli application to create and access VMs in other servers + +USAGE: + p2prc [global options] command [command options] [arguments...] + +VERSION: + 2.0.0 + +COMMANDS: + help, h Shows a list of commands or help for one command + +GLOBAL OPTIONS: + --Server, -s Starts server (default: false) [$SERVER] + --UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST] + --ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS] + --AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER] + --ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES] + --CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM] + --ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME] + --BaseImage value, --bi value Specifying the docker base image to template the dockerfile [$CONTAINER_NAME] + --RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM] + --ID value, --id value Docker Container ID [$ID] + --Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS] + --GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU] + --Specification value, --specs value Specs of the server node [$SPECS] + --SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG] + --NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE] + --ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN] + --TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS] + --ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN] + --CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP] + --Group value, --group value group flag with argument group ID [$GROUP] + --Groups, --groups View all groups (default: false) [$GROUPS] + --RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP] + --RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP] + --MAPPort value, --mp value Maps port for a specific port provided as the parameter [$MAPPORT] + --DomainName value, --dn value While mapping ports allows to set a domain name to create a mapping in the proxy server [$DOMAINNAME] + --Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE] + --ModuleName value, --mod value New go project module name [$MODULENAME] + --PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN] + --RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN] + --AddMetaData value, --amd value Adds metadata about the current node in the p2p network which is then propagated through the network [$ADDMETADATA] + --help, -h show help (default: false) + --version, -v print the version (default: false) +#+end_example + +-------------- + +* Using basic commands +:PROPERTIES: +:CUSTOM_ID: using-basic-commands +:END: +*** Start as a server +:PROPERTIES: +:CUSTOM_ID: start-as-a-server +:END: + +#+begin_example +p2prc -s +#+end_example + +*** View server Specification +:PROPERTIES: +:CUSTOM_ID: view-server-specification +:END: +#+begin_example +p2prc --specs= +#+end_example + +*** Run container +:PROPERTIES: +:CUSTOM_ID: run-container +:END: +use the =--gpu= if you know the other machine has a gpu. + +#+begin_example +p2prc --touch= -p --gpu +#+end_example + +*** Remove container +:PROPERTIES: +:CUSTOM_ID: remove-container +:END: +The docker id is present in the output where you create a container + +#+begin_example +p2prc --rm= --id= +#+end_example + +*** Adding servers to ip table +:PROPERTIES: +:CUSTOM_ID: adding-servers-to-ip-table +:END: +#+begin_example +p2prc --as= +#+end_example + +*** Update ip table +:PROPERTIES: +:CUSTOM_ID: update-ip-table +:END: +#+begin_example +p2prc --us +#+end_example + +*** List Servers +:PROPERTIES: +:CUSTOM_ID: list-servers +:END: +#+begin_example +p2prc --ls +#+end_example + +*** View Network interfaces +:PROPERTIES: +:CUSTOM_ID: view-network-interfaces +:END: +#+begin_example +p2prc --ni +#+end_example + +*** Viewing Containers created Client side +:PROPERTIES: +:CUSTOM_ID: viewing-containers-created-client-side +:END: +#+begin_example +p2prc --tc +#+end_example + +*** Running plugin +:PROPERTIES: +:CUSTOM_ID: running-plugin +:END: +#+begin_example +p2prc --plugin --id +#+end_example + +*** Create group +:PROPERTIES: +:CUSTOM_ID: create-group +:END: +#+begin_example +p2prc --cgroup +#+end_example + +*** Add container to group +:PROPERTIES: +:CUSTOM_ID: add-container-to-group +:END: +#+begin_example +p2prc --group --id +#+end_example + +*** View groups +:PROPERTIES: +:CUSTOM_ID: view-groups +:END: +#+begin_example +p2prc --groups +#+end_example + +*** View specific group +:PROPERTIES: +:CUSTOM_ID: view-specific-group +:END: +#+begin_example +p2prc --group +#+end_example + +*** Delete container from group +:PROPERTIES: +:CUSTOM_ID: delete-container-from-group +:END: +#+begin_example +p2prc --rmcgroup --group --id +#+end_example + +*** Delete entire group +:PROPERTIES: +:CUSTOM_ID: delete-entire-group +:END: +#+begin_example +p2prc --rmgroup +#+end_example + +*** Pulling plugin from a remote repo +:PROPERTIES: +:CUSTOM_ID: pulling-plugin-from-a-remote-repo +:END: +#+begin_example +p2prc --pp +#+end_example + +*** Deleting plugin from the plugin directory +:PROPERTIES: +:CUSTOM_ID: deleting-plugin-from-the-plugin-directory +:END: +#+begin_example +p2prc --rp +#+end_example + +*** Added custom metadata about the current node +:PROPERTIES: +:CUSTOM_ID: added-custom-metadata-about-the-current-node +:END: +#+begin_example +p2prc --amd "custom metadata" +#+end_example + +*** MapPort and link to domain name +Allows to expose remote ports from a machine in the P2P network. +#+begin_example +p2prc --mp --dn +#+end_example +**** MapPort in remote machine +This is to ensure ports on remote machines on the P2PRC can be easily opened. +#+begin_example +p2prc --mp --dn --ra +#+end_example + +-------------- + +* P2P Module Implementation +:PROPERTIES: +:CUSTOM_ID: p2p-module-implementation +:END: +The P2P module is for managing server information within the network. +It maintains and updates the IP table, ensuring accuracy by preventing duplicates and removing +entries for unreachable servers. Furthermore, the module conducts speed tests on the listed servers +to determine upload and download speeds. This valuable information enables users to identify nearby +servers with optimal performance, enhancing their overall network experience. + +#+caption: UML diagram of P2P module +[[file:Docs/images/p2pmoduleArch.png] + +The peer to peer implementation was built from scratch. This is because +other peer to peer libraries were on the implementation of the +Distributed hash table. At the current moment all those heavy features +are not needed because the objective is to search and list all possible +servers available. The limitation being that to be a part of the network +the user has to know at least 1 server. The advantage of building from +scratch makes the module super light and possibility for custom +functions and structs. The sub topics below will mention the +implementations of each functionality in depth. + +** IP Table +:PROPERTIES: +:CUSTOM_ID: ip-table +:END: +The ip table file is a json as the format with a list of servers ip +addresses, latencies, downloads and uploads speeds. The functions +implemented include read file, write file and remove duplicate IP +addresses. The remove duplicate IP address function exists because +sometimes servers IP tables can have the same ip addresses as what the +client has. The path of the IP table json file is received from the +configuration module. + +#+begin_src json +{ + "ip_address": [ + { + "Name": "", + "MachineUsername": "", + "IPV4": "", + "IPV6": "", + "Latency": , + "Download": , + "Upload": , + "ServerPort": "", + "BareMetalSSHPort": "", + "NAT": "", + "EscapeImplementation": "", + "ProxyServer": "", + "UnSafeMode": ", + "PublicKey": "", + "CustomInformation": "" + } + ] +} +#+end_src + +*** Latency +:PROPERTIES: +:CUSTOM_ID: latency +:END: +The latency is measured in milliseconds. The route /server_info is +called from the server and time it takes to provide a json response is +recorded. + +** NAT Traversal +:PROPERTIES: +:CUSTOM_ID: nat-traversal +:END: +P2PRC currently supports TURN for NAT traversal. + +** TURN +:PROPERTIES: +:CUSTOM_ID: turn +:END: +The current TURN implementation used is FRP. The TURN server is also +required when a P2PRC node is acting as a Server. The TURN server is +determined based on the Node with the least amount of latency based on +the Nodes available on the IPTable. Once a TURN server is determined +there are 2 actions performed. The first one is =/FRPPort= to the TURN +server to receive a port which is used to generate the external port +from the TURN server. The flow below describes the workflow. + +*** Client mode +:PROPERTIES: +:CUSTOM_ID: client-mode +:END: +- Call =/FRPPort= + +#+begin_example +http://:/FRPport +#+end_example + +- Call the TURN server in the following manner. The following is a + sample code snippet below. + +#+begin_src go +import ( + "github.com/Akilan1999/p2p-rendering-computation/p2p/frp" +) + +func main() { + serverPort, err := frp.GetFRPServerPort("http://" + + ":" + lowestLatencyIpAddress.ServerPort) + if err != nil { + return nil, err + } + // Create 1 second delay to allow FRP server to start + time.Sleep(1 * time.Second) + // Starts FRP as a client with + proxyPort, err := frp.StartFRPClientForServer(, serverPort, ) + if err != nil { + return nil, err + } +} +#+end_src + +* Language Bindings +:PROPERTIES: +:CUSTOM_ID: language-bindings +:END: +[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]] +refers to wrappers to bridge 2 programming languages. This is used in +P2PRC to extend calling P2PRC functions in other programming languages. +Currently this is done by generating =.so= and =.h= from the Go +compiler. + +** How to build shared object files +:PROPERTIES: +:CUSTOM_ID: how-to-build-shared-object-files +:END: +**** The easier way +:PROPERTIES: +:CUSTOM_ID: the-easier-way +:END: +#+begin_src sh +# Run +make sharedObjects +#+end_src + +**** Or the direct way +:PROPERTIES: +:CUSTOM_ID: or-the-direct-way +:END: +#+begin_src sh +# Run +cd Bindings && go build -buildmode=c-shared -o p2prc.so +#+end_src + +**** If successfully built: +:PROPERTIES: +:CUSTOM_ID: if-successfully-built +:END: +#+begin_src sh +# Enter into the Bindings directory +cd Bindings +# List files +ls +# Find files +p2prc.h p2prc.so +#+end_src + +** Workings under the hood +:PROPERTIES: +:CUSTOM_ID: workings-under-the-hood +:END: +Below are a sample set of commands to open the bindings implementation. + +#+begin_example +# run +cd Bindings/ +# list files +ls +# search for file +Client.go +#+end_example + +*** In Client go +:PROPERTIES: +:CUSTOM_ID: in-client-go +:END: +There a few things to notice which are different from your standard Go +programs: + +**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required. +:PROPERTIES: +:CUSTOM_ID: we-import-c-which-means-cgo-is-required. +:END: +#+begin_src go +import "C" +#+end_src + +**** 2. All functions which are required to be called from other programming languages have comment such as. +:PROPERTIES: +:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as. +:END: +#+begin_src go +//export + +// ------------ Example ---------------- +// The function below allows to externally +// to call the P2PRC function to start containers +// in a specific node in the know list of nodes +// in the p2p network. +// Note: the comment "//export StartContainer". + +//export StartContainer +func StartContainer(IP string) (output *C.char) { + container, err := client.StartContainer(IP, 0, false, "", "") + if err != nil { + return C.CString(err.Error()) + } + return ConvertStructToJSONString(container) + } +#+end_src + +**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure). +:PROPERTIES: +:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure. +:END: +#+begin_src go +// --------- Example ------------ + +//export StartContainer +func StartContainer(IP string) (output *C.char) { + container, err := client.StartContainer(IP, 0, false, "", "") + if err != nil { + return C.CString(err.Error()) + } + return ConvertStructToJSONString(container) +} + +//export ViewPlugin +func ViewPlugin() (output *C.char) { + plugins, err := plugin.DetectPlugins() + if err != nil { + return C.CString(err.Error()) + } + return ConvertStructToJSONString(plugins) +} +#+end_src + +**** It is easy to notice that: +:PROPERTIES: +:CUSTOM_ID: it-is-easy-to-notice-that +:END: +- =ConvertStructToJSONString()=: This is a helper function + that convert a go object to JSON string initially and converts it to + =CString=. +- =(output *C.char)=: This is the return type for most of the functions. + +**** A Pseudo code to refer to the common function implementation shape could be represented as: +:PROPERTIES: +:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as +:END: +#+begin_example +func (output *C.char) { + , := () + if != nil { + return C.CString(.Error()) + } + return ConvertStructToJSONString() +} +#+end_example + +** Current languages supported +:PROPERTIES: +:CUSTOM_ID: current-languages-supported +:END: +*** Python + +**** Build sample python program +:PROPERTIES: +:CUSTOM_ID: build-sample-python-program +:END: +The easier way + +#+begin_src sh +# Run +make python +# Expected ouput +Output is in the Directory Bindings/python/export/ +# Run +cd Bindings/python/export/ +# list files +ls +# Expected output +SharedObjects/ library.py requirements.txt +#+end_src + +Above shows a generated folder which consists of a folder called +"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files. +=p2prc.py= refers to a sample python script calling P2PRC go functions. +To start an any project to extend P2PRC with python, This generated +folder can copied and created as a new git repo for P2PRC extensions +scripted or used a reference point as proof of concept that P2PRC can be +called from other programming languages. + +*** Haskell +P2PRC officially supports Haskell bindings and will further support +project using Haskell to build orchestrators on top of P2PRC. + +[[https://p2prc.akilan.io/Docs/haskell][Read more...]] + +* Config Implementation +:PROPERTIES: +:CUSTOM_ID: config-implementation +:END: +The configuration module is responsible to store basic information of +absolute paths of files being called in the Go code. In a full-fledged +Cli the configuration file can be found in the directory /etc/ and from +there points to location such as where the IP table file is located. In +the future implementation the config file will have information such as +number of hops and other parameters to tweak and to improve the +effectiveness of the peer to peer network. The configuration module was +implemented using the library Viper. The Viper library automates +features such as searching in default paths to find out if the +configuration file is present. If the configuration file is not present +in the default paths then it auto generates the configuration file. The +configurations file can be in any format. In this project the +configuration file was generated using JSON format. + +#+begin_src json +{ + "MachineName": "pc-74-120.customer.ask4.lan", + "IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json", + "DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/", + "DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/", + "SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin", + "IPV6Address": "", + "PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy", + "TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json", + "ServerPort": "8088", + "GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json", + "FRPServerPort": "True", + "BehindNAT": "True", + "CustomConfig": null +} +#+end_src + +* Abstractions +:PROPERTIES: +:CUSTOM_ID: abstractions +:END: + +The Abstractions package consists of black-boxed functions for P2PRC. + +** Functions +:PROPERTIES: +:CUSTOM_ID: functions +:END: +- =Init()=: Initializes P2PRC with all the needed + configurations. +- =Start()=: Starts p2prc as a server and makes it possible to extend by + adding other routes and functionality to P2PRC. +- =MapPort()=: On the local machine the port you want to export + to world. +- =StartContainer()=: The machine on the p2p network where + you want to spin up a docker container. +- =RemoveContainer(,)=: Terminate container + based on the IP address and container name. +- =GetSpecs()=: Get specs of a machine on the network based + on the IP address. +- =ViewIPTable()=: View the IP table which about nodes in the network. +- =UpdateIPTable()=: Force update IP table to learn about new nodes + faster. + +* NAT Traversal +:PROPERTIES: +:CUSTOM_ID: nat-traversal +:END: +P2PRC currently supports TURN for NAT traversal. + +** TURN +:PROPERTIES: +:CUSTOM_ID: turn +:END: +The current TURN implementation used is FRP. The TURN server is also +required when a P2PRC node is acting as a Server. The TURN server is +determined based on the Node with the least amount of latency based on +the Nodes available on the IPTable. Once a TURN server is determined +there are 2 actions performed. The first one is =/FRPPort= to the TURN +server to receive a port which is used to generate the external port +from the TURN server. The flow below describes the workflow. + +* Client mode +:PROPERTIES: +:CUSTOM_ID: client-mode +:END: +- Call =/FRPPort= + +#+begin_example +http://:/FRPport +#+end_example + +- Call the TURN server in the following manner. The following is a + sample code snippet below. + +#+begin_src go +import ( + "github.com/Akilan1999/p2p-rendering-computation/p2p/frp" +) + +func main() { + serverPort, err := frp.GetFRPServerPort("http://" + + ":" + lowestLatencyIpAddress.ServerPort) + if err != nil { + return nil, err + } + // Create 1 second delay to allow FRP server to start + time.Sleep(1 * time.Second) + // Starts FRP as a client with + proxyPort, err := frp.StartFRPClientForServer(, serverPort, ) + if err != nil { + return nil, err + } +} +#+end_src + + + + +* Blog posts +** Self host within 5 minutes any program + + - Author: [[http://akilan.io/][Akilan Selvacoumar]] + - Date: 28-01-2025 + + + - Video tutorial: + [[https://youtu.be/rN4SiVowg5E][https://i3.ytimg.com/vi/rN4SiVowg5E/maxresdefault.jpg]] + +This is a fun expirement for anyone to try to quickly run a server and +quickly do a map port and domain name mapping in a single command. + +*** 1. Find a program you want to run + Let's try to setup a really easy program (Let's do with Linkwarden + with docker compose :) ). This is under the assumption you have docker + compose installed on your local machine. + +**** Let's run Linkwarden using docker compose and P2PRC + [[https://docs.linkwarden.app/self-hosting/installation][Installation instructions]]: + #+BEGIN_SRC +mkdir linkwarden && cd linkwarden +curl -O https://raw.githubusercontent.com/linkwarden/linkwarden/refs/heads/main/docker-compose.yml +curl -L https://raw.githubusercontent.com/linkwarden/linkwarden/refs/heads/main/.env.sample -o ".env" + #+END_SRC + + Environment configuration + #+BEGIN_SRC +vim .env + +# Change values +NEXTAUTH_URL=https:///api/v1/auth +NEXTAUTH_SECRET=VERY_SENSITIVE_SECRET +POSTGRES_PASSWORD=CUSTOM_POSTGRES_PASSWORD + #+END_SRC + Run linkwarden! + #+BEGIN_SRC +docker compose up + #+END_SRC + + If setup correctly linkwarden should + be running. + Local link: http://localhost:3000 + + Time to setup P2PRC + [[https://p2prc.akilan.io/Docs/#build-project-and-install-project][Installation Instructions]] + + Run p2prc as a background + #+BEGIN_SRC +p2prc -s & + #+END_SRC + + Run map port and domain mapping + #+BEGIN_SRC +p2prc --mp 3000 --dn + #+END_SRC + + Sample response + #+BEGIN_SRC + { + "IPAddress": "217.76.63.222", + "PortNo": "61582", + "EntireAddress": "217.76.63.222:61582" +} + #+END_SRC + + Add DNS entry + #+BEGIN_SRC +A entry 217.76.63.222 + #+END_SRC + + Your done now just head to the DOMAIN NAME you added. + ex: https://linkwarden.akilan.io + + + + + + + + + + +* Ideas for future potencial features +Consists of personal loideas for the future of P2PRC. +At moment only has main contributors writiing to this. + +** To support hetrogenous set of Nodes that cannot run P2PRC +This stems from a personal issue I have when doing research +on [[https://github.com/CTSRD-CHERI/cheribsd][CheriBSD]] kernel. For my research I am using the ARM morello +which is a 128bit ARMv8 processor. At the moment Go programs +can cannot compile and run inside the CPU. This means I cannot +run P2PRC at the moment inside the ARM morello to remotely access +it when it's behind NAT using P2PRC. This would indeed be a common +problem when running against various Architectures that do not +support running P2PRC. As you will see soon this also creates +oppurtunity space to scale faster to nodes in a local network +and would introduce a new layer fault tolerance within a local +network nodes. + +*** Assumptions: + - I have a Morello board that cannot run P2PRC + - The Morello has a local IP address (ex: 192.168.0.10) + - I have 2 laptops running P2PRC in that local network. + - This means I have 2 ways to access the Morello board: Which is to SSH + into either 2 laptops and then SSH into 192.168.0.10 to gain access + to the board. Wouldn't it be great to automate this whole layer and + as well look into custom tasks into the hetrogenous hardware. +*** Set of interesting possible: + We build a cool set possibilities before and use this to build up the implementation + plan. + - We can use P2PRC access the morello board remotely in a single command. + - We can use the P2PRC protocol to run servers inside the morello board via traversed + node locally which can access that Node. + - Spin servers on node not running P2PRC using the P2PRC standard abstractions. + - Auto-setup P2PRC nodes with just SSH access via potencially a DSL. + - A neat use case for CHERI for instance would be use the architecture to run light + weight hypervisors. +*** Implementation + - To use implementations similar to [[https://linux.die.net/man/1/socat][socat]] to ensure we can bind address of local + nodes to a node running P2PRC and the node running P2PRC can do a local map port. + - We are working on hardening the implementation of the --mp (Map port) to even + map ports to machines which remotely running P2PRC. This means of instance I + can issue a command to the Morello board without the morello board being in + my local network. + - We would want to implement the exsisting P2PRC public key mechanism as well so that + other nodes can access the Morello board who have permission access. + +#+CAPTION: Implementation idea (To be improved upon) + [[./Docs/images/P2PRCRemoteNodes.png]] diff --git a/style.css b/style.css index 1686a4f..0642a1a 100644 --- a/style.css +++ b/style.css @@ -1,32 +1,29 @@ -body { - font-family: sans-serif; +#table-of-contents h2{ + z-index: 200; + background-color: #b96c29; + text-align: center; + padding: 0.809em; + display: block; + color: #fcfcfc; + font-size: 100%; + margin-bottom: 0.809em; } -span#discordJoin { - color: red; +#search-results li { + background-color: #b96c29; + color: white; + margin-bottom: 5px; + padding: 5px; + border-radius: 4px; + cursor: pointer; + font-size: 14px; } -/* Dark mode */ -@media (prefers-color-scheme: dark) { - /* General Page Changing */ - body { - background: #121212; /* Background Color */ - color: #FFFFFF; /* Text Color */ - } - - a:link{ /* Unvisited Link */ - color: #03DAC5; - } - - a:visited { /* Visited Link */ - color: #BB86FC; - } - - a:hover { /* Mouse over Link */ - color: #018786; - } - - span#discordJoin { /* Text Color Change */ - color: #cc0227; - } +h4,h5,h6{ + color: #b96c29; + font-weight:300; } + +.video { + box-shadow: 10px 10px 25px rgba(0, 0, 0, 0.3); + }