me/p2p-rendering-computation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

converted entire docs to org mode

Browse Source
This commit is contained in:
Akilan Selvacoumar
2025-01-19 01:02:45 +00:00
parent fee39d0afa
commit 6863b1386b
50 changed files with 1886 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
Docs/DocsDeprecated/Abstractions.md Normal file
View File

@@ -0,0 +1,20 @@
# Abstractions
| [◀ Previous](Installation.md) | [Next ▶](Implementation.md) |
|:-----------:|---------|
The Abstractions package consists of black-boxed functions for P2PRC.
## Functions
- ```Init(<Project name>)```: Initializes P2PRC with all the needed configurations.
- ```Start()```: Starts p2prc as a server and makes it possible to extend by adding other routes and functionality to P2PRC.
- ```MapPort(<port no>)```: On the local machine the port you want to export to world.
- ```StartContainer(<ip address>)```: The machine on the p2p network where you want to spin up a docker container.
- ```RemoveContainer(<ip address>,<container id>)```: Terminate container based on the IP address and container name.
- ```GetSpecs(<ip address>)```: Get specs of a machine on the network based on the IP address.
- ```ViewIPTable()```: View the IP table which about nodes in the network.
- ```UpdateIPTable()```: Force update IP table to learn about new nodes faster.
---
### Next Chapter: [Implementation](Implementation.md)

35
Docs/DocsDeprecated/Abstractions.org Normal file
View File

@@ -0,0 +1,35 @@
* Abstractions
:PROPERTIES:
:CUSTOM_ID: abstractions
:END:
| [[file:Installation.md][◀ Previous]] | [[file:Implementation.md][Next ▶]] |
|--------------------------------------+------------------------------------|
The Abstractions package consists of black-boxed functions for P2PRC.
** Functions
:PROPERTIES:
:CUSTOM_ID: functions
:END:
- =Init(<Project name>)=: Initializes P2PRC with all the needed
configurations.
- =Start()=: Starts p2prc as a server and makes it possible to extend by
adding other routes and functionality to P2PRC.
- =MapPort(<port no>)=: On the local machine the port you want to export
to world.
- =StartContainer(<ip address>)=: The machine on the p2p network where
you want to spin up a docker container.
- =RemoveContainer(<ip address>,<container id>)=: Terminate container
based on the IP address and container name.
- =GetSpecs(<ip address>)=: Get specs of a machine on the network based
on the IP address.
- =ViewIPTable()=: View the IP table which about nodes in the network.
- =UpdateIPTable()=: Force update IP table to learn about new nodes
faster.
--------------
*** Next Chapter: [[file:Implementation.org][Implementation]]
:PROPERTIES:
:CUSTOM_ID: next-chapter-implementation
:END:

139
Docs/DocsDeprecated/Bindings.md Normal file
View File

@@ -0,0 +1,139 @@
# Language Bindings
[Language bindings](https://en.wikipedia.org/wiki/Language_binding) refers to wrappers to bridge 2 programming languages. This is used in P2PRC to extend calling P2PRC functions in other programming languages. Currently this is done by generating ```.so``` and ```.h``` from the Go compiler.
<br>
## How to build shared object files
#### The easier way
```bash
# Run
make sharedObjects
```
#### Or the direct way
```bash
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
```
#### If successfully built:
```bash
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
```
<br>
## Workings under the hood
Below are a sample set of commands to
open the bindings implementation.
```
# run
cd Bindings/
# list files
ls
# search for file
Client.go
```
### In Client go
There a few things to notice which are different from
your standard Go programs:
#### 1. We import "C" which means [Cgo](https://pkg.go.dev/cmd/cgo) is required.
```go
import "C"
```
#### 2. All functions which are required to be called from other programming languages have comment such as.
```go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
```
#### 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
```go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
```
#### It is easy to notice that:
- ```ConvertStructToJSONString(<go object>)```: This is a helper function that convert
a go object to JSON string initially and converts it to ```CString```.
- ```(output *C.char)```: This is the return type for most of the functions.
#### A Pseudo code to refer to the common function implementation shape could be represented as:
```
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
```
<br>
## Current languages supported
- Python
### Build sample python program
The easier way
```bash
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ p2prc.py
```
Above shows a generated folder which consists of a folder
called "SharedObjects/" which consists of ```p2prc.so```
and ```p2prc.h``` files. ```p2prc.py``` refers to a
sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python,
This generated folder can copied and created as a new
git repo for P2PRC extensions scripted or used a reference
point as proof of concept that P2PRC can be called from
other programming languages.

180
Docs/DocsDeprecated/Bindings.org Normal file
View File

@@ -0,0 +1,180 @@
* Language Bindings
:PROPERTIES:
:CUSTOM_ID: language-bindings
:END:
[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]]
refers to wrappers to bridge 2 programming languages. This is used in
P2PRC to extend calling P2PRC functions in other programming languages.
Currently this is done by generating =.so= and =.h= from the Go
compiler.
** How to build shared object files
:PROPERTIES:
:CUSTOM_ID: how-to-build-shared-object-files
:END:
**** The easier way
:PROPERTIES:
:CUSTOM_ID: the-easier-way
:END:
#+begin_src sh
# Run
make sharedObjects
#+end_src
**** Or the direct way
:PROPERTIES:
:CUSTOM_ID: or-the-direct-way
:END:
#+begin_src sh
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
#+end_src
**** If successfully built:
:PROPERTIES:
:CUSTOM_ID: if-successfully-built
:END:
#+begin_src sh
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
#+end_src
** Workings under the hood
:PROPERTIES:
:CUSTOM_ID: workings-under-the-hood
:END:
Below are a sample set of commands to open the bindings implementation.
#+begin_example
# run
cd Bindings/
# list files
ls
# search for file
Client.go
#+end_example
*** In Client go
:PROPERTIES:
:CUSTOM_ID: in-client-go
:END:
There a few things to notice which are different from your standard Go
programs:
**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required.
:PROPERTIES:
:CUSTOM_ID: we-import-c-which-means-cgo-is-required.
:END:
#+begin_src go
import "C"
#+end_src
**** 2. All functions which are required to be called from other programming languages have comment such as.
:PROPERTIES:
:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as.
:END:
#+begin_src go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
#+end_src
**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
:PROPERTIES:
:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure.
:END:
#+begin_src go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
#+end_src
**** It is easy to notice that:
:PROPERTIES:
:CUSTOM_ID: it-is-easy-to-notice-that
:END:
- =ConvertStructToJSONString(<go object>)=: This is a helper function
that convert a go object to JSON string initially and converts it to
=CString=.
- =(output *C.char)=: This is the return type for most of the functions.
**** A Pseudo code to refer to the common function implementation shape could be represented as:
:PROPERTIES:
:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as
:END:
#+begin_example
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
#+end_example
** Current languages supported
:PROPERTIES:
:CUSTOM_ID: current-languages-supported
:END:
- Python
*** Build sample python program
:PROPERTIES:
:CUSTOM_ID: build-sample-python-program
:END:
The easier way
#+begin_src sh
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ p2prc.py
#+end_src
Above shows a generated folder which consists of a folder called
"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files.
=p2prc.py= refers to a sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python, This generated
folder can copied and created as a new git repo for P2PRC extensions
scripted or used a reference point as proof of concept that P2PRC can be
called from other programming languages.

17
Docs/DocsDeprecated/CliImplementation.md Normal file
View File

@@ -0,0 +1,17 @@
# Cli module
The Cli (i.e Command Line Interface) is the only one in which the user can directly interact with the
modules in the project. The objective when building the Cli was to have the least amount of
commands as possible. The cli was built using the library called urfave cli v2 . They were 2
major files created named as flags.go and actions.go.
### Flags.go
The flags .go file is responsible to create the appropriate flags for the cli. There are 2 types of flags
called boolean and string. Each of the flags outputs are assigned to a
variable to be handled. The flags can also detect environment variables set. This feature is useful
because if the user wants to call certain flags in a repeated sequence it only has to be initialized
once.
### Actions.go
The actions.go file is implemented to call the appropriate functions when the flags are called. It
interacts directly with the modules in the project. Action.go checks if variables
are not empty string or the boolean value is true.

24
Docs/DocsDeprecated/CliImplementation.org Normal file
View File

@@ -0,0 +1,24 @@
* Cli module
:PROPERTIES:
:CUSTOM_ID: cli-module
:END:
The Cli (i.e Command Line Interface) is the only one in which the user
can directly interact with the modules in the project. The objective
when building the Cli was to have the least amount of commands as
possible. The cli was built using the library called urfave cli v2 .
They were 2 major files created named as flags.go and actions.go. ###
Flags.go The flags .go file is responsible to create the appropriate
flags for the cli. There are 2 types of flags called boolean and string.
Each of the flags outputs are assigned to a variable to be handled. The
flags can also detect environment variables set. This feature is useful
because if the user wants to call certain flags in a repeated sequence
it only has to be initialized once.
*** Actions.go
:PROPERTIES:
:CUSTOM_ID: actions.go
:END:
The actions.go file is implemented to call the appropriate functions
when the flags are called. It interacts directly with the modules in the
project. Action.go checks if variables are not empty string or the
boolean value is true.

17
Docs/DocsDeprecated/Client.md Normal file
View File

@@ -0,0 +1,17 @@
# Client Module
This module is incharge of communicating with the server and receiving the appropriate information back from the server.
## Functions of the Client Module
<!-- - [Interact with the Server Api](#functions-of-the-client-module) -->
- [Decision maker on how the ip table is created or updated](#decision-maker-on-how-the-ip-table-is-created-or-updated)
## Decision maker on how the IP table is created or updated
- Does a local speedtest to verify and see if the server IP's in the IP table
are pingable.
- Tries to ping the servers IP Table addresses.
- If it's pingable then it's added as a new entry in the IP table.
- The following steps occurs in the clients IP table.
- To ensure that the same servers are not being called to update the IP table. There is
a temporary list of IP address which have already been called in relation to updating the
IP table.
- Based on the current implementation there will 3 hops done to update the IP table.

33
Docs/DocsDeprecated/Client.org Normal file
View File

@@ -0,0 +1,33 @@
* Client Module
:PROPERTIES:
:CUSTOM_ID: client-module
:END:
This module is incharge of communicating with the server and receiving
the appropriate information back from the server.
** Functions of the Client Module
:PROPERTIES:
:CUSTOM_ID: functions-of-the-client-module
:END:
#+begin_html
<!-- - [Interact with the Server Api](#functions-of-the-client-module) -->
#+end_html
- [[#decision-maker-on-how-the-ip-table-is-created-or-updated][Decision
maker on how the ip table is created or updated]]
** Decision maker on how the IP table is created or updated
:PROPERTIES:
:CUSTOM_ID: decision-maker-on-how-the-ip-table-is-created-or-updated
:END:
- Does a local speedtest to verify and see if the server IP's in the IP
table are pingable.
- Tries to ping the servers IP Table addresses.
- If it's pingable then it's added as a new entry in the IP table.
- The following steps occurs in the clients IP table.
- To ensure that the same servers are not being called to update the IP
table. There is a temporary list of IP address which have already been
called in relation to updating the IP table.
- Based on the current implementation there will 3 hops done to update
the IP table.

12
Docs/DocsDeprecated/ClientArchitecture.md Normal file
View File

@@ -0,0 +1,12 @@
# Client Module Architecture
The Client Module interacts with the P2P module and Server Module. It is responsible for
interacting with the server module and appropriately updating the IP table on the client side. It
connects to the server using the server's REST Apis. It is also the primary decision maker on how
the IP table is updated is on the client side. This is because each user can have requirements like
how many number of hops they would want to do to update their IP table. Hops is the number of
times the client is going to download the IP table from different servers ,once it gets the IP tables
from the previous servers.
![Visual demonstration of hops](images/NumOfHops.png)
![UML diagram of client module](images/clientmoduleArch.png)

15
Docs/DocsDeprecated/ClientArchitecture.org Normal file
View File

@@ -0,0 +1,15 @@
* Client Module Architecture
:PROPERTIES:
:CUSTOM_ID: client-module-architecture
:END:
The Client Module interacts with the P2P module and Server Module. It is
responsible for interacting with the server module and appropriately
updating the IP table on the client side. It connects to the server
using the server's REST Apis. It is also the primary decision maker on
how the IP table is updated is on the client side. This is because each
user can have requirements like how many number of hops they would want
to do to update their IP table. Hops is the number of times the client
is going to download the IP table from different servers ,once it gets
the IP tables from the previous servers.
[[file:images/NumOfHops.png]] [[file:images/clientmoduleArch.png]]

88
Docs/DocsDeprecated/ClientImplementation.md Normal file
View File

@@ -0,0 +1,88 @@
# Client Module Implementation
The Client Module interacts with the P2P module and Server Module. It is responsible for interacting with the server module and appropriately updating the IP table on the client side. It connects to the server using the server's REST Apis. It is also the primary decision maker on how the IP table is updated is on the client side. This is because each user can have requirements like how many number of hops they would want to do to update their IP table. Hops is the number of times the client is going to download the IP table from different servers ,once it gets the IP tables from the previous servers.
![Visual demonstration of hops](images/NumOfHops.png)
![UML diagram of client module](images/clientmoduleArch.png)
## Topics
1. [Updating the IP table](#updating-the-IP-table)
2. [Reading server specifications](#reading-server-specifications)
3. [Client creating and removing container](#Client-creating-and-removing-container)
4. [Tracking Containers](#Tracking-Containers)
5. [Grouping Containers](#Grouping-Containers)
This section focuses in depth on how the client module works. The client module is incharge of communicating with
different servers based on the IP addresses provided to the user. The IP addresses are derived
from peer to peer modules. The objective here is how the client module interacts with peer to peer module
and server module.
### Updating the IP table
The client module calls the peer to peer module to get the local IP table initially, Based on the
servers IP addresses available it calls the speedtest function from the peer to peer module to
update IP addresses with information such as latencies, download and upload speeds. Once this is
done the client module does a Rest Api call to the server to download its IP Table. Once the hops are
done it writes the appropriate results to the Local IP table. Once this is done it prints out the results.
To derive parameters such as current the public IP address the url “http://ip-api.com/json/” was called.
This url returns json response of the current public IP address. This feature will be used in the future
to ensure that the user's current IP address will not be used for a speed test.
Clients IP table is updated to the server using a form of type multipart.
### Reading server specifications
The client module calls the route /server_specs and reads the json response. If the json response
was successful then it just calls the pretty print function which just prints the json output in the
terminal.
### Client creating and removing container
The client module uses the servers Rest apis to create and delete containers. To create a container
the client requires 3 parameters being the server ip address, the number of the ports the user
wants to open and if the user wants it connected to the GPU or not. The 3 parameters are sent as a
GET request to the server and the server responds with a json file which has information such as
the container ID, ports open , SSH username, SSH password, VNC username and VNC password.
At the moment the username and password are hard coded from the server side for both SSH and
VNC.
To remove a container the client module only requires the server IP address and the container ID.
The client prints the response from the server Rest api.
### Tracking Containers
Clients create docker images in multiple machines. This means if the client (i.e user) has many
containers created there needs to be a way to track them. To track containers there is a file
called ```trackcontainers.json``` which tracks all the containers running. The snippet below
show a sample structure of file ```trackcontainer.json```.
```
{
"TrackContainer": [
{
"ID": "<ID>",
"Container": {<docker.DockerVM struct>},
"IpAddress": "<IP Address>"
}
]
}
```
The default path to the container tracker is ```client/trackcontainers/trackcontainers.json```.
### Grouping Containers
When starting a set container possibility to be able to group them.
The benefit this would be that when executing plugins the group ID would be enough to execute
plugin in a set of containers. This provides the possibility to execute repetitive tasks in containers in
a single cli command. To store groups there is a file called ```grouptrackcontainer.json``` which tracks all
the groups currently present set by the client. The snippet below
show a sample structure of file ```grouptrackcontainer.json```.
```
{
"Groups": [
{
"ID": "grp<Random UUID>",
"TrackContainer": [{client.TrackContainers struct}]
}
]
}
```
The default path to the container tracker is ```client/trackcontainers/grouptrackcontainer.json```.
> [!NOTE]
> The group id will be auto-generated and will have its own prefix in the start which will mostly be ```grp<UUID>```.
> When a container is removed using the command. ```p2prc --rm <IP Address> --id <Container id>```. It will be automatically deleted from the groups it exists in.

124
Docs/DocsDeprecated/ClientImplementation.org Normal file
View File

@@ -0,0 +1,124 @@
* Client Module Implementation
:PROPERTIES:
:CUSTOM_ID: client-module-implementation
:END:
The Client Module interacts with the P2P module and Server Module. It is
responsible for interacting with the server module and appropriately
updating the IP table on the client side. It connects to the server
using the server's REST Apis. It is also the primary decision maker on
how the IP table is updated is on the client side. This is because each
user can have requirements like how many number of hops they would want
to do to update their IP table. Hops is the number of times the client
is going to download the IP table from different servers ,once it gets
the IP tables from the previous servers.
[[file:images/NumOfHops.png]] [[file:images/clientmoduleArch.png]]
This section focuses in depth on how the client module works. The client
module is incharge of communicating with different servers based on the
IP addresses provided to the user. The IP addresses are derived from
peer to peer modules. The objective here is how the client module
interacts with peer to peer module and server module.
*** Updating the IP table
:PROPERTIES:
:CUSTOM_ID: updating-the-ip-table
:END:
The client module calls the peer to peer module to get the local IP
table initially, Based on the servers IP addresses available it calls
the speedtest function from the peer to peer module to update IP
addresses with information such as latencies, download and upload
speeds. Once this is done the client module does a Rest Api call to the
server to download its IP Table. Once the hops are done it writes the
appropriate results to the Local IP table. Once this is done it prints
out the results. To derive parameters such as current the public IP
address the url "http://ip-api.com/json/" was called. This url returns
json response of the current public IP address. This feature will be
used in the future to ensure that the user's current IP address will not
be used for a speed test. Clients IP table is updated to the server
using a form of type multipart.
*** Reading server specifications
:PROPERTIES:
:CUSTOM_ID: reading-server-specifications
:END:
The client module calls the route /server_specs and reads the json
response. If the json response was successful then it just calls the
pretty print function which just prints the json output in the terminal.
*** Client creating and removing container
:PROPERTIES:
:CUSTOM_ID: client-creating-and-removing-container
:END:
The client module uses the servers Rest apis to create and delete
containers. To create a container the client requires 3 parameters being
the server ip address, the number of the ports the user wants to open
and if the user wants it connected to the GPU or not. The 3 parameters
are sent as a GET request to the server and the server responds with a
json file which has information such as the container ID, ports open ,
SSH username, SSH password, VNC username and VNC password. At the moment
the username and password are hard coded from the server side for both
SSH and VNC. To remove a container the client module only requires the
server IP address and the container ID. The client prints the response
from the server Rest api.
*** Tracking Containers
:PROPERTIES:
:CUSTOM_ID: tracking-containers
:END:
Clients create docker images in multiple machines. This means if the
client (i.e user) has many containers created there needs to be a way to
track them. To track containers there is a file called
=trackcontainers.json= which tracks all the containers running. The
snippet below show a sample structure of file =trackcontainer.json=.
#+begin_example
{
"TrackContainer": [
{
"ID": "<ID>",
"Container": {<docker.DockerVM struct>},
"IpAddress": "<IP Address>"
}
]
}
#+end_example
The default path to the container tracker is
=client/trackcontainers/trackcontainers.json=.
*** Grouping Containers
:PROPERTIES:
:CUSTOM_ID: grouping-containers
:END:
When starting a set container possibility to be able to group them. The
benefit this would be that when executing plugins the group ID would be
enough to execute plugin in a set of containers. This provides the
possibility to execute repetitive tasks in containers in a single cli
command. To store groups there is a file called
=grouptrackcontainer.json= which tracks all the groups currently present
set by the client. The snippet below show a sample structure of file
=grouptrackcontainer.json=.
#+begin_example
{
"Groups": [
{
"ID": "grp<Random UUID>",
"TrackContainer": [{client.TrackContainers struct}]
}
]
}
#+end_example
The default path to the container tracker is
=client/trackcontainers/grouptrackcontainer.json=.
#+begin_quote
[!NOTE] The group id will be auto-generated and will have its own prefix
in the start which will mostly be =grp<UUID>=.\\
When a container is removed using the command.
=p2prc --rm <IP Address> --id <Container id>=. It will be automatically
deleted from the groups it exists in.
#+end_quote

31
Docs/DocsDeprecated/ConfigImplementation.md Normal file
View File

@@ -0,0 +1,31 @@
# Config Implementation
The configuration module is responsible to store basic information of absolute paths of files being
called in the Go code. In a full-fledged Cli the configuration file can be found in the directory
/etc/<project name> and from there points to location such as where the IP table file is located. In
the future implementation the config file will have information such as number of hops and other
parameters to tweak and to improve the effectiveness of the peer to peer network. The
configuration module was implemented using the library Viper. The Viper library automates
features such as searching in default paths to find out if the configuration file is present. If the
configuration file is not present in the default paths then it auto generates the configuration file.
The configurations file can be in any format. In this project the configuration file was generated using
JSON format.
```json
{
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
```

35
Docs/DocsDeprecated/ConfigImplementation.org Normal file
View File

@@ -0,0 +1,35 @@
* Config Implementation
:PROPERTIES:
:CUSTOM_ID: config-implementation
:END:
The configuration module is responsible to store basic information of
absolute paths of files being called in the Go code. In a full-fledged
Cli the configuration file can be found in the directory /etc/ and from
there points to location such as where the IP table file is located. In
the future implementation the config file will have information such as
number of hops and other parameters to tweak and to improve the
effectiveness of the peer to peer network. The configuration module was
implemented using the library Viper. The Viper library automates
features such as searching in default paths to find out if the
configuration file is present. If the configuration file is not present
in the default paths then it auto generates the configuration file. The
configurations file can be in any format. In this project the
configuration file was generated using JSON format.
#+begin_src json
{
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
#+end_src

14
Docs/DocsDeprecated/DesignArchtectureIntro.md Normal file
View File

@@ -0,0 +1,14 @@
# Design Architecture
This chapter focuses on architecture of the dissertation. The objective would be to have a good
understanding on the purpose of each module and how they interact with each other. The design
architecture was inspired and based on the linux kernel design. The project is segmented into
various modules. Each module is responsible for certain tasks in the project. The modules are
highly dependent on each other hence the entire codebase can be considered as a huge monolithic
chuck which acts as its own library. The following sub topics below talk about the main modules
and how they function with appropriate diagrams.
### 1. [Client Module](ClientArchitecture.md)
### 2. [P2P Module](P2PArchitecture.md)
### 3. [Server Module](ServerArchitecture.md)

26
Docs/DocsDeprecated/DesignArchtectureIntro.org Normal file
View File

@@ -0,0 +1,26 @@
* Design Architecture
:PROPERTIES:
:CUSTOM_ID: design-architecture
:END:
This chapter focuses on architecture of the dissertation. The objective
would be to have a good understanding on the purpose of each module and
how they interact with each other. The design architecture was inspired
and based on the linux kernel design. The project is segmented into
various modules. Each module is responsible for certain tasks in the
project. The modules are highly dependent on each other hence the entire
codebase can be considered as a huge monolithic chuck which acts as its
own library. The following sub topics below talk about the main modules
and how they function with appropriate diagrams.
*** 1. [[file:ClientArchitecture.md][Client Module]]
:PROPERTIES:
:CUSTOM_ID: client-module
:END:
*** 2. [[file:P2PArchitecture.md][P2P Module]]
:PROPERTIES:
:CUSTOM_ID: p2p-module
:END:
*** 3. [[file:ServerArchitecture.md][Server Module]]
:PROPERTIES:
:CUSTOM_ID: server-module
:END:

2
Docs/DocsDeprecated/DomainNameMappingsImplementation.md Normal file
View File

@@ -0,0 +1,2 @@
# Domain name mappings
This

6
Docs/DocsDeprecated/DomainNameMappingsImplementation.org Normal file
View File

@@ -0,0 +1,6 @@
* Domain name mappings
:PROPERTIES:
:CUSTOM_ID: domain-name-mappings
:END:
Todo be written.

23
Docs/DocsDeprecated/Implementation.md Normal file
View File

@@ -0,0 +1,23 @@
# Implementation
| [◀ Previous](Introduction.md) | [Back to TOC](README.md) |
|:-----------:|---------|
This chapter describes how the project was built. It talks in depth of the implementation
performed to give a better understanding of the project.
## Programming langauge used
The programming language used for this project was [Golang](https://go.dev/). The reason Go lang was chosen was
because it is a compiled language.<br>
The entire codebase is just a single binary file. When
distributing to other linux distributing the only requirement would be the binary file to run the
code. It is easy to write independant modules and be monolithic at the sametime using Go.<br>
Using Go.mod makes it very easy to handle external libraries and modularise code. The go.mod name for
the project is [git.sr.ht/~akilan1999/p2p-rendering-computation](https://git.sr.ht/~akilan1999/p2p-rendering-computation).
- ## [Cli Module](CliImplementation.md)
- ## [Config Module](ConfigImplementation.md)
- ## [Server Module](ServerImplementation.md)
- ## [Client Module](ClientImplementation.md)
- ## [P2P Module](P2PImplementation.md)
- ## [Plugin Module](PluginImplementation.md)
- ## [Generate Module](GenerateImplementation.md)

21
Docs/DocsDeprecated/Implementation.org Normal file
View File

@@ -0,0 +1,21 @@
* Implementation
:PROPERTIES:
:CUSTOM_ID: implementation
:END
This chapter describes how the project was built. It talks in depth of
the implementation performed to give a better understanding of the
project.
** Programming language used
:PROPERTIES:
:CUSTOM_ID: programming-langauge-used
:END:
The programming language used for this project was
[[https://go.dev/][Golang]]. The reason Go lang was chosen was because
it is a compiled language. The entire codebase is just a single binary
file. When distributing to other linux distributing the only requirement
would be the binary file to run the code. It is easy to write
independant modules and be monolithic at the sametime using Go. Using
Go.mod makes it very easy to handle external libraries and modularise
code. The go.mod name for the project is
[[https://git.sr.ht/~akilan1999/p2p-rendering-computation][git.sr.ht/~akilan1999/p2p-rendering-computation]].

244
Docs/DocsDeprecated/Installation.md Normal file
View File

@@ -0,0 +1,244 @@
# Installation
| [◀ Previous](Introduction.md) | [Next ▶](Abstractions.md) |
|:-----------:|---------|
Over here we will cover the basic steps to get the server and client side running.
## Latest release install
https://github.com/Akilan1999/p2p-rendering-computation/releases
## Install from Github master branch
### Install Go lang
The entire the implementation of this project is done using Go lang.
Thus, we need go lang to compile to code to a binary file.
[Instructions to install Go lang](https://golang.org/doc/install)
### Install Docker
In this project the choice of virtualization is Docker due to it's wide usage
in the developer community. In the server module we use the Docker Go API to create and
interact with the containers.
[Instructions to install docker](https://docs.docker.com/get-docker/)
[Instructions to install docker GPU](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker)
````
// Do ensure that the docker command does not need sudo to run
sudo chmod 666 /var/run/docker.sock
````
### Build Project and install project
To set up the internal dependencies and build the entire go code
into a single binary
```
make install
```
#### For Windows
To set up P2PRC on Windows, simply run this batch file.
**Make sure you are not in admin mode when running this.**
```
.\install.bat
```
### Add appropriate paths to `.bashrc`
```
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
```
### Set up configuration file
```
make configfile
```
Open the config file ```config.json``` and add the IPv6 address
if you have one.
### Test if binary works
```
p2prc --help
```
#### Output:
```
NAME:
p2p-rendering-computation - p2p cli application to create and access VMs in other servers
USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
<version no>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--Server, -s Starts server (default: false) [$SERVER]
--UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST]
--ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS]
--AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER]
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
--Specification value, --specs value Specs of the server node [$SPECS]
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--help, -h show help (default: false)
--version, -v print the version (default: false)
```
<br>
--------------
<br>
# Using basic commands
### Start as a server
Do ensure you have Docker installed for this
```
p2prc -s
```
### View server Specification
```
p2prc --specs=<ip address>
```
### Run container
use the ```--gpu``` if you know the other machine has a gpu.
```
p2prc --touch=<server ip address> -p <number of ports> --gpu
```
### Remove container
The docker id is present in the output where you create a container
```
p2prc --rm=<server ip address> --id=<docker container id>
```
### Adding servers to ip table
```
p2prc --as=<server ip address you want to add>
```
### Update ip table
```
p2prc --us
```
### List Servers
```
p2prc --ls
```
### View Network interfaces
```
p2prc --ni
```
### Viewing Containers created Client side
```
p2prc --tc
```
[read more on tracking containers](ClientImplementation.md#tracking-containers)
### Running plugin
```
p2prc --plugin <plugin name> --id <container id or group id>
```
### Create group
```
p2prc --cgroup
```
### Add container to group
```
p2prc --group <group id> --id <container id>
```
### View groups
```
p2prc --groups
```
### View specific group
```
p2prc --group <group id>
```
### Delete container from group
```
p2prc --rmcgroup --group <group id> --id <container id>
```
### Delete entire group
```
p2prc --rmgroup <group id>
```
[read more on grouping containers](ClientImplementation.md#Grouping-Containers)
### Extending usecase of P2PRC (Requires a go compiler to run)
```
p2prc --gen <project name> --mod <go module name>
```
[read more about the generate module](GenerateImplementation.md)
### Pulling plugin from a remote repo
```
p2prc --pp <repo link>
```
### Deleting plugin from the plugin directory
```
p2prc --rp <plugin name>
```
### Added custom metadata about the current node
```
p2prc --amd "custom metadata"
```
<br>
--------------
<br>
# Using Plugins
This feature is still Under Development:
[Read more on the implementation](PluginImplementation.md)
#### Dependencies
- Ansible:
- Debian/ubuntu: ```sudo apt install ansible```
- Others: [Installation link](https://ansible-tips-and-tricks.readthedocs.io/en/latest/ansible/install/)
#### Run Test Cases
- Generate Test Case Ansible file
- ```make testcases```
- Enter inside plugin directory and run tests.<br>
> [!NOTE]
> That docker needs to installed and needs to run without
> sudo. Refer the section [Install Docker](#install-docker).
> - ```cd plugin```
> - ```go test .```
---
### Next Chapter: [Abstractions](Abstractions.md)

352
Docs/DocsDeprecated/Installation.org Normal file
View File

@@ -0,0 +1,352 @@
* Installation
:PROPERTIES:
:CUSTOM_ID: installation
:END:
Over here we will cover the basic steps to get the server and client
side running.
** Latest release install
:PROPERTIES:
:CUSTOM_ID: latest-release-install
:END:
https://github.com/Akilan1999/p2p-rendering-computation/releases
** Install from Github master branch
:PROPERTIES:
:CUSTOM_ID: install-from-github-master-branch
:END:
*** Install Go lang
:PROPERTIES:
:CUSTOM_ID: install-go-lang
:END:
The entire the implementation of this project is done using Go lang.
Thus, we need go lang to compile to code to a binary file.
[[https://golang.org/doc/install][Instructions to install Go lang]]
*** Install Docker
:PROPERTIES:
:CUSTOM_ID: install-docker
:END:
In this project the choice of virtualization is Docker due to it's wide
usage in the developer community. In the server module we use the Docker
Go API to create and interact with the containers.
[[https://docs.docker.com/get-docker/][Instructions to install docker]]
[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions
to install docker GPU]]
#+begin_example
// Do ensure that the docker command does not need sudo to run
sudo chmod 666 /var/run/docker.sock
#+end_example
*** Build Project and install project
:PROPERTIES:
:CUSTOM_ID: build-project-and-install-project
:END:
To set up the internal dependencies and build the entire go code into a
single binary
#+begin_example
make install
#+end_example
**** For Windows
:PROPERTIES:
:CUSTOM_ID: for-windows
:END:
To set up P2PRC on Windows, simply run this batch file. *Make sure you
are not in admin mode when running this.*
#+begin_example
.\install.bat
#+end_example
*** Add appropriate paths to =.bashrc=
:PROPERTIES:
:CUSTOM_ID: add-appropriate-paths-to-.bashrc
:END:
#+begin_example
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
#+end_example
*** Set up configuration file
:PROPERTIES:
:CUSTOM_ID: set-up-configuration-file
:END:
#+begin_example
make configfile
#+end_example
Open the config file =config.json= and add the IPv6 address if you have
one.
*** Test if binary works
:PROPERTIES:
:CUSTOM_ID: test-if-binary-works
:END:
#+begin_example
p2prc --help
#+end_example
**** Output:
:PROPERTIES:
:CUSTOM_ID: output
:END:
#+begin_example
NAME:
p2p-rendering-computation - p2p cli application to create and access VMs in other servers
USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
<version no>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--Server, -s Starts server (default: false) [$SERVER]
--UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST]
--ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS]
--AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER]
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
--Specification value, --specs value Specs of the server node [$SPECS]
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--help, -h show help (default: false)
--version, -v print the version (default: false)
#+end_example
--------------
* Using basic commands
:PROPERTIES:
:CUSTOM_ID: using-basic-commands
:END:
*** Start as a server
:PROPERTIES:
:CUSTOM_ID: start-as-a-server
:END:
Do ensure you have Docker installed for this
#+begin_example
p2prc -s
#+end_example
*** View server Specification
:PROPERTIES:
:CUSTOM_ID: view-server-specification
:END:
#+begin_example
p2prc --specs=<ip address>
#+end_example
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
use the =--gpu= if you know the other machine has a gpu.
#+begin_example
p2prc --touch=<server ip address> -p <number of ports> --gpu
#+end_example
*** Remove container
:PROPERTIES:
:CUSTOM_ID: remove-container
:END:
The docker id is present in the output where you create a container
#+begin_example
p2prc --rm=<server ip address> --id=<docker container id>
#+end_example
*** Adding servers to ip table
:PROPERTIES:
:CUSTOM_ID: adding-servers-to-ip-table
:END:
#+begin_example
p2prc --as=<server ip address you want to add>
#+end_example
*** Update ip table
:PROPERTIES:
:CUSTOM_ID: update-ip-table
:END:
#+begin_example
p2prc --us
#+end_example
*** List Servers
:PROPERTIES:
:CUSTOM_ID: list-servers
:END:
#+begin_example
p2prc --ls
#+end_example
*** View Network interfaces
:PROPERTIES:
:CUSTOM_ID: view-network-interfaces
:END:
#+begin_example
p2prc --ni
#+end_example
*** Viewing Containers created Client side
:PROPERTIES:
:CUSTOM_ID: viewing-containers-created-client-side
:END:
#+begin_example
p2prc --tc
#+end_example
[[file:ClientImplementation.md#tracking-containers][read more on
tracking containers]]
*** Running plugin
:PROPERTIES:
:CUSTOM_ID: running-plugin
:END:
#+begin_example
p2prc --plugin <plugin name> --id <container id or group id>
#+end_example
*** Create group
:PROPERTIES:
:CUSTOM_ID: create-group
:END:
#+begin_example
p2prc --cgroup
#+end_example
*** Add container to group
:PROPERTIES:
:CUSTOM_ID: add-container-to-group
:END:
#+begin_example
p2prc --group <group id> --id <container id>
#+end_example
*** View groups
:PROPERTIES:
:CUSTOM_ID: view-groups
:END:
#+begin_example
p2prc --groups
#+end_example
*** View specific group
:PROPERTIES:
:CUSTOM_ID: view-specific-group
:END:
#+begin_example
p2prc --group <group id>
#+end_example
*** Delete container from group
:PROPERTIES:
:CUSTOM_ID: delete-container-from-group
:END:
#+begin_example
p2prc --rmcgroup --group <group id> --id <container id>
#+end_example
*** Delete entire group
:PROPERTIES:
:CUSTOM_ID: delete-entire-group
:END:
#+begin_example
p2prc --rmgroup <group id>
#+end_example
[[file:ClientImplementation.md#Grouping-Containers][read more on
grouping containers]] ### Extending usecase of P2PRC (Requires a go
compiler to run)
#+begin_example
p2prc --gen <project name> --mod <go module name>
#+end_example
[[file:GenerateImplementation.md][read more about the generate module]]
*** Pulling plugin from a remote repo
:PROPERTIES:
:CUSTOM_ID: pulling-plugin-from-a-remote-repo
:END:
#+begin_example
p2prc --pp <repo link>
#+end_example
*** Deleting plugin from the plugin directory
:PROPERTIES:
:CUSTOM_ID: deleting-plugin-from-the-plugin-directory
:END:
#+begin_example
p2prc --rp <plugin name>
#+end_example
*** Added custom metadata about the current node
:PROPERTIES:
:CUSTOM_ID: added-custom-metadata-about-the-current-node
:END:
#+begin_example
p2prc --amd "custom metadata"
#+end_example
--------------
* Using Plugins
:PROPERTIES:
:CUSTOM_ID: using-plugins
:END:
This feature is still Under Development:
[[file:PluginImplementation.md][Read more on the implementation]]
**** Dependencies
:PROPERTIES:
:CUSTOM_ID: dependencies
:END:
- Ansible:
- Debian/ubuntu: =sudo apt install ansible=
- Others:
[[https://ansible-tips-and-tricks.readthedocs.io/en/latest/ansible/install/][Installation
link]]
**** Run Test Cases
:PROPERTIES:
:CUSTOM_ID: run-test-cases
:END:
- Generate Test Case Ansible file
- =make testcases=
- Enter inside plugin directory and run tests.
#+begin_quote
[!NOTE] That docker needs to installed and needs to run without sudo.
Refer the section [[#install-docker][Install Docker]]. - =cd plugin= -
=go test .=
#+end_quote

43
Docs/DocsDeprecated/Introduction.md Normal file
View File

@@ -0,0 +1,43 @@
# Chapter 1: Introduction
| [◀ Back to TOC](README.md) | [Next ▶](Installation.md) |
|:-----------:|---------|
## Abstract
This project focuses on creating a framework on running heavy tasks that a regular computer
cannot run easily such as graphically demanding video games, rendering 3D animations , protein
folding simulations. In this project the major focus will not be on the financial incentive part. A peer
to peer network will be created to help run tasks decentrally, increasing bandwidth for running
tasks. To ensure the tasks in the peer to peer network do not corrupt the server 0S (Operating
System), they will be executed in a virtual environment in the server.
The main aim of this project was to create a custom peer to peer network. The user acting as the
client has total flexibility on how to batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the containers at any point of time.
## Motivation
Many of the users rely on our PC / Laptop or servers that belong to a server farm to run heavy
tasks and with the demand of high creativity requires higher computing power. Buying a powerful
computer every few years to run a bunch of heavy tasks which are not executed as frequently to
reap the benefits can be inefficient utilization of hardware. On the other end, renting servers to
run these heavy tasks can be really useful. Ethically speaking this is leading to monopolisation of
computing power similar to what is happening in the web server area. By using peer to peer
principles it is possible to remove the monopolisation factor and increase the bandwidth between
the client and server.
<!--
## Aim
This project aims to create a peer to peer (p2p) network, where a user can use the p2p network to
act as a client (i.e sending tasks) or the server (i.e executing the tasks). A prototype application will
be developed, which comes bundled with a p2p module and possible to execute docker containers
or virtual environments across selected nodes.
## Objectives
- Background review on peer to peer network, virtual environments, decentralized
rendering tools and tools to batch any sort of tasks.
- Creating p2p network
- Server to create a containerised environment
- The client node to run tasks on Server containerised node -->
---
### Next Chapter: [Installation](Installation.md)

39
Docs/DocsDeprecated/Introduction.org Normal file
View File

@@ -0,0 +1,39 @@
* Chapter 1: Introduction
:PROPERTIES:
:CUSTOM_ID: chapter-1-introduction
:END:
** Abstract
:PROPERTIES:
:CUSTOM_ID: abstract
:END:
This project focuses on creating a framework on running heavy tasks that
a regular computer cannot run easily such as graphically demanding video
games, rendering 3D animations , protein folding simulations. In this
project the major focus will not be on the financial incentive part. A
peer to peer network will be created to help run tasks decentrally,
increasing bandwidth for running tasks. To ensure the tasks in the peer
to peer network do not corrupt the server 0S (Operating System), they
will be executed in a virtual environment in the server.
The main aim of this project was to create a custom peer to peer
network. The user acting as the client has total flexibility on how to
batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the
containers at any point of time.
** Motivation
:PROPERTIES:
:CUSTOM_ID: motivation
:END:
Many of the users rely on our PC / Laptop or servers that belong to a
server farm to run heavy tasks and with the demand of high creativity
requires higher computing power. Buying a powerful computer every few
years to run a bunch of heavy tasks which are not executed as frequently
to reap the benefits can be inefficient utilization of hardware. On the
other end, renting servers to run these heavy tasks can be really
useful. Ethically speaking this is leading to monopolisation of
computing power similar to what is happening in the web server area. By
using peer to peer principles it is possible to remove the
monopolisation factor and increase the bandwidth between the client and
server.

37
Docs/DocsDeprecated/NAT-Traveral.md Normal file
View File

@@ -0,0 +1,37 @@
# NAT Traversal
P2PRC currently supports TURN for NAT traversal.
## TURN
The current TURN implementation used is FRP. The TURN server is also required when
a P2PRC node is acting as a Server. The TURN server is determined based on the Node
with the least amount of latency based on the Nodes available on the IPTable.
Once a TURN server is determined there are 2 actions performed. The first one is
```/FRPPort``` to the TURN server to receive a port which is used to generate the external
port from the TURN server. The flow below describes the workflow.
### Client mode
- Call ```/FRPPort```
```
http://<turn server ip>:<server port no>/FRPport
```
- Call the TURN server in the following manner. The following is a sample code snippet below.
```go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
```

50
Docs/DocsDeprecated/NAT-Traveral.org Normal file
View File

@@ -0,0 +1,50 @@
* NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src

107
Docs/DocsDeprecated/P2P-testing.md Normal file
View File

@@ -0,0 +1,107 @@
# Testing P2P network
The objective would be to test the p2p network, and the effectiveness of updating
the ip tables. The objective of would be to give the impression to the client and
server of a Zero configuration setting. For testing there will be a
test network set. In the testing scenario all will be client and
server because the IP table does not store clients IP addresses. At current
number of hopes would be 3 as default.
### Test Network Scenario 1
The test network consists of 5 nodes acting as a client and server.
The objective would be to have the entire IP table Updated in each node
with interacting with only 1 node once. Each node has knowledge of
one node only.
![p2pscenario1](https://user-images.githubusercontent.com/31743758/115069627-e4aa8c80-9f04-11eb-8402-706a3407f0e8.png)
Fig 1.0 Visual Representation of testnet scenario 1
#### Result
All nodes except node 1 where able to have information of IP addresses in the test net. This was due to the reason of 3 hops
set as default. Node 1 had in it's IP table IP addresses of Node 2, Node 3, Node 4. Once the number of hops was set to 4 objective
of the test was acheived.
### Test Network Scenario 2
The second test network has a scenario of a single peer which all the
other nodes connect too. The scenario being when the other nodes
connect to the single server they download information about nodes
that have connected to the server node before.
### Testing Broadcast Module
For testing the broadcast module 2 types of servers will be
tested. One with a CPU only , another one with a CPU and GPU.
The expected result being that the appropriate results are
visible.
#### Results (CPU and GPU):
```
{
"Hostname": "akilan-Lenovo-IdeaPad-Y510P",
"Platform": "ubuntu",
"CPU": "Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz",
"RAM": 7872,
"Disk": 937367,
"GPU": {
"DriveVersion": "390.141",
"Gpu": {
"GpuName": "GeForce GT 755M",
"BiosVersion": "80.07.A8.00.0F",
"FanSpeed": "N/A",
"Utilization": {
"GpuUsage": "N/A",
"MemoryUsage": "N/A"
},
"Temperature": {
"GpuTemp": "66 C"
},
"Clock": {
"GpuClock": "N/A",
"GpuMemClock": "N/A"
}
}
}
}
```
At the moment of the current implementation v1.0. Nvidia GPU
are only compatible. As the Go code calls the command ``nvidia-smi``
to get information about the GPU available.
#### Results (CPU only)
```
{
"Hostname": "sv-t1.small.x86-01",
"Platform": "ubuntu",
"CPU": "Intel(R) Atom(TM) CPU C2750 @ 2.40GHz",
"RAM": 7944,
"Disk": 138793,
"GPU": null
}
```
As the ``nvidia-smi`` interface was not detected it only broadcasts
the CPU specs available.
### SpeedTests
The speed test has 3 parameters which are Ping , upload and download. The tests check if
the results returned are approximately correct. The ping at the moment returns the correct
result. The upload and download returned are inccorect at the moment, This is due incorrect
implementation in for timer and will be patched in future versions.
### Unit tests
All functions implemented on the P2P module returns type error.
The units test call certain functions and check if the functions
return an error or not. This proved sufficient as the point of
the units tests was code coverage to check if certain functions
return an error.
#### Functions tested
This sections talks about the function called and represents
code coverage.
1. ``TestServer_SpeedTest``: Function called LocalSpeedTestIpTable()
2. ``TestReadIpTable``: Function called ReadIpTable()
The P2P module has a 100% code coverage in unit tests as both the unit
tests call directly or call within the function all the functions used
in the P2P module.

136
Docs/DocsDeprecated/P2P-testing.org Normal file
View File

@@ -0,0 +1,136 @@
* Testing P2P network
:PROPERTIES:
:CUSTOM_ID: testing-p2p-network
:END:
The objective would be to test the p2p network, and the effectiveness of
updating the ip tables. The objective of would be to give the impression
to the client and server of a Zero configuration setting. For testing
there will be a test network set. In the testing scenario all will be
client and server because the IP table does not store clients IP
addresses. At current number of hopes would be 3 as default.
*** Test Network Scenario 1
:PROPERTIES:
:CUSTOM_ID: test-network-scenario-1
:END:
The test network consists of 5 nodes acting as a client and server. The
objective would be to have the entire IP table Updated in each node with
interacting with only 1 node once. Each node has knowledge of one node
only.
[[https://user-images.githubusercontent.com/31743758/115069627-e4aa8c80-9f04-11eb-8402-706a3407f0e8.png]]
Fig 1.0 Visual Representation of testnet scenario 1
**** Result
:PROPERTIES:
:CUSTOM_ID: result
:END:
All nodes except node 1 where able to have information of IP addresses
in the test net. This was due to the reason of 3 hops set as default.
Node 1 had in it's IP table IP addresses of Node 2, Node 3, Node 4. Once
the number of hops was set to 4 objective of the test was acheived.
*** Test Network Scenario 2
:PROPERTIES:
:CUSTOM_ID: test-network-scenario-2
:END:
The second test network has a scenario of a single peer which all the
other nodes connect too. The scenario being when the other nodes connect
to the single server they download information about nodes that have
connected to the server node before.
*** Testing Broadcast Module
:PROPERTIES:
:CUSTOM_ID: testing-broadcast-module
:END:
For testing the broadcast module 2 types of servers will be tested. One
with a CPU only , another one with a CPU and GPU. The expected result
being that the appropriate results are visible.
**** Results (CPU and GPU):
:PROPERTIES:
:CUSTOM_ID: results-cpu-and-gpu
:END:
#+begin_example
{
"Hostname": "akilan-Lenovo-IdeaPad-Y510P",
"Platform": "ubuntu",
"CPU": "Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz",
"RAM": 7872,
"Disk": 937367,
"GPU": {
"DriveVersion": "390.141",
"Gpu": {
"GpuName": "GeForce GT 755M",
"BiosVersion": "80.07.A8.00.0F",
"FanSpeed": "N/A",
"Utilization": {
"GpuUsage": "N/A",
"MemoryUsage": "N/A"
},
"Temperature": {
"GpuTemp": "66 C"
},
"Clock": {
"GpuClock": "N/A",
"GpuMemClock": "N/A"
}
}
}
}
#+end_example
At the moment of the current implementation v1.0. Nvidia GPU are only
compatible. As the Go code calls the command =nvidia-smi= to get
information about the GPU available.
**** Results (CPU only)
:PROPERTIES:
:CUSTOM_ID: results-cpu-only
:END:
#+begin_example
{
"Hostname": "sv-t1.small.x86-01",
"Platform": "ubuntu",
"CPU": "Intel(R) Atom(TM) CPU C2750 @ 2.40GHz",
"RAM": 7944,
"Disk": 138793,
"GPU": null
}
#+end_example
As the =nvidia-smi= interface was not detected it only broadcasts the
CPU specs available.
*** SpeedTests
:PROPERTIES:
:CUSTOM_ID: speedtests
:END:
The speed test has 3 parameters which are Ping , upload and download.
The tests check if the results returned are approximately correct. The
ping at the moment returns the correct result. The upload and download
returned are inccorect at the moment, This is due incorrect
implementation in for timer and will be patched in future versions.
*** Unit tests
:PROPERTIES:
:CUSTOM_ID: unit-tests
:END:
All functions implemented on the P2P module returns type error. The
units test call certain functions and check if the functions return an
error or not. This proved sufficient as the point of the units tests was
code coverage to check if certain functions return an error.
**** Functions tested
:PROPERTIES:
:CUSTOM_ID: functions-tested
:END:
This sections talks about the function called and represents code
coverage.
1. =TestServer_SpeedTest=: Function called LocalSpeedTestIpTable()
2. =TestReadIpTable=: Function called ReadIpTable()
The P2P module has a 100% code coverage in unit tests as both the unit
tests call directly or call within the function all the functions used
in the P2P module.

27
Docs/DocsDeprecated/P2P.md Normal file
View File

@@ -0,0 +1,27 @@
# P2P (Peer to Peer module)
In this repository the P2P module has been designed from sratch at the point of this implementation.
[More about function implementation](https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation@v0.0.0-20210404191839-6a046babcb02/p2p)
## Terminology
1. IPTable: Refers to a json file which stores information about the current servers avaliable with the speedtest results ran from the Node that triggered it.
```
{
"ip_address": [
{
"ipv4": "localhost",
"latency": 14981051,
"download": 8142.122540206258,
"upload": 3578.766512629995,
}
]
}
```
## Responsibility
- To ensure the IP table has nodes which are pingable
- Taking to nodes behind NAT. [More about the implementation](NAT-Traversal)...
> [!NOTE]
> If you are running in server mode it is recommended to use [DMZ](https://routerguide.net/when-and-how-to-setup-dmz-host-for-home-use/) to bypass the [NAT](https://en.wikipedia.org/wiki/Network_address_translation).

45
Docs/DocsDeprecated/P2P.org Normal file
View File

@@ -0,0 +1,45 @@
* P2P (Peer to Peer module)
:PROPERTIES:
:CUSTOM_ID: p2p-peer-to-peer-module
:END:
In this repository the P2P module has been designed from sratch at the
point of this implementation.
[[https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation@v0.0.0-20210404191839-6a046babcb02/p2p][More
about function implementation]]
** Terminology
:PROPERTIES:
:CUSTOM_ID: terminology
:END:
1. IPTable: Refers to a json file which stores information about the
current servers avaliable with the speedtest results ran from the
Node that triggered it.
#+begin_example
{
"ip_address": [
{
"ipv4": "localhost",
"latency": 14981051,
"download": 8142.122540206258,
"upload": 3578.766512629995,
}
]
}
#+end_example
** Responsibility
:PROPERTIES:
:CUSTOM_ID: responsibility
:END:
- To ensure the IP table has nodes which are pingable
- Taking to nodes behind NAT. [[file:NAT-Traversal][More about the
implementation]]...
#+begin_quote
[!NOTE] If you are running in server mode it is recommended to use
[[https://routerguide.net/when-and-how-to-setup-dmz-host-for-home-use/][DMZ]]
to bypass the
[[https://en.wikipedia.org/wiki/Network_address_translation][NAT]].
#+end_quote

11
Docs/DocsDeprecated/P2PArchitecture.md Normal file
View File

@@ -0,0 +1,11 @@
# P2P Module Architecture
The P2P module (i.e Peer to Peer Module) is responsible for storing the IP table and interacting
with the IP table. In the following implementation of the P2P module ,the IP table stores
information about servers available in the network. The other functionality the P2P module takes
care of is doing the appropriate speed tests to the servers in the IP table. This is for informing the
users about nodes which are close by and nodes which have quicker uploads and downloads
speeds. The module is responsible to ensure that there are no duplicate server IPs in the IP table
and to remove all server IPs which are not pingable.
![UML diagram of P2P module](images/p2pmoduleArch.png)

16
Docs/DocsDeprecated/P2PArchitecture.org Normal file
View File

@@ -0,0 +1,16 @@
* P2P Module Architecture
:PROPERTIES:
:CUSTOM_ID: p2p-module-architecture
:END:
The P2P module (i.e Peer to Peer Module) is responsible for storing the
IP table and interacting with the IP table. In the following
implementation of the P2P module ,the IP table stores information about
servers available in the network. The other functionality the P2P module
takes care of is doing the appropriate speed tests to the servers in the
IP table. This is for informing the users about nodes which are close by
and nodes which have quicker uploads and downloads speeds. The module is
responsible to ensure that there are no duplicate server IPs in the IP
table and to remove all server IPs which are not pingable.
#+caption: UML diagram of P2P module
[[file:images/p2pmoduleArch.png]]

85
Docs/DocsDeprecated/P2PImplementation.md Normal file
View File

@@ -0,0 +1,85 @@
# P2P Module Implementation
The P2P module (i.e Peer to Peer Module) is responsible for storing the IP table and interacting
with the IP table. In the following implementation of the P2P module ,the IP table stores
information about servers available in the network. The other functionality the P2P module takes
care of is doing the appropriate speed tests to the servers in the IP table. This is for informing the
users about nodes which are close by and nodes which have quicker uploads and downloads
speeds. The module is responsible to ensure that there are no duplicate server IPs in the IP table
and to remove all server IPs which are not pingable.
![UML diagram of P2P module](images/p2pmoduleArch.png)
The peer to peer implementation was built from scratch. This is because other peer to peer
libraries were on the implementation of the Distributed hash table. At the current moment all
those heavy features are not needed because the objective is to search and list all possible servers
available. The limitation being that to be a part of the network the user has to know at least 1
server. The advantage of building from scratch makes the module super light and
possibility for custom functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
## IP Table
The ip table file is a json as the format with a list of servers ip addresses, latencies, downloads and
uploads speeds. The functions implemented include read
file, write file and remove duplicate IP addresses. The remove duplicate IP address function exists
because sometimes servers IP tables can have the same ip addresses as what the client has. The
path of the IP table json file is received from the configuration module.
```json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>",
}
]
}
```
### Latency
The latency is measured in milliseconds. The route /server_info is called from the
server and time it takes to provide a json response is recorded.
## NAT Traversal
P2PRC currently supports TURN for NAT traversal.
## TURN
The current TURN implementation used is FRP. The TURN server is also required when
a P2PRC node is acting as a Server. The TURN server is determined based on the Node
with the least amount of latency based on the Nodes available on the IPTable.
Once a TURN server is determined there are 2 actions performed. The first one is
```/FRPPort``` to the TURN server to receive a port which is used to generate the external
port from the TURN server. The flow below describes the workflow.
### Client mode
- Call ```/FRPPort```
```
http://<turn server ip>:<server port no>/FRPport
```
- Call the TURN server in the following manner. The following is a sample code snippet below.
```go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
```

111
Docs/DocsDeprecated/P2PImplementation.org Normal file
View File

@@ -0,0 +1,111 @@
* P2P Module Implementation
:PROPERTIES:
:CUSTOM_ID: p2p-module-implementation
:END:
The P2P module (i.e Peer to Peer Module) is responsible for storing the
IP table and interacting with the IP table. In the following
implementation of the P2P module ,the IP table stores information about
servers available in the network. The other functionality the P2P module
takes care of is doing the appropriate speed tests to the servers in the
IP table. This is for informing the users about nodes which are close by
and nodes which have quicker uploads and downloads speeds. The module is
responsible to ensure that there are no duplicate server IPs in the IP
table and to remove all server IPs which are not pingable.
#+caption: UML diagram of P2P module
[[file:images/p2pmoduleArch.png]]
The peer to peer implementation was built from scratch. This is because
other peer to peer libraries were on the implementation of the
Distributed hash table. At the current moment all those heavy features
are not needed because the objective is to search and list all possible
servers available. The limitation being that to be a part of the network
the user has to know at least 1 server. The advantage of building from
scratch makes the module super light and possibility for custom
functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
** IP Table
:PROPERTIES:
:CUSTOM_ID: ip-table
:END:
The ip table file is a json as the format with a list of servers ip
addresses, latencies, downloads and uploads speeds. The functions
implemented include read file, write file and remove duplicate IP
addresses. The remove duplicate IP address function exists because
sometimes servers IP tables can have the same ip addresses as what the
client has. The path of the IP table json file is received from the
configuration module.
#+begin_src json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>",
}
]
}
#+end_src
*** Latency
:PROPERTIES:
:CUSTOM_ID: latency
:END:
The latency is measured in milliseconds. The route /server_info is
called from the server and time it takes to provide a json response is
recorded.
** NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src

156
Docs/DocsDeprecated/PluginImplementation.md Normal file
View File

@@ -0,0 +1,156 @@
# Plugin Module Implementation
## Topics
1. [Introduciton](#introduction)
2. [Site.yml](#site-File-Template)
3. [Host](#hosts-file)
4. [Description](#description-file)
5. [Automatic port allocations](#automatic-port-allocations)
6. [Sample plugins implemented](#sample-plugins-implemented)
## Introduction
The plugin module is designed to ensure clients can execute instructions in a declarative manner across different
containers created. This means the user (i.e client) needs to write the instruction only once, and these instructions
can be executed across different nodes in a repetitive manner.
In the scenario of this project Ansibles will be used as the way the users can create these instructions.
- [Setup instruction](Installation.md#Using-Plugins)
The plugin module introduces a new path to the config file known as pluginpath. This path by defaults points to
```${P2PRC}/plugin/deploy```. Any file/folder inside ```plugin/deploy``` is part of the .gitginore. Plugins are
detected by folder names inside the ```plugin/deploy```.
```
plugin
|___ Deploy
|___<plugin name>
|___ site.yml
|___ hosts
|___ ports.json
|___ description.txt
.
.
.
n: n number of plugins possible
```
## Site File Template
The site file is also known as the Ansible playbook and is incharge of executing
instructions in a declarative manner. The below example specifies how to make one.
```
- hosts: all
tasks:
- name: <task name>
<ansible task>
debug:
msg: <debug message>
```
Read more about ansible tasks: https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html#about-playbooks
## Hosts file
hosts file is also known as the inventory file. This file consists of all the information required to connect to other
nodes to execute Ansible instructions. In this project this file needs to be set in a certain configuration because the
go code or binary will populate this file automatically with the appropriate information required to connect to local or
remote containers.
> [!NOTE]
> Add as exactly specified below:
> ```
>all:
> vars:
> ansible_python_interpreter: /usr/bin/python3 // Path to your python 3 interpreter
>main:
> hosts:
> host1:
> // Note: These values will be automatically overwritten
> // by the Go functions
> ansible_host: 0.0.0.0
> ansible_port: 39269
> ansible_user: master
> ansible_ssh_pass: password
> ansible_sudo_pass: password
>```
## Ports.json
The ```ports.json``` file is intended to mention the number of ports required
by the plugin.
```
{
"NumOfPorts": <number of ports>
}
```
## Description file
This is a simple text file used to describe what the module does.
When the client is looking at various commands via the ClI.
The description is displayed along-side the plugin name.
Ex: When the flag ```--ViewPlugins``` or ```--vp``` is called
```
{
"PluginsDetected": [
{
"FolderName": "<name of the plugin>",
"PluginDescription": "<description of the plugin>"
}
]
}
```
## Automatic port allocations
P2PRC would be in-charge to set to the ports to various TCP ports
opened. Due to this implementation the plugin being executed is
copied to the tmp directory with a unique UUID.
```
Command: ls /tmp
output: Semantic <UUID>_<Plugin Name>
2e6d76c4-0ed1-4b55-9385-79a58d4f0492_p2prc-vscode-browser
7b631e08-62ee-4c1c-a2a4-c05857b9aa7d_p2prc-vscode-browser
```
Once the copy of the plugin is added to the /tmp directory
the site.yml file inside the appropriate yaml is modified
with the appropriate ports assigned to the container.
### Ex:
1. Create container called c1 with an automatic generated TCP port
3313 (external) - 3313 (internal)
2. Assumption of plugin p1 exists. p1 has one server which needs to
be mapped to a free open TCP port in container c1. Below shows
an implementation of a sample site.yml file.
```
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:{{index . 0}}
```
Notice there is the following {{index . 0}}. {{index . 0}} does not belong to
Ansible but rather is a way to mention where to add the external free port
of the container. We use the golang [template library](https://pkg.go.dev/text/template)
to parse and populate the site.yml with the appropriate open ports. An array of ints
which consists of open free ports are sent to the site.yml. 0 in {{index . 0}} refers
to the index in the int array passed on.
After the port is automatically it's ready to run !
```
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:3313
```
### Sample plugins implemented:
- [VSCode Plugin](https://github.com/Akilan1999/p2prc-vscode-browser)
## Pull Plugins
The following allows us to pull plugins from a remote git repository and store them
in the default plugins directory. The implementation uses a Go git library to pull the
git repo and automatically save it as a folder in the plugin path.
## Delete Plugins
We delete the plugin folder based on the plugin name provided as an argument on the cli command.
Once the folder is deleted, the plugin manager automatically knows that the plugin does not exist anymore.

203
Docs/DocsDeprecated/PluginImplementation.org Normal file
View File

@@ -0,0 +1,203 @@
* Plugin Module Implementation
:PROPERTIES:
:CUSTOM_ID: plugin-module-implementation
:END:
** Introduction
:PROPERTIES:
:CUSTOM_ID: introduction
:END:
The plugin module is designed to ensure clients can execute instructions
in a declarative manner across different containers created. This means
the user (i.e client) needs to write the instruction only once, and
these instructions can be executed across different nodes in a
repetitive manner.
In the scenario of this project Ansibles will be used as the way the
users can create these instructions.
- [[file:Installation.md#Using-Plugins][Setup instruction]]
The plugin module introduces a new path to the config file known as
pluginpath. This path by defaults points to =${P2PRC}/plugin/deploy=.
Any file/folder inside =plugin/deploy= is part of the .gitginore.
Plugins are detected by folder names inside the =plugin/deploy=.
#+begin_example
plugin
|___ Deploy
|___<plugin name>
|___ site.yml
|___ hosts
|___ ports.json
|___ description.txt
.
.
.
n: n number of plugins possible
#+end_example
** Site File Template
:PROPERTIES:
:CUSTOM_ID: site-file-template
:END:
The site file is also known as the Ansible playbook and is incharge of
executing instructions in a declarative manner. The below example
specifies how to make one.
#+begin_example
- hosts: all
tasks:
- name: <task name>
<ansible task>
debug:
msg: <debug message>
#+end_example
Read more about ansible tasks:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html#about-playbooks
** Hosts file
:PROPERTIES:
:CUSTOM_ID: hosts-file
:END:
hosts file is also known as the inventory file. This file consists of
all the information required to connect to other nodes to execute
Ansible instructions. In this project this file needs to be set in a
certain configuration because the go code or binary will populate this
file automatically with the appropriate information required to connect
to local or remote containers.
#+begin_quote
[!NOTE] Add as exactly specified below:
#+begin_example
all:
vars:
ansible_python_interpreter: /usr/bin/python3 // Path to your python 3 interpreter
main:
hosts:
host1:
// Note: These values will be automatically overwritten
// by the Go functions
ansible_host: 0.0.0.0
ansible_port: 39269
ansible_user: master
ansible_ssh_pass: password
ansible_sudo_pass: password
#+end_example
#+end_quote
** Ports.json
:PROPERTIES:
:CUSTOM_ID: ports.json
:END:
The =ports.json= file is intended to mention the number of ports
required by the plugin.
#+begin_example
{
"NumOfPorts": <number of ports>
}
#+end_example
** Description file
:PROPERTIES:
:CUSTOM_ID: description-file
:END:
This is a simple text file used to describe what the module does. When
the client is looking at various commands via the ClI. The description
is displayed along-side the plugin name.
Ex: When the flag =--ViewPlugins= or =--vp= is called
#+begin_example
{
"PluginsDetected": [
{
"FolderName": "<name of the plugin>",
"PluginDescription": "<description of the plugin>"
}
]
}
#+end_example
** Automatic port allocations
:PROPERTIES:
:CUSTOM_ID: automatic-port-allocations
:END:
P2PRC would be in-charge to set to the ports to various TCP ports
opened. Due to this implementation the plugin being executed is copied
to the tmp directory with a unique UUID.
#+begin_example
Command: ls /tmp
output: Semantic <UUID>_<Plugin Name>
2e6d76c4-0ed1-4b55-9385-79a58d4f0492_p2prc-vscode-browser
7b631e08-62ee-4c1c-a2a4-c05857b9aa7d_p2prc-vscode-browser
#+end_example
Once the copy of the plugin is added to the /tmp directory the site.yml
file inside the appropriate yaml is modified with the appropriate ports
assigned to the container.
*** Ex:
:PROPERTIES:
:CUSTOM_ID: ex
:END:
1. Create container called c1 with an automatic generated TCP port 3313
(external) - 3313 (internal)
2. Assumption of plugin p1 exists. p1 has one server which needs to be
mapped to a free open TCP port in container c1. Below shows an
implementation of a sample site.yml file.
#+begin_example
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:{{index . 0}}
#+end_example
Notice there is the following {{index . 0}}. {{index . 0}} does not
belong to Ansible but rather is a way to mention where to add the
external free port of the container. We use the golang
[[https://pkg.go.dev/text/template][template library]] to parse and
populate the site.yml with the appropriate open ports. An array of ints
which consists of open free ports are sent to the site.yml. 0 in
{{index . 0}} refers to the index in the int array passed on.
After the port is automatically it's ready to run !
#+begin_example
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:3313
#+end_example
*** Sample plugins implemented:
:PROPERTIES:
:CUSTOM_ID: sample-plugins-implemented
:END:
- [[https://github.com/Akilan1999/p2prc-vscode-browser][VSCode Plugin]]
** Pull Plugins
:PROPERTIES:
:CUSTOM_ID: pull-plugins
:END:
The following allows us to pull plugins from a remote git repository and
store them in the default plugins directory. The implementation uses a
Go git library to pull the git repo and automatically save it as a
folder in the plugin path.
** Delete Plugins
:PROPERTIES:
:CUSTOM_ID: delete-plugins
:END:
We delete the plugin folder based on the plugin name provided as an
argument on the cli command. Once the folder is deleted, the plugin
manager automatically knows that the plugin does not exist anymore.

170
Docs/DocsDeprecated/README.md Normal file
View File

@@ -0,0 +1,170 @@
> [!TIP]
> Haskell bindings supported!: [Bindings documentaton](https://p2prc.akilan.io/Docs/haskell/P2PRC.html)
> [!NOTE]
> Fixing documentation to latest changes. If you have any questions setting up P2PRC either [create an issue](https://github.com/Akilan1999/p2p-rendering-computation/issues/new/choose) or send me an email (me AT akilan dot io).
> Currently HEAD is always intended to stay on a working state. It is recommended to always use HEAD in your go.mod file.
<h1 align="center">
<br>
<a href=""><img src="https://raw.githubusercontent.com/Akilan1999/p2p-rendering-computation/master/Docs/images/p2prclogo.png" alt="p2prc" width="400"></a>
<br>
</h1>
<!-- seperator -->
<div style="display:flex;flex-wrap:wrap;">
<a href="http://perso.crans.org/besson/LICENSE.html"><img alt="GPLv2 license" src="https://img.shields.io/badge/License-GPLv2-blue.svg" style="padding:5px;margin:5px;" /></a>
<a href="https://GitHub.com/Akilan1999/p2p-rendering-computation/graphs/commit-activity"><img alt="Maintenance" src="https://img.shields.io/badge/Maintained%3F-yes-green.svg" style="padding:5px;margin:5px;" /></a>
<a href="http://golang.org"><img alt="made-with-Go" src="https://img.shields.io/badge/Made%20with-Go-1f425f.svg" style="padding:5px;margin:5px;" /></a>
<a href="https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation"><img alt="GoDoc reference example" src="https://img.shields.io/badge/godoc-reference-blue.svg" style="padding:5px;margin:5px;" /></a>
</div>
The main aim of this project was to create a custom peer to peer network. The user acting as the
client has total flexibility on how to batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the containers at any point of time.
## Latest tutorial
[![IMAGE ALT TEXT](https://i.ytimg.com/vi/OMwCpedu5cs/hqdefault.jpg)](https://www.youtube.com/watch?v=OMwCpedu5cs")
<br>
## Table of contents in the current README
1. [Introduction](#Introduction)
2. [Installation](#extend-your-application-with-p2prc)
3. [Design Architecture](#Design-Architecture)
4. [Implementation](#Implementation)
5. [Find out more](#Find-out-more)
<br>
# Table of contents in the Docs folder
1. [Introduction](Docs/Introduction.md)
2. [Installation](Docs/Installation.md)
3. [Abstractions](Docs/Abstractions.md)
<!-- 3. [Design Architecture](DesignArchtectureIntro.md)
1. [Client Module](ClientArchitecture.md)
2. [P2P Module](P2PArchitecture.md)
3. [Server Module](ServerArchitecture.md) -->
4. [Implementation](Docs/Implementation.md)
1. [Client Module](Docs/ClientImplementation.md)
2. [P2P Module](Docs/P2PImplementation.md)
3. [Server Module](Docs/ServerImplementation.md)
4. [Config Module](Docs/ConfigImplementation.md)
5. [Cli Module](Docs/CliImplementation.md)
6. [Plugin Module](Docs/PluginImplementation.md)
7. [Language bindings](Docs/Bindings.md)
8. [Domain name mappings](Docs/Bindings.md)
5. Language bindings
1. [Haskell](Docs/haskell/)
<!-- 5. [Problems](https://github.com/Akilan1999/p2p-rendering-computation/issues) -->
<br>
## Introduction
This project aims to create a peer to peer (p2p) network, where a user can use the p2p network to act as a client (i.e sending tasks) or the server (i.e executing the tasks). A prototype application will be developed, which comes bundled with a p2p module and possible to execute docker containers or virtual environments across selected nodes.
### Objectives
- Background review on peer to peer network, virtual environments, decentralized rendering tools and tools to batch any sort of tasks.
- Creating p2p network
- Server to create a containerised environment
- The client node to run tasks on Server containerised node
[Read more on the introduction](Docs/Introduction.md)
<br>
## Extend your application with P2PRC
```go
package main
import (
"fmt"
"github.com/Akilan1999/p2p-rendering-computation/abstractions"
)
func main() {
_, err := abstractions.Init(nil)
if err != nil {
fmt.Println(err)
return
}
// start p2prc
_, err = abstractions.Start()
if err != nil {
fmt.Println(err)
return
}
// Run server till termination
for {
}
}
```
### Export once this is added export P2PRC as environment paths
```
export P2PRC=<PROJECT PATH>
export PATH=<PROJECT PATH>:${PATH}
```
[Read more](Docs/Abstractions.md) ...
## Installation from source
1. Ensure the Go compiler is installed
```
go version
```
3. Ensure docker is installed (Should run without sudo)
```
docker ps
```
3. Clone this repository
```
git clone https://github.com/Akilan1999/p2p-rendering-computation
```
4. Install and build the project
```
make install
```
- If you look closely you will get outputs such as:
```
// Add them to your .bashrc file
export P2PRC=/<path>/p2p-rendering-computation
export PATH=/<path>/p2p-rendering-computation:${PATH}
```
5. Test if it works
```
p2prc -h
```
or
```
./p2prc -h
```
[Read more on the installation and usage](Docs/Installation.md)
<br>
## Design Architecture
The design architecture was inspired and based on the linux kernel design. The project is segmented into various modules. Each module is responsible for certain tasks in the project. The modules are highly dependent on each other hence the entire codebase can be considered as a huge monolithic chuck which acts as its own library
[Read more on the Design Architecture](Docs/DesignArchtectureIntro.md)
<br>
## Implementation
The programming language used for this project was Golang. The reason Go lang was chosen was because it is a compiled language. The entire codebase is just a single binary file. When distributing to other linux distributing the only requirement would be the binary file to run the code. It is easy to write independant modules and be monolithic at the sametime using Go. Using Go.mod makes it very easy to handle external libraries and modularise code. The go.mod name for the project is git.sr.ht/~akilan1999/p2p-rendering-computation.
[Read more on the Implementation](Docs/Implementation.md)
<br>
## Find out more
As we are working on the open source project p2prc (i.e p2p network designed for computation).If you are interested in participating as a contributor
or just providing feedback on new features to build or even just curious about new features added to the project. We have decided to create a discord group.
[![Support Server](https://discordapp.com/api/guilds/854397492795277322/widget.png?style=banner2)](https://discord.gg/b4nRGTjYqy)
[![Star History Chart](https://api.star-history.com/svg?repos=Akilan1999/p2p-rendering-computation&type=Date)](https://github.com/Gaurav-Gosain)

18
Docs/DocsDeprecated/README.org Normal file
View File

@@ -0,0 +1,18 @@
* Table of contents
:PROPERTIES:
:CUSTOM_ID: table-of-contents
:END:
1.[[file:Introduction.org][ Introduction]]
2. [[file:Installation.org][Installation]]
3. [[file:Abstractions.org][Abstractions]]
4. [[file:Implementation.org][Implementation]]
1. [[file:ClientImplementation.org][Client Module]]
2. [[file:P2PImplementation.org][P2P Module]]
3. [[file:ServerImplementation.org][Server Module]]
4. [[file:ConfigImplementation.org][Config Module]]
5. [[file:CliImplementation.org][Cli Module]]
6. [[file:PluginImplementation.org][Plugin Module]]
7. [[file:Bindings.org][Language bindings]]
8. [[file:Bindings.org][Domain name mappings]]
5. Language bindings
1. [[file:haskell/][Haskell]]

8
Docs/DocsDeprecated/ServerArchitecture.md Normal file
View File

@@ -0,0 +1,8 @@
# Server Module Architecture
The server module takes care of setting and removing the virtualization environment (i.e
containers) for accessing and doing the appropriate computation. It also interacts with the peer to
peer module to update the IP table on the server side. The server module
accesses information regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to upload and download a 50mb.
![UML diagram of server module](images/servermoduleArch.png)

14
Docs/DocsDeprecated/ServerArchitecture.md.org Normal file
View File

@@ -0,0 +1,14 @@
* Server Module Architecture
:PROPERTIES:
:CUSTOM_ID: server-module-architecture
:END:
The server module takes care of setting and removing the virtualization
environment (i.e containers) for accessing and doing the appropriate
computation. It also interacts with the peer to peer module to update
the IP table on the server side. The server module accesses information
regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to
upload and download a 50mb.
#+caption: UML diagram of server module
[[file:images/servermoduleArch.png]]

200
Docs/DocsDeprecated/ServerImplementation.md Normal file
View File

@@ -0,0 +1,200 @@
# Server Module Implementation
This section focuses on an in-depth understanding of the server module implementation. To
understand the architecture of the server module refer. The server module can be split
into various sections. Each section will provide information on how a certain feature works.
The server module takes care of setting and removing the virtualization environment (i.e
containers) for accessing and doing the appropriate computation. It also interacts with the peer to
peer module to update the IP table on the server side. The server module
accesses information regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to upload and download a 50mb.
![UML diagram of server module](images/servermoduleArch.png)
## Web framework
The web framework used for the server module is called Gin. The reason Gin was chosen is due to
its wide use and strong documentation available on the official github repository. The default
port used is 8088. For version 1.0 of the project ,the server needs to keep port 8088 open to
ensure that other clients and servers can detect it. The possible requests available are GET and
POST for this implementation. The possible responses are either a string or json response or a file.
In the majority of routes a string response refers to an error when calling the following routes.
The following sub topics below will talk about the route implemented:
### /server_info
This route is responsible to get information about the specifications of the
server. The response of this route is in json if the call was successful.
### /50
This route is responsible for returning a randomly generated 50mb file. This is used to
calculate the download speed from the p2p module.
### /IpTable
This route is a POST request that is responsible to update the server IP table
based on the IP table the client provides. Once the server gets the IP table it checks if the
client is also a server. This is done by calling the url http://<client ip>:8088/server_info. If
the server_info route from the client responds back with computer specifications of the
client. Then the server initially appends the clients IP to the struct. After that the IP table
received from the client is uploaded to the struct. Once this is done the server passes the
struct to the peer to peer module function. The peer to peer module function will return the back with the
new struct with the valid server nodes. The server responds back to the new struct as a
json format. If a string is present in the response then there is probably an error on the
server side.
### /startcontainer
This route takes in a GET request with the number of TCP ports to open and
checks whether the docker container should be hooked to the GPU or not. This route talks
to the docker module implemented as a sub module in the server module. More
information on the docker module in section 5.4.3. This route calls docker the module to
start the container for the client. The docker module returns back a struct. This struct is
returned back to the client as the json response. This struct consists of information such as
docker id, ports numbers open , information regarding SSH and VNC connections to the
docker container created when the client created this request.
### /RemoveContainer
This route takes in a GET request as the container ID. Based on the
container ID provided ,it calls the docker module which deletes the container. If the
deletion is successful it returns back a string which says success.
## Server information/ Specification
This section provides information on how the server specifications are read. There are 2 major
implementations. The first implementation mentions how basic information such as RAM usage,
CPU specification are detected and the second implementation mentions how the GPU drivers are
detected and information is extracted. The client has to assume that the server is using default
docker settings in terms of CPU cycles and other parameters.
### Basic Information
The file name for these functions is called gopsutil.go. This codebase
uses the library gopsutil. Gopsutil has various packages or modules within the library
which have functions implemented to get system information. The following information is
stored in a struct and the function returns that struct.
```go
type SysInfo struct {
Hostname string `bson:hostname`
Platform string `bson:platform`
CPU string `bson:cpu`
RAM uint64 `bson:ram`
Disk uint64 `bson:disk`
GPU *Query `xml: GpuInfo`
}
```
### GPU Information
The file name for these functions is called GPU.go. This codebase checks
if the Nvidia driver exists and returns the driver information. To do this a shell
command called nvidia-smi is executed. This shell command is executed with a --xml as flag
to ensure that the output is in the XML format. If there is an output as a xml format, that
means there is an nvidia driver installed, and the function just reads the output and stores it
to the struct and returns the GPU information.
```go
type Query struct {
DriveVersion string `xml:"driver_version"`
Gpu Gpu `xml:"gpu"`
}
type Gpu struct{
GpuName string `xml:"product_name"`
BiosVersion string `xml:"vbios_version"`
FanSpeed string `xml:"fan_speed"`
Utilization GpuUtilization `xml:"utilization"`
Temperature GpuTemperature `xml:"temperature"`
Clock GpuClock `xml:"clocks"`
}
type GpuUtilization struct {
GpuUsage string `xml:"gpu_util"`
MemoryUsage string `xml:"memory_util"`
}
type GpuTemperature struct {
GpuTemp string `xml:"gpu_temp"`
}
type GpuClock struct {
GpuClock string `xml:"graphics_clock"`
GpuMemClock string `xml:"mem_clock"`
}
```
## Docker Module
This section provides information on how the server module interacts with the docker containers.
The server calls 2 routes which either creates or removes the docker container. Docker has a huge
advantage because it takes less than 20 seconds to spin up a new container once it’s built and
executed at least once. For docker operations a separate module/package has been created. The
following subtopics will provide more information on how this package works.
### Docker Api
For this the api has been taken from the official docker repository. To be more
specific it is the client module in the official docker repository. Docker was built using Go.
During this project Docker functions could be directly called from the docker repository.
The Docker api initially ensures that it can detect the docker environment variables. Once
detected, it can execute various functions from the docker client module. The reason the
docker api was selected was to detect and handle errors better.
### Docker Image
The docker image used to spin up the containers is called
ConSol/docker-headless-vnc-container. The following container was modified to open
SSH ports for an SSH connection. The following docker image runs ubuntu 16. The reason
this image was chosen as a default is because if the client wants to access the container in
the form of a desktop environment. This image would allow the client to do so from just a
browser.
### Build container
This function pulls the docker image locally and builds the image. Initially
there is a timeout function to ensure that building the image does not take too long to
build. The next phase would be based on the path to get the DockerFile. The tag name of
the container is set as p2p-ubuntu as default. Once the following is set then the docker
build command is executed.
### Run container
After building the container it needs to be executed for the user to access
the container and do certain operations. The docker package/module has a function to do
this. The function takes in the docker environment as a parameter and also the docker
struct. The docker struct has information such as the TCP ports which are supposed to be
open and whether the docker container should have the GPU hooked to it or not. Based on
the appropriate information provided ,the docker image gets started. The Image gets
started by interacting with the docker client modules. When hooking the GPU the docker
run command is called from the shell. This is because the docker Api does not support the
GPU module yet. When the container is executed for the first time it takes
more than 10 minutes to build. From the second time onwards it takes only 10 seconds to
run.
### Stop and remove container
This implementation here ensures that the docker is stopped, and the container is removed. This is to ensure
it does not utilize server resources when it is not being used, or the task that is intended to be executed is complete.
To run this function all that is needed is the docker container ID. If the function is successful it returns
a string that says success.
### Ports json file
This file will help map internal ports inside a container to external ports inside a container. A common example
would be the SSH port which is port 22 inside the docker container and is mapped to random TCP port outside container
so that any external machines can directly connect into the container. The below representation mentions of where
the ports.json file is located and also the format of that file.
```
|_ <Container name>
|_ Dockerfile
|_ description.txt
|_ ports.json // The ports file
```
Format of the ports.json file
```
{
"Port": [
{
"PortName": "<Port name>",
"InternalPort": <internal port>,
"Type": "<tcp/udp>",
"ExternalPort": <external port>,
"IsUsed": "<boolean value (i.e true or false)>",
"Description": "<description about the port>"
}, ... n
]
}
```

278
Docs/DocsDeprecated/ServerImplementation.md.org Normal file
View File

@@ -0,0 +1,278 @@
* Server Module Implementation
:PROPERTIES:
:CUSTOM_ID: server-module-implementation
:END:
This section focuses on an in-depth understanding of the server module
implementation. To understand the architecture of the server module
refer. The server module can be split into various sections. Each
section will provide information on how a certain feature works.
The server module takes care of setting and removing the virtualization
environment (i.e containers) for accessing and doing the appropriate
computation. It also interacts with the peer to peer module to update
the IP table on the server side. The server module accesses information
regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to
upload and download a 50mb.
#+caption: UML diagram of server module
[[file:images/servermoduleArch.png]]
** Web framework
:PROPERTIES:
:CUSTOM_ID: web-framework
:END:
The web framework used for the server module is called Gin. The reason
Gin was chosen is due to its wide use and strong documentation available
on the official github repository. The default port used is 8088. For
version 1.0 of the project ,the server needs to keep port 8088 open to
ensure that other clients and servers can detect it. The possible
requests available are GET and POST for this implementation. The
possible responses are either a string or json response or a file. In
the majority of routes a string response refers to an error when calling
the following routes. The following sub topics below will talk about the
route implemented:
*** /server_info
:PROPERTIES:
:CUSTOM_ID: server_info
:END:
This route is responsible to get information about the specifications of
the server. The response of this route is in json if the call was
successful.
*** /50
:PROPERTIES:
:CUSTOM_ID: section
:END:
This route is responsible for returning a randomly generated 50mb file.
This is used to calculate the download speed from the p2p module.
*** /IpTable
:PROPERTIES:
:CUSTOM_ID: iptable
:END:
This route is a POST request that is responsible to update the server IP
table based on the IP table the client provides. Once the server gets
the IP table it checks if the client is also a server. This is done by
calling the url http://:8088/server_info. If the server_info route from
the client responds back with computer specifications of the client.
Then the server initially appends the clients IP to the struct. After
that the IP table received from the client is uploaded to the struct.
Once this is done the server passes the struct to the peer to peer
module function. The peer to peer module function will return the back
with the new struct with the valid server nodes. The server responds
back to the new struct as a json format. If a string is present in the
response then there is probably an error on the server side.
*** /startcontainer
:PROPERTIES:
:CUSTOM_ID: startcontainer
:END:
This route takes in a GET request with the number of TCP ports to open
and checks whether the docker container should be hooked to the GPU or
not. This route talks to the docker module implemented as a sub module
in the server module. More information on the docker module in section
5.4.3. This route calls docker the module to start the container for the
client. The docker module returns back a struct. This struct is returned
back to the client as the json response. This struct consists of
information such as docker id, ports numbers open , information
regarding SSH and VNC connections to the docker container created when
the client created this request.
*** /RemoveContainer
:PROPERTIES:
:CUSTOM_ID: removecontainer
:END:
This route takes in a GET request as the container ID. Based on the
container ID provided ,it calls the docker module which deletes the
container. If the deletion is successful it returns back a string which
says success.
** Server information/ Specification
:PROPERTIES:
:CUSTOM_ID: server-information-specification
:END:
This section provides information on how the server specifications are
read. There are 2 major implementations. The first implementation
mentions how basic information such as RAM usage, CPU specification are
detected and the second implementation mentions how the GPU drivers are
detected and information is extracted. The client has to assume that the
server is using default docker settings in terms of CPU cycles and other
parameters.
*** Basic Information
:PROPERTIES:
:CUSTOM_ID: basic-information
:END:
The file name for these functions is called gopsutil.go. This codebase
uses the library gopsutil. Gopsutil has various packages or modules
within the library which have functions implemented to get system
information. The following information is stored in a struct and the
function returns that struct.
#+begin_src go
type SysInfo struct {
Hostname string `bson:hostname`
Platform string `bson:platform`
CPU string `bson:cpu`
RAM uint64 `bson:ram`
Disk uint64 `bson:disk`
GPU *Query `xml: GpuInfo`
}
#+end_src
*** GPU Information
:PROPERTIES:
:CUSTOM_ID: gpu-information
:END:
The file name for these functions is called GPU.go. This codebase checks
if the Nvidia driver exists and returns the driver information. To do
this a shell command called nvidia-smi is executed. This shell command
is executed with a --xml as flag to ensure that the output is in the XML
format. If there is an output as a xml format, that means there is an
nvidia driver installed, and the function just reads the output and
stores it to the struct and returns the GPU information.
#+begin_src go
type Query struct {
DriveVersion string `xml:"driver_version"`
Gpu Gpu `xml:"gpu"`
}
type Gpu struct{
GpuName string `xml:"product_name"`
BiosVersion string `xml:"vbios_version"`
FanSpeed string `xml:"fan_speed"`
Utilization GpuUtilization `xml:"utilization"`
Temperature GpuTemperature `xml:"temperature"`
Clock GpuClock `xml:"clocks"`
}
type GpuUtilization struct {
GpuUsage string `xml:"gpu_util"`
MemoryUsage string `xml:"memory_util"`
}
type GpuTemperature struct {
GpuTemp string `xml:"gpu_temp"`
}
type GpuClock struct {
GpuClock string `xml:"graphics_clock"`
GpuMemClock string `xml:"mem_clock"`
}
#+end_src
** Docker Module
:PROPERTIES:
:CUSTOM_ID: docker-module
:END:
This section provides information on how the server module interacts
with the docker containers. The server calls 2 routes which either
creates or removes the docker container. Docker has a huge advantage
because it takes less than 20 seconds to spin up a new container once
it's built and executed at least once. For docker operations a separate
module/package has been created. The following subtopics will provide
more information on how this package works.
*** Docker Api
:PROPERTIES:
:CUSTOM_ID: docker-api
:END:
For this the api has been taken from the official docker repository. To
be more specific it is the client module in the official docker
repository. Docker was built using Go. During this project Docker
functions could be directly called from the docker repository. The
Docker api initially ensures that it can detect the docker environment
variables. Once detected, it can execute various functions from the
docker client module. The reason the docker api was selected was to
detect and handle errors better.
*** Docker Image
:PROPERTIES:
:CUSTOM_ID: docker-image
:END:
The docker image used to spin up the containers is called
ConSol/docker-headless-vnc-container. The following container was
modified to open SSH ports for an SSH connection. The following docker
image runs ubuntu 16. The reason this image was chosen as a default is
because if the client wants to access the container in the form of a
desktop environment. This image would allow the client to do so from
just a browser.
*** Build container
:PROPERTIES:
:CUSTOM_ID: build-container
:END:
This function pulls the docker image locally and builds the image.
Initially there is a timeout function to ensure that building the image
does not take too long to build. The next phase would be based on the
path to get the DockerFile. The tag name of the container is set as
p2p-ubuntu as default. Once the following is set then the docker build
command is executed.
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
After building the container it needs to be executed for the user to
access the container and do certain operations. The docker
package/module has a function to do this. The function takes in the
docker environment as a parameter and also the docker struct. The docker
struct has information such as the TCP ports which are supposed to be
open and whether the docker container should have the GPU hooked to it
or not. Based on the appropriate information provided ,the docker image
gets started. The Image gets started by interacting with the docker
client modules. When hooking the GPU the docker run command is called
from the shell. This is because the docker Api does not support the GPU
module yet. When the container is executed for the first time it takes
more than 10 minutes to build. From the second time onwards it takes
only 10 seconds to run.
*** Stop and remove container
:PROPERTIES:
:CUSTOM_ID: stop-and-remove-container
:END:
This implementation here ensures that the docker is stopped, and the
container is removed. This is to ensure it does not utilize server
resources when it is not being used, or the task that is intended to be
executed is complete. To run this function all that is needed is the
docker container ID. If the function is successful it returns a string
that says success.
*** Ports json file
:PROPERTIES:
:CUSTOM_ID: ports-json-file
:END:
This file will help map internal ports inside a container to external
ports inside a container. A common example would be the SSH port which
is port 22 inside the docker container and is mapped to random TCP port
outside container so that any external machines can directly connect
into the container. The below representation mentions of where the
ports.json file is located and also the format of that file.
#+begin_example
|_ <Container name>
|_ Dockerfile
|_ description.txt
|_ ports.json // The ports file
#+end_example
Format of the ports.json file
#+begin_example
{
"Port": [
{
"PortName": "<Port name>",
"InternalPort": <internal port>,
"Type": "<tcp/udp>",
"ExternalPort": <external port>,
"IsUsed": "<boolean value (i.e true or false)>",
"Description": "<description about the port>"
}, ... n
]
}
#+end_example

12
Docs/DocsDeprecated/Virtualization Normal file
View File

@@ -0,0 +1,12 @@
Virtualization Design
======================
The virtualization tool will be treated as a separate module. In our implementation
we will use docker as it's easier to configure and set.
Methods to be created
- Build OS Image from DockerFile
- Run Image Built
- Possibility to kill image by server admin or client side user.
- Track stats of the docker container by server admin or client side user.

800
Docs/DocsDeprecated/docs.org Normal file
View File

@@ -0,0 +1,800 @@
yes* Chapter 1: Introduction
:PROPERTIES:
:CUSTOM_ID: chapter-1-introduction
:END:
** Abstract
:PROPERTIES:
:CUSTOM_ID: abstract
:END:
This project focuses on creating a framework on running heavy tasks that
a regular computer cannot run easily such as graphically demanding video
games, rendering 3D animations , protein folding simulations. In this
project the major focus will not be on the financial incentive part. A
peer to peer network will be created to help run tasks decentrally,
increasing bandwidth for running tasks. To ensure the tasks in the peer
to peer network do not corrupt the server 0S (Operating System), they
will be executed in a virtual environment in the server.
The main aim of this project was to create a custom peer to peer
network. The user acting as the client has total flexibility on how to
batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the
containers at any point of time.
** Motivation
:PROPERTIES:
:CUSTOM_ID: motivation
:END:
Many of the users rely on our PC / Laptop or servers that belong to a
server farm to run heavy tasks and with the demand of high creativity
requires higher computing power. Buying a powerful computer every few
years to run a bunch of heavy tasks which are not executed as frequently
to reap the benefits can be inefficient utilization of hardware. On the
other end, renting servers to run these heavy tasks can be really
useful. Ethically speaking this is leading to monopolisation of
computing power similar to what is happening in the web server area. By
using peer to peer principles it is possible to remove the
monopolisation factor and increase the bandwidth between the client and
server.
* Installation
:PROPERTIES:
:CUSTOM_ID: installation
:END:
Over here we will cover the basic steps to get the server and client
side running.
** Latest release install
:PROPERTIES:
:CUSTOM_ID: latest-release-install
:END:
https://github.com/Akilan1999/p2p-rendering-computation/releases
** Install from Github master branch
:PROPERTIES:
:CUSTOM_ID: install-from-github-master-branch
:END:
*** Install Go lang
:PROPERTIES:
:CUSTOM_ID: install-go-lang
:END:
The entire the implementation of this project is done using Go lang.
Thus, we need go lang to compile to code to a binary file.
[[https://golang.org/doc/install][Instructions to install Go lang]]
*** Install Docker
:PROPERTIES:
:CUSTOM_ID: install-docker
:END:
In this project the choice of virtualization is Docker due to it's wide
usage in the developer community. In the server module we use the Docker
Go API to create and interact with the containers.
[[https://docs.docker.com/get-docker/][Instructions to install docker]]
[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions
to install docker GPU]]
#+begin_example
// Do ensure that the docker command does not need sudo to run
sudo chmod 666 /var/run/docker.sock
#+end_example
*** Build Project and install project
:PROPERTIES:
:CUSTOM_ID: build-project-and-install-project
:END:
To set up the internal dependencies and build the entire go code into a
single binary
#+begin_example
make install
#+end_example
**** For Windows
:PROPERTIES:
:CUSTOM_ID: for-windows
:END:
To set up P2PRC on Windows, simply run this batch file. *Make sure you
are not in admin mode when running this.*
#+begin_example
.\install.bat
#+end_example
*** Add appropriate paths to =.bashrc=
:PROPERTIES:
:CUSTOM_ID: add-appropriate-paths-to-.bashrc
:END:
#+begin_example
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
#+end_example
*** Set up configuration file
:PROPERTIES:
:CUSTOM_ID: set-up-configuration-file
:END:
#+begin_example
make configfile
#+end_example
Open the config file =config.json= and add the IPv6 address if you have
one.
*** Test if binary works
:PROPERTIES:
:CUSTOM_ID: test-if-binary-works
:END:
#+begin_example
p2prc --help
#+end_example
**** Output:
:PROPERTIES:
:CUSTOM_ID: output
:END:
#+begin_example
NAME:
p2p-rendering-computation - p2p cli application to create and access VMs in other servers
USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
<version no>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--Server, -s Starts server (default: false) [$SERVER]
--UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST]
--ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS]
--AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER]
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
--Specification value, --specs value Specs of the server node [$SPECS]
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--help, -h show help (default: false)
--version, -v print the version (default: false)
#+end_example
--------------
* Using basic commands
:PROPERTIES:
:CUSTOM_ID: using-basic-commands
:END:
*** Start as a server
:PROPERTIES:
:CUSTOM_ID: start-as-a-server
:END:
Do ensure you have Docker installed for this
#+begin_example
p2prc -s
#+end_example
*** View server Specification
:PROPERTIES:
:CUSTOM_ID: view-server-specification
:END:
#+begin_example
p2prc --specs=<ip address>
#+end_example
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
use the =--gpu= if you know the other machine has a gpu.
#+begin_example
p2prc --touch=<server ip address> -p <number of ports> --gpu
#+end_example
*** Remove container
:PROPERTIES:
:CUSTOM_ID: remove-container
:END:
The docker id is present in the output where you create a container
#+begin_example
p2prc --rm=<server ip address> --id=<docker container id>
#+end_example
*** Adding servers to ip table
:PROPERTIES:
:CUSTOM_ID: adding-servers-to-ip-table
:END:
#+begin_example
p2prc --as=<server ip address you want to add>
#+end_example
*** Update ip table
:PROPERTIES:
:CUSTOM_ID: update-ip-table
:END:
#+begin_example
p2prc --us
#+end_example
*** List Servers
:PROPERTIES:
:CUSTOM_ID: list-servers
:END:
#+begin_example
p2prc --ls
#+end_example
*** View Network interfaces
:PROPERTIES:
:CUSTOM_ID: view-network-interfaces
:END:
#+begin_example
p2prc --ni
#+end_example
*** Viewing Containers created Client side
:PROPERTIES:
:CUSTOM_ID: viewing-containers-created-client-side
:END:
#+begin_example
p2prc --tc
#+end_example
[[file:ClientImplementation.md#tracking-containers][read more on
tracking containers]]
*** Running plugin
:PROPERTIES:
:CUSTOM_ID: running-plugin
:END:
#+begin_example
p2prc --plugin <plugin name> --id <container id or group id>
#+end_example
*** Create group
:PROPERTIES:
:CUSTOM_ID: create-group
:END:
#+begin_example
p2prc --cgroup
#+end_example
*** Add container to group
:PROPERTIES:
:CUSTOM_ID: add-container-to-group
:END:
#+begin_example
p2prc --group <group id> --id <container id>
#+end_example
*** View groups
:PROPERTIES:
:CUSTOM_ID: view-groups
:END:
#+begin_example
p2prc --groups
#+end_example
*** View specific group
:PROPERTIES:
:CUSTOM_ID: view-specific-group
:END:
#+begin_example
p2prc --group <group id>
#+end_example
*** Delete container from group
:PROPERTIES:
:CUSTOM_ID: delete-container-from-group
:END:
#+begin_example
p2prc --rmcgroup --group <group id> --id <container id>
#+end_example
*** Delete entire group
:PROPERTIES:
:CUSTOM_ID: delete-entire-group
:END:
#+begin_example
p2prc --rmgroup <group id>
#+end_example
[[file:ClientImplementation.md#Grouping-Containers][read more on
grouping containers]] ### Extending usecase of P2PRC (Requires a go
compiler to run)
#+begin_example
p2prc --gen <project name> --mod <go module name>
#+end_example
[[file:GenerateImplementation.md][read more about the generate module]]
*** Pulling plugin from a remote repo
:PROPERTIES:
:CUSTOM_ID: pulling-plugin-from-a-remote-repo
:END:
#+begin_example
p2prc --pp <repo link>
#+end_example
*** Deleting plugin from the plugin directory
:PROPERTIES:
:CUSTOM_ID: deleting-plugin-from-the-plugin-directory
:END:
#+begin_example
p2prc --rp <plugin name>
#+end_example
*** Added custom metadata about the current node
:PROPERTIES:
:CUSTOM_ID: added-custom-metadata-about-the-current-node
:END:
#+begin_example
p2prc --amd "custom metadata"
#+end_example
--------------
* Using Plugins
:PROPERTIES:
:CUSTOM_ID: using-plugins
:END:
This feature is still Under Development:
[[file:PluginImplementation.md][Read more on the implementation]]
**** Dependencies
:PROPERTIES:
:CUSTOM_ID: dependencies
:END:
- Ansible:
- Debian/ubuntu: =sudo apt install ansible=
- Others:
[[https://ansible-tips-and-tricks.readthedocs.io/en/latest/ansible/install/][Installation
link]]
**** Run Test Cases
:PROPERTIES:
:CUSTOM_ID: run-test-cases
:END:
- Generate Test Case Ansible file
- =make testcases=
- Enter inside plugin directory and run tests.
#+begin_quote
[!NOTE] That docker needs to installed and needs to run without sudo.
Refer the section [[#install-docker][Install Docker]]. - =cd plugin= -
=go test .=
#+end_quote
* P2P Module Implementation
:PROPERTIES:
:CUSTOM_ID: p2p-module-implementation
:END:
The P2P module (i.e Peer to Peer Module) is responsible for storing the
IP table and interacting with the IP table. In the following
implementation of the P2P module ,the IP table stores information about
servers available in the network. The other functionality the P2P module
takes care of is doing the appropriate speed tests to the servers in the
IP table. This is for informing the users about nodes which are close by
and nodes which have quicker uploads and downloads speeds. The module is
responsible to ensure that there are no duplicate server IPs in the IP
table and to remove all server IPs which are not pingable.
#+caption: UML diagram of P2P module
[[file:images/p2pmoduleArch.png]]
The peer to peer implementation was built from scratch. This is because
other peer to peer libraries were on the implementation of the
Distributed hash table. At the current moment all those heavy features
are not needed because the objective is to search and list all possible
servers available. The limitation being that to be a part of the network
the user has to know at least 1 server. The advantage of building from
scratch makes the module super light and possibility for custom
functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
** IP Table
:PROPERTIES:
:CUSTOM_ID: ip-table
:END:
The ip table file is a json as the format with a list of servers ip
addresses, latencies, downloads and uploads speeds. The functions
implemented include read file, write file and remove duplicate IP
addresses. The remove duplicate IP address function exists because
sometimes servers IP tables can have the same ip addresses as what the
client has. The path of the IP table json file is received from the
configuration module.
#+begin_src json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>",
}
]
}
#+end_src
*** Latency
:PROPERTIES:
:CUSTOM_ID: latency
:END:
The latency is measured in milliseconds. The route /server_info is
called from the server and time it takes to provide a json response is
recorded.
** NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src
* Language Bindings
:PROPERTIES:
:CUSTOM_ID: language-bindings
:END:
[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]]
refers to wrappers to bridge 2 programming languages. This is used in
P2PRC to extend calling P2PRC functions in other programming languages.
Currently this is done by generating =.so= and =.h= from the Go
compiler.
** How to build shared object files
:PROPERTIES:
:CUSTOM_ID: how-to-build-shared-object-files
:END:
**** The easier way
:PROPERTIES:
:CUSTOM_ID: the-easier-way
:END:
#+begin_src sh
# Run
make sharedObjects
#+end_src
**** Or the direct way
:PROPERTIES:
:CUSTOM_ID: or-the-direct-way
:END:
#+begin_src sh
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
#+end_src
**** If successfully built:
:PROPERTIES:
:CUSTOM_ID: if-successfully-built
:END:
#+begin_src sh
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
#+end_src
** Workings under the hood
:PROPERTIES:
:CUSTOM_ID: workings-under-the-hood
:END:
Below are a sample set of commands to open the bindings implementation.
#+begin_example
# run
cd Bindings/
# list files
ls
# search for file
Client.go
#+end_example
*** In Client go
:PROPERTIES:
:CUSTOM_ID: in-client-go
:END:
There a few things to notice which are different from your standard Go
programs:
**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required.
:PROPERTIES:
:CUSTOM_ID: we-import-c-which-means-cgo-is-required.
:END:
#+begin_src go
import "C"
#+end_src
**** 2. All functions which are required to be called from other programming languages have comment such as.
:PROPERTIES:
:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as.
:END:
#+begin_src go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
#+end_src
**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
:PROPERTIES:
:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure.
:END:
#+begin_src go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
#+end_src
**** It is easy to notice that:
:PROPERTIES:
:CUSTOM_ID: it-is-easy-to-notice-that
:END:
- =ConvertStructToJSONString(<go object>)=: This is a helper function
that convert a go object to JSON string initially and converts it to
=CString=.
- =(output *C.char)=: This is the return type for most of the functions.
**** A Pseudo code to refer to the common function implementation shape could be represented as:
:PROPERTIES:
:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as
:END:
#+begin_example
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
#+end_example
** Current languages supported
:PROPERTIES:
:CUSTOM_ID: current-languages-supported
:END:
- Python
*** Build sample python program
:PROPERTIES:
:CUSTOM_ID: build-sample-python-program
:END:
The easier way
#+begin_src sh
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ p2prc.py
#+end_src
Above shows a generated folder which consists of a folder called
"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files.
=p2prc.py= refers to a sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python, This generated
folder can copied and created as a new git repo for P2PRC extensions
scripted or used a reference point as proof of concept that P2PRC can be
called from other programming languages.
* Config Implementation
:PROPERTIES:
:CUSTOM_ID: config-implementation
:END:
The configuration module is responsible to store basic information of
absolute paths of files being called in the Go code. In a full-fledged
Cli the configuration file can be found in the directory /etc/ and from
there points to location such as where the IP table file is located. In
the future implementation the config file will have information such as
number of hops and other parameters to tweak and to improve the
effectiveness of the peer to peer network. The configuration module was
implemented using the library Viper. The Viper library automates
features such as searching in default paths to find out if the
configuration file is present. If the configuration file is not present
in the default paths then it auto generates the configuration file. The
configurations file can be in any format. In this project the
configuration file was generated using JSON format.
#+begin_src json
{
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
#+end_src
* Abstractions
:PROPERTIES:
:CUSTOM_ID: abstractions
:END:
The Abstractions package consists of black-boxed functions for P2PRC.
** Functions
:PROPERTIES:
:CUSTOM_ID: functions
:END:
- =Init(<Project name>)=: Initializes P2PRC with all the needed
configurations.
- =Start()=: Starts p2prc as a server and makes it possible to extend by
adding other routes and functionality to P2PRC.
- =MapPort(<port no>)=: On the local machine the port you want to export
to world.
- =StartContainer(<ip address>)=: The machine on the p2p network where
you want to spin up a docker container.
- =RemoveContainer(<ip address>,<container id>)=: Terminate container
based on the IP address and container name.
- =GetSpecs(<ip address>)=: Get specs of a machine on the network based
on the IP address.
- =ViewIPTable()=: View the IP table which about nodes in the network.
- =UpdateIPTable()=: Force update IP table to learn about new nodes
faster.
* NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src