From 3068e7d2655dbbc338af11ef781bd1110ec77d65 Mon Sep 17 00:00:00 2001 From: Akilan Date: Sun, 11 Apr 2021 01:56:35 +0400 Subject: [PATCH 1/5] added docs --- Docs/Client.md | 29 +++++++++++++++++++++ Docs/Installation.md | 60 ++++++++++++++++++++++++++++++++++++++++++++ Docs/P2P.md | 29 +++++++++++++++++++++ Docs/Problems.md | 35 ++++++++++++++++++++++++++ Docs/README.md | 7 ++++++ Docs/Virtualization | 12 +++++++++ 6 files changed, 172 insertions(+) create mode 100644 Docs/Client.md create mode 100644 Docs/Installation.md create mode 100644 Docs/P2P.md create mode 100644 Docs/Problems.md create mode 100644 Docs/README.md create mode 100644 Docs/Virtualization diff --git a/Docs/Client.md b/Docs/Client.md new file mode 100644 index 0000000..f8099e1 --- /dev/null +++ b/Docs/Client.md @@ -0,0 +1,29 @@ +# Client Module +This module is incharge of communicating with the server and receiving the appropriate information back from the server. +Note: To [read more about the functions](https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation@v0.0.0-20210404191839-6a046babcb02/client) + +## Functions of the Client Module +- [Interact with the Server Api](#functions-of-the-client-module) +- [Decision maker on how the ip table is created or updated](#decision-maker-on-how-the-ip-table-is-created-or-updated) + + +## Interact with the Server Api +This sections talks about the functionality implemented till now. +- The client can start docker containers using the function StartContainer. +This functions calls the route: +``` +http://:/startcontainer +``` +TODO: Outputs and how it's printed + +## Decision maker on how the IP table is created or updated +- Does a local speedtest to verify and see if the server IP's in the IP table +are pingable. +- Downloads the Servers IP table. +- Tries to ping the servers IP Table addresses. +- If it's pingable then it's added as a new entry in the IP table. +- The following steps occurs in the clients IP table. +- To ensure that the same servers are not being called to update the IP table. There is +a temporary list of IP address which have already been called in relation to updating the +IP table. +- Based on the current implementation there will 3 hops done to update the IP table. diff --git a/Docs/Installation.md b/Docs/Installation.md new file mode 100644 index 0000000..cfbc848 --- /dev/null +++ b/Docs/Installation.md @@ -0,0 +1,60 @@ +# Installation + +Over here we will cover the basic steps to get the server and client side running. + +### Install Go lang +The entire the implementation of this project is done using Go lang. +Thus, we need go lang to compile to code to a binary file. +[Instructions to install Go lang](https://golang.org/doc/install) +### Install Docker +In this project the choice of virtualization is Docker due to it's wide usage +in the developer community. In the server module we use the Docker Go API to create and +interact with the containers. + +[Instructions to install docker](https://docs.docker.com/get-docker/) +```` +Do ensure that the docker command does not need sudo to run +```` + +### Build Project +To set up the internal dependencies and build the entire go code +into a single binary +``` +make build +``` + +### Test if binary works +``` +./p2p-rendering-computation --help +``` +#### Output: +``` +NAME: + p2p-rendering-computation - p2p cli application to create and access VMs in other servers + +USAGE: + p2p-rendering-computation [global options] command [command options] [arguments...] + +VERSION: + 1.0.0 + +COMMANDS: + help, h Shows a list of commands or help for one command + +GLOBAL OPTIONS: + --Mode value Specifies mode of running (default: "client") [$P2P_MODE] + --ListServers List servers which can render tasks (default: false) [$LIST_SERVERS] + --CreateVM value Creates Docker container on the selected server [$CREATE_VM] + --FilePath Testing for absolute path (default: false) + --help, -h show help (default: false) + --version, -v print the version (default: false) + +``` + +### Add command to .bashrc or .zshrc +``` +export PATH=~//p2p-redering-computation:${PATH} +``` + + +#### Note: The steps on how to use the commands will be added later. diff --git a/Docs/P2P.md b/Docs/P2P.md new file mode 100644 index 0000000..e6a33b3 --- /dev/null +++ b/Docs/P2P.md @@ -0,0 +1,29 @@ +# P2P (Peer to Peer module) +In this repository the P2P module has been designed from sratch at the point of this implementation. +[More about function implementation](https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation@v0.0.0-20210404191839-6a046babcb02/p2p) + +## Terminology + 1. IPTable: Refers to a json file which stores information about the current servers avaliable with the speedtest results ran from the Node that triggered it. + ``` + { + "ip_address": [ + { + "ipv4": "localhost", + "latency": 14981051, + "download": 8142.122540206258, + "upload": 3578.766512629995, + } + ] +} + ``` + + +## Responsibility +- To perform speed test to determine best node to connect +- To ensure the IP table has nodes which are pingable +- Using techniques such as [UPNP](https://en.wikipedia.org/wiki/Universal_Plug_and_Play). Still under development +- Port Forwarding (To be introducted in a future release) + + +## Note: +If you are running in server mode it is recommended to use [DMZ](https://routerguide.net/when-and-how-to-setup-dmz-host-for-home-use/) to bypass the [NAT](https://en.wikipedia.org/wiki/Network_address_translation). diff --git a/Docs/Problems.md b/Docs/Problems.md new file mode 100644 index 0000000..0320f61 --- /dev/null +++ b/Docs/Problems.md @@ -0,0 +1,35 @@ +# Problems in implementation + +### Number of Devices in the network: +The current implementation has the major flaw which is that +the server has to be in port 8088 to detected in the public +network. As we know most personal networks have a single IP +address. This means we cannot have duplicate ports. A fix can +be to mention the open port on the IP table file. +(Ex: Possible feild to be added) +``` +{ + "ip_address": [ + { + "ipv4": "localhost", + "latency": 14981051, + "download": 8142.122540206258, + "upload": 3578.766512629995, + "port": 8088 + } + ] +} +``` + +### Broadcast of container specs +At the moment the container specs are not broadcasted rather +just the machines specs and this module has yet to be tested +rigorously. + +### Intergration with GPU +Certain machines have GPUs present in them which provide a +huge advantage for those who want to do rendering and certain +sort of computation. The Aim is to only allow it to be compatible +with Nvidia. But an better idea would be to provide compatability +with multiple GPU providers. + diff --git a/Docs/README.md b/Docs/README.md new file mode 100644 index 0000000..ca51cb8 --- /dev/null +++ b/Docs/README.md @@ -0,0 +1,7 @@ +# Table of contents +1. [Architecture](#NotImplemented) + 1. [Client](Client.md) + 2. [P2P](P2P.md) + 3. [Server](#NotImplemented) +2. [Installation](Installation.md) +3. [Problems](Problems.md) diff --git a/Docs/Virtualization b/Docs/Virtualization new file mode 100644 index 0000000..c0fa128 --- /dev/null +++ b/Docs/Virtualization @@ -0,0 +1,12 @@ + +Virtualization Design +====================== + +The virtualization tool will be treated as a separate module. In our implementation +we will use docker as it's easier to configure and set. + +Methods to be created +- Build OS Image from DockerFile +- Run Image Built +- Possibility to kill image by server admin or client side user. +- Track stats of the docker container by server admin or client side user. From 2a921a6dc25bfb4a03860fbbf38c1ceba2fe5d75 Mon Sep 17 00:00:00 2001 From: Akilan Date: Sun, 11 Apr 2021 02:00:11 +0400 Subject: [PATCH 2/5] updatesd docs --- Docs/Installation.md | 1 + 1 file changed, 1 insertion(+) diff --git a/Docs/Installation.md b/Docs/Installation.md index cfbc848..208361d 100644 --- a/Docs/Installation.md +++ b/Docs/Installation.md @@ -12,6 +12,7 @@ in the developer community. In the server module we use the Docker Go API to cre interact with the containers. [Instructions to install docker](https://docs.docker.com/get-docker/) +[Instructions to install docker GPU](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker) ```` Do ensure that the docker command does not need sudo to run ```` From 6f0063a26d2409bb09f60b1deebc7f7ef038f498 Mon Sep 17 00:00:00 2001 From: Akilan Date: Thu, 15 Apr 2021 15:55:17 +0400 Subject: [PATCH 3/5] added basics docs for testing p2p module --- Docs/P2P-testing.md | 90 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 Docs/P2P-testing.md diff --git a/Docs/P2P-testing.md b/Docs/P2P-testing.md new file mode 100644 index 0000000..c555baa --- /dev/null +++ b/Docs/P2P-testing.md @@ -0,0 +1,90 @@ +# Testing P2P network + +The objective would be to test the p2p network, and the effectiveness of updating +the ip tables. The objective of would be to give the impression to the client and +server of a Zero configuration setting. For testing there will be a +test network set. In the testing scenario all will be client and +server because the IP table does not store clients IP addresses. + +### Test Network Scenario 1 +The test network consists of 5 nodes acting as a client and server. +The objective would be to have the entire IP table Updated in each node +with interacting with only 1 node twice. Each node has knowledge of +one node only. + +### Test Network Scenario 2 +The second test network has a scenario of a single peer which all the +other nodes connect too. The scenario being when the other nodes +connect to the single server they download information about nodes +that have connected to the server node before. + +### Testing Broadcast Module +For testing the broadcast module 2 types of servers will be +tested. One with a CPU only , another one with a CPU and GPU. +The expected result being that the appropriate results are +visible. + +#### Results (CPU and GPU): +``` +{ + "Hostname": "akilan-Lenovo-IdeaPad-Y510P", + "Platform": "ubuntu", + "CPU": "Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz", + "RAM": 7872, + "Disk": 937367, + "GPU": { + "DriveVersion": "390.141", + "Gpu": { + "GpuName": "GeForce GT 755M", + "BiosVersion": "80.07.A8.00.0F", + "FanSpeed": "N/A", + "Utilization": { + "GpuUsage": "N/A", + "MemoryUsage": "N/A" + }, + "Temperature": { + "GpuTemp": "66 C" + }, + "Clock": { + "GpuClock": "N/A", + "GpuMemClock": "N/A" + } + } + } +} +``` +At the moment of the current implementation v1.0. Nvidia GPU +are only compatible. As the Go code calls the command ``nvidia-smi`` +to get information about the GPU available. + +#### Results (CPU only) +``` +{ + "Hostname": "sv-t1.small.x86-01", + "Platform": "ubuntu", + "CPU": "Intel(R) Atom(TM) CPU C2750 @ 2.40GHz", + "RAM": 7944, + "Disk": 138793, + "GPU": null +} +``` +As the ``nvidia-smi`` interface was not detected it only broadcasts +the CPU specs available. + +### Unit tests +All functions implemented on the P2P module returns type error. +The units test call certain functions and check if the functions +return an error or not. This proved sufficient as the point of +the units tests was code coverage to check if certain functions +return an error. + +#### Functions tested +This sections talks about the function called and represents +code coverage. + +1. ``TestServer_SpeedTest``: Function called LocalSpeedTestIpTable() +2. ``TestReadIpTable``: Function called ReadIpTable() + +The P2P module has a 100% code coverage in unit tests as both the unit +tests call directly or call within the function all the functions used +in the P2P module. From 85f7e926177201c12463be1a77b2d6d4c7881c80 Mon Sep 17 00:00:00 2001 From: Akilan Date: Fri, 16 Apr 2021 22:06:56 +0400 Subject: [PATCH 4/5] new changes --- Docs/P2P-testing.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Docs/P2P-testing.md b/Docs/P2P-testing.md index c555baa..6cdd008 100644 --- a/Docs/P2P-testing.md +++ b/Docs/P2P-testing.md @@ -4,12 +4,13 @@ The objective would be to test the p2p network, and the effectiveness of updatin the ip tables. The objective of would be to give the impression to the client and server of a Zero configuration setting. For testing there will be a test network set. In the testing scenario all will be client and -server because the IP table does not store clients IP addresses. +server because the IP table does not store clients IP addresses. At current +number of hopes would be 3 as default. ### Test Network Scenario 1 The test network consists of 5 nodes acting as a client and server. The objective would be to have the entire IP table Updated in each node -with interacting with only 1 node twice. Each node has knowledge of +with interacting with only 1 node once. Each node has knowledge of one node only. ### Test Network Scenario 2 From b8816590a0b9ce55baa9562370203f19abd11143 Mon Sep 17 00:00:00 2001 From: Akilan Selvacoumar <31743758+Akilan1999@users.noreply.github.com> Date: Sat, 17 Apr 2021 00:51:26 +0400 Subject: [PATCH 5/5] Update P2P-testing.md --- Docs/P2P-testing.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/Docs/P2P-testing.md b/Docs/P2P-testing.md index 6cdd008..fc2a873 100644 --- a/Docs/P2P-testing.md +++ b/Docs/P2P-testing.md @@ -13,6 +13,15 @@ The objective would be to have the entire IP table Updated in each node with interacting with only 1 node once. Each node has knowledge of one node only. +![p2pscenario1](https://user-images.githubusercontent.com/31743758/115069627-e4aa8c80-9f04-11eb-8402-706a3407f0e8.png) +Fig 1.0 Visual Representation of testnet scenario 1 + +#### Result +All nodes except node 1 where able to have information of IP addresses in the test net. This was due to the reason of 3 hops +set as default. Node 1 had in it's IP table IP addresses of Node 2, Node 3, Node 4. Once the number of hops was set to 4 objective +of the test was acheived. + + ### Test Network Scenario 2 The second test network has a scenario of a single peer which all the other nodes connect too. The scenario being when the other nodes @@ -70,7 +79,13 @@ to get information about the GPU available. } ``` As the ``nvidia-smi`` interface was not detected it only broadcasts -the CPU specs available. +the CPU specs available. + +### SpeedTests +The speed test has 3 parameters which are Ping , upload and download. The tests check if +the results returned are approximately correct. The ping at the moment returns the correct +result. The upload and download returned are inccorect at the moment, This is due incorrect +implementation in for timer and will be patched in future versions. ### Unit tests All functions implemented on the P2P module returns type error. @@ -89,3 +104,4 @@ code coverage. The P2P module has a 100% code coverage in unit tests as both the unit tests call directly or call within the function all the functions used in the P2P module. +