236 Commits

Author SHA1 Message Date
2283736cc6 added language bindings, python implementation and docs for the feature add
root node
2025-04-28 23:15:35 +01:00
51b253df15 added feature to add root node 2025-04-28 22:22:29 +01:00
b9f5dec9de added static server 2025-04-25 23:53:00 +01:00
30b464449c added basic static server 2025-04-25 23:50:56 +01:00
ffd711f448 added run docs server locally 2025-04-25 23:40:18 +01:00
0d01d70c2b renamed docs directory 2025-04-25 23:22:39 +01:00
6b1edd3d8a revert back docs directory 2025-04-25 23:18:19 +01:00
5cf1e144aa moved docs to root folder 2025-04-25 23:07:24 +01:00
Akilan Selvacoumar
f605e2e06b Merge pull request #126 from Akilan1999/simulation
Haskell and Documentation fixes
2025-04-25 22:31:32 +01:00
873caa911c modified docs 2025-04-25 22:27:39 +01:00
73a470e201 comment out config behavoir since only used for prints 2025-04-25 21:23:42 +01:00
805514edfa uniform variable naming haskell 2025-04-25 20:56:54 +01:00
Akilan Selvacoumar
5e3a4e8b7a Merge pull request #125 from Akilan1999/python-addcustom-information
Python addcustom information
2025-04-25 13:05:38 +01:00
5f8407a37f added comment 2025-04-24 23:28:19 +01:00
a6939ec35e added python function call add custom information 2025-04-24 23:15:02 +01:00
Akilan Selvacoumar
c445ac88c4 Merge pull request #124 from Akilan1999/python-bindings
Python bindings (Base function calls)
2025-04-24 22:17:02 +01:00
9075ab5967 added python bindings 2025-04-24 15:51:47 +01:00
125d113ade added base python bindings 2025-04-07 20:25:25 +01:00
e7e0641f31 removed syscall kill 2025-04-07 18:52:18 +01:00
05655e2da1 modified README 2025-02-27 17:21:21 +00:00
ed8d2be205 added version 3 2025-02-27 17:19:56 +00:00
Akilan Selvacoumar
30cd9c1901 Update README.md 2025-01-29 17:17:02 +00:00
d9d5ebe475 Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2025-01-29 17:12:00 +00:00
23abbc8f90 added base tutorial for running nix 2025-01-29 17:11:43 +00:00
Akilan Selvacoumar
5c0bcfd4cf Update README.md 2025-01-28 16:17:36 +00:00
9b255265c4 added docs changes 2025-01-28 16:15:42 +00:00
075d1850f6 Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2025-01-27 22:41:12 +00:00
38bef975d5 added initial blog post 2025-01-27 22:41:03 +00:00
Akilan Selvacoumar
5cbc9842b4 Update README.md 2025-01-27 21:26:44 +00:00
4479a79912 pushed commit for remote map port 2025-01-24 22:35:50 +00:00
88a4e02e30 added index.html 2025-01-21 21:25:58 +00:00
ddd6864741 latest git ignore changes 2025-01-21 21:25:21 +00:00
Akilan Selvacoumar
3735ab87e0 Merge pull request #122 from Akilan1999/p2prc-docs
P2PRC docs
2025-01-21 21:14:36 +00:00
07811b5b38 removed ODT file 2025-01-21 21:10:49 +00:00
a07f9588fb pushed final docs for initial merge 2025-01-21 21:08:22 +00:00
1bd3a35992 added future idea docs, apologies if there are any typos will be fixed in future versions 2025-01-21 15:29:39 +00:00
8f5a45362a added test doc.md 2025-01-20 17:45:46 +00:00
7ff0607801 modified README 2025-01-19 22:56:17 +00:00
f74e093067 resolved merge conflict 2025-01-19 22:32:03 +00:00
df50c0157e added changes to README 2025-01-19 22:31:10 +00:00
Akilan Selvacoumar
861ed0cb2b Update docs.org 2025-01-19 01:04:26 +00:00
Akilan Selvacoumar
1952010aa2 Update README.org 2025-01-19 01:03:43 +00:00
6863b1386b converted entire docs to org mode 2025-01-19 01:02:45 +00:00
fee39d0afa basic fixes to org files 2025-01-18 17:57:56 +00:00
ca46b20be1 did basic conversion to org mode for the docs folder 2025-01-18 17:30:10 +00:00
f38b6d2871 did a base conversion of P2PRC docs files to org mode 2025-01-18 17:28:08 +00:00
40ffdaec18 did a base conversion of P2PRC docs files to org mode 2025-01-18 17:23:17 +00:00
6d0646e5af Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2025-01-14 21:00:40 +00:00
e378ced781 Testing bindings 2025-01-14 21:00:25 +00:00
Akilan Selvacoumar
c24400f4c7 Merge pull request #121 from xecarlox94/master
Nix Flake development and build environment upgrade
2025-01-12 08:48:03 +00:00
084e1f4a7d finishing up code issues, updated documentation 2025-01-11 21:03:54 +00:00
a0e0a8f73e updating haskell module to nix build and development environment 2025-01-11 13:26:59 +00:00
406947f249 added p2prc package to haskell library 2025-01-11 02:40:08 +00:00
16917b375d trying to import nix flake above 2025-01-11 01:23:33 +00:00
6cce007ae5 adding p2prc import to haskell module 2025-01-11 01:07:13 +00:00
f47e3f33b1 adding flake to project 2025-01-11 00:56:07 +00:00
0c7874913a moved Haskell client library to Bindings folder 2025-01-11 00:44:06 +00:00
3cfd4c56dc Merge branch 'Akilan1999:master' into master 2025-01-11 00:28:19 +00:00
c6fd9f88ba fixing default.nix file 2025-01-07 21:35:17 +00:00
f1e86773f7 trying to wrap program 2025-01-07 21:18:08 +00:00
6683566b3c removing check phase 2025-01-07 20:38:33 +00:00
ea8e7d4073 uncommenting testing code 2025-01-07 20:19:47 +00:00
07dd26e11f fixed testing errors 2025-01-07 01:24:14 +00:00
0c7dadd2a0 fixing check phase 2025-01-05 18:20:20 +00:00
86f66b1595 fixing nix build 2025-01-05 17:43:53 +00:00
46e4ae91e9 Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2025-01-05 14:20:49 +00:00
26c5ffc453 added change in abstractions so that node remote map ports is possible 2025-01-05 14:20:40 +00:00
Akilan Selvacoumar
1e18896b8b Merge pull request #118 from xecarlox94/master
Nix Flake development environment
2025-01-05 14:17:37 +00:00
bedf1431fa nix development environment for P2PRC go program 2025-01-05 14:02:16 +00:00
111b4dd777 adding nix lock file 2025-01-05 13:41:59 +00:00
dfb606e59e moving flake file and additional gitignore changes to branch 2025-01-05 13:41:19 +00:00
Akilan Selvacoumar
67165d4bf6 Update README.md 2024-12-06 23:12:29 +00:00
Akilan Selvacoumar
c3e1f24f60 Update README.md 2024-12-06 23:12:05 +00:00
Akilan Selvacoumar
5cb8d5d50c Update README.md 2024-12-03 00:49:18 +00:00
Akilan Selvacoumar
e8b0834c36 Update README.md 2024-12-03 00:49:02 +00:00
Akilan Selvacoumar
b9c9ca5f42 Update README.md 2024-12-03 00:48:07 +00:00
Akilan Selvacoumar
9ca1587cd9 Update README.md 2024-12-03 00:47:51 +00:00
Akilan Selvacoumar
b20459224a Update README.md 2024-12-03 00:42:58 +00:00
Akilan Selvacoumar
92d50adb7c Update README.md 2024-12-03 00:42:32 +00:00
Akilan Selvacoumar
674a4b28d7 Merge pull request #116 from xecarlox94/haskell
Initial integration of Haskell orchestration module bindings
2024-12-03 00:36:20 +00:00
2ba8355fb1 finished first version of documentation 2024-12-03 00:33:28 +00:00
faabb4651f Merge branch 'Akilan1999:master' into haskell 2024-12-03 00:17:05 +00:00
2b170c7ea1 improving documentation 2024-12-02 00:35:48 +00:00
bf24c30470 improving documentation 2024-12-01 18:16:38 +00:00
0254bbbd43 improving documentation 2024-12-01 17:40:11 +00:00
Akilan Selvacoumar
de5042294f Update README.md 2024-11-25 14:48:25 +00:00
5394e8a657 working on documentation; finished upper structure 2024-11-21 21:58:23 +00:00
95620ba509 removing license 2024-11-21 19:35:37 +00:00
Akilan Selvacoumar
08d14b41a2 Merge pull request #115 from Akilan1999/setup-baremetal
Setup baremetal
2024-11-14 00:23:05 +00:00
6d6c997df5 working on documentation 2024-11-13 21:57:57 +00:00
55d1a5ec3f working on main page documentation 2024-11-13 21:01:15 +00:00
a9ec5332b4 working on documentation 2024-11-13 20:09:03 +00:00
c5ce6de385 working on documentation 2024-11-13 20:07:00 +00:00
277259a65a removed terminal channel check error prone 2024-11-13 19:16:18 +00:00
22c8fefd8d added base mechanism to add public keys from the propogated from the IPTable 2024-11-13 17:55:43 +00:00
623d7b93e0 for update IPTable reduce logic 2024-11-13 17:04:28 +00:00
eb0a7d6b34 for update IPTable reduce logic 2024-11-13 16:59:12 +00:00
2bd39cc450 fixed JSON compile time error 2024-11-13 16:31:01 +00:00
afacc339ec reverted IPTable logic 2024-11-13 16:29:21 +00:00
b3f5d63e16 fixed adding public key default path 2024-11-13 16:20:30 +00:00
f833eba32b fixed untracked files 2024-11-13 16:09:39 +00:00
b55134d194 fixed untracked files 2024-11-13 16:00:12 +00:00
0b0ae2df06 added optmized approach to update IPTable 2024-11-13 15:52:24 +00:00
ec2f169153 refactoring main imports 2024-11-12 17:41:16 +00:00
c3259e957b adding gitignore for haddocks 2024-11-10 20:29:51 +00:00
9265c616ef change module exports 2024-11-10 20:17:44 +00:00
1034a4c0b3 change module exports 2024-11-10 19:33:39 +00:00
030eb6d022 tested example 2024-11-10 19:23:23 +00:00
2e1ca51486 refactored library interface 2024-11-10 18:54:39 +00:00
c89e73eb3b small housekeeping tasks 2024-11-10 18:32:43 +00:00
a11efa624d finished moving code into their respective library modules 2024-11-09 21:47:27 +00:00
a9400d9079 moving API code 2024-11-09 21:28:31 +00:00
b6926e0998 moving code to different files 2024-11-09 21:19:17 +00:00
86dab8919e organised library for p2prc; moved Error, Environment and JSON logic to their own files 2024-11-09 20:50:41 +00:00
4a384f2c5c organising library files 2024-11-09 20:24:42 +00:00
4a103679ad moved code to library side 2024-11-09 19:52:56 +00:00
c481eb0b5c improving cabal build configuration 2024-11-09 19:28:05 +00:00
427f47cdf4 renamed app directory to src directory 2024-11-09 18:53:03 +00:00
780da3bb6d adding dummy library for haskell package 2024-11-09 18:51:10 +00:00
36a00f99b0 adding planning for next stage; adding github issues 2024-11-05 22:38:28 +00:00
6bb45f40fe improved p2prc API 2024-10-26 23:09:51 +01:00
1bbf06685c fixing parsing of json 2024-10-26 22:36:06 +01:00
b3ba2c906e parsing config file started 2024-10-26 19:24:16 +01:00
410d2ae111 implemented safer version of code for shell calls 2024-10-24 23:59:35 +01:00
98988454da fixing readProcess and spawnProcess 2024-10-24 21:44:48 +01:00
2a111bd22d improving safety commands 2024-10-23 23:43:23 +01:00
8081dd8b09 improving spawn command execution 2024-10-23 23:39:59 +01:00
46a21949df improving shell command execution 2024-10-23 23:02:37 +01:00
cc08e35d82 finished cleaning environment functionality 2024-10-21 01:14:13 +01:00
7a195d6c36 fixing gitignore confict 2024-10-20 22:48:22 +01:00
ff105bf9e0 added map port response as JSON output 2024-10-20 22:29:05 +01:00
7a86bb29c9 Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2024-10-20 22:20:19 +01:00
71b23b1e0d save documentation 2024-10-20 22:19:59 +01:00
7ab6471c75 fixing haskell API issue 2024-10-20 22:02:08 +01:00
7855859c3a fixing string issue 2024-10-20 21:54:14 +01:00
182f7fb10f working on cleaning environment and map port function 2024-10-20 21:33:43 +01:00
Akilan Selvacoumar
64f8d77a65 Merge pull request #112 from Akilan1999/cname-entry
Added possibility to add domain name for nodes on P2PRC
2024-10-19 23:45:57 +01:00
5766623d7d adding next API end point goal 2024-10-19 23:22:38 +01:00
02be206221 brainstorming: basic use case 2024-10-19 22:52:39 +01:00
be1e2ec384 finished initial API 2024-10-19 22:39:23 +01:00
10c1548b28 added stdInput and shell options 2024-10-19 20:36:17 +01:00
b2a03312a8 working on haskell API 2024-10-19 20:25:55 +01:00
4a9d1e237f fixed cmd location issue 2024-10-19 18:47:50 +01:00
1e773748fc finished parsing Server information data 2024-10-19 14:26:11 +01:00
65265c08b0 adding data types to inputs and outputs of shell commands 2024-10-19 13:12:38 +01:00
fcda233ddb adding data types to inputs and outputs of shell commands 2024-10-19 12:39:24 +01:00
35b16db6b9 adding todos 2024-10-18 11:37:13 +01:00
a11a6be51a integrating haskell orchestration 2024-10-18 10:59:17 +01:00
5a5985a573 enabled TLS 2024-08-23 04:38:17 +01:00
0a13327551 enabled TLS 2024-08-23 04:32:59 +01:00
00317008a2 fixes 2024-08-23 04:24:43 +01:00
ecd6b0cbf3 changes 2024-08-23 04:20:12 +01:00
dabba14741 changes 2024-08-23 04:18:28 +01:00
fc7ad63c06 debug changes 2024-08-23 04:06:36 +01:00
ac8a3dcdca debug changes 2024-08-23 03:52:41 +01:00
447834b556 debug changes 2024-08-23 03:50:28 +01:00
883b3a5cde debug changes 2024-08-23 03:47:19 +01:00
2c8ef44bd8 removed map check 2024-08-23 03:45:01 +01:00
63d079f71d added make for create map 2024-08-23 03:42:02 +01:00
a4cd96c72f fixed flipped pem files 2024-08-23 03:25:25 +01:00
07dea4ff27 bumped up go version in mod file and added automatic generation of pem certificates 2024-08-23 03:22:51 +01:00
a37f81a47f added defaults for root node 2024-08-23 02:24:54 +01:00
0c1e6edcbc changed state for Proxy default for IPTables 2024-08-23 02:18:06 +01:00
9192901e25 added automatic entry for nodes not behind NAT 2024-08-23 02:02:51 +01:00
5d20e688a8 added domain name as parameter 2024-08-23 01:46:18 +01:00
505ef3b982 added base reverse proxy 2024-08-22 17:14:13 +01:00
9d9e385909 removed 50mb binary file 2024-07-26 21:50:23 +01:00
aa4b18f39d added make clean 2024-03-27 08:28:18 +00:00
f9fb225b91 Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2024-03-27 08:06:44 +00:00
7a5cb2f1e3 changes to go.mod 2024-03-27 08:06:23 +00:00
Akilan Selvacoumar
8ff61422d7 Merge pull request #111 from cr2007/docs-markdown-alerts
Adds Alerts Markdown Support
2024-03-14 10:45:30 +00:00
CSK
0c86c6e673 Adds Markdown Alerts for Notes
Reference: https://github.com/orgs/community/discussions/16925

And adds some minor fixes
2024-03-12 14:41:29 +04:00
CSK
00629b604f Adds hyperlink for creating new issue
And some minor fixes in the README.md file
2024-03-12 13:55:37 +04:00
72484da67b Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2024-02-23 17:50:11 +00:00
7f1e9cbbae added base abstraction functions and docs to P2PRC 2024-02-23 17:49:55 +00:00
Akilan Selvacoumar
c14a0521c9 Update README.md for extending server. 2024-02-10 22:18:46 +00:00
504cdb41cf added easier use of abstractions 2024-02-10 22:05:47 +00:00
c815ed34d7 fixed mapPort for abstractions 2024-02-10 19:59:46 +00:00
60c4bba380 changing root node 2024-01-10 03:55:11 +00:00
3a2cb47b6e Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2023-12-22 00:14:09 +00:00
8c708c6f11 added map port and baremetal mode 2023-12-22 00:13:57 +00:00
83653866e6 added support for baremetal mode 2023-12-20 04:35:04 +00:00
Akilan Selvacoumar
9c94f20ea8 Update README.md 2023-12-16 03:35:20 +00:00
Akilan Selvacoumar
18ae29946d Update README.md 2023-12-16 03:33:17 +00:00
Akilan Selvacoumar
05936a5486 Update README.md 2023-12-16 03:26:56 +00:00
Akilan Selvacoumar
7d5e5cb619 Update README.md 2023-12-16 03:08:03 +00:00
Akilan Selvacoumar
04eea717a6 Update README.md 2023-12-16 03:05:06 +00:00
Akilan Selvacoumar
b6db9147c8 Update README.md 2023-12-16 03:03:49 +00:00
Akilan Selvacoumar
599f7ba2c6 Update README.md 2023-12-16 03:02:03 +00:00
Akilan Selvacoumar
38a05d6ab5 Update README.md 2023-12-16 03:01:10 +00:00
Akilan Selvacoumar
ea08b5c3ea Update Bindings.md 2023-12-16 02:57:44 +00:00
Akilan Selvacoumar
1c496d8be1 Update Bindings.md 2023-12-16 02:55:54 +00:00
Akilan Selvacoumar
a4b9634dc2 Update Bindings.md 2023-12-16 02:39:22 +00:00
Akilan Selvacoumar
65b90d7ce9 Update Bindings.md 2023-12-16 02:17:25 +00:00
Akilan Selvacoumar
95ab45bf62 Update Bindings.md 2023-12-16 02:08:26 +00:00
751f1b15a6 added binding docs 2023-12-15 18:33:28 +00:00
8ccdcaefb5 Merge branch 'master' of github.com:Akilan1999/p2p-rendering-computation 2023-12-15 17:16:10 +00:00
213838715b fixed adding public key as parameter when spawning a container 2023-12-15 17:15:57 +00:00
Akilan Selvacoumar
f899b4e6bb Update README.md 2023-12-15 03:45:00 +00:00
2704caa5e4 added public-private keys for all docker containers 2023-12-15 03:42:06 +00:00
2b6c7a5963 added support to generate Public and Private keys 2023-12-15 02:08:11 +00:00
b78c009f9f modifed shared object calls for python bindings 2023-11-16 15:17:05 +00:00
708eacd1db added support for python bindings 2023-11-10 21:00:24 +00:00
979e856272 pushed changes relating P2PRC changes for python bindings 2023-11-10 18:17:28 +00:00
614a3412c5 pushed changes relating P2PRC changes for python bindings 2023-11-10 18:17:15 +00:00
ea90348fd2 added shared starter tests and added to Makefile to add compile .h and .so files 2023-10-05 17:07:21 +01:00
Akilan Selvacoumar
a07c91c888 Merge pull request #107 from xecarlox94/master
modifying nix config
2023-09-27 17:40:48 +01:00
636646b0a1 modifying nix files, adding adding building, and docker (VIM as well) 2023-09-27 17:38:50 +01:00
3933b29979 merged pull request 2023-09-27 17:28:28 +01:00
96e755bc19 added flag to add custom metadata 2023-09-27 17:27:52 +01:00
Akilan Selvacoumar
7f0a43625b Merge pull request #106 from xecarlox94/master
adding nix shell development environment
2023-09-27 16:29:16 +01:00
29eff7a3af adding nix shell development environment, tested 2023-09-27 16:27:17 +01:00
7ed25ef4b3 added JSON response for IPTables 2023-09-27 15:27:25 +01:00
6b287f2cf1 added docker templating and introducted possbility to change the baseimage name 2023-08-23 22:45:46 +01:00
dfa8713aea fixed tracked container issue 2023-07-29 15:42:05 +01:00
263275d80c remove nodes not pingable 2023-07-15 00:07:14 +01:00
9caf6cde07 saving current changes 2023-07-14 17:54:56 +01:00
d71bd991b0 modified README files and changed version to v2.0.0 2023-07-14 16:51:48 +01:00
e21465b1eb modified README files and changed version to v2.0.0 2023-07-14 16:49:43 +01:00
fb606cf4ff merged from master 2023-07-14 16:31:17 +01:00
136243d6a5 added new IPTable entries 2023-07-14 16:30:20 +01:00
Akilan Selvacoumar
2f6116993b Merge pull request #101 from cr2007/master
Updates the `CITATION.cff` file
2023-07-11 10:27:35 +01:00
Chandrashekhar R
0665a82815 Merge branch 'Akilan1999:master' into master 2023-07-07 09:31:33 +04:00
Chandrashekhar R
a7d741909c Updates the version, release date in CITATION.cff 2023-07-07 09:31:25 +04:00
Akilan Selvacoumar
232c13aca0 Merge pull request #100 from cr2007/master
Polishes up the Documentation
2023-07-06 15:39:38 +01:00
Chandrashekhar R
70aa82a946 Adds buttons to go to the previous/next section of documentation 2023-07-06 14:24:42 +04:00
Akilan Selvacoumar
8f4c2623e3 Update ConfigImplementation.md 2023-06-08 16:50:50 +01:00
Akilan Selvacoumar
a038bf9f20 Update CliImplementation.md 2023-06-08 16:49:27 +01:00
Akilan Selvacoumar
683e20953b Update ServerImplementation.md 2023-06-08 16:43:59 +01:00
Akilan Selvacoumar
8e6a6e246c Update P2PImplementation.md 2023-06-08 16:41:42 +01:00
Akilan Selvacoumar
06e5e1e67f Update ClientImplementation.md 2023-06-08 16:07:38 +01:00
Akilan Selvacoumar
2b191af951 Update ClientImplementation.md 2023-06-08 16:06:38 +01:00
Akilan Selvacoumar
85ac36d51b Update Client.md 2023-06-08 16:04:28 +01:00
Akilan Selvacoumar
64225b85b7 Update Client.md 2023-06-08 16:02:28 +01:00
Akilan Selvacoumar
66b26c5ea2 Update Introduction.md 2023-06-08 16:01:29 +01:00
4d2b903a0f updated docs 2023-06-08 03:39:05 +01:00
155 changed files with 15387 additions and 2350 deletions

BIN
.DS_Store vendored

Binary file not shown.

View File

@@ -1,49 +0,0 @@
name: Deploy Jekyll with GitHub Pages dependencies preinstalled
on:
# Runs on pushes targeting the default branch
push:
branches: ["main", "master"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow one concurrent deployment
concurrency:
group: "pages"
cancel-in-progress: true
jobs:
# Build job
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Pages
uses: actions/configure-pages@v2
- name: Build with Jekyll
uses: actions/jekyll-build-pages@v1
with:
source: ./
destination: ./_site
- name: Upload artifact
uses: actions/upload-pages-artifact@v1
# Deployment job
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1

25
.gitignore vendored
View File

@@ -21,6 +21,31 @@ generate/Test
#ignore windows exe files
*.exe
dist/
# Ignore any sort of logs
logs/
# ignore docker image files
server/docker/containers/
*.h
*.so
# generic folder to ignore
export/
# Any testing file
*test
# Ignore public and private keys
p2prc.publicKey
p2prc.privateKey
p2prc.PublicKeyBareMetal
# Ignore pem files
*.pem
# ignore virtual env file
venv
# Nix and Nix flake files
result
result-*

187
Bindings/Client.go Normal file
View File

@@ -0,0 +1,187 @@
package main
import "C"
import (
"encoding/json"
"time"
"github.com/Akilan1999/p2p-rendering-computation/abstractions"
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
// The Client package where data-types
// are manually converted to the
// to a string so that it can
// be export
// --------------------------------- Container Control ----------------------------------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := abstractions.StartContainer(IP)
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export RemoveContainer
func RemoveContainer(IP string, ID string) (output *C.char) {
err := abstractions.RemoveContainer(IP, ID)
if err != nil {
return C.CString(err.Error())
}
return C.CString("Success")
}
// --------------------------------- Plugin Control ----------------------------------------
// DEPRECATED
////export ViewPlugin
//func ViewPlugin() (output *C.char) {
// plugins, err := plugin.DetectPlugins()
// if err != nil {
// return C.CString(err.Error())
// }
// return ConvertStructToJSONString(plugins)
//}
//
////export PullPlugin
//func PullPlugin(pluginUrl string) (output *C.char) {
// err := plugin.DownloadPlugin(pluginUrl)
// if err != nil {
// return C.CString(err.Error())
// }
// return C.CString("Success")
//}
//
////export DeletePlugin
//func DeletePlugin(pluginName string) (output *C.char) {
// err := plugin.DeletePlugin(pluginName)
// if err != nil {
// return C.CString(err.Error())
// }
// return C.CString("Success")
//}
//
////export ExecutePlugin
//func ExecutePlugin(pluginname string, ContainerID string) (output *C.char) {
// err := plugin.RunPluginContainer(pluginname, ContainerID)
// if err != nil {
// return C.CString(err.Error())
// }
// return C.CString("Success")
//}
// --------------------------------- Get Specs ----------------------------------------
//export GetSpecs
func GetSpecs(IP string) (output *C.char) {
specs, err := abstractions.GetSpecs(IP)
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(specs)
}
//export Init
func Init(customConfig string) (output *C.char) {
init, err := abstractions.Init(customConfig)
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(init)
}
// --------------------------------- P2P Controls -----------------------------------
//export ViewIPTable
func ViewIPTable() (output *C.char) {
table, err := abstractions.ViewIPTable()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(table)
}
//export UpdateIPTable
func UpdateIPTable() (output *C.char) {
err := abstractions.UpdateIPTable()
if err != nil {
return C.CString(err.Error())
}
return C.CString("Success")
}
//export EscapeFirewall
func EscapeFirewall(HostOutsideNATIP string, HostOutsideNATPort string, internalPort string) (output *C.char) {
// Get free port from P2PRC server node
serverPort, err := frp.GetFRPServerPort("http://" + HostOutsideNATIP + ":" + HostOutsideNATPort)
if err != nil {
return C.CString(err.Error())
}
time.Sleep(5 * time.Second)
ExposedPort, err := frp.StartFRPClientForServer(HostOutsideNATIP+":"+HostOutsideNATPort, serverPort, internalPort, "")
if err != nil {
return C.CString(err.Error())
}
return C.CString(ExposedPort)
}
//export MapPort
func MapPort(Port string, DomainName string, ServerAddress string) *C.char {
Address, err := abstractions.MapPort(Port, DomainName, ServerAddress)
if err != nil {
return C.CString(err.Error())
}
return C.CString(Address.EntireAddress)
}
//export CustomInformation
func CustomInformation(CustomInformation string) *C.char {
err := abstractions.AddCustomInformation(CustomInformation)
if err != nil {
return C.CString(err.Error())
}
return C.CString("Success")
}
//export AddRootNode
func AddRootNode(IP string, Port string) *C.char {
err := abstractions.AddRootNode(IP, Port)
if err != nil {
return C.CString(err.Error())
}
return C.CString("Success")
}
// --------------------------------- Controlling Server ----------------------------------------
//export Server
func Server() (output *C.char) {
_, err := abstractions.Start()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString("")
}
// --------------------------------- Helper Functions ----------------------------------------
func ConvertStructToJSONString(Struct interface{}) *C.char {
jsonBytes, err := json.Marshal(Struct)
if err != nil {
return C.CString(err.Error())
}
// Convert the JSON bytes to a string
return C.CString(string(jsonBytes))
}
func main() {}

15
Bindings/Haskell/.gitignore vendored Normal file
View File

@@ -0,0 +1,15 @@
CHANGELOG.md
client
config.json
dist-newstyle
p2p
p2prc.privateKey
p2prc.PublicKeyBareMetal
plugin
server
haddocks/
key.pem
cert.pem
cabal.project.local

2
Bindings/Haskell/.mhsi Normal file
View File

@@ -0,0 +1,2 @@
exit
Exit

120
Bindings/Haskell/README.md Normal file
View File

@@ -0,0 +1,120 @@
<div id="package-header">
<span class="caption">p2prc-0.1.0.0: P2PRC haskell library</span>
- [Contents](index.html)
- [Index](doc-index.html)
</div>
<div id="content">
<div id="module-header">
| | |
|--------------|-----------------------------------------|
| Copyright | Copyright (C) 2006-2024 John MacFarlane |
| License | GNU GPL, version 2 or above |
| Maintainer | John MacFarlane \<jgm@berkeley.edu\> |
| Stability | alpha |
| Portability | portable |
| Safe Haskell | Safe-Inferred |
| Language | GHC2021 |
P2PRC
</div>
<div id="description">
Description
<div class="doc">
This helper module exports the main writers, readers, and data structure
definitions from the Pandoc libraries.
A typical application will chain together a reader and a writer to
convert strings from one format to another. For example, the following
simple program will act as a filter converting markdown fragments to
reStructuredText, using reference-style links instead of inline links:
module Main where
import Text.Pandoc
import Data.Text (Text)
import qualified Data.Text.IO as T
mdToRST :: Text -> IO Text
mdToRST txt = runIOorExplode $
readMarkdown def txt
>>= writeRST def{ writerReferenceLinks = True }
main :: IO ()
main = do
T.getContents >>= mdToRST >>= T.putStrLn
</div>
</div>
<div id="synopsis">
Synopsis
- [runP2PRC](#v:runP2PRC) ::
[MapPortRequest](P2PRC.html#t:MapPortRequest "P2PRC") -\>
[IO]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/System-IO.html#t:IO "System.IO")
()
- <span class="keyword">data</span> [MapPortRequest](#t:MapPortRequest)
= [MkMapPortRequest](#v:MkMapPortRequest)
[Int]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-Int.html#t:Int "Data.Int")
[String]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-String.html#t:String "Data.String")
</div>
<div id="interface">
# Documentation
<div class="top">
<span id="v:runP2PRC" class="def">runP2PRC</span> ::
[MapPortRequest](P2PRC.html#t:MapPortRequest "P2PRC") -\>
[IO]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/System-IO.html#t:IO "System.IO")
() <a href="#v:runP2PRC" class="selflink">#</a>
<div class="doc">
Hello World
</div>
</div>
<div class="top">
<span class="keyword">data</span> <span id="t:MapPortRequest"
class="def">MapPortRequest</span>
<a href="#t:MapPortRequest" class="selflink">#</a>
<div class="subs constructors">
Constructors
| | |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----|
| <span id="v:MkMapPortRequest" class="def">MkMapPortRequest</span> [Int]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-Int.html#t:Int "Data.Int") [String]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-String.html#t:String "Data.String") |   |
</div>
</div>
</div>
</div>
<div id="footer">
Produced by [Haddock](http://www.haskell.org/haddock/) version 2.29.2
</div>

View File

@@ -0,0 +1,4 @@
packages: ./p2prc.cabal
index-state: 2024-11-09T17:56:52Z

3
Bindings/Haskell/dev_run.sh Executable file
View File

@@ -0,0 +1,3 @@
rm -rf *pem client/ plugin/ server/ p2p/ p2prc.[pP]* config.json dist-newstyle/ &&\
cabal clean &&\
cabal run

149
Bindings/Haskell/flake.lock generated Normal file
View File

@@ -0,0 +1,149 @@
{
"nodes": {
"flake-utils": {
"inputs": {
"systems": "systems"
},
"locked": {
"lastModified": 1731533236,
"narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"gomod2nix": {
"inputs": {
"flake-utils": [
"p2prc",
"flake-utils"
],
"nixpkgs": [
"p2prc",
"nixpkgs"
]
},
"locked": {
"lastModified": 1733668782,
"narHash": "sha256-tPsqU00FhgdFr0JiQUiBMgPVbl1jbPCY5gbFiJycL3I=",
"owner": "nix-community",
"repo": "gomod2nix",
"rev": "514283ec89c39ad0079ff2f3b1437404e4cba608",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "gomod2nix",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 0,
"narHash": "sha256-m/lh6hYMIWDYHCAsn81CDAiXoT3gmxXI9J987W5tZrE=",
"path": "/nix/store/wj2qla569hnxwqfc26imv5hqbxc1rc27-source",
"type": "path"
},
"original": {
"id": "nixpkgs",
"type": "indirect"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1735834308,
"narHash": "sha256-dklw3AXr3OGO4/XT1Tu3Xz9n/we8GctZZ75ZWVqAVhk=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "6df24922a1400241dae323af55f30e4318a6ca65",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"p2prc": {
"inputs": {
"flake-utils": "flake-utils",
"gomod2nix": "gomod2nix",
"nixpkgs": "nixpkgs_2"
},
"locked": {
"lastModified": 0,
"narHash": "sha256-f4e4JcQzBS+R+8HqJbQVeecInm2ggInJuSSymj6gtQo=",
"path": "/nix/store/hmk8v0lxifsscll054qsdnzxqfql1998-source",
"type": "path"
},
"original": {
"path": "/nix/store/hmk8v0lxifsscll054qsdnzxqfql1998-source",
"type": "path"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs",
"p2prc": "p2prc",
"utils": "utils"
}
},
"systems": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"systems_2": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"utils": {
"inputs": {
"systems": "systems_2"
},
"locked": {
"lastModified": 1731533236,
"narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
}
},
"root": "root",
"version": 7
}

View File

@@ -0,0 +1,24 @@
{
description = "Nix flake for P2PRC Haskell library";
inputs = {
p2prc.url = "../../";
utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, utils, p2prc }: utils.lib.eachDefaultSystem (system:
let
pkgs = nixpkgs.legacyPackages.${system};
in
{
devShell = pkgs.mkShell {
buildInputs = with pkgs; [
cabal-install
haskell.compiler.ghc96
zlib.dev
p2prc.outputs.packages.${system}.default
];
};
}
);
}

11
Bindings/Haskell/gen_docs.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/bin/bash
rm -rf dist-newstyle/
cabal haddock
rm -rf ../../Docs/haskell
cp -r \
./dist-newstyle/build/x86_64-linux/ghc-9.6.6/p2prc-0.1.0.0/doc/html/p2prc/ \
../../Docs/haskell

155
Bindings/Haskell/lib/API.hs Normal file
View File

@@ -0,0 +1,155 @@
module API
( P2PRCapi(..)
, MapPortRequest(..)
, p2prcAPI
)
where
import System.Directory ( getCurrentDirectory )
import System.Process ( ProcessHandle )
import Data.Aeson ( FromJSON )
import Error
( IOEitherError
)
import JSON
( IPAddressTable(..)
, MapPortResponse(..)
, P2PRCConfig
)
import CLI
( StdInput(..)
, CLIOpt(..)
, eitherErrDecode
, p2prcCmdName
, eitherExecProcess
, eitherExecProcessParser
, spawnProcP2PRC
)
-- import System.Environment (lookupEnv)
-- | Lower level P2PRC Haskell api that exposes basic functionality necessary to joint the network.
data P2PRCapi
= MkP2PRCapi
{ startServer :: IOEitherError ProcessHandle
-- ^ Start server
-- , execInitConfig :: IOEitherError P2PRCConfig
-- ^ Instantiate server configuration
, execListServers :: IOEitherError IPAddressTable
-- ^ List servers in network
, execMapPort :: MapPortRequest -> IOEitherError MapPortResponse
-- ^ Exposes and associates a local TCP port with a remote DNS address
}
-- | This defines the request required to create an association between a TCP socket port and a DNS server in the network. If successful, it makes a resource available in the network.
data MapPortRequest =
MkMapPortRequest -- ^ P2PRC's port allocation request value
Int -- ^ TCP socket number
String -- ^ Network domain name
{-|
This function intiates a pure P2PRC runtime state and builds up a 'P2PRCapi' API instance. It allows a developer to create computing orchestration algorithms using the API primitives.
==== __Example__
The following example show how this function can be used to expose the runtime functionalities:
@
module Main where
import P2PRC
( p2prcAPI
, P2PRCapi(..)
)
main :: IO ()
main =
print "Hello P2PRC"
-- your code logic goes here
where
MkP2PRCapi
{ startServer=startServer
, execMapPort=execMapPort
, execListServers=execListServers
, execInitConfig=execInitConfig
} = p2prcAPI
@
-}
{-# WARNING p2prcAPI "This function is currently unstable because the configuration reading is dependent on the following issue: https://github.com/Akilan1999/p2p-rendering-computation/issues/120" #-}
p2prcAPI :: P2PRCapi
p2prcAPI =
MkP2PRCapi
{ startServer = spawnProcP2PRC p2prcCmdName [ MkOptAtomic "--s" ]
, execListServers =
execProcP2PRCParser [ MkOptAtomic "--ls" ] MkEmptyStdInput
, execMapPort =
\ (MkMapPortRequest portNumber domainName) ->
execProcP2PRCParser
[ MkOptTuple
( "--mp"
, show portNumber
)
, MkOptTuple
( "--dn"
, domainName
)
]
MkEmptyStdInput
-- , execInitConfig = do
-- confInitRes <- execProcP2PRC [ MkOptAtomic "--dc" ] MkEmptyStdInput
-- case confInitRes of
-- (Right _) -> do
-- maybeValue <- lookupEnv "P2PRC"
-- -- TODO: get config file name dynamically
-- --
-- currDirectory <- getCurrentDirectory
-- -- TODO: change values before loading file
-- let fname = currDirectory ++ "/config.json" :: FilePath
-- -- TODO: read config check if file exists
-- configContent <- readFile fname
-- pure $ eitherErrDecode configContent
-- (Left err) -> pure $ Left err
}
where
execProcP2PRCParser ::
FromJSON a =>
[CLIOpt] -> StdInput -> IOEitherError a
execProcP2PRCParser = eitherExecProcessParser p2prcCmdName
-- TODO: GHC question, why does it scope down instead staying generic
-- execProcP2PRC = eitherExecProcess p2prcCmdName

120
Bindings/Haskell/lib/CLI.hs Normal file
View File

@@ -0,0 +1,120 @@
module CLI where
import System.Process
( readProcessWithExitCode
, proc
, createProcess
, terminateProcess
, ProcessHandle
)
import System.Exit
( ExitCode(ExitFailure)
)
import Error
( IOEitherError
, Error(..)
, assignError
)
import Data.Aeson
( FromJSON
, eitherDecode
)
import qualified Data.ByteString.Lazy.Char8 as LBC8
data StdInput
= MkEmptyStdInput
| MkStdInputVal String
instance Show StdInput where
show MkEmptyStdInput = ""
show (MkStdInputVal v) = v
data CLIOpt
= MkEmptyOpts
| MkOptAtomic String
| MkOptTuple (String, String)
type CLIOptsInput = [String]
type CLICmd = String
eitherExecProcessParser ::
FromJSON a =>
CLICmd -> [CLIOpt] -> StdInput -> IOEitherError a
eitherExecProcessParser p2prcCmd opts stdInput =
do
val <- eitherExecProcess p2prcCmd opts stdInput
pure $ case val of
(Right v) -> eitherErrDecode v
(Left e) -> Left e
eitherErrDecode ::
FromJSON a =>
String -> Either Error a
eitherErrDecode = eitherErrorDecode . eitherDecode . LBC8.pack
eitherExecProcess :: CLICmd -> [CLIOpt] -> StdInput -> IOEitherError String
eitherExecProcess cmd opts input =
do
(code, out, err) <-
readProcessWithExitCode
cmd
(optsToCLI opts)
(show input)
pure $ case code of
ExitFailure i -> Left $ MkCLISystemError i cmd err
_ -> Right out
optsToCLI :: [CLIOpt] -> CLIOptsInput
optsToCLI = concatMap _optToCLI
where
_optToCLI :: CLIOpt -> CLIOptsInput
_optToCLI MkEmptyOpts = []
_optToCLI (MkOptAtomic o) = [o]
_optToCLI (MkOptTuple (o, v)) = [o, v]
spawnProcP2PRC :: CLICmd -> [CLIOpt] -> IOEitherError ProcessHandle
spawnProcP2PRC cmd opts =
do
let prc = proc cmd $ optsToCLI opts
creationResult <- createProcess prc
let (_, _, _, ph) = creationResult
case creationResult of
(_, _, Just _, _) -> do
terminateProcess ph
pure $ Left $ MkErrorSpawningProcess cmd
_-> pure $ Right ph
eitherErrorDecode :: Either String a -> Either Error a
eitherErrorDecode esa =
case esa of
(Left s) -> Left $ assignError s
(Right v) -> Right v
-- assumes the program is ran inside the haskell module in p2prc's repo
-- assumes that last path segment is "haskell" and that p2prc binary's name is "p2p-rendering-computation"
p2prcCmdName :: String
p2prcCmdName = "p2prc"

View File

@@ -0,0 +1,161 @@
{-# LANGUAGE OverloadedStrings #-}
module Engine
( runP2PRC
)
where
import Control.Concurrent
import Data.Char (toLower)
import System.IO
import Control.Monad (when)
import System.Process ( terminateProcess )
import API
( P2PRCapi(..)
, MapPortRequest(..)
, p2prcAPI
)
-- URGENT TASKS
--
--
-- TODO: add Haddock documentation
--
-- TODO: P2PRC setup
-- check version P2PRC: only run if version if above a certain value
-- setup p2prc command
-- check if p2prc command is available in environment first
-- otherwise check folder above
--
-- TODO: Fix API TODOS
--
-- TODO: Fix JSON TODOS
--
-- create DSL to start and orchestrate network
--
-- TODO: publish haskell library
--
{-|
This function starts and bootstraps the P2PRC runtime that associates the a specific host's machine port to a DNS address to expose a certain application to the P2PRC network. You will only need to also import the 'MkMapPortRequest' data constructor to represent the this port request.
==== __Example__
This example demonstrates how it can be ran on the IO context:
@
example :: IO ()
example = do
runP2PRC
( MkMapPortRequest 8080 "jose.akilan.io"
)
@
-}
runP2PRC
:: MapPortRequest -- ^ TCP Port Request
-> IO ()
runP2PRC
( MkMapPortRequest
portNumber
domainName
)
=
let
--
-- TODO: add quickcheck testing (quickchecking-dynamic)
--
-- TODO: Change Standard Library
-- TODO: add GDTA syntax to data types
--
-- TODO: need monad transformers to refactor the code
--
--
-- TODO: parse IO arguments;
-- TODO: create DSL from the standard input
--
-- TODO: add use case examples (extra-source_files)
--
-- TODO: setup nix flake package
-- Nix p2prc runtime packaging
-- Perhaps create internal script to run P2PRC from nix flake
-- "nix flake run ..."
-- simplify packaging
--
-- Extra:
--
-- TODO: Error
-- assign error: should parse other error
( MkP2PRCapi
{ startServer = startServer
-- , execInitConfig = execInitConfig
, execListServers = execListServers
, execMapPort = execMapPort
}
) = p2prcAPI
in do
let
-- configValue <- execInitConfig
-- TODO: get name of host server from config json
-- print configValue
-- putStrLn "\n\n\n"
eitherStartProcessHandle <- startServer
case eitherStartProcessHandle of
(Right startProcessHandle) -> do
let sleepNSecs i = threadDelay (i * 1000000)
sleepNSecs 5
outputStr <- execListServers
print outputStr
mapPortOut <- execMapPort $ MkMapPortRequest portNumber domainName
case mapPortOut of
(Right v) -> print v
(Left e) -> print e
-- TODO: work on looping function
--
-- Loop (Run replica of haskell program on different $NODES)
-- - Start server
-- - wait 4 seconds
-- - Identify new node running p2prc with SSH external port exposed
-- - SSH into machine
-- - Use simple File transfer to setup files
-- - Run server
-- - Use remote machine p2prc cmd to map a port using --mp
-- - Return back the exposed public IP and port number back to stdout
exitOnQ $ terminateProcess startProcessHandle
(Left err) -> print err
where
exitOnQ :: IO () -> IO ()
exitOnQ exitF = do
hSetBuffering stdin NoBuffering
c <- getChar
when (toLower c /= 'q') $ exitOnQ exitF
exitF

View File

@@ -0,0 +1,36 @@
module Error
( Error(..)
, IOEitherError
, assignError
) where
{- |
Haskell-side Error value. This type is designed to parse and track System and P2PRC's error signals in a safe and effective manner.
It does have an 'MkUnknownError' value which is meant to be warn about new kinds of error not yet accounted in this client. Github issues and pull requests are very welcome to improve error handling by parsing more types of errors.
-}
data Error
= MkCLISystemError -- ^ This is a CLI System Error
Int -- ^ System error code
String -- ^ Command name executed
String -- ^ Error output
| MkErrorSpawningProcess -- ^ Spawing process error
String -- ^ Spawning executable name
| MkUnknownError -- ^ This is an unparsed P2PRC's error
String -- ^ Unparsed error message
deriving Show
-- | Type synonym for an IO action with either returns an Error or a parsed value
type IOEitherError a = IO ( Either Error a)
assignError :: String -> Error
assignError = MkUnknownError
--
-- TODO: add megaparsec to parse Error Messages
--
-- TODO: add error when internet connection is off
--
-- Left (MkUnknownError "Unexpected end-of-input, expecting JSON value")
-- MkSystemError 1 "/home/xecarlox/Desktop/p2p-rendering-computation/p2p-rendering-computation" "2024/11/09 21:08:06 Get \"http://0.0.0.0:8088/MAPPort?port=3333&domain_name=\": dial tcp 0.0.0.0:8088: connect: connection refused\n"

View File

@@ -0,0 +1,202 @@
{-# LANGUAGE OverloadedStrings #-}
module JSON
( P2PRCConfig(..)
, IPAddressTable(..)
, IPAddress(..)
, ServerInfo(..)
, MapPortResponse(..)
)
where
import Control.Monad ( mzero )
import qualified Data.Text as T
import Data.Aeson
{-# WARNING MapPortResponse "This newtype is unstable at the moment due to the P2PRC's library error handling bug. For more information visit: https://github.com/Akilan1999/p2p-rendering-computation/issues/114#issuecomment-2474737015" #-}
-- ^ This represents P2PRC's response to the TCP port and DNS address allocation. This value will confirm the successful allocation and return information about it.
newtype MapPortResponse
= MkMapPortResponse -- ^ Allocation information value
String -- ^ Column separated Host's IP address and Port String
deriving Show
instance FromJSON MapPortResponse where
parseJSON (Object o) = do
ipAddress <- o .: "IPAddress"
pure $ MkMapPortResponse ipAddress
parseJSON _ = mzero
{-# WARNING P2PRCConfig "This type is unstable at the moment due to the P2PRC's library error handling bug. For more information visit: https://github.com/Akilan1999/p2p-rendering-computation/issues/114#issuecomment-2474737015" #-}
-- | This represents the server configuration that defines its attributes and behaviours in the network, as well as, the location of the runtime persistence artifacts.
newtype P2PRCConfig
= MkP2PRCConfig
{ machineName :: String -- ^ Machine Name
} deriving Show
-- , hostServerPort :: Int
-- , iPV6Address :: Maybe String
-- , proxyPort :: Maybe Int
-- , fRPServerPort :: Bool
-- , behindNAT :: Bool
-- , iPTableKey :: String
-- , bareMetal :: Bool
-- , customConfig :: String
--
-- , iPTable :: String -- File
-- , dockerContainers :: String -- Directory
-- , defaultDockerFile :: String -- Directory
-- , dockerRunLogs :: String -- Directory
-- , speedTestFile :: String -- File
-- , pluginPath :: String -- Directory
-- , trackContainersPath :: String -- File
-- , publicKeyFile :: String -- File
-- , privateKeyFile :: String -- File
-- , pemFile :: String -- File
-- , keyFile :: String -- File
-- , groupTrackContainersPath :: String -- File
-- TODO: p2prc API
--
-- ListServers
-- remove "ip_address" root field if not needed
-- "Nat field" returning a JSON Boolean
-- serverPort as a JSON number
-- baremetalPort as a JSON number
-- have either IPV4 or IPV6 field visible
-- remove "customInformation" if not needed anymore
-- remove "escapeImplementation" if not needed anymore
--
-- Config file
--
-- Fix JSON number: ServerPort
-- Fix: IPV6Address dont show if value does not exist
-- Fix JSON number: ProxyPort to number (dont show if it does not exist)
-- Fix JSON number: fRPServerPort
-- Fix JSON boolean: fRPServerPort
-- Fix JSON boolean: behindNAT
-- Fix JSON boolean: bareMetal
-- remove "customConfig" if not needed
--
-- MapPort
-- to have a dedicated ip address (with type either IPV6 or IPV4 fields)
-- to have a dedicated port field
instance FromJSON P2PRCConfig where
parseJSON (Object o) = do
machineName <- o .: "MachineName"
pure
$ MkP2PRCConfig
{ machineName=machineName
}
parseJSON _ = mzero
{-# WARNING IPAddressTable "This newtype is highly unstable due to undergoing work on improving P2PRC's server api. For more information, visit: https://github.com/Akilan1999/p2p-rendering-computation/issues/114" #-}
-- | This is a wrapper value that parses a json key value from the list of ip addresses in the network.
newtype IPAddressTable
= MkIPAddressTable -- ^ Wrapping constructor
[ServerInfo] -- ^ List current network servers
deriving Show
instance FromJSON IPAddressTable where
parseJSON = withObject "IPAdressTable" $
\ v ->
MkIPAddressTable <$> v .: "ip_address"
{-# WARNING ServerInfo "This type is highly unstable due to undergoing work on improving P2PRC's server api. For more information, visit: https://github.com/Akilan1999/p2p-rendering-computation/issues/114" #-}
{- |
This is a record that keeps track of the current state of every node in the network. It is crucial information required for orchestration strategies.
-}
data ServerInfo =
MkServerInfo
{ name :: T.Text -- ^ Machine name
, ip :: IPAddress -- ^ Machine IP address
, latency :: Int -- ^ Response latency
, download :: Int -- ^ Download speed
, upload :: Int -- ^ Upload speed
, serverPort :: Int -- ^ Server port number
, bareMetalSSHPort :: Maybe Int -- ^ SSH machine port number
, nat :: Bool -- ^ Checking if node is behind a NAT
, escapeImplementation :: Maybe T.Text -- ^ Type of NAT trasversal technique
, customInformation :: Maybe T.Text -- ^ Custom String information
}
deriving Show
-- | This is a simple representation of the IP address of nodes in the network.
data IPAddress
= MkIPv4 String -- ^ IP version 4 address
| MkIPv6 String -- ^ IP version 6 address
deriving Show
instance FromJSON ServerInfo where
parseJSON = withObject "ServerInfo" $
\ o -> do
name <- o .: "Name"
ip4str <- o .: "IPV4"
ip6str <- o .: "IPV6"
latency <- o .: "Latency"
download <- o .: "Download"
upload <- o .: "Upload"
serverPort <- o .: "ServerPort"
bmSshPort <- o .: "BareMetalSSHPort"
nat <- o .: "NAT"
mEscImpl <- o .: "EscapeImplementation"
custInfo <- o .: "CustomInformation"
pure $
MkServerInfo
{ name = name
, ip = getIPAddress ip4str ip6str
, latency = latency
, download = download
, upload = upload
, serverPort = getPortUNSAFE serverPort
, bareMetalSSHPort = getBMShhPort bmSshPort
, nat = getNat nat
, escapeImplementation = mEscImpl
, customInformation = custInfo -- TODO: deal with null value
}
where
getNat :: String -> Bool -- TODO: Change it to normal JSON
getNat ('T':_) = True
getNat _ = False
getBMShhPort :: String -> Maybe Int -- TODO: Dangerous partial function call !!!!!!!!!!!!!!!!!!!
getBMShhPort [] = Nothing
getBMShhPort bmSshPort = Just $ getPortUNSAFE bmSshPort
getPortUNSAFE :: String -> Int -- TODO: Dangerous partial function call !!!!!!!!!!!!!!!!!!!
getPortUNSAFE = read
getIPAddress :: String -> String -> IPAddress
getIPAddress [] ip6 = MkIPv6 ip6
getIPAddress ip4 _ = MkIPv4 ip4

View File

@@ -0,0 +1,94 @@
{-# OPTIONS_HADDOCK show-extensions #-}
{- |
Module : P2PRC
Copyright : Copyright (C) 2024-2024 Jose Fernandes
License : GNU GPL, version 2 or above
Maintainer : Jose Fernandes <jf94.uk@gmail.com>
Stability : beta
Portability : portable
This library provides an interface to the P2Prc runtime.
This Module intends to export the main functions and data type definitions necessary to get started with the P2PRC api.
A minimal application will require the import of 'runP2PRC' function that accepts a 'MapPortRequest' value that exposes a specific port number and associates it with a domain name in the internet.
This is a small template to get quickly get started with this interface. We assume the user has already an application listening on the tcp socket "8080".
> module Main where
>
> import P2PRC
> ( runP2PRC
> , MapPortRequest(MkMapPortRequest)
> )
>
> main :: IO ()
> main =
> runP2PRC
> ( MkMapPortRequest 8080 "jose.akilan.io"
> )
-}
module P2PRC
(
-- * Functions
{- | These are the available functions available to interact with the P2Prc environment, at a lower level of abstraction.
It is intended this way to give freedom to the developer to implement their own orchestration strategies.
-}
runP2PRC
, p2prcAPI
-- * Data Types
{- | This section describes and explains the library's type system, more specifically, the interfaces and primitive types.
-}
-- ** Interface data types
-- | This section gives an overview on the runtime and host machine interfaces.
, P2PRCapi(..)
, P2PRCConfig(..)
, MapPortRequest(..)
, MapPortResponse(..)
-- ** Primitive data types
-- | These types represent the core data that is communicated between requests and the runtime.
, IPAddressTable(..)
, ServerInfo(..)
, IPAddress(..)
, Error(..)
-- ** Type Synonyms
-- | This section is reserved to some useful type synonyms that add significant ergonomics.
, IOEitherError
)
where
import Engine
( runP2PRC
)
import JSON
( IPAddressTable(..)
, ServerInfo(..)
, IPAddress(..)
, MapPortResponse(..)
, P2PRCConfig(..)
)
import API
( MapPortRequest(..)
, P2PRCapi(..)
, p2prcAPI
)
import Error
( Error(..)
, IOEitherError
)

View File

@@ -0,0 +1,66 @@
cabal-version: 3.0
name: p2prc
-- See the Haskell package versioning policy (PVP) for standards
-- guiding when and how versions should be incremented.
-- https://pvp.haskell.org
-- PVP summary: +-+------- breaking API changes
-- | | +----- non-breaking API additions
-- | | | +--- code changes with no API change
version: 0.1.0.0
synopsis: P2PRC haskell library
description: Implements a client interface to the P2PRC networking runtime
author: xecarlox94
maintainer: jf94.uk@gmail.com
category: Network
build-type: Simple
extra-doc-files: README.md
common warnings
ghc-options: -Wall
executable p2prc
import: warnings
hs-source-dirs: src
main-is: Main.hs
other-extensions: OverloadedStrings
build-depends: base
, p2prc
default-language: GHC2021
library
import: warnings
build-depends: base
, text
, aeson
, process
, bytestring
, directory
hs-source-dirs: lib
exposed-modules: P2PRC
other-modules: API
, Engine
, CLI
, JSON
, Error
default-language: GHC2021

View File

@@ -0,0 +1,12 @@
module Main where
import P2PRC
( runP2PRC
, MapPortRequest(MkMapPortRequest)
)
main :: IO ()
main =
runP2PRC
( MkMapPortRequest 8080 "jose.akilan.io"
)

113
Bindings/docs.md Normal file
View File

@@ -0,0 +1,113 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# Bindings
```go
import "github.com/Akilan1999/p2p-rendering-computation/Bindings"
```
## Index
- [func ConvertStructToJSONString\(Struct interface\{\}\) \*C.char](<#ConvertStructToJSONString>)
- [func EscapeFirewall\(HostOutsideNATIP string, HostOutsideNATPort string, internalPort string\) \(output \*C.char\)](<#EscapeFirewall>)
- [func GetSpecs\(IP string\) \(output \*C.char\)](<#GetSpecs>)
- [func Init\(customConfig string\) \(output \*C.char\)](<#Init>)
- [func MapPort\(Port string, DomainName string, ServerAddress string\) \*C.char](<#MapPort>)
- [func RemoveContainer\(IP string, ID string\) \(output \*C.char\)](<#RemoveContainer>)
- [func Server\(\) \(output \*C.char\)](<#Server>)
- [func StartContainer\(IP string\) \(output \*C.char\)](<#StartContainer>)
- [func UpdateIPTable\(\) \(output \*C.char\)](<#UpdateIPTable>)
- [func ViewIPTable\(\) \(output \*C.char\)](<#ViewIPTable>)
<a name="ConvertStructToJSONString"></a>
## func [ConvertStructToJSONString](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L159>)
```go
func ConvertStructToJSONString(Struct interface{}) *C.char
```
<a name="EscapeFirewall"></a>
## func [EscapeFirewall](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L118>)
```go
func EscapeFirewall(HostOutsideNATIP string, HostOutsideNATPort string, internalPort string) (output *C.char)
```
<a name="GetSpecs"></a>
## func [GetSpecs](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L79>)
```go
func GetSpecs(IP string) (output *C.char)
```
<a name="Init"></a>
## func [Init](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L88>)
```go
func Init(customConfig string) (output *C.char)
```
<a name="MapPort"></a>
## func [MapPort](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L137>)
```go
func MapPort(Port string, DomainName string, ServerAddress string) *C.char
```
<a name="RemoveContainer"></a>
## func [RemoveContainer](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L29>)
```go
func RemoveContainer(IP string, ID string) (output *C.char)
```
<a name="Server"></a>
## func [Server](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L148>)
```go
func Server() (output *C.char)
```
<a name="StartContainer"></a>
## func [StartContainer](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L20>)
```go
func StartContainer(IP string) (output *C.char)
```
<a name="UpdateIPTable"></a>
## func [UpdateIPTable](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L109>)
```go
func UpdateIPTable() (output *C.char)
```
<a name="ViewIPTable"></a>
## func [ViewIPTable](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/Bindings/Client.go#L100>)
```go
func ViewIPTable() (output *C.char)
```
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

BIN
Bindings/python/.DS_Store vendored Normal file

Binary file not shown.

129
Bindings/python/library.py Normal file
View File

@@ -0,0 +1,129 @@
import ctypes
from ctypes import Structure, c_char_p, c_int, cdll
from dataclasses import dataclass, astuple
import time
import json
from urllib.parse import urlparse
from typing import List
import subprocess
import uuid
import dacite
import os
import sqlalchemy
import dataclasses
from typing import Union
import schedule
import threading
import requests
p2prc = ctypes.CDLL("SharedObjects/p2prc.so")
p2prc.Init("")
# Start running schedule
schedule.run_pending()
# # Create Sqlite database to track processes
# engine=sqlalchemy.create_engine(f'sqlite:///homeserver.db')
# Global variable
# A global variable will be populated on runtime.
# It is read from P2PRC directly or from a local
# database.
# Node information
# Generated using: https://jsonformatter.org/json-to-python
@dataclass
class Node:
Name: str
MachineUsername: str
IPV4: str
IPV6: str
Latency: int
Download: int
Upload: int
ServerPort: str
BareMetalSSHPort: str
NAT: bool
EscapeImplementation: str
ProxyServer: bool
UnSafeMode: bool
PublicKey: str
CustomInformation: str
@dataclass
class IPAddress:
ip_address: List[Node]
# ----------------------------------------------------------------------------
# ----------------------------- Helper functions -----------------------------
# ----------------------------------------------------------------------------
def StartServer():
# Starting P2PRC as a server mode
p2prc.Server()
# Class to create string to pass as string function
# parameter to shared object file
class go_string(Structure):
_fields_ = [
("p", c_char_p),
("n", c_int)]
# Ensuring local port can escape NAT and responds
# with the public ip and port no.
# If the domain name is specified then the public IP
# can be used as an A name entry.
def P2PRCMapPort(port="",domainname="",serveraddress=""):
port = go_string(c_char_p(port.encode('utf-8')), len(port))
domainname = go_string(c_char_p(domainname.encode('utf-8')), len(domainname))
serveraddress = go_string(c_char_p(serveraddress.encode('utf-8')), len(serveraddress))
# Defining the response type of the GoLang function
# function
p2prc.MapPort.restype = c_char_p
# Calling the Go function
address = p2prc.MapPort(port,domainname,serveraddress)
res = str(address).strip("b'")
return res
# Lists all avaliable nodes in the network
def ListNodes():
# View IP Table information
p2prc.ViewIPTable.restype = c_char_p
ipTable = p2prc.ViewIPTable()
# View IP Table as
ipTableObject = json.loads((str(ipTable).strip("b'")))
dat: IPAddress = dacite.from_dict(IPAddress,ipTableObject)
return dat
# Add a root node to P2RRC and overwrites all other nodes.
# To be only added before the network started and with
# the intention of a fresh protocol.
def AddRootNode(ip="", port="") -> bool:
ip = go_string(c_char_p(ip.encode('utf-8')), len(ip))
port = go_string(c_char_p(port.encode('utf-8')), len(port))
p2prc.AddRootNode.restype = c_char_p
res = p2prc.AddRootNode(ip, port)
if str(res).strip("b'") == "Success":
return True
return False
# python function to pass-through custom
# information to interpret which can be
# interpreted as a DSL.
def AddCustomInformation(message="") -> bool:
message = go_string(c_char_p(message.encode('utf-8')), len(message))
p2prc.CustomInformation.restype = c_char_p
status = p2prc.CustomInformation(message)
if str(status).strip("b'") == "Success":
return True
return False

View File

@@ -0,0 +1,4 @@
dacite
schedule
sqlalchemy
requests

14
Bindings/python/test.py Normal file
View File

@@ -0,0 +1,14 @@
from library import *
if __name__ == "__main__":
P2PRCNodes = ListNodes()
# Print nodes in the network
print(P2PRCNodes)
# Add custom information to the network
if AddCustomInformation("Test"):
print("It worked")
if AddRootNode("0.0.0.0", "8081"):
print("It worked for adding root node")

View File

@@ -5,5 +5,5 @@ authors:
given-names: Akilan
title: P2PRC
license: "GPL-2.0"
version: 1.0.0
date-released: 2021-07-22
version: 2.0.0
date-released: 2023-06-08

72
Docs/.gitignore vendored Normal file
View File

@@ -0,0 +1,72 @@
# Don't bother tracking a bunch of stuff when building and installing
# Org from the master git repository.
# ...by ignoring everything created by 'make', 'make doc', `make info'
# `make html_manual', `make release'
*.aux
*.bak
*.cp
*.cps
*.diff
*.dvi
*.elc
*.fn
*.fns
*.info
*.ky
*.kys
*.log
*.patch
*.pdf
*.pg
*.pgs
*.ps
*.toc
*.tex
*.tp
*.vr
*.vrs
orgcard_letter.tex
orgcard.txt
org
org-loaddefs.el
org-version.el
doc/org-version.inc
org-*.tar*
orgplus-*.tar*
org-*.zip
version.mk
manual
org_dual_license.texi
ORGWEBPAGE/Changes.txt
local*.mk
.gitattributes
mk/x11idle
# texi2pdf --tidy
doc/*.t2d
# aspell word and replacement lists
.aspell.org.pws
.aspell.org.prepl
# allow tmp and test directories that will not be tracked
test
t
auto
tmp
TODO
# and collateral damage from Emacs
*~
.DS_Store
*#
.#*
#
# Local variables:
# End:

View File

@@ -1,6 +0,0 @@
# Abstractions
The Abstractions package consists of black-boxed functions for P2PRC.
## Functions
- ```Init(<Project name>)```: Initializes P2PRC with all the needed configurations.
- ```Start()```: Starts p2prc as a server and makes it possible to extend by adding other routes and functionality to P2PRC.

Binary file not shown.

After

Width:  |  Height:  |  Size: 276 KiB

View File

@@ -0,0 +1,20 @@
# Abstractions
| [◀ Previous](Installation.md) | [Next ▶](Implementation.md) |
|:-----------:|---------|
The Abstractions package consists of black-boxed functions for P2PRC.
## Functions
- ```Init(<Project name>)```: Initializes P2PRC with all the needed configurations.
- ```Start()```: Starts p2prc as a server and makes it possible to extend by adding other routes and functionality to P2PRC.
- ```MapPort(<port no>)```: On the local machine the port you want to export to world.
- ```StartContainer(<ip address>)```: The machine on the p2p network where you want to spin up a docker container.
- ```RemoveContainer(<ip address>,<container id>)```: Terminate container based on the IP address and container name.
- ```GetSpecs(<ip address>)```: Get specs of a machine on the network based on the IP address.
- ```ViewIPTable()```: View the IP table which about nodes in the network.
- ```UpdateIPTable()```: Force update IP table to learn about new nodes faster.
---
### Next Chapter: [Implementation](Implementation.md)

View File

@@ -0,0 +1,35 @@
* Abstractions
:PROPERTIES:
:CUSTOM_ID: abstractions
:END:
| [[file:Installation.md][◀ Previous]] | [[file:Implementation.md][Next ▶]] |
|--------------------------------------+------------------------------------|
The Abstractions package consists of black-boxed functions for P2PRC.
** Functions
:PROPERTIES:
:CUSTOM_ID: functions
:END:
- =Init(<Project name>)=: Initializes P2PRC with all the needed
configurations.
- =Start()=: Starts p2prc as a server and makes it possible to extend by
adding other routes and functionality to P2PRC.
- =MapPort(<port no>)=: On the local machine the port you want to export
to world.
- =StartContainer(<ip address>)=: The machine on the p2p network where
you want to spin up a docker container.
- =RemoveContainer(<ip address>,<container id>)=: Terminate container
based on the IP address and container name.
- =GetSpecs(<ip address>)=: Get specs of a machine on the network based
on the IP address.
- =ViewIPTable()=: View the IP table which about nodes in the network.
- =UpdateIPTable()=: Force update IP table to learn about new nodes
faster.
--------------
*** Next Chapter: [[file:Implementation.org][Implementation]]
:PROPERTIES:
:CUSTOM_ID: next-chapter-implementation
:END:

View File

@@ -0,0 +1,139 @@
# Language Bindings
[Language bindings](https://en.wikipedia.org/wiki/Language_binding) refers to wrappers to bridge 2 programming languages. This is used in P2PRC to extend calling P2PRC functions in other programming languages. Currently this is done by generating ```.so``` and ```.h``` from the Go compiler.
<br>
## How to build shared object files
#### The easier way
```bash
# Run
make sharedObjects
```
#### Or the direct way
```bash
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
```
#### If successfully built:
```bash
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
```
<br>
## Workings under the hood
Below are a sample set of commands to
open the bindings implementation.
```
# run
cd Bindings/
# list files
ls
# search for file
Client.go
```
### In Client go
There a few things to notice which are different from
your standard Go programs:
#### 1. We import "C" which means [Cgo](https://pkg.go.dev/cmd/cgo) is required.
```go
import "C"
```
#### 2. All functions which are required to be called from other programming languages have comment such as.
```go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
```
#### 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
```go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
```
#### It is easy to notice that:
- ```ConvertStructToJSONString(<go object>)```: This is a helper function that convert
a go object to JSON string initially and converts it to ```CString```.
- ```(output *C.char)```: This is the return type for most of the functions.
#### A Pseudo code to refer to the common function implementation shape could be represented as:
```
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
```
<br>
## Current languages supported
- Python
### Build sample python program
The easier way
```bash
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ p2prc.py
```
Above shows a generated folder which consists of a folder
called "SharedObjects/" which consists of ```p2prc.so```
and ```p2prc.h``` files. ```p2prc.py``` refers to a
sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python,
This generated folder can copied and created as a new
git repo for P2PRC extensions scripted or used a reference
point as proof of concept that P2PRC can be called from
other programming languages.

View File

@@ -0,0 +1,180 @@
* Language Bindings
:PROPERTIES:
:CUSTOM_ID: language-bindings
:END:
[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]]
refers to wrappers to bridge 2 programming languages. This is used in
P2PRC to extend calling P2PRC functions in other programming languages.
Currently this is done by generating =.so= and =.h= from the Go
compiler.
** How to build shared object files
:PROPERTIES:
:CUSTOM_ID: how-to-build-shared-object-files
:END:
**** The easier way
:PROPERTIES:
:CUSTOM_ID: the-easier-way
:END:
#+begin_src sh
# Run
make sharedObjects
#+end_src
**** Or the direct way
:PROPERTIES:
:CUSTOM_ID: or-the-direct-way
:END:
#+begin_src sh
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
#+end_src
**** If successfully built:
:PROPERTIES:
:CUSTOM_ID: if-successfully-built
:END:
#+begin_src sh
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
#+end_src
** Workings under the hood
:PROPERTIES:
:CUSTOM_ID: workings-under-the-hood
:END:
Below are a sample set of commands to open the bindings implementation.
#+begin_example
# run
cd Bindings/
# list files
ls
# search for file
Client.go
#+end_example
*** In Client go
:PROPERTIES:
:CUSTOM_ID: in-client-go
:END:
There a few things to notice which are different from your standard Go
programs:
**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required.
:PROPERTIES:
:CUSTOM_ID: we-import-c-which-means-cgo-is-required.
:END:
#+begin_src go
import "C"
#+end_src
**** 2. All functions which are required to be called from other programming languages have comment such as.
:PROPERTIES:
:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as.
:END:
#+begin_src go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
#+end_src
**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
:PROPERTIES:
:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure.
:END:
#+begin_src go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
#+end_src
**** It is easy to notice that:
:PROPERTIES:
:CUSTOM_ID: it-is-easy-to-notice-that
:END:
- =ConvertStructToJSONString(<go object>)=: This is a helper function
that convert a go object to JSON string initially and converts it to
=CString=.
- =(output *C.char)=: This is the return type for most of the functions.
**** A Pseudo code to refer to the common function implementation shape could be represented as:
:PROPERTIES:
:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as
:END:
#+begin_example
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
#+end_example
** Current languages supported
:PROPERTIES:
:CUSTOM_ID: current-languages-supported
:END:
- Python
*** Build sample python program
:PROPERTIES:
:CUSTOM_ID: build-sample-python-program
:END:
The easier way
#+begin_src sh
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ p2prc.py
#+end_src
Above shows a generated folder which consists of a folder called
"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files.
=p2prc.py= refers to a sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python, This generated
folder can copied and created as a new git repo for P2PRC extensions
scripted or used a reference point as proof of concept that P2PRC can be
called from other programming languages.

View File

@@ -6,7 +6,7 @@ commands as possible. The cli was built using the library called urfave cli v2 .
major files created named as flags.go and actions.go.
### Flags.go
The flags .go file is responsible to create the appropriate flags for the cli. There are 2 types of flags
called boolean and string as described in Fig 5.3.1. Each of the flags outputs are assigned to a
called boolean and string. Each of the flags outputs are assigned to a
variable to be handled. The flags can also detect environment variables set. This feature is useful
because if the user wants to call certain flags in a repeated sequence it only has to be initialized
once.

View File

@@ -0,0 +1,24 @@
* Cli module
:PROPERTIES:
:CUSTOM_ID: cli-module
:END:
The Cli (i.e Command Line Interface) is the only one in which the user
can directly interact with the modules in the project. The objective
when building the Cli was to have the least amount of commands as
possible. The cli was built using the library called urfave cli v2 .
They were 2 major files created named as flags.go and actions.go. ###
Flags.go The flags .go file is responsible to create the appropriate
flags for the cli. There are 2 types of flags called boolean and string.
Each of the flags outputs are assigned to a variable to be handled. The
flags can also detect environment variables set. This feature is useful
because if the user wants to call certain flags in a repeated sequence
it only has to be initialized once.
*** Actions.go
:PROPERTIES:
:CUSTOM_ID: actions.go
:END:
The actions.go file is implemented to call the appropriate functions
when the flags are called. It interacts directly with the modules in the
project. Action.go checks if variables are not empty string or the
boolean value is true.

View File

@@ -1,25 +1,13 @@
# Client Module
This module is incharge of communicating with the server and receiving the appropriate information back from the server.
Note: To [read more about the functions](https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation@v0.0.0-20210404191839-6a046babcb02/client)
## Functions of the Client Module
- [Interact with the Server Api](#functions-of-the-client-module)
<!-- - [Interact with the Server Api](#functions-of-the-client-module) -->
- [Decision maker on how the ip table is created or updated](#decision-maker-on-how-the-ip-table-is-created-or-updated)
## Interact with the Server Api
This sections talks about the functionality implemented till now.
- The client can start docker containers using the function StartContainer.
This functions calls the route:
```
http://<server IP address>:<server port>/startcontainer
```
TODO: Outputs and how it's printed
## Decision maker on how the IP table is created or updated
- Does a local speedtest to verify and see if the server IP's in the IP table
are pingable.
- Downloads the Servers IP table.
- Tries to ping the servers IP Table addresses.
- If it's pingable then it's added as a new entry in the IP table.
- The following steps occurs in the clients IP table.

View File

@@ -0,0 +1,33 @@
* Client Module
:PROPERTIES:
:CUSTOM_ID: client-module
:END:
This module is incharge of communicating with the server and receiving
the appropriate information back from the server.
** Functions of the Client Module
:PROPERTIES:
:CUSTOM_ID: functions-of-the-client-module
:END:
#+begin_html
<!-- - [Interact with the Server Api](#functions-of-the-client-module) -->
#+end_html
- [[#decision-maker-on-how-the-ip-table-is-created-or-updated][Decision
maker on how the ip table is created or updated]]
** Decision maker on how the IP table is created or updated
:PROPERTIES:
:CUSTOM_ID: decision-maker-on-how-the-ip-table-is-created-or-updated
:END:
- Does a local speedtest to verify and see if the server IP's in the IP
table are pingable.
- Tries to ping the servers IP Table addresses.
- If it's pingable then it's added as a new entry in the IP table.
- The following steps occurs in the clients IP table.
- To ensure that the same servers are not being called to update the IP
table. There is a temporary list of IP address which have already been
called in relation to updating the IP table.
- Based on the current implementation there will 3 hops done to update
the IP table.

View File

@@ -0,0 +1,15 @@
* Client Module Architecture
:PROPERTIES:
:CUSTOM_ID: client-module-architecture
:END:
The Client Module interacts with the P2P module and Server Module. It is
responsible for interacting with the server module and appropriately
updating the IP table on the client side. It connects to the server
using the server's REST Apis. It is also the primary decision maker on
how the IP table is updated is on the client side. This is because each
user can have requirements like how many number of hops they would want
to do to update their IP table. Hops is the number of times the client
is going to download the IP table from different servers ,once it gets
the IP tables from the previous servers.
[[file:images/NumOfHops.png]] [[file:images/clientmoduleArch.png]]

View File

@@ -1,5 +1,10 @@
# Client Module Implementation
The Client Module interacts with the P2P module and Server Module. It is responsible for interacting with the server module and appropriately updating the IP table on the client side. It connects to the server using the server's REST Apis. It is also the primary decision maker on how the IP table is updated is on the client side. This is because each user can have requirements like how many number of hops they would want to do to update their IP table. Hops is the number of times the client is going to download the IP table from different servers ,once it gets the IP tables from the previous servers.
![Visual demonstration of hops](images/NumOfHops.png)
![UML diagram of client module](images/clientmoduleArch.png)
## Topics
1. [Updating the IP table](#updating-the-IP-table)
2. [Reading server specifications](#reading-server-specifications)
@@ -78,8 +83,6 @@ show a sample structure of file ```grouptrackcontainer.json```.
```
The default path to the container tracker is ```client/trackcontainers/grouptrackcontainer.json```.
### Note:
The group id will be auto-generated and will have its own prefix in the start which will mostly be ```grp<UUID>```.
When a container is removed using the command. ```p2prc --rm <IP Address> --id <Container id>```. It will be automatically deleted
from the groups it exists in.
> [!NOTE]
> The group id will be auto-generated and will have its own prefix in the start which will mostly be ```grp<UUID>```.
> When a container is removed using the command. ```p2prc --rm <IP Address> --id <Container id>```. It will be automatically deleted from the groups it exists in.

View File

@@ -0,0 +1,124 @@
* Client Module Implementation
:PROPERTIES:
:CUSTOM_ID: client-module-implementation
:END:
The Client Module interacts with the P2P module and Server Module. It is
responsible for interacting with the server module and appropriately
updating the IP table on the client side. It connects to the server
using the server's REST Apis. It is also the primary decision maker on
how the IP table is updated is on the client side. This is because each
user can have requirements like how many number of hops they would want
to do to update their IP table. Hops is the number of times the client
is going to download the IP table from different servers ,once it gets
the IP tables from the previous servers.
[[file:images/NumOfHops.png]] [[file:images/clientmoduleArch.png]]
This section focuses in depth on how the client module works. The client
module is incharge of communicating with different servers based on the
IP addresses provided to the user. The IP addresses are derived from
peer to peer modules. The objective here is how the client module
interacts with peer to peer module and server module.
*** Updating the IP table
:PROPERTIES:
:CUSTOM_ID: updating-the-ip-table
:END:
The client module calls the peer to peer module to get the local IP
table initially, Based on the servers IP addresses available it calls
the speedtest function from the peer to peer module to update IP
addresses with information such as latencies, download and upload
speeds. Once this is done the client module does a Rest Api call to the
server to download its IP Table. Once the hops are done it writes the
appropriate results to the Local IP table. Once this is done it prints
out the results. To derive parameters such as current the public IP
address the url "http://ip-api.com/json/" was called. This url returns
json response of the current public IP address. This feature will be
used in the future to ensure that the user's current IP address will not
be used for a speed test. Clients IP table is updated to the server
using a form of type multipart.
*** Reading server specifications
:PROPERTIES:
:CUSTOM_ID: reading-server-specifications
:END:
The client module calls the route /server_specs and reads the json
response. If the json response was successful then it just calls the
pretty print function which just prints the json output in the terminal.
*** Client creating and removing container
:PROPERTIES:
:CUSTOM_ID: client-creating-and-removing-container
:END:
The client module uses the servers Rest apis to create and delete
containers. To create a container the client requires 3 parameters being
the server ip address, the number of the ports the user wants to open
and if the user wants it connected to the GPU or not. The 3 parameters
are sent as a GET request to the server and the server responds with a
json file which has information such as the container ID, ports open ,
SSH username, SSH password, VNC username and VNC password. At the moment
the username and password are hard coded from the server side for both
SSH and VNC. To remove a container the client module only requires the
server IP address and the container ID. The client prints the response
from the server Rest api.
*** Tracking Containers
:PROPERTIES:
:CUSTOM_ID: tracking-containers
:END:
Clients create docker images in multiple machines. This means if the
client (i.e user) has many containers created there needs to be a way to
track them. To track containers there is a file called
=trackcontainers.json= which tracks all the containers running. The
snippet below show a sample structure of file =trackcontainer.json=.
#+begin_example
{
"TrackContainer": [
{
"ID": "<ID>",
"Container": {<docker.DockerVM struct>},
"IpAddress": "<IP Address>"
}
]
}
#+end_example
The default path to the container tracker is
=client/trackcontainers/trackcontainers.json=.
*** Grouping Containers
:PROPERTIES:
:CUSTOM_ID: grouping-containers
:END:
When starting a set container possibility to be able to group them. The
benefit this would be that when executing plugins the group ID would be
enough to execute plugin in a set of containers. This provides the
possibility to execute repetitive tasks in containers in a single cli
command. To store groups there is a file called
=grouptrackcontainer.json= which tracks all the groups currently present
set by the client. The snippet below show a sample structure of file
=grouptrackcontainer.json=.
#+begin_example
{
"Groups": [
{
"ID": "grp<Random UUID>",
"TrackContainer": [{client.TrackContainers struct}]
}
]
}
#+end_example
The default path to the container tracker is
=client/trackcontainers/grouptrackcontainer.json=.
#+begin_quote
[!NOTE] The group id will be auto-generated and will have its own prefix
in the start which will mostly be =grp<UUID>=.\\
When a container is removed using the command.
=p2prc --rm <IP Address> --id <Container id>=. It will be automatically
deleted from the groups it exists in.
#+end_quote

View File

@@ -13,9 +13,19 @@ JSON format.
```json
{
"dockerfile": "/<path>/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"iptable": "/<path>/p2p-rendering-computation/p2p/ip_table.json",
"speedtestfile": "/<path>/p2p-rendering-computation/p2p/50.bin"
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
```

View File

@@ -0,0 +1,35 @@
* Config Implementation
:PROPERTIES:
:CUSTOM_ID: config-implementation
:END:
The configuration module is responsible to store basic information of
absolute paths of files being called in the Go code. In a full-fledged
Cli the configuration file can be found in the directory /etc/ and from
there points to location such as where the IP table file is located. In
the future implementation the config file will have information such as
number of hops and other parameters to tweak and to improve the
effectiveness of the peer to peer network. The configuration module was
implemented using the library Viper. The Viper library automates
features such as searching in default paths to find out if the
configuration file is present. If the configuration file is not present
in the default paths then it auto generates the configuration file. The
configurations file can be in any format. In this project the
configuration file was generated using JSON format.
#+begin_src json
{
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
#+end_src

View File

@@ -0,0 +1,26 @@
* Design Architecture
:PROPERTIES:
:CUSTOM_ID: design-architecture
:END:
This chapter focuses on architecture of the dissertation. The objective
would be to have a good understanding on the purpose of each module and
how they interact with each other. The design architecture was inspired
and based on the linux kernel design. The project is segmented into
various modules. Each module is responsible for certain tasks in the
project. The modules are highly dependent on each other hence the entire
codebase can be considered as a huge monolithic chuck which acts as its
own library. The following sub topics below talk about the main modules
and how they function with appropriate diagrams.
*** 1. [[file:ClientArchitecture.md][Client Module]]
:PROPERTIES:
:CUSTOM_ID: client-module
:END:
*** 2. [[file:P2PArchitecture.md][P2P Module]]
:PROPERTIES:
:CUSTOM_ID: p2p-module
:END:
*** 3. [[file:ServerArchitecture.md][Server Module]]
:PROPERTIES:
:CUSTOM_ID: server-module
:END:

View File

@@ -0,0 +1,2 @@
# Domain name mappings
This

View File

@@ -0,0 +1,6 @@
* Domain name mappings
:PROPERTIES:
:CUSTOM_ID: domain-name-mappings
:END:
Todo be written.

View File

@@ -0,0 +1,23 @@
# Implementation
| [◀ Previous](Introduction.md) | [Back to TOC](README.md) |
|:-----------:|---------|
This chapter describes how the project was built. It talks in depth of the implementation
performed to give a better understanding of the project.
## Programming langauge used
The programming language used for this project was [Golang](https://go.dev/). The reason Go lang was chosen was
because it is a compiled language.<br>
The entire codebase is just a single binary file. When
distributing to other linux distributing the only requirement would be the binary file to run the
code. It is easy to write independant modules and be monolithic at the sametime using Go.<br>
Using Go.mod makes it very easy to handle external libraries and modularise code. The go.mod name for
the project is [git.sr.ht/~akilan1999/p2p-rendering-computation](https://git.sr.ht/~akilan1999/p2p-rendering-computation).
- ## [Cli Module](CliImplementation.md)
- ## [Config Module](ConfigImplementation.md)
- ## [Server Module](ServerImplementation.md)
- ## [Client Module](ClientImplementation.md)
- ## [P2P Module](P2PImplementation.md)
- ## [Plugin Module](PluginImplementation.md)
- ## [Generate Module](GenerateImplementation.md)

View File

@@ -0,0 +1,21 @@
* Implementation
:PROPERTIES:
:CUSTOM_ID: implementation
:END
This chapter describes how the project was built. It talks in depth of
the implementation performed to give a better understanding of the
project.
** Programming language used
:PROPERTIES:
:CUSTOM_ID: programming-langauge-used
:END:
The programming language used for this project was
[[https://go.dev/][Golang]]. The reason Go lang was chosen was because
it is a compiled language. The entire codebase is just a single binary
file. When distributing to other linux distributing the only requirement
would be the binary file to run the code. It is easy to write
independant modules and be monolithic at the sametime using Go. Using
Go.mod makes it very easy to handle external libraries and modularise
code. The go.mod name for the project is
[[https://git.sr.ht/~akilan1999/p2p-rendering-computation][git.sr.ht/~akilan1999/p2p-rendering-computation]].

View File

@@ -1,9 +1,12 @@
# Installation
| [◀ Previous](Introduction.md) | [Next ▶](Abstractions.md) |
|:-----------:|---------|
Over here we will cover the basic steps to get the server and client side running.
## Alpha release install
https://github.com/Akilan1999/p2p-rendering-computation/releases/tag/v1.0.0-alpha
## Latest release install
https://github.com/Akilan1999/p2p-rendering-computation/releases
## Install from Github master branch
@@ -39,7 +42,7 @@ To set up P2PRC on Windows, simply run this batch file.
.\install.bat
```
### Add appropriate paths to .bashrc
### Add appropriate paths to `.bashrc`
```
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
@@ -65,7 +68,7 @@ USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
1.0.0
<version no>
COMMANDS:
help, h Shows a list of commands or help for one command
@@ -78,7 +81,7 @@ GLOBAL OPTIONS:
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container [$REMOVE_VM]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
@@ -86,10 +89,20 @@ GLOBAL OPTIONS:
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--help, -h show help (default: false)
--version, -v print the version (default: false)
```
<br>
@@ -101,7 +114,7 @@ GLOBAL OPTIONS:
# Using basic commands
### Start as a server
Do ensure you have docker installed for this
Do ensure you have Docker installed for this
```
p2prc -s
```
@@ -195,6 +208,11 @@ p2prc --pp <repo link>
p2prc --rp <plugin name>
```
### Added custom metadata about the current node
```
p2prc --amd "custom metadata"
```
<br>
--------------
@@ -210,23 +228,17 @@ This feature is still Under Development:
- Debian/ubuntu: ```sudo apt install ansible```
- Others: [Installation link](https://ansible-tips-and-tricks.readthedocs.io/en/latest/ansible/install/)
#### Set ansible host_key_checking to false
- On linux
- ```sudo nano /etc/ansible/ansible.cfg```: Open the following file. If this file is not found then where
ever the file ```ansible.cfg``` is located.
- Add or uncomment ```host_key_checking = False```
#### Run Test Cases
- Generate Test Case Ansible file
- ```make testcases```
- Enter inside plugin directory and run tests.
Note: That docker needs to installed and needs to run without
sudo. Refer the section install Docker.
- ```cd plugin```
- ```go test .```
- Enter inside plugin directory and run tests.<br>
> [!NOTE]
> That docker needs to installed and needs to run without
> sudo. Refer the section [Install Docker](#install-docker).
> - ```cd plugin```
> - ```go test .```
---
### Next Chapter: [Abstractions](Abstractions.md)

View File

@@ -0,0 +1,352 @@
* Installation
:PROPERTIES:
:CUSTOM_ID: installation
:END:
Over here we will cover the basic steps to get the server and client
side running.
** Latest release install
:PROPERTIES:
:CUSTOM_ID: latest-release-install
:END:
https://github.com/Akilan1999/p2p-rendering-computation/releases
** Install from Github master branch
:PROPERTIES:
:CUSTOM_ID: install-from-github-master-branch
:END:
*** Install Go lang
:PROPERTIES:
:CUSTOM_ID: install-go-lang
:END:
The entire the implementation of this project is done using Go lang.
Thus, we need go lang to compile to code to a binary file.
[[https://golang.org/doc/install][Instructions to install Go lang]]
*** Install Docker
:PROPERTIES:
:CUSTOM_ID: install-docker
:END:
In this project the choice of virtualization is Docker due to it's wide
usage in the developer community. In the server module we use the Docker
Go API to create and interact with the containers.
[[https://docs.docker.com/get-docker/][Instructions to install docker]]
[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions
to install docker GPU]]
#+begin_example
// Do ensure that the docker command does not need sudo to run
sudo chmod 666 /var/run/docker.sock
#+end_example
*** Build Project and install project
:PROPERTIES:
:CUSTOM_ID: build-project-and-install-project
:END:
To set up the internal dependencies and build the entire go code into a
single binary
#+begin_example
make install
#+end_example
**** For Windows
:PROPERTIES:
:CUSTOM_ID: for-windows
:END:
To set up P2PRC on Windows, simply run this batch file. *Make sure you
are not in admin mode when running this.*
#+begin_example
.\install.bat
#+end_example
*** Add appropriate paths to =.bashrc=
:PROPERTIES:
:CUSTOM_ID: add-appropriate-paths-to-.bashrc
:END:
#+begin_example
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
#+end_example
*** Set up configuration file
:PROPERTIES:
:CUSTOM_ID: set-up-configuration-file
:END:
#+begin_example
make configfile
#+end_example
Open the config file =config.json= and add the IPv6 address if you have
one.
*** Test if binary works
:PROPERTIES:
:CUSTOM_ID: test-if-binary-works
:END:
#+begin_example
p2prc --help
#+end_example
**** Output:
:PROPERTIES:
:CUSTOM_ID: output
:END:
#+begin_example
NAME:
p2p-rendering-computation - p2p cli application to create and access VMs in other servers
USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
<version no>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--Server, -s Starts server (default: false) [$SERVER]
--UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST]
--ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS]
--AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER]
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
--Specification value, --specs value Specs of the server node [$SPECS]
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--help, -h show help (default: false)
--version, -v print the version (default: false)
#+end_example
--------------
* Using basic commands
:PROPERTIES:
:CUSTOM_ID: using-basic-commands
:END:
*** Start as a server
:PROPERTIES:
:CUSTOM_ID: start-as-a-server
:END:
Do ensure you have Docker installed for this
#+begin_example
p2prc -s
#+end_example
*** View server Specification
:PROPERTIES:
:CUSTOM_ID: view-server-specification
:END:
#+begin_example
p2prc --specs=<ip address>
#+end_example
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
use the =--gpu= if you know the other machine has a gpu.
#+begin_example
p2prc --touch=<server ip address> -p <number of ports> --gpu
#+end_example
*** Remove container
:PROPERTIES:
:CUSTOM_ID: remove-container
:END:
The docker id is present in the output where you create a container
#+begin_example
p2prc --rm=<server ip address> --id=<docker container id>
#+end_example
*** Adding servers to ip table
:PROPERTIES:
:CUSTOM_ID: adding-servers-to-ip-table
:END:
#+begin_example
p2prc --as=<server ip address you want to add>
#+end_example
*** Update ip table
:PROPERTIES:
:CUSTOM_ID: update-ip-table
:END:
#+begin_example
p2prc --us
#+end_example
*** List Servers
:PROPERTIES:
:CUSTOM_ID: list-servers
:END:
#+begin_example
p2prc --ls
#+end_example
*** View Network interfaces
:PROPERTIES:
:CUSTOM_ID: view-network-interfaces
:END:
#+begin_example
p2prc --ni
#+end_example
*** Viewing Containers created Client side
:PROPERTIES:
:CUSTOM_ID: viewing-containers-created-client-side
:END:
#+begin_example
p2prc --tc
#+end_example
[[file:ClientImplementation.md#tracking-containers][read more on
tracking containers]]
*** Running plugin
:PROPERTIES:
:CUSTOM_ID: running-plugin
:END:
#+begin_example
p2prc --plugin <plugin name> --id <container id or group id>
#+end_example
*** Create group
:PROPERTIES:
:CUSTOM_ID: create-group
:END:
#+begin_example
p2prc --cgroup
#+end_example
*** Add container to group
:PROPERTIES:
:CUSTOM_ID: add-container-to-group
:END:
#+begin_example
p2prc --group <group id> --id <container id>
#+end_example
*** View groups
:PROPERTIES:
:CUSTOM_ID: view-groups
:END:
#+begin_example
p2prc --groups
#+end_example
*** View specific group
:PROPERTIES:
:CUSTOM_ID: view-specific-group
:END:
#+begin_example
p2prc --group <group id>
#+end_example
*** Delete container from group
:PROPERTIES:
:CUSTOM_ID: delete-container-from-group
:END:
#+begin_example
p2prc --rmcgroup --group <group id> --id <container id>
#+end_example
*** Delete entire group
:PROPERTIES:
:CUSTOM_ID: delete-entire-group
:END:
#+begin_example
p2prc --rmgroup <group id>
#+end_example
[[file:ClientImplementation.md#Grouping-Containers][read more on
grouping containers]] ### Extending usecase of P2PRC (Requires a go
compiler to run)
#+begin_example
p2prc --gen <project name> --mod <go module name>
#+end_example
[[file:GenerateImplementation.md][read more about the generate module]]
*** Pulling plugin from a remote repo
:PROPERTIES:
:CUSTOM_ID: pulling-plugin-from-a-remote-repo
:END:
#+begin_example
p2prc --pp <repo link>
#+end_example
*** Deleting plugin from the plugin directory
:PROPERTIES:
:CUSTOM_ID: deleting-plugin-from-the-plugin-directory
:END:
#+begin_example
p2prc --rp <plugin name>
#+end_example
*** Added custom metadata about the current node
:PROPERTIES:
:CUSTOM_ID: added-custom-metadata-about-the-current-node
:END:
#+begin_example
p2prc --amd "custom metadata"
#+end_example
--------------
* Using Plugins
:PROPERTIES:
:CUSTOM_ID: using-plugins
:END:
This feature is still Under Development:
[[file:PluginImplementation.md][Read more on the implementation]]
**** Dependencies
:PROPERTIES:
:CUSTOM_ID: dependencies
:END:
- Ansible:
- Debian/ubuntu: =sudo apt install ansible=
- Others:
[[https://ansible-tips-and-tricks.readthedocs.io/en/latest/ansible/install/][Installation
link]]
**** Run Test Cases
:PROPERTIES:
:CUSTOM_ID: run-test-cases
:END:
- Generate Test Case Ansible file
- =make testcases=
- Enter inside plugin directory and run tests.
#+begin_quote
[!NOTE] That docker needs to installed and needs to run without sudo.
Refer the section [[#install-docker][Install Docker]]. - =cd plugin= -
=go test .=
#+end_quote

View File

@@ -1,5 +1,8 @@
# Chapter 1: Introduction
| [◀ Back to TOC](README.md) | [Next ▶](Installation.md) |
|:-----------:|---------|
## Abstract
This project focuses on creating a framework on running heavy tasks that a regular computer
cannot run easily such as graphically demanding video games, rendering 3D animations , protein
@@ -21,7 +24,7 @@ run these heavy tasks can be really useful. Ethically speaking this is leading t
computing power similar to what is happening in the web server area. By using peer to peer
principles it is possible to remove the monopolisation factor and increase the bandwidth between
the client and server.
<!--
## Aim
This project aims to create a peer to peer (p2p) network, where a user can use the p2p network to
act as a client (i.e sending tasks) or the server (i.e executing the tasks). A prototype application will
@@ -33,4 +36,8 @@ or virtual environments across selected nodes.
rendering tools and tools to batch any sort of tasks.
- Creating p2p network
- Server to create a containerised environment
- The client node to run tasks on Server containerised node
- The client node to run tasks on Server containerised node -->
---
### Next Chapter: [Installation](Installation.md)

View File

@@ -0,0 +1,39 @@
* Chapter 1: Introduction
:PROPERTIES:
:CUSTOM_ID: chapter-1-introduction
:END:
** Abstract
:PROPERTIES:
:CUSTOM_ID: abstract
:END:
This project focuses on creating a framework on running heavy tasks that
a regular computer cannot run easily such as graphically demanding video
games, rendering 3D animations , protein folding simulations. In this
project the major focus will not be on the financial incentive part. A
peer to peer network will be created to help run tasks decentrally,
increasing bandwidth for running tasks. To ensure the tasks in the peer
to peer network do not corrupt the server 0S (Operating System), they
will be executed in a virtual environment in the server.
The main aim of this project was to create a custom peer to peer
network. The user acting as the client has total flexibility on how to
batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the
containers at any point of time.
** Motivation
:PROPERTIES:
:CUSTOM_ID: motivation
:END:
Many of the users rely on our PC / Laptop or servers that belong to a
server farm to run heavy tasks and with the demand of high creativity
requires higher computing power. Buying a powerful computer every few
years to run a bunch of heavy tasks which are not executed as frequently
to reap the benefits can be inefficient utilization of hardware. On the
other end, renting servers to run these heavy tasks can be really
useful. Ethically speaking this is leading to monopolisation of
computing power similar to what is happening in the web server area. By
using peer to peer principles it is possible to remove the
monopolisation factor and increase the bandwidth between the client and
server.

View File

@@ -0,0 +1,50 @@
* NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src

View File

@@ -0,0 +1,136 @@
* Testing P2P network
:PROPERTIES:
:CUSTOM_ID: testing-p2p-network
:END:
The objective would be to test the p2p network, and the effectiveness of
updating the ip tables. The objective of would be to give the impression
to the client and server of a Zero configuration setting. For testing
there will be a test network set. In the testing scenario all will be
client and server because the IP table does not store clients IP
addresses. At current number of hopes would be 3 as default.
*** Test Network Scenario 1
:PROPERTIES:
:CUSTOM_ID: test-network-scenario-1
:END:
The test network consists of 5 nodes acting as a client and server. The
objective would be to have the entire IP table Updated in each node with
interacting with only 1 node once. Each node has knowledge of one node
only.
[[https://user-images.githubusercontent.com/31743758/115069627-e4aa8c80-9f04-11eb-8402-706a3407f0e8.png]]
Fig 1.0 Visual Representation of testnet scenario 1
**** Result
:PROPERTIES:
:CUSTOM_ID: result
:END:
All nodes except node 1 where able to have information of IP addresses
in the test net. This was due to the reason of 3 hops set as default.
Node 1 had in it's IP table IP addresses of Node 2, Node 3, Node 4. Once
the number of hops was set to 4 objective of the test was acheived.
*** Test Network Scenario 2
:PROPERTIES:
:CUSTOM_ID: test-network-scenario-2
:END:
The second test network has a scenario of a single peer which all the
other nodes connect too. The scenario being when the other nodes connect
to the single server they download information about nodes that have
connected to the server node before.
*** Testing Broadcast Module
:PROPERTIES:
:CUSTOM_ID: testing-broadcast-module
:END:
For testing the broadcast module 2 types of servers will be tested. One
with a CPU only , another one with a CPU and GPU. The expected result
being that the appropriate results are visible.
**** Results (CPU and GPU):
:PROPERTIES:
:CUSTOM_ID: results-cpu-and-gpu
:END:
#+begin_example
{
"Hostname": "akilan-Lenovo-IdeaPad-Y510P",
"Platform": "ubuntu",
"CPU": "Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz",
"RAM": 7872,
"Disk": 937367,
"GPU": {
"DriveVersion": "390.141",
"Gpu": {
"GpuName": "GeForce GT 755M",
"BiosVersion": "80.07.A8.00.0F",
"FanSpeed": "N/A",
"Utilization": {
"GpuUsage": "N/A",
"MemoryUsage": "N/A"
},
"Temperature": {
"GpuTemp": "66 C"
},
"Clock": {
"GpuClock": "N/A",
"GpuMemClock": "N/A"
}
}
}
}
#+end_example
At the moment of the current implementation v1.0. Nvidia GPU are only
compatible. As the Go code calls the command =nvidia-smi= to get
information about the GPU available.
**** Results (CPU only)
:PROPERTIES:
:CUSTOM_ID: results-cpu-only
:END:
#+begin_example
{
"Hostname": "sv-t1.small.x86-01",
"Platform": "ubuntu",
"CPU": "Intel(R) Atom(TM) CPU C2750 @ 2.40GHz",
"RAM": 7944,
"Disk": 138793,
"GPU": null
}
#+end_example
As the =nvidia-smi= interface was not detected it only broadcasts the
CPU specs available.
*** SpeedTests
:PROPERTIES:
:CUSTOM_ID: speedtests
:END:
The speed test has 3 parameters which are Ping , upload and download.
The tests check if the results returned are approximately correct. The
ping at the moment returns the correct result. The upload and download
returned are inccorect at the moment, This is due incorrect
implementation in for timer and will be patched in future versions.
*** Unit tests
:PROPERTIES:
:CUSTOM_ID: unit-tests
:END:
All functions implemented on the P2P module returns type error. The
units test call certain functions and check if the functions return an
error or not. This proved sufficient as the point of the units tests was
code coverage to check if certain functions return an error.
**** Functions tested
:PROPERTIES:
:CUSTOM_ID: functions-tested
:END:
This sections talks about the function called and represents code
coverage.
1. =TestServer_SpeedTest=: Function called LocalSpeedTestIpTable()
2. =TestReadIpTable=: Function called ReadIpTable()
The P2P module has a 100% code coverage in unit tests as both the unit
tests call directly or call within the function all the functions used
in the P2P module.

View File

@@ -19,11 +19,9 @@ In this repository the P2P module has been designed from sratch at the point of
## Responsibility
- To perform speed test to determine best node to connect
- To ensure the IP table has nodes which are pingable
- Using techniques such as [UPNP](https://en.wikipedia.org/wiki/Universal_Plug_and_Play). Still under development
- Port Forwarding (To be introducted in a future release)
- Taking to nodes behind NAT. [More about the implementation](NAT-Traversal)...
## Note:
If you are running in server mode it is recommended to use [DMZ](https://routerguide.net/when-and-how-to-setup-dmz-host-for-home-use/) to bypass the [NAT](https://en.wikipedia.org/wiki/Network_address_translation).
> [!NOTE]
> If you are running in server mode it is recommended to use [DMZ](https://routerguide.net/when-and-how-to-setup-dmz-host-for-home-use/) to bypass the [NAT](https://en.wikipedia.org/wiki/Network_address_translation).

View File

@@ -0,0 +1,45 @@
* P2P (Peer to Peer module)
:PROPERTIES:
:CUSTOM_ID: p2p-peer-to-peer-module
:END:
In this repository the P2P module has been designed from sratch at the
point of this implementation.
[[https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation@v0.0.0-20210404191839-6a046babcb02/p2p][More
about function implementation]]
** Terminology
:PROPERTIES:
:CUSTOM_ID: terminology
:END:
1. IPTable: Refers to a json file which stores information about the
current servers avaliable with the speedtest results ran from the
Node that triggered it.
#+begin_example
{
"ip_address": [
{
"ipv4": "localhost",
"latency": 14981051,
"download": 8142.122540206258,
"upload": 3578.766512629995,
}
]
}
#+end_example
** Responsibility
:PROPERTIES:
:CUSTOM_ID: responsibility
:END:
- To ensure the IP table has nodes which are pingable
- Taking to nodes behind NAT. [[file:NAT-Traversal][More about the
implementation]]...
#+begin_quote
[!NOTE] If you are running in server mode it is recommended to use
[[https://routerguide.net/when-and-how-to-setup-dmz-host-for-home-use/][DMZ]]
to bypass the
[[https://en.wikipedia.org/wiki/Network_address_translation][NAT]].
#+end_quote

View File

@@ -0,0 +1,16 @@
* P2P Module Architecture
:PROPERTIES:
:CUSTOM_ID: p2p-module-architecture
:END:
The P2P module (i.e Peer to Peer Module) is responsible for storing the
IP table and interacting with the IP table. In the following
implementation of the P2P module ,the IP table stores information about
servers available in the network. The other functionality the P2P module
takes care of is doing the appropriate speed tests to the servers in the
IP table. This is for informing the users about nodes which are close by
and nodes which have quicker uploads and downloads speeds. The module is
responsible to ensure that there are no duplicate server IPs in the IP
table and to remove all server IPs which are not pingable.
#+caption: UML diagram of P2P module
[[file:images/p2pmoduleArch.png]]

View File

@@ -0,0 +1,85 @@
# P2P Module Implementation
The P2P module (i.e Peer to Peer Module) is responsible for storing the IP table and interacting
with the IP table. In the following implementation of the P2P module ,the IP table stores
information about servers available in the network. The other functionality the P2P module takes
care of is doing the appropriate speed tests to the servers in the IP table. This is for informing the
users about nodes which are close by and nodes which have quicker uploads and downloads
speeds. The module is responsible to ensure that there are no duplicate server IPs in the IP table
and to remove all server IPs which are not pingable.
![UML diagram of P2P module](images/p2pmoduleArch.png)
The peer to peer implementation was built from scratch. This is because other peer to peer
libraries were on the implementation of the Distributed hash table. At the current moment all
those heavy features are not needed because the objective is to search and list all possible servers
available. The limitation being that to be a part of the network the user has to know at least 1
server. The advantage of building from scratch makes the module super light and
possibility for custom functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
## IP Table
The ip table file is a json as the format with a list of servers ip addresses, latencies, downloads and
uploads speeds. The functions implemented include read
file, write file and remove duplicate IP addresses. The remove duplicate IP address function exists
because sometimes servers IP tables can have the same ip addresses as what the client has. The
path of the IP table json file is received from the configuration module.
```json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>",
}
]
}
```
### Latency
The latency is measured in milliseconds. The route /server_info is called from the
server and time it takes to provide a json response is recorded.
## NAT Traversal
P2PRC currently supports TURN for NAT traversal.
## TURN
The current TURN implementation used is FRP. The TURN server is also required when
a P2PRC node is acting as a Server. The TURN server is determined based on the Node
with the least amount of latency based on the Nodes available on the IPTable.
Once a TURN server is determined there are 2 actions performed. The first one is
```/FRPPort``` to the TURN server to receive a port which is used to generate the external
port from the TURN server. The flow below describes the workflow.
### Client mode
- Call ```/FRPPort```
```
http://<turn server ip>:<server port no>/FRPport
```
- Call the TURN server in the following manner. The following is a sample code snippet below.
```go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
```

View File

@@ -0,0 +1,111 @@
* P2P Module Implementation
:PROPERTIES:
:CUSTOM_ID: p2p-module-implementation
:END:
The P2P module (i.e Peer to Peer Module) is responsible for storing the
IP table and interacting with the IP table. In the following
implementation of the P2P module ,the IP table stores information about
servers available in the network. The other functionality the P2P module
takes care of is doing the appropriate speed tests to the servers in the
IP table. This is for informing the users about nodes which are close by
and nodes which have quicker uploads and downloads speeds. The module is
responsible to ensure that there are no duplicate server IPs in the IP
table and to remove all server IPs which are not pingable.
#+caption: UML diagram of P2P module
[[file:images/p2pmoduleArch.png]]
The peer to peer implementation was built from scratch. This is because
other peer to peer libraries were on the implementation of the
Distributed hash table. At the current moment all those heavy features
are not needed because the objective is to search and list all possible
servers available. The limitation being that to be a part of the network
the user has to know at least 1 server. The advantage of building from
scratch makes the module super light and possibility for custom
functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
** IP Table
:PROPERTIES:
:CUSTOM_ID: ip-table
:END:
The ip table file is a json as the format with a list of servers ip
addresses, latencies, downloads and uploads speeds. The functions
implemented include read file, write file and remove duplicate IP
addresses. The remove duplicate IP address function exists because
sometimes servers IP tables can have the same ip addresses as what the
client has. The path of the IP table json file is received from the
configuration module.
#+begin_src json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>",
}
]
}
#+end_src
*** Latency
:PROPERTIES:
:CUSTOM_ID: latency
:END:
The latency is measured in milliseconds. The route /server_info is
called from the server and time it takes to provide a json response is
recorded.
** NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src

View File

@@ -54,22 +54,23 @@ nodes to execute Ansible instructions. In this project this file needs to be set
go code or binary will populate this file automatically with the appropriate information required to connect to local or
remote containers.
#### Note: Add as exactly specified below
```
all:
vars:
ansible_python_interpreter: /usr/bin/python3 // Path to your python 3 interpreter
main:
hosts:
host1:
// Note: These values will be automatically overwritten
// by the Go functions
ansible_host: 0.0.0.0
ansible_port: 39269
ansible_user: master
ansible_ssh_pass: password
ansible_sudo_pass: password
```
> [!NOTE]
> Add as exactly specified below:
> ```
>all:
> vars:
> ansible_python_interpreter: /usr/bin/python3 // Path to your python 3 interpreter
>main:
> hosts:
> host1:
> // Note: These values will be automatically overwritten
> // by the Go functions
> ansible_host: 0.0.0.0
> ansible_port: 39269
> ansible_user: master
> ansible_ssh_pass: password
> ansible_sudo_pass: password
>```
## Ports.json
The ```ports.json``` file is intended to mention the number of ports required

View File

@@ -0,0 +1,203 @@
* Plugin Module Implementation
:PROPERTIES:
:CUSTOM_ID: plugin-module-implementation
:END:
** Introduction
:PROPERTIES:
:CUSTOM_ID: introduction
:END:
The plugin module is designed to ensure clients can execute instructions
in a declarative manner across different containers created. This means
the user (i.e client) needs to write the instruction only once, and
these instructions can be executed across different nodes in a
repetitive manner.
In the scenario of this project Ansibles will be used as the way the
users can create these instructions.
- [[file:Installation.md#Using-Plugins][Setup instruction]]
The plugin module introduces a new path to the config file known as
pluginpath. This path by defaults points to =${P2PRC}/plugin/deploy=.
Any file/folder inside =plugin/deploy= is part of the .gitginore.
Plugins are detected by folder names inside the =plugin/deploy=.
#+begin_example
plugin
|___ Deploy
|___<plugin name>
|___ site.yml
|___ hosts
|___ ports.json
|___ description.txt
.
.
.
n: n number of plugins possible
#+end_example
** Site File Template
:PROPERTIES:
:CUSTOM_ID: site-file-template
:END:
The site file is also known as the Ansible playbook and is incharge of
executing instructions in a declarative manner. The below example
specifies how to make one.
#+begin_example
- hosts: all
tasks:
- name: <task name>
<ansible task>
debug:
msg: <debug message>
#+end_example
Read more about ansible tasks:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html#about-playbooks
** Hosts file
:PROPERTIES:
:CUSTOM_ID: hosts-file
:END:
hosts file is also known as the inventory file. This file consists of
all the information required to connect to other nodes to execute
Ansible instructions. In this project this file needs to be set in a
certain configuration because the go code or binary will populate this
file automatically with the appropriate information required to connect
to local or remote containers.
#+begin_quote
[!NOTE] Add as exactly specified below:
#+begin_example
all:
vars:
ansible_python_interpreter: /usr/bin/python3 // Path to your python 3 interpreter
main:
hosts:
host1:
// Note: These values will be automatically overwritten
// by the Go functions
ansible_host: 0.0.0.0
ansible_port: 39269
ansible_user: master
ansible_ssh_pass: password
ansible_sudo_pass: password
#+end_example
#+end_quote
** Ports.json
:PROPERTIES:
:CUSTOM_ID: ports.json
:END:
The =ports.json= file is intended to mention the number of ports
required by the plugin.
#+begin_example
{
"NumOfPorts": <number of ports>
}
#+end_example
** Description file
:PROPERTIES:
:CUSTOM_ID: description-file
:END:
This is a simple text file used to describe what the module does. When
the client is looking at various commands via the ClI. The description
is displayed along-side the plugin name.
Ex: When the flag =--ViewPlugins= or =--vp= is called
#+begin_example
{
"PluginsDetected": [
{
"FolderName": "<name of the plugin>",
"PluginDescription": "<description of the plugin>"
}
]
}
#+end_example
** Automatic port allocations
:PROPERTIES:
:CUSTOM_ID: automatic-port-allocations
:END:
P2PRC would be in-charge to set to the ports to various TCP ports
opened. Due to this implementation the plugin being executed is copied
to the tmp directory with a unique UUID.
#+begin_example
Command: ls /tmp
output: Semantic <UUID>_<Plugin Name>
2e6d76c4-0ed1-4b55-9385-79a58d4f0492_p2prc-vscode-browser
7b631e08-62ee-4c1c-a2a4-c05857b9aa7d_p2prc-vscode-browser
#+end_example
Once the copy of the plugin is added to the /tmp directory the site.yml
file inside the appropriate yaml is modified with the appropriate ports
assigned to the container.
*** Ex:
:PROPERTIES:
:CUSTOM_ID: ex
:END:
1. Create container called c1 with an automatic generated TCP port 3313
(external) - 3313 (internal)
2. Assumption of plugin p1 exists. p1 has one server which needs to be
mapped to a free open TCP port in container c1. Below shows an
implementation of a sample site.yml file.
#+begin_example
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:{{index . 0}}
#+end_example
Notice there is the following {{index . 0}}. {{index . 0}} does not
belong to Ansible but rather is a way to mention where to add the
external free port of the container. We use the golang
[[https://pkg.go.dev/text/template][template library]] to parse and
populate the site.yml with the appropriate open ports. An array of ints
which consists of open free ports are sent to the site.yml. 0 in
{{index . 0}} refers to the index in the int array passed on.
After the port is automatically it's ready to run !
#+begin_example
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:3313
#+end_example
*** Sample plugins implemented:
:PROPERTIES:
:CUSTOM_ID: sample-plugins-implemented
:END:
- [[https://github.com/Akilan1999/p2prc-vscode-browser][VSCode Plugin]]
** Pull Plugins
:PROPERTIES:
:CUSTOM_ID: pull-plugins
:END:
The following allows us to pull plugins from a remote git repository and
store them in the default plugins directory. The implementation uses a
Go git library to pull the git repo and automatically save it as a
folder in the plugin path.
** Delete Plugins
:PROPERTIES:
:CUSTOM_ID: delete-plugins
:END:
We delete the plugin folder based on the plugin name provided as an
argument on the cli command. Once the folder is deleted, the plugin
manager automatically knows that the plugin does not exist anymore.

View File

@@ -0,0 +1,170 @@
> [!TIP]
> Haskell bindings supported!: [Bindings documentaton](https://p2prc.akilan.io/Docs/haskell/P2PRC.html)
> [!NOTE]
> Fixing documentation to latest changes. If you have any questions setting up P2PRC either [create an issue](https://github.com/Akilan1999/p2p-rendering-computation/issues/new/choose) or send me an email (me AT akilan dot io).
> Currently HEAD is always intended to stay on a working state. It is recommended to always use HEAD in your go.mod file.
<h1 align="center">
<br>
<a href=""><img src="https://raw.githubusercontent.com/Akilan1999/p2p-rendering-computation/master/Docs/images/p2prclogo.png" alt="p2prc" width="400"></a>
<br>
</h1>
<!-- seperator -->
<div style="display:flex;flex-wrap:wrap;">
<a href="http://perso.crans.org/besson/LICENSE.html"><img alt="GPLv2 license" src="https://img.shields.io/badge/License-GPLv2-blue.svg" style="padding:5px;margin:5px;" /></a>
<a href="https://GitHub.com/Akilan1999/p2p-rendering-computation/graphs/commit-activity"><img alt="Maintenance" src="https://img.shields.io/badge/Maintained%3F-yes-green.svg" style="padding:5px;margin:5px;" /></a>
<a href="http://golang.org"><img alt="made-with-Go" src="https://img.shields.io/badge/Made%20with-Go-1f425f.svg" style="padding:5px;margin:5px;" /></a>
<a href="https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation"><img alt="GoDoc reference example" src="https://img.shields.io/badge/godoc-reference-blue.svg" style="padding:5px;margin:5px;" /></a>
</div>
The main aim of this project was to create a custom peer to peer network. The user acting as the
client has total flexibility on how to batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the containers at any point of time.
## Latest tutorial
[![IMAGE ALT TEXT](https://i.ytimg.com/vi/OMwCpedu5cs/hqdefault.jpg)](https://www.youtube.com/watch?v=OMwCpedu5cs")
<br>
## Table of contents in the current README
1. [Introduction](#Introduction)
2. [Installation](#extend-your-application-with-p2prc)
3. [Design Architecture](#Design-Architecture)
4. [Implementation](#Implementation)
5. [Find out more](#Find-out-more)
<br>
# Table of contents in the Docs folder
1. [Introduction](Docs/Introduction.md)
2. [Installation](Docs/Installation.md)
3. [Abstractions](Docs/Abstractions.md)
<!-- 3. [Design Architecture](DesignArchtectureIntro.md)
1. [Client Module](ClientArchitecture.md)
2. [P2P Module](P2PArchitecture.md)
3. [Server Module](ServerArchitecture.md) -->
4. [Implementation](Docs/Implementation.md)
1. [Client Module](Docs/ClientImplementation.md)
2. [P2P Module](Docs/P2PImplementation.md)
3. [Server Module](Docs/ServerImplementation.md)
4. [Config Module](Docs/ConfigImplementation.md)
5. [Cli Module](Docs/CliImplementation.md)
6. [Plugin Module](Docs/PluginImplementation.md)
7. [Language bindings](Docs/Bindings.md)
8. [Domain name mappings](Docs/Bindings.md)
5. Language bindings
1. [Haskell](Docs/haskell/)
<!-- 5. [Problems](https://github.com/Akilan1999/p2p-rendering-computation/issues) -->
<br>
## Introduction
This project aims to create a peer to peer (p2p) network, where a user can use the p2p network to act as a client (i.e sending tasks) or the server (i.e executing the tasks). A prototype application will be developed, which comes bundled with a p2p module and possible to execute docker containers or virtual environments across selected nodes.
### Objectives
- Background review on peer to peer network, virtual environments, decentralized rendering tools and tools to batch any sort of tasks.
- Creating p2p network
- Server to create a containerised environment
- The client node to run tasks on Server containerised node
[Read more on the introduction](Docs/Introduction.md)
<br>
## Extend your application with P2PRC
```go
package main
import (
"fmt"
"github.com/Akilan1999/p2p-rendering-computation/abstractions"
)
func main() {
_, err := abstractions.Init(nil)
if err != nil {
fmt.Println(err)
return
}
// start p2prc
_, err = abstractions.Start()
if err != nil {
fmt.Println(err)
return
}
// Run server till termination
for {
}
}
```
### Export once this is added export P2PRC as environment paths
```
export P2PRC=<PROJECT PATH>
export PATH=<PROJECT PATH>:${PATH}
```
[Read more](Docs/Abstractions.md) ...
## Installation from source
1. Ensure the Go compiler is installed
```
go version
```
3. Ensure docker is installed (Should run without sudo)
```
docker ps
```
3. Clone this repository
```
git clone https://github.com/Akilan1999/p2p-rendering-computation
```
4. Install and build the project
```
make install
```
- If you look closely you will get outputs such as:
```
// Add them to your .bashrc file
export P2PRC=/<path>/p2p-rendering-computation
export PATH=/<path>/p2p-rendering-computation:${PATH}
```
5. Test if it works
```
p2prc -h
```
or
```
./p2prc -h
```
[Read more on the installation and usage](Docs/Installation.md)
<br>
## Design Architecture
The design architecture was inspired and based on the linux kernel design. The project is segmented into various modules. Each module is responsible for certain tasks in the project. The modules are highly dependent on each other hence the entire codebase can be considered as a huge monolithic chuck which acts as its own library
[Read more on the Design Architecture](Docs/DesignArchtectureIntro.md)
<br>
## Implementation
The programming language used for this project was Golang. The reason Go lang was chosen was because it is a compiled language. The entire codebase is just a single binary file. When distributing to other linux distributing the only requirement would be the binary file to run the code. It is easy to write independant modules and be monolithic at the sametime using Go. Using Go.mod makes it very easy to handle external libraries and modularise code. The go.mod name for the project is git.sr.ht/~akilan1999/p2p-rendering-computation.
[Read more on the Implementation](Docs/Implementation.md)
<br>
## Find out more
As we are working on the open source project p2prc (i.e p2p network designed for computation).If you are interested in participating as a contributor
or just providing feedback on new features to build or even just curious about new features added to the project. We have decided to create a discord group.
[![Support Server](https://discordapp.com/api/guilds/854397492795277322/widget.png?style=banner2)](https://discord.gg/b4nRGTjYqy)
[![Star History Chart](https://api.star-history.com/svg?repos=Akilan1999/p2p-rendering-computation&type=Date)](https://github.com/Gaurav-Gosain)

View File

@@ -0,0 +1,18 @@
* Table of contents
:PROPERTIES:
:CUSTOM_ID: table-of-contents
:END:
1.[[file:Introduction.org][ Introduction]]
2. [[file:Installation.org][Installation]]
3. [[file:Abstractions.org][Abstractions]]
4. [[file:Implementation.org][Implementation]]
1. [[file:ClientImplementation.org][Client Module]]
2. [[file:P2PImplementation.org][P2P Module]]
3. [[file:ServerImplementation.org][Server Module]]
4. [[file:ConfigImplementation.org][Config Module]]
5. [[file:CliImplementation.org][Cli Module]]
6. [[file:PluginImplementation.org][Plugin Module]]
7. [[file:Bindings.org][Language bindings]]
8. [[file:Bindings.org][Domain name mappings]]
5. Language bindings
1. [[file:haskell/][Haskell]]

View File

@@ -0,0 +1,14 @@
* Server Module Architecture
:PROPERTIES:
:CUSTOM_ID: server-module-architecture
:END:
The server module takes care of setting and removing the virtualization
environment (i.e containers) for accessing and doing the appropriate
computation. It also interacts with the peer to peer module to update
the IP table on the server side. The server module accesses information
regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to
upload and download a 50mb.
#+caption: UML diagram of server module
[[file:images/servermoduleArch.png]]

View File

@@ -4,6 +4,14 @@ This section focuses on an in-depth understanding of the server module implement
understand the architecture of the server module refer. The server module can be split
into various sections. Each section will provide information on how a certain feature works.
The server module takes care of setting and removing the virtualization environment (i.e
containers) for accessing and doing the appropriate computation. It also interacts with the peer to
peer module to update the IP table on the server side. The server module
accesses information regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to upload and download a 50mb.
![UML diagram of server module](images/servermoduleArch.png)
## Web framework
The web framework used for the server module is called Gin. The reason Gin was chosen is due to
its wide use and strong documentation available on the official github repository. The default

View File

@@ -0,0 +1,278 @@
* Server Module Implementation
:PROPERTIES:
:CUSTOM_ID: server-module-implementation
:END:
This section focuses on an in-depth understanding of the server module
implementation. To understand the architecture of the server module
refer. The server module can be split into various sections. Each
section will provide information on how a certain feature works.
The server module takes care of setting and removing the virtualization
environment (i.e containers) for accessing and doing the appropriate
computation. It also interacts with the peer to peer module to update
the IP table on the server side. The server module accesses information
regarding CPU and GPU specifications of the machine running the server
module. To do Speed tests the server has routes which allows it to
upload and download a 50mb.
#+caption: UML diagram of server module
[[file:images/servermoduleArch.png]]
** Web framework
:PROPERTIES:
:CUSTOM_ID: web-framework
:END:
The web framework used for the server module is called Gin. The reason
Gin was chosen is due to its wide use and strong documentation available
on the official github repository. The default port used is 8088. For
version 1.0 of the project ,the server needs to keep port 8088 open to
ensure that other clients and servers can detect it. The possible
requests available are GET and POST for this implementation. The
possible responses are either a string or json response or a file. In
the majority of routes a string response refers to an error when calling
the following routes. The following sub topics below will talk about the
route implemented:
*** /server_info
:PROPERTIES:
:CUSTOM_ID: server_info
:END:
This route is responsible to get information about the specifications of
the server. The response of this route is in json if the call was
successful.
*** /50
:PROPERTIES:
:CUSTOM_ID: section
:END:
This route is responsible for returning a randomly generated 50mb file.
This is used to calculate the download speed from the p2p module.
*** /IpTable
:PROPERTIES:
:CUSTOM_ID: iptable
:END:
This route is a POST request that is responsible to update the server IP
table based on the IP table the client provides. Once the server gets
the IP table it checks if the client is also a server. This is done by
calling the url http://:8088/server_info. If the server_info route from
the client responds back with computer specifications of the client.
Then the server initially appends the clients IP to the struct. After
that the IP table received from the client is uploaded to the struct.
Once this is done the server passes the struct to the peer to peer
module function. The peer to peer module function will return the back
with the new struct with the valid server nodes. The server responds
back to the new struct as a json format. If a string is present in the
response then there is probably an error on the server side.
*** /startcontainer
:PROPERTIES:
:CUSTOM_ID: startcontainer
:END:
This route takes in a GET request with the number of TCP ports to open
and checks whether the docker container should be hooked to the GPU or
not. This route talks to the docker module implemented as a sub module
in the server module. More information on the docker module in section
5.4.3. This route calls docker the module to start the container for the
client. The docker module returns back a struct. This struct is returned
back to the client as the json response. This struct consists of
information such as docker id, ports numbers open , information
regarding SSH and VNC connections to the docker container created when
the client created this request.
*** /RemoveContainer
:PROPERTIES:
:CUSTOM_ID: removecontainer
:END:
This route takes in a GET request as the container ID. Based on the
container ID provided ,it calls the docker module which deletes the
container. If the deletion is successful it returns back a string which
says success.
** Server information/ Specification
:PROPERTIES:
:CUSTOM_ID: server-information-specification
:END:
This section provides information on how the server specifications are
read. There are 2 major implementations. The first implementation
mentions how basic information such as RAM usage, CPU specification are
detected and the second implementation mentions how the GPU drivers are
detected and information is extracted. The client has to assume that the
server is using default docker settings in terms of CPU cycles and other
parameters.
*** Basic Information
:PROPERTIES:
:CUSTOM_ID: basic-information
:END:
The file name for these functions is called gopsutil.go. This codebase
uses the library gopsutil. Gopsutil has various packages or modules
within the library which have functions implemented to get system
information. The following information is stored in a struct and the
function returns that struct.
#+begin_src go
type SysInfo struct {
Hostname string `bson:hostname`
Platform string `bson:platform`
CPU string `bson:cpu`
RAM uint64 `bson:ram`
Disk uint64 `bson:disk`
GPU *Query `xml: GpuInfo`
}
#+end_src
*** GPU Information
:PROPERTIES:
:CUSTOM_ID: gpu-information
:END:
The file name for these functions is called GPU.go. This codebase checks
if the Nvidia driver exists and returns the driver information. To do
this a shell command called nvidia-smi is executed. This shell command
is executed with a --xml as flag to ensure that the output is in the XML
format. If there is an output as a xml format, that means there is an
nvidia driver installed, and the function just reads the output and
stores it to the struct and returns the GPU information.
#+begin_src go
type Query struct {
DriveVersion string `xml:"driver_version"`
Gpu Gpu `xml:"gpu"`
}
type Gpu struct{
GpuName string `xml:"product_name"`
BiosVersion string `xml:"vbios_version"`
FanSpeed string `xml:"fan_speed"`
Utilization GpuUtilization `xml:"utilization"`
Temperature GpuTemperature `xml:"temperature"`
Clock GpuClock `xml:"clocks"`
}
type GpuUtilization struct {
GpuUsage string `xml:"gpu_util"`
MemoryUsage string `xml:"memory_util"`
}
type GpuTemperature struct {
GpuTemp string `xml:"gpu_temp"`
}
type GpuClock struct {
GpuClock string `xml:"graphics_clock"`
GpuMemClock string `xml:"mem_clock"`
}
#+end_src
** Docker Module
:PROPERTIES:
:CUSTOM_ID: docker-module
:END:
This section provides information on how the server module interacts
with the docker containers. The server calls 2 routes which either
creates or removes the docker container. Docker has a huge advantage
because it takes less than 20 seconds to spin up a new container once
it's built and executed at least once. For docker operations a separate
module/package has been created. The following subtopics will provide
more information on how this package works.
*** Docker Api
:PROPERTIES:
:CUSTOM_ID: docker-api
:END:
For this the api has been taken from the official docker repository. To
be more specific it is the client module in the official docker
repository. Docker was built using Go. During this project Docker
functions could be directly called from the docker repository. The
Docker api initially ensures that it can detect the docker environment
variables. Once detected, it can execute various functions from the
docker client module. The reason the docker api was selected was to
detect and handle errors better.
*** Docker Image
:PROPERTIES:
:CUSTOM_ID: docker-image
:END:
The docker image used to spin up the containers is called
ConSol/docker-headless-vnc-container. The following container was
modified to open SSH ports for an SSH connection. The following docker
image runs ubuntu 16. The reason this image was chosen as a default is
because if the client wants to access the container in the form of a
desktop environment. This image would allow the client to do so from
just a browser.
*** Build container
:PROPERTIES:
:CUSTOM_ID: build-container
:END:
This function pulls the docker image locally and builds the image.
Initially there is a timeout function to ensure that building the image
does not take too long to build. The next phase would be based on the
path to get the DockerFile. The tag name of the container is set as
p2p-ubuntu as default. Once the following is set then the docker build
command is executed.
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
After building the container it needs to be executed for the user to
access the container and do certain operations. The docker
package/module has a function to do this. The function takes in the
docker environment as a parameter and also the docker struct. The docker
struct has information such as the TCP ports which are supposed to be
open and whether the docker container should have the GPU hooked to it
or not. Based on the appropriate information provided ,the docker image
gets started. The Image gets started by interacting with the docker
client modules. When hooking the GPU the docker run command is called
from the shell. This is because the docker Api does not support the GPU
module yet. When the container is executed for the first time it takes
more than 10 minutes to build. From the second time onwards it takes
only 10 seconds to run.
*** Stop and remove container
:PROPERTIES:
:CUSTOM_ID: stop-and-remove-container
:END:
This implementation here ensures that the docker is stopped, and the
container is removed. This is to ensure it does not utilize server
resources when it is not being used, or the task that is intended to be
executed is complete. To run this function all that is needed is the
docker container ID. If the function is successful it returns a string
that says success.
*** Ports json file
:PROPERTIES:
:CUSTOM_ID: ports-json-file
:END:
This file will help map internal ports inside a container to external
ports inside a container. A common example would be the SSH port which
is port 22 inside the docker container and is mapped to random TCP port
outside container so that any external machines can directly connect
into the container. The below representation mentions of where the
ports.json file is located and also the format of that file.
#+begin_example
|_ <Container name>
|_ Dockerfile
|_ description.txt
|_ ports.json // The ports file
#+end_example
Format of the ports.json file
#+begin_example
{
"Port": [
{
"PortName": "<Port name>",
"InternalPort": <internal port>,
"Type": "<tcp/udp>",
"ExternalPort": <external port>,
"IsUsed": "<boolean value (i.e true or false)>",
"Description": "<description about the port>"
}, ... n
]
}
#+end_example

View File

@@ -0,0 +1,800 @@
yes* Chapter 1: Introduction
:PROPERTIES:
:CUSTOM_ID: chapter-1-introduction
:END:
** Abstract
:PROPERTIES:
:CUSTOM_ID: abstract
:END:
This project focuses on creating a framework on running heavy tasks that
a regular computer cannot run easily such as graphically demanding video
games, rendering 3D animations , protein folding simulations. In this
project the major focus will not be on the financial incentive part. A
peer to peer network will be created to help run tasks decentrally,
increasing bandwidth for running tasks. To ensure the tasks in the peer
to peer network do not corrupt the server 0S (Operating System), they
will be executed in a virtual environment in the server.
The main aim of this project was to create a custom peer to peer
network. The user acting as the client has total flexibility on how to
batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the
containers at any point of time.
** Motivation
:PROPERTIES:
:CUSTOM_ID: motivation
:END:
Many of the users rely on our PC / Laptop or servers that belong to a
server farm to run heavy tasks and with the demand of high creativity
requires higher computing power. Buying a powerful computer every few
years to run a bunch of heavy tasks which are not executed as frequently
to reap the benefits can be inefficient utilization of hardware. On the
other end, renting servers to run these heavy tasks can be really
useful. Ethically speaking this is leading to monopolisation of
computing power similar to what is happening in the web server area. By
using peer to peer principles it is possible to remove the
monopolisation factor and increase the bandwidth between the client and
server.
* Installation
:PROPERTIES:
:CUSTOM_ID: installation
:END:
Over here we will cover the basic steps to get the server and client
side running.
** Latest release install
:PROPERTIES:
:CUSTOM_ID: latest-release-install
:END:
https://github.com/Akilan1999/p2p-rendering-computation/releases
** Install from Github master branch
:PROPERTIES:
:CUSTOM_ID: install-from-github-master-branch
:END:
*** Install Go lang
:PROPERTIES:
:CUSTOM_ID: install-go-lang
:END:
The entire the implementation of this project is done using Go lang.
Thus, we need go lang to compile to code to a binary file.
[[https://golang.org/doc/install][Instructions to install Go lang]]
*** Install Docker
:PROPERTIES:
:CUSTOM_ID: install-docker
:END:
In this project the choice of virtualization is Docker due to it's wide
usage in the developer community. In the server module we use the Docker
Go API to create and interact with the containers.
[[https://docs.docker.com/get-docker/][Instructions to install docker]]
[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions
to install docker GPU]]
#+begin_example
// Do ensure that the docker command does not need sudo to run
sudo chmod 666 /var/run/docker.sock
#+end_example
*** Build Project and install project
:PROPERTIES:
:CUSTOM_ID: build-project-and-install-project
:END:
To set up the internal dependencies and build the entire go code into a
single binary
#+begin_example
make install
#+end_example
**** For Windows
:PROPERTIES:
:CUSTOM_ID: for-windows
:END:
To set up P2PRC on Windows, simply run this batch file. *Make sure you
are not in admin mode when running this.*
#+begin_example
.\install.bat
#+end_example
*** Add appropriate paths to =.bashrc=
:PROPERTIES:
:CUSTOM_ID: add-appropriate-paths-to-.bashrc
:END:
#+begin_example
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
#+end_example
*** Set up configuration file
:PROPERTIES:
:CUSTOM_ID: set-up-configuration-file
:END:
#+begin_example
make configfile
#+end_example
Open the config file =config.json= and add the IPv6 address if you have
one.
*** Test if binary works
:PROPERTIES:
:CUSTOM_ID: test-if-binary-works
:END:
#+begin_example
p2prc --help
#+end_example
**** Output:
:PROPERTIES:
:CUSTOM_ID: output
:END:
#+begin_example
NAME:
p2p-rendering-computation - p2p cli application to create and access VMs in other servers
USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
<version no>
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--Server, -s Starts server (default: false) [$SERVER]
--UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST]
--ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS]
--AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER]
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
--Specification value, --specs value Specs of the server node [$SPECS]
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--help, -h show help (default: false)
--version, -v print the version (default: false)
#+end_example
--------------
* Using basic commands
:PROPERTIES:
:CUSTOM_ID: using-basic-commands
:END:
*** Start as a server
:PROPERTIES:
:CUSTOM_ID: start-as-a-server
:END:
Do ensure you have Docker installed for this
#+begin_example
p2prc -s
#+end_example
*** View server Specification
:PROPERTIES:
:CUSTOM_ID: view-server-specification
:END:
#+begin_example
p2prc --specs=<ip address>
#+end_example
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
use the =--gpu= if you know the other machine has a gpu.
#+begin_example
p2prc --touch=<server ip address> -p <number of ports> --gpu
#+end_example
*** Remove container
:PROPERTIES:
:CUSTOM_ID: remove-container
:END:
The docker id is present in the output where you create a container
#+begin_example
p2prc --rm=<server ip address> --id=<docker container id>
#+end_example
*** Adding servers to ip table
:PROPERTIES:
:CUSTOM_ID: adding-servers-to-ip-table
:END:
#+begin_example
p2prc --as=<server ip address you want to add>
#+end_example
*** Update ip table
:PROPERTIES:
:CUSTOM_ID: update-ip-table
:END:
#+begin_example
p2prc --us
#+end_example
*** List Servers
:PROPERTIES:
:CUSTOM_ID: list-servers
:END:
#+begin_example
p2prc --ls
#+end_example
*** View Network interfaces
:PROPERTIES:
:CUSTOM_ID: view-network-interfaces
:END:
#+begin_example
p2prc --ni
#+end_example
*** Viewing Containers created Client side
:PROPERTIES:
:CUSTOM_ID: viewing-containers-created-client-side
:END:
#+begin_example
p2prc --tc
#+end_example
[[file:ClientImplementation.md#tracking-containers][read more on
tracking containers]]
*** Running plugin
:PROPERTIES:
:CUSTOM_ID: running-plugin
:END:
#+begin_example
p2prc --plugin <plugin name> --id <container id or group id>
#+end_example
*** Create group
:PROPERTIES:
:CUSTOM_ID: create-group
:END:
#+begin_example
p2prc --cgroup
#+end_example
*** Add container to group
:PROPERTIES:
:CUSTOM_ID: add-container-to-group
:END:
#+begin_example
p2prc --group <group id> --id <container id>
#+end_example
*** View groups
:PROPERTIES:
:CUSTOM_ID: view-groups
:END:
#+begin_example
p2prc --groups
#+end_example
*** View specific group
:PROPERTIES:
:CUSTOM_ID: view-specific-group
:END:
#+begin_example
p2prc --group <group id>
#+end_example
*** Delete container from group
:PROPERTIES:
:CUSTOM_ID: delete-container-from-group
:END:
#+begin_example
p2prc --rmcgroup --group <group id> --id <container id>
#+end_example
*** Delete entire group
:PROPERTIES:
:CUSTOM_ID: delete-entire-group
:END:
#+begin_example
p2prc --rmgroup <group id>
#+end_example
[[file:ClientImplementation.md#Grouping-Containers][read more on
grouping containers]] ### Extending usecase of P2PRC (Requires a go
compiler to run)
#+begin_example
p2prc --gen <project name> --mod <go module name>
#+end_example
[[file:GenerateImplementation.md][read more about the generate module]]
*** Pulling plugin from a remote repo
:PROPERTIES:
:CUSTOM_ID: pulling-plugin-from-a-remote-repo
:END:
#+begin_example
p2prc --pp <repo link>
#+end_example
*** Deleting plugin from the plugin directory
:PROPERTIES:
:CUSTOM_ID: deleting-plugin-from-the-plugin-directory
:END:
#+begin_example
p2prc --rp <plugin name>
#+end_example
*** Added custom metadata about the current node
:PROPERTIES:
:CUSTOM_ID: added-custom-metadata-about-the-current-node
:END:
#+begin_example
p2prc --amd "custom metadata"
#+end_example
--------------
* Using Plugins
:PROPERTIES:
:CUSTOM_ID: using-plugins
:END:
This feature is still Under Development:
[[file:PluginImplementation.md][Read more on the implementation]]
**** Dependencies
:PROPERTIES:
:CUSTOM_ID: dependencies
:END:
- Ansible:
- Debian/ubuntu: =sudo apt install ansible=
- Others:
[[https://ansible-tips-and-tricks.readthedocs.io/en/latest/ansible/install/][Installation
link]]
**** Run Test Cases
:PROPERTIES:
:CUSTOM_ID: run-test-cases
:END:
- Generate Test Case Ansible file
- =make testcases=
- Enter inside plugin directory and run tests.
#+begin_quote
[!NOTE] That docker needs to installed and needs to run without sudo.
Refer the section [[#install-docker][Install Docker]]. - =cd plugin= -
=go test .=
#+end_quote
* P2P Module Implementation
:PROPERTIES:
:CUSTOM_ID: p2p-module-implementation
:END:
The P2P module (i.e Peer to Peer Module) is responsible for storing the
IP table and interacting with the IP table. In the following
implementation of the P2P module ,the IP table stores information about
servers available in the network. The other functionality the P2P module
takes care of is doing the appropriate speed tests to the servers in the
IP table. This is for informing the users about nodes which are close by
and nodes which have quicker uploads and downloads speeds. The module is
responsible to ensure that there are no duplicate server IPs in the IP
table and to remove all server IPs which are not pingable.
#+caption: UML diagram of P2P module
[[file:images/p2pmoduleArch.png]]
The peer to peer implementation was built from scratch. This is because
other peer to peer libraries were on the implementation of the
Distributed hash table. At the current moment all those heavy features
are not needed because the objective is to search and list all possible
servers available. The limitation being that to be a part of the network
the user has to know at least 1 server. The advantage of building from
scratch makes the module super light and possibility for custom
functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
** IP Table
:PROPERTIES:
:CUSTOM_ID: ip-table
:END:
The ip table file is a json as the format with a list of servers ip
addresses, latencies, downloads and uploads speeds. The functions
implemented include read file, write file and remove duplicate IP
addresses. The remove duplicate IP address function exists because
sometimes servers IP tables can have the same ip addresses as what the
client has. The path of the IP table json file is received from the
configuration module.
#+begin_src json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>",
}
]
}
#+end_src
*** Latency
:PROPERTIES:
:CUSTOM_ID: latency
:END:
The latency is measured in milliseconds. The route /server_info is
called from the server and time it takes to provide a json response is
recorded.
** NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src
* Language Bindings
:PROPERTIES:
:CUSTOM_ID: language-bindings
:END:
[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]]
refers to wrappers to bridge 2 programming languages. This is used in
P2PRC to extend calling P2PRC functions in other programming languages.
Currently this is done by generating =.so= and =.h= from the Go
compiler.
** How to build shared object files
:PROPERTIES:
:CUSTOM_ID: how-to-build-shared-object-files
:END:
**** The easier way
:PROPERTIES:
:CUSTOM_ID: the-easier-way
:END:
#+begin_src sh
# Run
make sharedObjects
#+end_src
**** Or the direct way
:PROPERTIES:
:CUSTOM_ID: or-the-direct-way
:END:
#+begin_src sh
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
#+end_src
**** If successfully built:
:PROPERTIES:
:CUSTOM_ID: if-successfully-built
:END:
#+begin_src sh
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
#+end_src
** Workings under the hood
:PROPERTIES:
:CUSTOM_ID: workings-under-the-hood
:END:
Below are a sample set of commands to open the bindings implementation.
#+begin_example
# run
cd Bindings/
# list files
ls
# search for file
Client.go
#+end_example
*** In Client go
:PROPERTIES:
:CUSTOM_ID: in-client-go
:END:
There a few things to notice which are different from your standard Go
programs:
**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required.
:PROPERTIES:
:CUSTOM_ID: we-import-c-which-means-cgo-is-required.
:END:
#+begin_src go
import "C"
#+end_src
**** 2. All functions which are required to be called from other programming languages have comment such as.
:PROPERTIES:
:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as.
:END:
#+begin_src go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
#+end_src
**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
:PROPERTIES:
:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure.
:END:
#+begin_src go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
#+end_src
**** It is easy to notice that:
:PROPERTIES:
:CUSTOM_ID: it-is-easy-to-notice-that
:END:
- =ConvertStructToJSONString(<go object>)=: This is a helper function
that convert a go object to JSON string initially and converts it to
=CString=.
- =(output *C.char)=: This is the return type for most of the functions.
**** A Pseudo code to refer to the common function implementation shape could be represented as:
:PROPERTIES:
:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as
:END:
#+begin_example
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
#+end_example
** Current languages supported
:PROPERTIES:
:CUSTOM_ID: current-languages-supported
:END:
- Python
*** Build sample python program
:PROPERTIES:
:CUSTOM_ID: build-sample-python-program
:END:
The easier way
#+begin_src sh
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ p2prc.py
#+end_src
Above shows a generated folder which consists of a folder called
"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files.
=p2prc.py= refers to a sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python, This generated
folder can copied and created as a new git repo for P2PRC extensions
scripted or used a reference point as proof of concept that P2PRC can be
called from other programming languages.
* Config Implementation
:PROPERTIES:
:CUSTOM_ID: config-implementation
:END:
The configuration module is responsible to store basic information of
absolute paths of files being called in the Go code. In a full-fledged
Cli the configuration file can be found in the directory /etc/ and from
there points to location such as where the IP table file is located. In
the future implementation the config file will have information such as
number of hops and other parameters to tweak and to improve the
effectiveness of the peer to peer network. The configuration module was
implemented using the library Viper. The Viper library automates
features such as searching in default paths to find out if the
configuration file is present. If the configuration file is not present
in the default paths then it auto generates the configuration file. The
configurations file can be in any format. In this project the
configuration file was generated using JSON format.
#+begin_src json
{
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
#+end_src
* Abstractions
:PROPERTIES:
:CUSTOM_ID: abstractions
:END:
The Abstractions package consists of black-boxed functions for P2PRC.
** Functions
:PROPERTIES:
:CUSTOM_ID: functions
:END:
- =Init(<Project name>)=: Initializes P2PRC with all the needed
configurations.
- =Start()=: Starts p2prc as a server and makes it possible to extend by
adding other routes and functionality to P2PRC.
- =MapPort(<port no>)=: On the local machine the port you want to export
to world.
- =StartContainer(<ip address>)=: The machine on the p2p network where
you want to spin up a docker container.
- =RemoveContainer(<ip address>,<container id>)=: Terminate container
based on the IP address and container name.
- =GetSpecs(<ip address>)=: Get specs of a machine on the network based
on the IP address.
- =ViewIPTable()=: View the IP table which about nodes in the network.
- =UpdateIPTable()=: Force update IP table to learn about new nodes
faster.
* NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src

View File

@@ -1,116 +0,0 @@
# Generate Module
P2PRC is a great layer of abstraction. This means that in many cases it is not an end product but rather
a tool that customized as an end product. An example would be writing your own billing module to monetize
the computation power available. The generate module copies the current with the appropriate git histories
and keeps only the go files which would be useful to edit. To use the generate module the user will need
to have a go compiler present in his computer. Due to the introduction of this module there will 2 releases:
- Regular Release (Consists of only the build binary and cli command cannot access the generate module)
- Developer Release (Consists of important Go files and the cli can access the generate module)
## How does this work ?
### [Struct information](https://github.com/Akilan1999/p2p-rendering-computation/blob/9d69aed8ce0fe5273aaff2828f7d51c3d5ac2ce4/generate/generate.go#L19)
- ### ```Generate.go```:
This file creates a local copy of P2PRC from where the CLI was called from.
This go file also does various stuff like instruction of file should be ignored when copying and
which of should not be. Now let's understand this. Below is a sample code which does the following:
```go
//----------------------------------------------------------------
// Action performed:
// - Ensuring main.go file exists
// - Skipping all .go files apart from the ones listed above
// - Skipping .idea/ directory
// - Skipping Makefile file
//----------------------------------------------------------------
Options.Skip = func(src string) (bool, error) {
switch {
case strings.HasSuffix(src, "main.go"):
return false, nil
case strings.HasSuffix(src, ".go"):
return true, nil
case strings.HasSuffix(src, ".idea"):
return true, nil
case strings.HasSuffix(src, "Makefile"):
return true, nil
default:
return false, nil
}
}
// Doing the copy
err = copy.Copy("<P2PRC folder you want to copy from>", "<PATH to the directory>", Options)
```
Unfortunately currently this will have to be manually edited in the ```Generate.go``` file. When using the generate
module the user also creates their own Go module which is the modified version of P2PRC. This means
if the 1 modified package is using another modified package then the appropriate import have to be modified
in the file where the import is called:
Ex:
```go
//Sample Project module name = Test
//Package names:
//- Test/Genius
//- Test/GeGeGenuis
//
// When we call the generate function with the new project with the module name = MicDrop
// The new package name would be:
// - MicDrop/Genius
// - MicDrop/GeGeGenuis
// Test/Genius code depends on the package Test/GeGeGenuis
import (
"Test/GeGeGenuis"
)
// When we create a new module with the copy of the
// existing project we need change:
import (
"MicDrop/GeGeGenuis"
)
```
To do this we have built functions which can modify import names in the Go file provided.
To customize the use case of your generate module you would need to manually add your own
imports which are supposed to be replaced and in which files they are supposed to be replaced
in.
```go
// 1.0 - Test/Genius.go -> GeGeGenuis module
// a is struct of type NewProject
a.FileNameAST = "<path to project to copy from>/Test/Genius.go"
// Get AST information of the file
err := a.GetASTGoFile()
if err != nil {
return err
}
// Change the appropriate Go file
err = a.ChangeImports("Test/GeGeGenuis", "MicDrop/GeGeGenuis")
if err != nil {
return err
}
// Writes the change to the appropriate file
err = a.WriteGoAst()
if err != nil {
return err
}
```
Higher order of execution of ```Generate.go```:
1. Copy entire P2PRC project and ignores files which are not meant to be copied
2. The folder name will be based on the new project name and the module name based on the new
module name provided.
3. Modifies the appropriate imports in the project as instructed in the code.
4. Creates a commit with the new changes in the new project.
- ### ``` modifyGenerate.go```:
This a really simple implementation where we replace the imports
in certain files as instructed from ```generate.go```. To do we create an AST (i.e Abstract Syntax tree)
from new file we want to change the imports in. AST create a tree structure of expression. To change the
import we can just traverse to the appropriate expression and change the value of that expression in
the case of modifying imports. This approach is more simple than using templates.

View File

@@ -1,20 +0,0 @@
# Implementation
This chapter describes how the project was built. It talks in depth of the implementation
performed to give a better understanding of the project.
## Programming langauge used
The programming language used for this project was Golang. The reason Go lang was chosen was
because it is a compiled language.
The entire codebase is just a single binary file. When
distributing to other linux distributing the only requirement would be the binary file to run the
code. It is easy to write independant modules and be monolithic at the sametime using Go. Using
Go.mod makes it very easy to handle external libraries and modularise code. The go.mod name for
the project is git.sr.ht/~akilan1999/p2p-rendering-computation.
## [Cli Module](CliImplementation.md)
## [Config Module](ConfigImplementation.md)
## [Server Module](ServerImplementation.md)
## [Client Module](ClientImplementation.md)
## [P2P Module](P2PImplementation.md)
## [Plugin Module](PluginImplementation.md)
## [Generate Module](GenerateImplementation.md)

View File

@@ -1,49 +0,0 @@
# P2P Module Implementation
The peer to peer implementation was built from scratch. This is because other peer to peer
libraries were on the implementation of the Distributed hash table. At the current moment all
those heavy features are not needed because the objective is to search and list all possible servers
available. The limitation being that to be a part of the network the user has to know at least 1
server and has to have DMZ enabled from the router if the user wants to act as a server out of the
users local network. The advantage of building from scratch makes the module super light and
possibility for custom functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
## IP Table
The ip table file is a json as the format with a list of servers ip addresses, latencies, downloads and
uploads speeds. The functions implemented include read
file, write file and remove duplicate IP addresses. The remove duplicate IP address function exists
because sometimes servers IP tables can have the same ip addresses as what the client has. The
path of the IP table json file is received from the configuration module.
```json
{
"ip_address": [
{
"ipv4": "<ipv4 address>",
"latency": "<latency>",
"download": "<download>",
"upload": "<upload>"
"port no": "<server port no>"
}
]
}
```
## Speed Test
The speed test functions populate the fields which are latency, download, upload speed. Before the
speed test begins for each server IP address. The p2p module ensures that each server IP address
is pingable. If the server IP address is not pingable then it removes that IP address from the struct.
### Latency
The latency is measured in milliseconds. The route /server_info is called from the
server and time it takes to provide a json response is recorded.
### Download speed
The download speed is measured as (<file size>/<time taken to
download>)*8. This gives the result in megabits per second. The file downloaded is a 50 mb
auto generated file.
### Upload speed
The upload speed is measured as (<file size>/<time taken to upload>)*8. This
gives the results in megabits per second. The file uploaded is a 50 mb auto generated file.
The route /upload is called from the server side to upload the file.

View File

@@ -1,35 +0,0 @@
# Problems in implementation
### Number of Devices in the network:
The current implementation has the major flaw which is that
the server has to be in port 8088 to detected in the public
network. As we know most personal networks have a single IP
address. This means we cannot have duplicate ports. A fix can
be to mention the open port on the IP table file.
(Ex: Possible feild to be added)
```
{
"ip_address": [
{
"ipv4": "localhost",
"latency": 14981051,
"download": 8142.122540206258,
"upload": 3578.766512629995,
"port": 8088
}
]
}
```
### Broadcast of container specs
At the moment the container specs are not broadcasted rather
just the machines specs and this module has yet to be tested
rigorously.
### Intergration with GPU
Certain machines have GPUs present in them which provide a
huge advantage for those who want to do rendering and certain
sort of computation. The Aim is to only allow it to be compatible
with Nvidia. But an better idea would be to provide compatability
with multiple GPU providers.

View File

@@ -1,18 +0,0 @@
# Table of contents
1. [Introduction](Introduction.md)
2. [Installation](Installation.md)
3. [Abstractions](Abstractions.md)
3. [Design Architecture](DesignArchtectureIntro.md)
1. [Client Module](ClientArchitecture.md)
2. [P2P Module](P2PArchitecture.md)
3. [Server Module](ServerArchitecture.md)
4. [Implementation](Implementation.md)
1. [Client Module](ClientImplementation.md)
2. [P2P Module](P2PImplementation.md)
3. [Server Module](ServerImplementation.md)
4. [Config Module](ConfigImplementation.md)
5. [Cli Module](CliImplementation.md)
6. [Plugin Module](PluginImplementation.md)
5. [Problems](https://github.com/Akilan1999/p2p-rendering-computation/issues)

View File

@@ -0,0 +1,82 @@
<mxfile host="app.diagrams.net" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15" version="26.0.6">
<diagram name="Page-1" id="D9MDhD-JVaS0pL1oCc8_">
<mxGraphModel dx="954" dy="647" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
<mxCell id="qpEPyUIO_lfUjYMVJA4N-15" value="" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
<mxGeometry x="370" y="305" width="150" height="115" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-14" value="" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
<mxGeometry x="370" y="310" width="150" height="120" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-1" value="" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
<mxGeometry x="330" y="250" width="440" height="220" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-2" value="&lt;h1 style=&quot;margin-top: 0px;&quot;&gt;Home network&lt;/h1&gt;" style="text;html=1;whiteSpace=wrap;overflow=hidden;rounded=0;" vertex="1" parent="1">
<mxGeometry x="460" y="260" width="180" height="50" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-3" value="P2PRC n1&amp;nbsp;" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
<mxGeometry x="380" y="320" width="120" height="30" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-4" value="P2PRC n2&amp;nbsp;" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
<mxGeometry x="380" y="380" width="120" height="30" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-6" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0.25;entryDx=0;entryDy=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=3;" edge="1" parent="1" target="qpEPyUIO_lfUjYMVJA4N-3">
<mxGeometry relative="1" as="geometry">
<mxPoint x="585" y="365" as="sourcePoint" />
<mxPoint x="515" y="335" as="targetPoint" />
</mxGeometry>
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-5" value="Morello board" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
<mxGeometry x="570" y="350" width="120" height="30" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-7" value="" style="endArrow=classic;html=1;rounded=0;entryX=1;entryY=0.25;entryDx=0;entryDy=0;exitX=0;exitY=0.5;exitDx=0;exitDy=0;" edge="1" parent="1" source="qpEPyUIO_lfUjYMVJA4N-5" target="qpEPyUIO_lfUjYMVJA4N-4">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="570" y="370" as="sourcePoint" />
<mxPoint x="440" y="310" as="targetPoint" />
<Array as="points">
<mxPoint x="540" y="365" />
<mxPoint x="540" y="390" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-8" value="Group of Root Nodes" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
<mxGeometry x="110" y="130" width="170" height="50" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-9" value="Node Accessing Morello&amp;nbsp;&lt;div&gt;board outside the home network&amp;nbsp;&lt;/div&gt;" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
<mxGeometry x="20" y="300" width="150" height="50" as="geometry" />
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-11" value="" style="endArrow=classic;startArrow=classic;html=1;rounded=0;exitX=0.46;exitY=-0.02;exitDx=0;exitDy=0;exitPerimeter=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=2;" edge="1" parent="1" source="qpEPyUIO_lfUjYMVJA4N-9">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="390" y="360" as="sourcePoint" />
<mxPoint x="110" y="150" as="targetPoint" />
<Array as="points">
<mxPoint x="90" y="150" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-13" value="" style="endArrow=classic;startArrow=classic;html=1;rounded=0;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="qpEPyUIO_lfUjYMVJA4N-4" target="qpEPyUIO_lfUjYMVJA4N-8">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="390" y="360" as="sourcePoint" />
<mxPoint x="260" y="370" as="targetPoint" />
<Array as="points">
<mxPoint x="300" y="395" />
<mxPoint x="300" y="155" />
</Array>
</mxGeometry>
</mxCell>
<mxCell id="qpEPyUIO_lfUjYMVJA4N-17" value="" style="endArrow=classic;startArrow=classic;html=1;rounded=0;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=3;" edge="1" parent="1" source="qpEPyUIO_lfUjYMVJA4N-3" target="qpEPyUIO_lfUjYMVJA4N-8">
<mxGeometry width="50" height="50" relative="1" as="geometry">
<mxPoint x="390" y="360" as="sourcePoint" />
<mxPoint x="290" y="150" as="targetPoint" />
<Array as="points">
<mxPoint x="300" y="335" />
<mxPoint x="300" y="155" />
</Array>
</mxGeometry>
</mxCell>
</root>
</mxGraphModel>
</diagram>
</mxfile>

40
Docs/haskell/P2PRC.html Normal file

File diff suppressed because one or more lines are too long

120
Docs/haskell/README.md Normal file
View File

@@ -0,0 +1,120 @@
<div id="package-header">
<span class="caption">p2prc-0.1.0.0: P2PRC haskell library</span>
- [Contents](index.html)
- [Index](doc-index.html)
</div>
<div id="content">
<div id="module-header">
| | |
|--------------|-----------------------------------------|
| Copyright | Copyright (C) 2006-2024 John MacFarlane |
| License | GNU GPL, version 2 or above |
| Maintainer | John MacFarlane \<jgm@berkeley.edu\> |
| Stability | alpha |
| Portability | portable |
| Safe Haskell | Safe-Inferred |
| Language | GHC2021 |
P2PRC
</div>
<div id="description">
Description
<div class="doc">
This helper module exports the main writers, readers, and data structure
definitions from the Pandoc libraries.
A typical application will chain together a reader and a writer to
convert strings from one format to another. For example, the following
simple program will act as a filter converting markdown fragments to
reStructuredText, using reference-style links instead of inline links:
module Main where
import Text.Pandoc
import Data.Text (Text)
import qualified Data.Text.IO as T
mdToRST :: Text -> IO Text
mdToRST txt = runIOorExplode $
readMarkdown def txt
>>= writeRST def{ writerReferenceLinks = True }
main :: IO ()
main = do
T.getContents >>= mdToRST >>= T.putStrLn
</div>
</div>
<div id="synopsis">
Synopsis
- [runP2PRC](#v:runP2PRC) ::
[MapPortRequest](P2PRC.html#t:MapPortRequest "P2PRC") -\>
[IO]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/System-IO.html#t:IO "System.IO")
()
- <span class="keyword">data</span> [MapPortRequest](#t:MapPortRequest)
= [MkMapPortRequest](#v:MkMapPortRequest)
[Int]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-Int.html#t:Int "Data.Int")
[String]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-String.html#t:String "Data.String")
</div>
<div id="interface">
# Documentation
<div class="top">
<span id="v:runP2PRC" class="def">runP2PRC</span> ::
[MapPortRequest](P2PRC.html#t:MapPortRequest "P2PRC") -\>
[IO]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/System-IO.html#t:IO "System.IO")
() <a href="#v:runP2PRC" class="selflink">#</a>
<div class="doc">
Hello World
</div>
</div>
<div class="top">
<span class="keyword">data</span> <span id="t:MapPortRequest"
class="def">MapPortRequest</span>
<a href="#t:MapPortRequest" class="selflink">#</a>
<div class="subs constructors">
Constructors
| | |
|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----|
| <span id="v:MkMapPortRequest" class="def">MkMapPortRequest</span> [Int]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-Int.html#t:Int "Data.Int") [String]($%7Bpkgroot%7D/../../../../dcnyq1a8qi8x59n5p53d0dx42cl8hf8x-ghc-9.6.5-doc/share/doc/ghc/html/libraries/base-4.18.2.1/Data-String.html#t:String "Data.String") |   |
</div>
</div>
</div>
</div>
<div id="footer">
Produced by [Haddock](http://www.haskell.org/haddock/) version 2.29.2
</div>

File diff suppressed because one or more lines are too long

2
Docs/haskell/haddock-bundle.min.js vendored Normal file

File diff suppressed because one or more lines are too long

1
Docs/haskell/index.html Normal file
View File

@@ -0,0 +1 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><meta name="viewport" content="width=device-width, initial-scale=1" /><title>p2prc-0.1.0.0: P2PRC haskell library</title><link href="linuwial.css" rel="stylesheet" type="text/css" title="Linuwial" /><link rel="stylesheet" type="text/css" href="quick-jump.css" /><link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=PT+Sans:400,400i,700" /><script src="haddock-bundle.min.js" async="async" type="text/javascript"></script><script type="text/x-mathjax-config">MathJax.Hub.Config({ tex2jax: { processClass: "mathjax", ignoreClass: ".*" } });</script><script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript"></script></head><body><div id="package-header"><span class="caption">p2prc-0.1.0.0: P2PRC haskell library</span><ul class="links" id="page-menu"><li><a href="index.html">Contents</a></li><li><a href="doc-index.html">Index</a></li></ul></div><div id="content"><div id="description"><h1>p2prc-0.1.0.0: P2PRC haskell library</h1><div class="doc"><p>Implements a client interface to the P2PRC networking runtime</p></div></div><div id="module-list"><p class="caption">Modules</p><div id="module-list"><p class="caption">p2prc-0.1.0.0</p><ul><li><span class="module"><span class="noexpander">&nbsp;</span><a href="P2PRC.html">P2PRC</a></span></li></ul></div></div></div><div id="footer"><p>Produced by <a href="http://www.haskell.org/haddock/">Haddock</a> version 2.29.2</p></div></body></html>

881
Docs/haskell/linuwial.css Normal file
View File

@@ -0,0 +1,881 @@
/* @group Fundamentals */
* { margin: 0; padding: 0 }
/* Is this portable? */
html {
background-color: white;
width: 100%;
height: 100%;
}
body {
background: #fefefe;
color: #111;
text-align: left;
min-height: 100vh;
position: relative;
-webkit-text-size-adjust: 100%;
-webkit-font-feature-settings: "kern" 1, "liga" 0;
-moz-font-feature-settings: "kern" 1, "liga" 0;
-o-font-feature-settings: "kern" 1, "liga" 0;
font-feature-settings: "kern" 1, "liga" 0;
letter-spacing: 0.0015rem;
}
#content a {
overflow-wrap: break-word;
}
p {
margin: 0.8em 0;
}
ul, ol {
margin: 0.8em 0 0.8em 2em;
}
dl {
margin: 0.8em 0;
}
dt {
font-weight: bold;
}
dd {
margin-left: 2em;
}
a { text-decoration: none; }
a[href]:link { color: #9E358F; }
a[href]:visited {color: #6F5F9C; }
a[href]:hover { text-decoration:underline; }
a[href].def:link, a[href].def:visited { color: rgba(69, 59, 97, 0.8); }
a[href].def:hover { color: rgb(78, 98, 114); }
/* @end */
/* @group Show and hide with JS */
body.js-enabled .hide-when-js-enabled {
display: none;
}
/* @end */
/* @group responsive */
#package-header .caption {
margin: 0px 1em 0 2em;
}
@media only screen and (min-width: 1280px) {
#content {
width: 63vw;
max-width: 1450px;
}
#table-of-contents {
position: fixed;
max-width: 10vw;
top: 10.2em;
left: 2em;
bottom: 1em;
overflow-y: auto;
}
#synopsis {
display: block;
position: fixed;
float: left;
top: 5em;
bottom: 1em;
right: 0;
max-width: 65vw;
overflow-y: auto;
/* Ensure that synopsis covers everything (including MathJAX markup) */
z-index: 1;
}
#synopsis .show {
border: 1px solid #5E5184;
padding: 0.7em;
max-height: 65vh;
}
}
@media only screen and (max-width: 1279px) {
#content {
width: 80vw;
}
#synopsis {
display: block;
padding: 0;
position: relative;
margin: 0;
width: 100%;
}
}
@media only screen and (max-width: 999px) {
#content {
width: 93vw;
}
}
/* menu for wider screens
Display the package name at the left and the menu links at the right,
inline with each other:
The package name Source . Contents . Index
*/
@media only screen and (min-width: 1000px) {
#package-header {
text-align: left;
white-space: nowrap;
height: 40px;
padding: 4px 1.5em 0px 1.5em;
overflow: visible;
display: flex;
justify-content: space-between;
align-items: center;
}
#package-header .caption {
display: inline-block;
margin: 0;
}
#package-header ul.links {
margin: 0;
display: inline-table;
}
#package-header .caption + ul.links {
margin-left: 1em;
}
}
/* menu for smaller screens
Display the package name on top of the menu links and center both elements:
The package name
Source . Contents . Index
*/
@media only screen and (max-width: 999px) {
#package-header {
text-align: center;
padding: 6px 0 4px 0;
overflow: hidden;
}
#package-header ul.links {
display: block;
text-align: center;
margin: 0;
/* Hide scrollbar but allow scrolling menu links horizontally */
white-space: nowrap;
overflow-x: auto;
overflow-y: hidden;
margin-bottom: -17px;
height: 50px;
}
#package-header .caption {
display: block;
margin: 4px 0;
text-align: center;
}
#package-header ul.links::-webkit-scrollbar {
display: none;
}
#package-header ul.links li:first-of-type {
padding-left: 1em;
}
#package-header ul.links li:last-of-type {
/*
The last link of the menu should offer the same distance to the right
as the #package-header enforces at the left.
*/
padding-right: 1em;
}
#package-header .caption + ul.links {
padding-top: 9px;
}
#module-header table.info {
float: none;
top: 0;
margin: 0 auto;
overflow: hidden;
max-width: 80vw;
}
}
/* @end */
/* @group Fonts & Sizes */
/* Basic technique & IE workarounds from YUI 3
For reasons, see:
http://yui.yahooapis.com/3.1.1/build/cssfonts/fonts.css
*/
body, button {
font: 400 14px/1.4 'PT Sans',
/* Fallback Font Stack */
-apple-system,
BlinkMacSystemFont,
'Segoe UI',
Roboto,
Oxygen-Sans,
Cantarell,
'Helvetica Neue',
sans-serif;
*font-size: medium; /* for IE */
*font:x-small; /* for IE in quirks mode */
}
h1 { font-size: 146.5%; /* 19pt */ }
h2 { font-size: 131%; /* 17pt */ }
h3 { font-size: 116%; /* 15pt */ }
h4 { font-size: 100%; /* 13pt */ }
h5 { font-size: 100%; /* 13pt */ }
table {
font-size:inherit;
font:100%;
}
pre, code, kbd, samp, tt, .src {
font-family:monospace;
}
.links, .link {
font-size: 85%; /* 11pt */
}
#module-header .caption {
font-size: 182%; /* 24pt */
}
#module-header .caption sup {
font-size: 80%;
font-weight: normal;
}
#package-header #page-menu a:link, #package-header #page-menu a:visited { color: white; }
.info {
font-size: 90%;
}
/* @end */
/* @group Common */
.caption, h1, h2, h3, h4, h5, h6, summary {
font-weight: bold;
color: #5E5184;
margin: 1.5em 0 1em 0;
}
* + h1, * + h2, * + h3, * + h4, * + h5, * + h6 {
margin-top: 2em;
}
h1 + h2, h2 + h3, h3 + h4, h4 + h5, h5 + h6 {
margin-top: inherit;
}
ul li + li {
margin-top: 0.2rem;
}
ul + p {
margin-top: 0.93em;
}
p + ul {
margin-top: 0.5em;
}
p {
margin-top: 0.7rem;
}
ul, ol {
margin: 0.8em 0 0.8em 2em;
}
ul.links {
list-style: none;
text-align: left;
font-size: 0.95em;
}
#package-header ul.links, #package-header ul.links button {
font-size: 1rem;
}
ul.links li {
display: inline;
white-space: nowrap;
padding: 0;
}
ul.links > li + li:before {
content: '\00B7';
}
ul.links li a {
padding: 0.2em 0.5em;
}
.hide { display: none; }
.show { display: inherit; }
.clear { clear: both; }
.collapser:before, .expander:before, .noexpander:before {
font-size: 1.2em;
color: #9C5791;
display: inline-block;
padding-right: 7px;
}
.collapser:before {
content: '▿';
}
.expander:before {
content: '▹';
}
.noexpander:before {
content: '▿';
visibility: hidden;
}
.collapser, .expander {
cursor: pointer;
}
.instance.collapser, .instance.expander {
margin-left: 0px;
background-position: left center;
min-width: 9px;
min-height: 9px;
}
summary {
cursor: pointer;
outline: none;
}
pre {
padding: 0.5rem 1rem;
margin: 1em 0 0 0;
background-color: #f7f7f7;
overflow: auto;
border: 1px solid #ddd;
border-radius: 0.3em;
}
pre + p {
margin-top: 1em;
}
pre + pre {
margin-top: 0.5em;
}
blockquote {
border-left: 3px solid #c7a5d3;
background-color: #eee4f1;
margin: 0.5em;
padding: 0.0005em 0.3em 0.5em 0.5em;
}
.src {
background: #f2f2f2;
padding: 0.2em 0.5em;
}
.keyword { font-weight: normal; }
.def { font-weight: bold; }
@media print {
#footer { display: none; }
}
/* @end */
/* @group Page Structure */
#content {
margin: 3em auto 6em auto;
padding: 0;
}
#package-header {
background: #5E5184;
border-bottom: 5px solid rgba(69, 59, 97, 0.5);
color: #ddd;
position: relative;
font-size: 1.2em;
text-align: left;
margin: 0 auto;
}
#package-header .caption {
color: white;
font-style: normal;
font-size: 1rem;
font-weight: bold;
}
#module-header .caption {
font-weight: bold;
border-bottom: 1px solid #ddd;
}
table.info {
float: right;
padding: 0.5em 1em;
border: 1px solid #ddd;
color: rgb(78,98,114);
background-color: #fff;
max-width: 60%;
border-spacing: 0;
position: relative;
top: -0.78em;
margin: 0 0 0 2em;
}
.info th {
padding: 0 1em 0 0;
text-align: right;
}
#style-menu li {
display: block;
border-style: none;
list-style-type: none;
}
#footer {
background: #ededed;
border-top: 1px solid #aaa;
padding: 0.5em 0;
color: #222;
text-align: center;
width: 100%;
height: 3em;
margin-top: 3em;
position: relative;
clear: both;
}
/* @end */
/* @group Front Matter */
#synopsis .caption,
#contents-list .caption {
font-size: 1rem;
}
#synopsis, #table-of-contents {
font-size: 16px;
}
#contents-list {
background: #f4f4f4;
padding: 1em;
margin: 0;
}
#contents-list .caption {
text-align: left;
margin: 0;
}
#contents-list ul {
list-style: none;
margin: 0;
margin-top: 10px;
font-size: 14px;
}
#contents-list ul ul {
margin-left: 1.5em;
}
#description .caption {
display: none;
}
#synopsis summary {
display: block;
float: right;
width: 29px;
color: rgba(255,255,255,0);
height: 110px;
margin: 0;
font-size: 1px;
padding: 0;
background: url(synopsis.png) no-repeat 0px -8px;
}
#synopsis details[open] > summary {
background: url(synopsis.png) no-repeat -75px -8px;
}
#synopsis details:not([open]) > ul {
visibility: hidden;
}
#synopsis ul {
height: 100%;
overflow: auto;
padding: 0.5em;
margin: 0;
}
#synopsis ul ul {
overflow: hidden;
}
#synopsis ul,
#synopsis ul li.src {
background-color: rgb(250,247,224);
white-space: nowrap;
list-style: none;
margin-left: 0;
}
#interface td.src {
white-space: nowrap;
}
/* @end */
/* @group Main Content */
#interface div.top + div.top {
margin-top: 1.5em;
}
#interface p + div.top,
#interface h1 + div.top,
#interface h2 + div.top,
#interface h3 + div.top,
#interface h4 + div.top,
#interface h5 + div.top {
margin-top: 1em;
}
#interface .src .selflink,
#interface .src .link {
float: right;
color: #888;
padding: 0 7px;
-moz-user-select: none;
font-weight: bold;
line-height: 30px;
}
#interface .src .selflink {
margin: 0 -0.5em 0 0.5em;
}
#interface span.fixity {
color: #919191;
border-left: 1px solid #919191;
padding: 0.2em 0.5em 0.2em 0.5em;
margin: 0 -1em 0 1em;
}
#interface span.rightedge {
border-left: 1px solid #919191;
padding: 0.2em 0 0.2em 0;
margin: 0 0 0 1em;
}
#interface table { border-spacing: 2px; }
#interface td {
vertical-align: top;
padding-left: 0.5em;
}
#interface td.doc p {
margin: 0;
}
#interface td.doc p + p {
margin-top: 0.8em;
}
.doc table {
border-collapse: collapse;
border-spacing: 0px;
}
.doc th,
.doc td {
padding: 5px;
border: 1px solid #ddd;
}
.doc th {
background-color: #f0f0f0;
}
.clearfix:after {
clear: both;
content: " ";
display: block;
height: 0;
visibility: hidden;
}
.subs, .top > .doc, .subs > .doc {
padding-left: 1em;
border-left: 1px solid gainsboro;
margin-bottom: 1em;
}
.top .subs {
margin-bottom: 0.6em;
}
.subs.fields ul {
list-style: none;
display: table;
margin: 0;
}
.subs.fields ul li {
display: table-row;
}
.subs ul li dfn {
display: table-cell;
font-style: normal;
font-weight: bold;
margin: 1px 0;
white-space: nowrap;
}
.subs ul li > .doc {
display: table-cell;
padding-left: 0.5em;
margin-bottom: 0.5em;
}
.subs ul li > .doc p {
margin: 0;
}
.subs .subs p.src {
border: none;
background-color: #f8f8f8;
}
.subs .subs .caption {
margin-top: 1em ;
margin-bottom: 0px;
}
.subs p.caption {
margin-top: 0;
}
.subs .subs .caption + .src {
margin: 0px;
margin-top: 8px;
}
.subs .subs .src + .src {
margin: 7px 0 0 0;
}
/* Render short-style data instances */
.inst ul {
height: 100%;
padding: 0.5em;
margin: 0;
}
.inst, .inst li {
list-style: none;
margin-left: 1em;
}
/* Workaround for bug in Firefox (issue #384) */
.inst-left {
float: left;
}
.top p.src {
border-bottom: 3px solid #e5e5e5;
line-height: 2rem;
margin-bottom: 1em;
}
.warning {
color: red;
}
.arguments {
margin-top: -0.4em;
}
.arguments .caption {
display: none;
}
.fields { padding-left: 1em; }
.fields .caption { display: none; }
.fields p { margin: 0 0; }
/* this seems bulky to me
.methods, .constructors {
background: #f8f8f8;
border: 1px solid #eee;
}
*/
/* @end */
/* @group Auxillary Pages */
.extension-list {
list-style-type: none;
margin-left: 0;
}
#mini {
margin: 0 auto;
padding: 0 1em 1em;
}
#mini > * {
font-size: 93%; /* 12pt */
}
#mini #module-list .caption,
#mini #module-header .caption {
font-size: 125%; /* 15pt */
}
#mini #interface h1,
#mini #interface h2,
#mini #interface h3,
#mini #interface h4 {
font-size: 109%; /* 13pt */
margin: 1em 0 0;
}
#mini #interface .top,
#mini #interface .src {
margin: 0;
}
#mini #module-list ul {
list-style: none;
margin: 0;
}
#alphabet ul {
list-style: none;
padding: 0;
margin: 0.5em 0 0;
text-align: center;
}
#alphabet li {
display: inline;
margin: 0 0.25em;
}
#alphabet a {
font-weight: bold;
}
#index .caption,
#module-list .caption { font-size: 131%; /* 17pt */ }
#index table {
margin-left: 2em;
}
#index .src {
font-weight: bold;
}
#index .alt {
font-size: 77%; /* 10pt */
font-style: italic;
padding-left: 2em;
}
#index td + td {
padding-left: 1em;
}
#module-list ul {
list-style: none;
margin: 0 0 0 2em;
}
#module-list li {
clear: right;
}
#module-list span.collapser,
#module-list span.expander {
background-position: 0 0.3em;
}
#module-list .package {
float: right;
}
:target {
background: -webkit-linear-gradient(top, transparent 0%, transparent 65%, #fbf36d 60%, #fbf36d 100%);
background: -moz-linear-gradient(top, transparent 0%, transparent 65%, #fbf36d 60%, #fbf36d 100%);
background: -o-linear-gradient(top, transparent 0%, transparent 65%, #fbf36d 60%, #fbf36d 100%);
background: -ms-linear-gradient(top, transparent 0%, transparent 65%, #fbf36d 60%, #fbf36d 100%);
background: linear-gradient(to bottom, transparent 0%, transparent 65%, #fbf36d 60%, #fbf36d 100%);
}
:target:hover {
background: -webkit-linear-gradient(top, transparent 0%, transparent 0%, #fbf36d 0%, #fbf36d 100%);
background: -moz-linear-gradient(top, transparent 0%, transparent 0%, #fbf36d 0%, #fbf36d 100%);
background: -o-linear-gradient(top, transparent 0%, transparent 0%, #fbf36d 0%, #fbf36d 100%);
background: -ms-linear-gradient(top, transparent 0%, transparent 0%, #fbf36d 0%, #fbf36d 100%);
background: linear-gradient(to bottom, transparent 0%, transparent 0%, #fbf36d 0%, #fbf36d 100%);
}
/* @end */
/* @group Dropdown menus */
#preferences-menu, #style-menu {
width: 25em;
overflow-y: auto;
}
/* @end */

1
Docs/haskell/meta.json Normal file
View File

@@ -0,0 +1 @@
{"haddock_version":"2.29.2"}

BIN
Docs/haskell/p2prc.haddock Normal file

Binary file not shown.

221
Docs/haskell/quick-jump.css Normal file
View File

@@ -0,0 +1,221 @@
/* @group Fundamentals */
.hidden {
display: none;
}
/* @end */
/* @group Search box layout */
#search {
position: fixed;
top: 3.2em;
bottom: 0;
left: calc(50% - 22em);
width: 44em;
z-index: 1000;
overflow-y: auto;
}
@media only screen and (max-width: 999px) {
#search {
top: 5.7em;
}
}
#search-form, #search-results {
box-shadow: 2px 2px 6px rgb(199, 204, 208);
pointer-events: all;
}
#search-form input {
font-size: 1.25em; line-height: 2.3em; height: 2.4em;
display: block;
box-sizing: border-box;
width: 100%;
margin: 0;
padding: 0 0.75em;
border: 0.05em solid rgb(151, 179, 202);
}
#search input:focus {
outline: none;
}
#search p.error {
color: rgb(107, 24, 24);
font-weight: bold;
}
#search-results {
box-sizing: border-box;
border: 0.05em solid #b2d5fb;
background: #e8f3ff;
max-height: 80%;
overflow: scroll;
}
#search-form input + #search-results {
border-top: none;
top: 3em;
max-height: calc(100% - 3em);
}
/* @end */
/* @group search results */
#search-results > ul {
margin: 0;
list-style: none;
}
#search-results > ul > li,
#search-results > p,
#search-results > table {
padding: 0.5em 1em;
margin: 0;
}
#search-results > ul > li {
border-bottom: 1px solid #b2d5fb;
}
#search-results > ul > li > ul {
list-style: none;
}
.search-module h4 {
margin: 0;
}
.search-module > ul {
margin: 0.5em 0 0.5em 2em;
}
.search-module > ul > li > a[href] {
display: block;
color: inherit;
padding: 0.25em 0.5em;
}
.search-module > ul > li > a[href].active-link {
background: #faf9dc;
}
.search-module a[href]:hover {
text-decoration: none;
}
.search-result a a {
pointer-events: none;
}
.search-result ul.subs {
display: inline-block;
margin: 0; padding: 0;
}
.search-result ul.subs li {
display: none;
}
.search-result ul.subs::after {
display: inline-block;
content: "...";
color: rgb(78,98,114);
margin: 0 0.25em;
}
.more-results {
color: rgb(99, 141, 173);
position: relative;
}
.more-results::before {
content: "+";
display: inline-block;
color: #b2d5fb;
font-weight: bold;
font-size: 1.25em; line-height: inherit;
position: absolute;
left: -1em;
}
/* @end */
/* @group Keyboard shortcuts table */
.keyboard-shortcuts {
line-height: 1.6em;
}
.keyboard-shortcuts th {
color: rgb(78,98,114);
}
.keyboard-shortcuts td:first-child,
.keyboard-shortcuts th:first-child {
text-align: right;
padding-right: 0.6em;
}
.key {
display: inline-block;
font-size: 0.9em;
min-width: 0.8em; line-height: 1.2em;
text-align: center;
background: #b2d5fb;
border: 1px solid #74a3d6;
padding: 0 0.2em;
margin: 0 0.1em;
}
/* @end */
/* @group Dropdown menus */
/* Based on #search styling above. */
.dropdown-menu {
position: fixed;
/* Not robust to window size changes. */
top: 3.2em;
right: 0;
/* To display on top of synopsis menu on right side. */
z-index: 1000;
border: 0.05em solid #b2d5fb;
background: #e8f3ff;
}
@media only screen and (max-width: 999px) {
.dropdown-menu {
top: 5.7em;
}
}
.dropdown-menu * {
margin: 0.1em;
}
.dropdown-menu button {
border: 1px #5E5184 solid;
border-radius: 3px;
background: #5E5184;
padding: 3px;
color: #f4f4f4;
min-width: 6em;
}
.dropdown-menu button:hover {
color: #5E5184;
background: #f4f4f4;
}
.dropdown-menu button:active {
color: #f4f4f4;
background: #5E5184;
}
/* @end */

BIN
Docs/haskell/synopsis.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

1268
Docs/index.html Normal file

File diff suppressed because it is too large Load Diff

920
Docs/index.org Normal file
View File

@@ -0,0 +1,920 @@
#+SETUPFILE: https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="https://fniessen.github.io/org-html-themes/src/readtheorg_theme/css/search.css"/>
#+HTML_HEAD: <link rel="stylesheet" type="text/css" href="style.css"/>
#+attr_html: :width 300px
[[./Colored-On-Light-Image.png]]
* Guide through video
*** The video below shows the setup and usage of P2PRC.
#+attr_html: :class video
[[https://www.youtube.com/watch?v=OMwCpedu5cs][https://i3.ytimg.com/vi/OMwCpedu5cs/maxresdefault.jpg]]
*** Source code: https://github.com/Akilan1999/p2p-rendering-computation
* Introduction
:PROPERTIES:
:CUSTOM_ID: chapter1-introduction
:END:
This project focuses on creating a framework for running heavy computational tasks that a regular
computer cannot handle easily. These tasks may include graphically demanding video games, rendering
3D animations, and performing complex protein folding simulations. The major focus of this project
is not on financial incentives but rather on building a robust and efficient peer-to-peer (P2P)
network to decentralise task execution and increase the computational bandwidth available for
such tasks.
The P2PRC framework serves as a foundation for decentralised rendering and computation,
providing insights into how tasks can be distributed efficiently across a network of peers.
Leveraging the P2PRC approach, this project extends its capabilities to handle a
wider range of computationally intensive tasks.
** Motivation
:PROPERTIES:
:CUSTOM_ID: motivation
:END:
Many of the users rely on our PC / Laptop or servers that belong to a
server farm to run heavy tasks and with the demand of high creativity
requires higher computing power. Buying a powerful computer every few
years to run a bunch of heavy tasks which are not executed as frequently
to reap the benefits can be inefficient utilization of hardware. On the
other end, renting servers to run these heavy tasks can be really
useful. Ethically speaking this is leading to monopolisation of
computing power similar to what is happening in the web server area. By
using peer to peer principles it is possible to remove the
monopolisation factor and increase the bandwidth between the client and
server.
* Installation
:PROPERTIES:
:CUSTOM_ID: installation
:END:
Over here we will cover the basic steps to get the server and client
side running.
** Latest release install
:PROPERTIES:
:CUSTOM_ID: latest-release-install
:END:
https://github.com/Akilan1999/p2p-rendering-computation/releases
** Install from Github master branch
:PROPERTIES:
:CUSTOM_ID: install-from-github-master-branch
:END:
*** Install Go lang
:PROPERTIES:
:CUSTOM_ID: install-go-lang
:END:
The entire the implementation of this project is done using Go lang.
Thus, we need go lang to compile to code to a binary file.
[[https://golang.org/doc/install][Instructions to install Go lang]]
*** Install Docker
:PROPERTIES:
:CUSTOM_ID: install-docker
:END:
In this project the choice of virtualization is Docker due to it's wide
usage in the developer community. In the server module we use the Docker
Go API to create and interact with the containers.
[[https://docs.docker.com/get-docker/][Instructions to install docker]]
[[https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker][Instructions
to install docker GPU]]
#+begin_example
// Do ensure that the docker command does not need sudo to run
sudo chmod 666 /var/run/docker.sock
#+end_example
*** Build Project and install project
:PROPERTIES:
:CUSTOM_ID: build-project-and-install-project
:END:
To set up the internal dependencies and build the entire go code into a
single binary
#+begin_example
make
#+end_example
**** For Windows
:PROPERTIES:
:CUSTOM_ID: for-windows
:END:
To set up P2PRC on Windows, simply run this batch file. *Make sure you
are not in admin mode when running this.*
#+begin_example
.\install.bat
#+end_example
*** Add appropriate paths to =.bashrc=
:PROPERTIES:
:CUSTOM_ID: add-appropriate-paths-to-.bashrc
:END:
#+begin_example
export P2PRC=/<PATH>/p2p-rendering-computation
export PATH=/<PATH>/p2p-rendering-computation:${PATH}
#+end_example
*** Test if binary works
:PROPERTIES:
:CUSTOM_ID: test-if-binary-works
:END:
#+begin_example
p2prc --help
#+end_example
**** Output:
:PROPERTIES:
:CUSTOM_ID: output
:END:
#+begin_example
NAME:
p2p-rendering-computation - p2p cli application to create and access VMs in other servers
USAGE:
p2prc [global options] command [command options] [arguments...]
VERSION:
2.0.0
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--Server, -s Starts server (default: false) [$SERVER]
--UpdateServerList, --us Update List of Server available based on servers iptables (default: false) [$UPDATE_SERVER_LIST]
--ListServers, --ls List servers which can render tasks (default: false) [$LIST_SERVERS]
--AddServer value, --as value Adds server IP address to iptables [$ADD_SERVER]
--ViewImages value, --vi value View images available on the server IP address [$VIEW_IMAGES]
--CreateVM value, --touch value Creates Docker container on the selected server [$CREATE_VM]
--ContainerName value, --cn value Specifying the container run on the server side [$CONTAINER_NAME]
--BaseImage value, --bi value Specifying the docker base image to template the dockerfile [$CONTAINER_NAME]
--RemoveVM value, --rm value Stop and Remove Docker container (IP:port) accompanied by container ID via --ID or --id [$REMOVE_VM]
--ID value, --id value Docker Container ID [$ID]
--Ports value, -p value Number of ports to open for the Docker Container [$NUM_PORTS]
--GPU, --gpu Create Docker Containers to access GPU (default: false) [$USE_GPU]
--Specification value, --specs value Specs of the server node [$SPECS]
--SetDefaultConfig, --dc Sets a default configuration file (default: false) [$SET_DEFAULT_CONFIG]
--NetworkInterfaces, --ni Shows the network interface in your computer (default: false) [$NETWORK_INTERFACE]
--ViewPlugins, --vp Shows plugins available to be executed (default: false) [$VIEW_PLUGIN]
--TrackedContainers, --tc View (currently running) containers which have been created from the client side (default: false) [$TRACKED_CONTAINERS]
--ExecutePlugin value, --plugin value Plugin which needs to be executed [$EXECUTE_PLUGIN]
--CreateGroup, --cgroup Creates a new group (default: false) [$CREATE_GROUP]
--Group value, --group value group flag with argument group ID [$GROUP]
--Groups, --groups View all groups (default: false) [$GROUPS]
--RemoveContainerGroup, --rmcgroup Remove specific container in the group (default: false) [$REMOVE_CONTAINER_GROUP]
--RemoveGroup value, --rmgroup value Removes the entire group [$REMOVE_GROUP]
--MAPPort value, --mp value Maps port for a specific port provided as the parameter [$MAPPORT]
--DomainName value, --dn value While mapping ports allows to set a domain name to create a mapping in the proxy server [$DOMAINNAME]
--Generate value, --gen value Generates a new copy of P2PRC which can be modified based on your needs [$GENERATE]
--ModuleName value, --mod value New go project module name [$MODULENAME]
--PullPlugin value, --pp value Pulls plugin from git repos [$PULLPLUGIN]
--RemovePlugin value, --rp value Removes plugin [$REMOVEPLUGIN]
--AddMetaData value, --amd value Adds metadata about the current node in the p2p network which is then propagated through the network [$ADDMETADATA]
--help, -h show help (default: false)
--version, -v print the version (default: false)
#+end_example
--------------
* Using basic commands
:PROPERTIES:
:CUSTOM_ID: using-basic-commands
:END:
*** Start as a server
:PROPERTIES:
:CUSTOM_ID: start-as-a-server
:END:
#+begin_example
p2prc -s
#+end_example
*** View server Specification
:PROPERTIES:
:CUSTOM_ID: view-server-specification
:END:
#+begin_example
p2prc --specs=<ip address>
#+end_example
*** Run container
:PROPERTIES:
:CUSTOM_ID: run-container
:END:
use the =--gpu= if you know the other machine has a gpu.
#+begin_example
p2prc --touch=<server ip address> -p <number of ports> --gpu
#+end_example
*** Remove container
:PROPERTIES:
:CUSTOM_ID: remove-container
:END:
The docker id is present in the output where you create a container
#+begin_example
p2prc --rm=<server ip address> --id=<docker container id>
#+end_example
*** Adding servers to ip table
:PROPERTIES:
:CUSTOM_ID: adding-servers-to-ip-table
:END:
#+begin_example
p2prc --as=<server ip address you want to add>
#+end_example
*** Update ip table
:PROPERTIES:
:CUSTOM_ID: update-ip-table
:END:
#+begin_example
p2prc --us
#+end_example
*** List Servers
:PROPERTIES:
:CUSTOM_ID: list-servers
:END:
#+begin_example
p2prc --ls
#+end_example
*** View Network interfaces
:PROPERTIES:
:CUSTOM_ID: view-network-interfaces
:END:
#+begin_example
p2prc --ni
#+end_example
*** Viewing Containers created Client side
:PROPERTIES:
:CUSTOM_ID: viewing-containers-created-client-side
:END:
#+begin_example
p2prc --tc
#+end_example
*** Running plugin
:PROPERTIES:
:CUSTOM_ID: running-plugin
:END:
#+begin_example
p2prc --plugin <plugin name> --id <container id or group id>
#+end_example
*** Create group
:PROPERTIES:
:CUSTOM_ID: create-group
:END:
#+begin_example
p2prc --cgroup
#+end_example
*** Add container to group
:PROPERTIES:
:CUSTOM_ID: add-container-to-group
:END:
#+begin_example
p2prc --group <group id> --id <container id>
#+end_example
*** View groups
:PROPERTIES:
:CUSTOM_ID: view-groups
:END:
#+begin_example
p2prc --groups
#+end_example
*** View specific group
:PROPERTIES:
:CUSTOM_ID: view-specific-group
:END:
#+begin_example
p2prc --group <group id>
#+end_example
*** Delete container from group
:PROPERTIES:
:CUSTOM_ID: delete-container-from-group
:END:
#+begin_example
p2prc --rmcgroup --group <group id> --id <container id>
#+end_example
*** Delete entire group
:PROPERTIES:
:CUSTOM_ID: delete-entire-group
:END:
#+begin_example
p2prc --rmgroup <group id>
#+end_example
*** Pulling plugin from a remote repo
:PROPERTIES:
:CUSTOM_ID: pulling-plugin-from-a-remote-repo
:END:
#+begin_example
p2prc --pp <repo link>
#+end_example
*** Deleting plugin from the plugin directory
:PROPERTIES:
:CUSTOM_ID: deleting-plugin-from-the-plugin-directory
:END:
#+begin_example
p2prc --rp <plugin name>
#+end_example
*** Added custom metadata about the current node
:PROPERTIES:
:CUSTOM_ID: added-custom-metadata-about-the-current-node
:END:
#+begin_example
p2prc --amd "custom metadata"
#+end_example
*** MapPort and link to domain name
Allows to expose remote ports from a machine in the P2P network.
#+begin_example
p2prc --mp <port no to map> --dn <domain name to link Mapped port against>
#+end_example
**** MapPort in remote machine
This is to ensure ports on remote machines on the P2PRC can be easily opened.
#+begin_example
p2prc --mp <port no to map> --dn <domain name to link Mapped port against> --ra <remote server address>
#+end_example
--------------
*** Add root node
Adds a root node to P2RRC and overwrites all other nodes in the ip table.
To be only added before the network is started and with
the intention of a fresh instance.
#+begin_example
p2prc --arn --ip <root node ip address> -p <root node port no>
#+end_example
* P2P Module Implementation
:PROPERTIES:
:CUSTOM_ID: p2p-module-implementation
:END:
The P2P module is for managing server information within the network.
It maintains and updates the IP table, ensuring accuracy by preventing duplicates and removing
entries for unreachable servers. Furthermore, the module conducts speed tests on the listed servers
to determine upload and download speeds. This valuable information enables users to identify nearby
servers with optimal performance, enhancing their overall network experience.
#+caption: UML diagram of P2P module
[[file:./images/p2pmoduleArch.png]
The peer to peer implementation was built from scratch. This is because
other peer to peer libraries were on the implementation of the
Distributed hash table. At the current moment all those heavy features
are not needed because the objective is to search and list all possible
servers available. The limitation being that to be a part of the network
the user has to know at least 1 server. The advantage of building from
scratch makes the module super light and possibility for custom
functions and structs. The sub topics below will mention the
implementations of each functionality in depth.
** IP Table
:PROPERTIES:
:CUSTOM_ID: ip-table
:END:
The ip table file is a json as the format with a list of servers ip
addresses, latencies, downloads and uploads speeds. The functions
implemented include read file, write file and remove duplicate IP
addresses. The remove duplicate IP address function exists because
sometimes servers IP tables can have the same ip addresses as what the
client has. The path of the IP table json file is received from the
configuration module.
#+begin_src json
{
"ip_address": [
{
"Name": "<hostname of the machine>",
"MachineUsername": "<machine username>",
"IPV4": "<ipv4 address>",
"IPV6": "<ipv6 address (Not used)>",
"Latency": <latency to the server>,
"Download": <download speed (Not used)>,
"Upload": <upload speed (Not used)>,
"ServerPort": "<server port no>",
"BareMetalSSHPort": "<Baremetal ssh port no>",
"NAT": "<boolean representing if the node is behind NAT or not>",
"EscapeImplementation": "<NAT traversal implementation>",
"ProxyServer": "<If the node listed is acting as a proxy server>",
"UnSafeMode": <Unsafe mode if turned on will allow all nodes in the network public keys to be
added to that particular node>",
"PublicKey": "<Public key of that particular node>",
"CustomInformation": "<custom information passed in through all the nodes in the network>"
}
]
}
#+end_src
*** Latency
:PROPERTIES:
:CUSTOM_ID: latency
:END:
The latency is measured in milliseconds. The route /server_info is
called from the server and time it takes to provide a json response is
recorded.
** NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
*** Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src
* Language Bindings
:PROPERTIES:
:CUSTOM_ID: language-bindings
:END:
[[https://en.wikipedia.org/wiki/Language_binding][Language bindings]]
refers to wrappers to bridge 2 programming languages. This is used in
P2PRC to extend calling P2PRC functions in other programming languages.
Currently this is done by generating =.so= and =.h= from the Go
compiler.
** How to build shared object files
:PROPERTIES:
:CUSTOM_ID: how-to-build-shared-object-files
:END:
**** The easier way
:PROPERTIES:
:CUSTOM_ID: the-easier-way
:END:
#+begin_src sh
# Run
make sharedObjects
#+end_src
**** Or the direct way
:PROPERTIES:
:CUSTOM_ID: or-the-direct-way
:END:
#+begin_src sh
# Run
cd Bindings && go build -buildmode=c-shared -o p2prc.so
#+end_src
**** If successfully built:
:PROPERTIES:
:CUSTOM_ID: if-successfully-built
:END:
#+begin_src sh
# Enter into the Bindings directory
cd Bindings
# List files
ls
# Find files
p2prc.h p2prc.so
#+end_src
** Workings under the hood
:PROPERTIES:
:CUSTOM_ID: workings-under-the-hood
:END:
Below are a sample set of commands to open the bindings implementation.
#+begin_example
# run
cd Bindings/
# list files
ls
# search for file
Client.go
#+end_example
*** In Client go
:PROPERTIES:
:CUSTOM_ID: in-client-go
:END:
There a few things to notice which are different from your standard Go
programs:
**** 1. We import "C" which means [[https://pkg.go.dev/cmd/cgo][Cgo]] is required.
:PROPERTIES:
:CUSTOM_ID: we-import-c-which-means-cgo-is-required.
:END:
#+begin_src go
import "C"
#+end_src
**** 2. All functions which are required to be called from other programming languages have comment such as.
:PROPERTIES:
:CUSTOM_ID: all-functions-which-are-required-to-be-called-from-other-programming-languages-have-comment-such-as.
:END:
#+begin_src go
//export <function name>
// ------------ Example ----------------
// The function below allows to externally
// to call the P2PRC function to start containers
// in a specific node in the know list of nodes
// in the p2p network.
// Note: the comment "//export StartContainer".
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
#+end_src
**** 3. While looking through the file (If 2 files are compared it is pretty trivial to notice a common structure).
:PROPERTIES:
:CUSTOM_ID: while-looking-through-the-file-if-2-files-are-compared-it-is-pretty-trivial-to-notice-a-common-structure.
:END:
#+begin_src go
// --------- Example ------------
//export StartContainer
func StartContainer(IP string) (output *C.char) {
container, err := client.StartContainer(IP, 0, false, "", "")
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(container)
}
//export ViewPlugin
func ViewPlugin() (output *C.char) {
plugins, err := plugin.DetectPlugins()
if err != nil {
return C.CString(err.Error())
}
return ConvertStructToJSONString(plugins)
}
#+end_src
**** It is easy to notice that:
:PROPERTIES:
:CUSTOM_ID: it-is-easy-to-notice-that
:END:
- =ConvertStructToJSONString(<go object>)=: This is a helper function
that convert a go object to JSON string initially and converts it to
=CString=.
- =(output *C.char)=: This is the return type for most of the functions.
**** A Pseudo code to refer to the common function implementation shape could be represented as:
:PROPERTIES:
:CUSTOM_ID: a-pseudo-code-to-refer-to-the-common-function-implementation-shape-could-be-represented-as
:END:
#+begin_example
func <Function name> (output *C.char) {
<response>,<error> := <P2PRC function name>(<parameters if needed>)
if <error> != nil {
return C.CString(<error>.Error())
}
return ConvertStructToJSONString(<response>)
}
#+end_example
** Current languages supported
:PROPERTIES:
:CUSTOM_ID: current-languages-supported
:END:
*** Python
**** Build sample python program
:PROPERTIES:
:CUSTOM_ID: build-sample-python-program
:END:
The easier way
#+begin_src sh
# Run
make python
# Expected ouput
Output is in the Directory Bindings/python/export/
# Run
cd Bindings/python/export/
# list files
ls
# Expected output
SharedObjects/ library.py requirements.txt
#+end_src
Above shows a generated folder which consists of a folder called
"SharedObjects/" which consists of =p2prc.so= and =p2prc.h= files.
=p2prc.py= refers to a sample python script calling P2PRC go functions.
To start an any project to extend P2PRC with python, This generated
folder can copied and created as a new git repo for P2PRC extensions
scripted or used a reference point as proof of concept that P2PRC can be
called from other programming languages.
*** Haskell
P2PRC officially supports Haskell bindings and will further support
project using Haskell to build orchestrators on top of P2PRC.
[[https://p2prc.akilan.io/Docs/haskell][Read more...]]
* Config Implementation
:PROPERTIES:
:CUSTOM_ID: config-implementation
:END:
The configuration module is responsible to store basic information of
absolute paths of files being called in the Go code. In a full-fledged
Cli the configuration file can be found in the directory /etc/ and from
there points to location such as where the IP table file is located. In
the future implementation the config file will have information such as
number of hops and other parameters to tweak and to improve the
effectiveness of the peer to peer network. The configuration module was
implemented using the library Viper. The Viper library automates
features such as searching in default paths to find out if the
configuration file is present. If the configuration file is not present
in the default paths then it auto generates the configuration file. The
configurations file can be in any format. In this project the
configuration file was generated using JSON format.
#+begin_src json
{
"MachineName": "pc-74-120.customer.ask4.lan",
"IPTable": "/Users/akilan/Documents/p2p-rendering-computation/p2p/iptable/ip_table.json",
"DockerContainers": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/",
"DefaultDockerFile": "/Users/akilan/Documents/p2p-rendering-computation/server/docker/containers/docker-ubuntu-sshd/",
"SpeedTestFile": "/Users/akilan/Documents/p2p-rendering-computation/p2p/50.bin",
"IPV6Address": "",
"PluginPath": "/Users/akilan/Documents/p2p-rendering-computation/plugin/deploy",
"TrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/trackcontainers.json",
"ServerPort": "8088",
"GroupTrackContainersPath": "/Users/akilan/Documents/p2p-rendering-computation/client/trackcontainers/grouptrackcontainers.json",
"FRPServerPort": "True",
"BehindNAT": "True",
"CustomConfig": null
}
#+end_src
* Abstractions
:PROPERTIES:
:CUSTOM_ID: abstractions
:END:
The Abstractions package consists of black-boxed functions for P2PRC.
** Functions
:PROPERTIES:
:CUSTOM_ID: functions
:END:
- =Init(<Project name>)=: Initializes P2PRC with all the needed
configurations.
- =Start()=: Starts p2prc as a server and makes it possible to extend by
adding other routes and functionality to P2PRC.
- =MapPort(<port no>)=: On the local machine the port you want to export
to world.
- =StartContainer(<ip address>)=: The machine on the p2p network where
you want to spin up a docker container.
- =RemoveContainer(<ip address>,<container id>)=: Terminate container
based on the IP address and container name.
- =GetSpecs(<ip address>)=: Get specs of a machine on the network based
on the IP address.
- =ViewIPTable()=: View the IP table which about nodes in the network.
- =UpdateIPTable()=: Force update IP table to learn about new nodes
faster.
* NAT Traversal
:PROPERTIES:
:CUSTOM_ID: nat-traversal
:END:
P2PRC currently supports TURN for NAT traversal.
** TURN
:PROPERTIES:
:CUSTOM_ID: turn
:END:
The current TURN implementation used is FRP. The TURN server is also
required when a P2PRC node is acting as a Server. The TURN server is
determined based on the Node with the least amount of latency based on
the Nodes available on the IPTable. Once a TURN server is determined
there are 2 actions performed. The first one is =/FRPPort= to the TURN
server to receive a port which is used to generate the external port
from the TURN server. The flow below describes the workflow.
* Client mode
:PROPERTIES:
:CUSTOM_ID: client-mode
:END:
- Call =/FRPPort=
#+begin_example
http://<turn server ip>:<server port no>/FRPport
#+end_example
- Call the TURN server in the following manner. The following is a
sample code snippet below.
#+begin_src go
import (
"github.com/Akilan1999/p2p-rendering-computation/p2p/frp"
)
func main() {
serverPort, err := frp.GetFRPServerPort("http://" + <lowestLatencyIpAddress.Ipv4> + ":" + lowestLatencyIpAddress.ServerPort)
if err != nil {
return nil, err
}
// Create 1 second delay to allow FRP server to start
time.Sleep(1 * time.Second)
// Starts FRP as a client with
proxyPort, err := frp.StartFRPClientForServer(<lowestLatencyIpAddress.Ipv4>, serverPort, <the port you want to expose externally>)
if err != nil {
return nil, err
}
}
#+end_src
* Blog posts
** Self host within 5 minutes any program
- Author: [[http://akilan.io/][Akilan Selvacoumar]]
- Date: 28-01-2025
- Video tutorial:
#+attr_html: :class video
[[https://youtu.be/rN4SiVowg5E][https://i3.ytimg.com/vi/rN4SiVowg5E/maxresdefault.jpg]]
This is a fun expirement for anyone to try to quickly run a server and
quickly do a map port and domain name mapping in a single command.
*** 1. Find a program you want to run
Let's try to setup a really easy program (Let's do with Linkwarden
with docker compose :) ). This is under the assumption you have docker
compose installed on your local machine.
**** Let's run Linkwarden using docker compose and P2PRC
[[https://docs.linkwarden.app/self-hosting/installation][Installation instructions]]:
#+BEGIN_SRC
mkdir linkwarden && cd linkwarden
curl -O https://raw.githubusercontent.com/linkwarden/linkwarden/refs/heads/main/docker-compose.yml
curl -L https://raw.githubusercontent.com/linkwarden/linkwarden/refs/heads/main/.env.sample -o ".env"
#+END_SRC
Environment configuration
#+BEGIN_SRC
vim .env
# Change values
NEXTAUTH_URL=https://<DOMAIN NAME>/api/v1/auth
NEXTAUTH_SECRET=VERY_SENSITIVE_SECRET
POSTGRES_PASSWORD=CUSTOM_POSTGRES_PASSWORD
#+END_SRC
Run linkwarden!
#+BEGIN_SRC
docker compose up
#+END_SRC
If setup correctly linkwarden should
be running.
Local link: http://localhost:3000
Time to setup P2PRC
[[https://p2prc.akilan.io/Docs/#build-project-and-install-project][Installation Instructions]]
Run p2prc as a background
#+BEGIN_SRC
p2prc -s &
#+END_SRC
Run map port and domain mapping
#+BEGIN_SRC
p2prc --mp 3000 --dn <DOMAIN NAME>
#+END_SRC
Sample response
#+BEGIN_SRC
{
"IPAddress": "217.76.63.222",
"PortNo": "61582",
"EntireAddress": "217.76.63.222:61582"
}
#+END_SRC
Add DNS entry
#+BEGIN_SRC
A entry 217.76.63.222
#+END_SRC
Your done now just head to the DOMAIN NAME you added.
ex: https://linkwarden.akilan.io
* Ideas for future potencial features
Consists of personal loideas for the future of P2PRC.
At moment only has main contributors writiing to this.
** To support hetrogenous set of Nodes that cannot run P2PRC
This stems from a personal issue I have when doing research
on [[https://github.com/CTSRD-CHERI/cheribsd][CheriBSD]] kernel. For my research I am using the ARM morello
which is a 128bit ARMv8 processor. At the moment Go programs
can cannot compile and run inside the CPU. This means I cannot
run P2PRC at the moment inside the ARM morello to remotely access
it when it's behind NAT using P2PRC. This would indeed be a common
problem when running against various Architectures that do not
support running P2PRC. As you will see soon this also creates
oppurtunity space to scale faster to nodes in a local network
and would introduce a new layer fault tolerance within a local
network nodes.
*** Assumptions:
- I have a Morello board that cannot run P2PRC
- The Morello has a local IP address (ex: 192.168.0.10)
- I have 2 laptops running P2PRC in that local network.
- This means I have 2 ways to access the Morello board: Which is to SSH
into either 2 laptops and then SSH into 192.168.0.10 to gain access
to the board. Wouldn't it be great to automate this whole layer and
as well look into custom tasks into the hetrogenous hardware.
*** Set of interesting possible:
We build a cool set possibilities before and use this to build up the implementation
plan.
- We can use P2PRC access the morello board remotely in a single command.
- We can use the P2PRC protocol to run servers inside the morello board via traversed
node locally which can access that Node.
- Spin servers on node not running P2PRC using the P2PRC standard abstractions.
- Auto-setup P2PRC nodes with just SSH access via potencially a DSL.
- A neat use case for CHERI for instance would be use the architecture to run light
weight hypervisors.
*** Implementation
- To use implementations similar to [[https://linux.die.net/man/1/socat][socat]] to ensure we can bind address of local
nodes to a node running P2PRC and the node running P2PRC can do a local map port.
- We are working on hardening the implementation of the --mp (Map port) to even
map ports to machines which remotely running P2PRC. This means of instance I
can issue a command to the Morello board without the morello board being in
my local network.
- We would want to implement the exsisting P2PRC public key mechanism as well so that
other nodes can access the Morello board who have permission access.
#+CAPTION: Implementation idea (To be improved upon)
[[./images/P2PRCRemoteNodes.png]]

1
Docs/run-docs.sh Normal file
View File

@@ -0,0 +1 @@
go run staticServer.go &

28
Docs/staticServer.go Normal file
View File

@@ -0,0 +1,28 @@
/*
Serve is a very simple static file server in go
Usage:
-p="8100": port to serve on
-d=".": the directory of static files to host
Navigating to http://localhost:8100 will display the index.html or directory
listing file.
*/
package main
import (
"flag"
"log"
"net/http"
)
func main() {
port := flag.String("p", "8083", "port to serve on")
directory := flag.String("d", ".", "the directory of static file to host")
flag.Parse()
http.Handle("/", http.FileServer(http.Dir(*directory)))
log.Printf("Serving %s on HTTP port: %s\n", *directory, *port)
log.Fatal(http.ListenAndServe(":"+*port, nil))
}

29
Docs/style.css Normal file
View File

@@ -0,0 +1,29 @@
#table-of-contents h2{
z-index: 200;
background-color: #b96c29;
text-align: center;
padding: 0.809em;
display: block;
color: #fcfcfc;
font-size: 100%;
margin-bottom: 0.809em;
}
#search-results li {
background-color: #b96c29;
color: white;
margin-bottom: 5px;
padding: 5px;
border-radius: 4px;
cursor: pointer;
font-size: 14px;
}
h4,h5,h6{
color: #b96c29;
font-weight:300;
}
.video {
box-shadow: 10px 10px 25px rgba(0, 0, 0, 0.3);
}

View File

@@ -1,4 +1,3 @@
SHELL := /bin/bash
install:
sh install.sh p2prc
@@ -9,3 +8,12 @@ testcases:
run:
go run main.go
sharedObjects:
sh build-bindings.sh
python:
sh build-python-package.sh
clean:
go clean -modcache
rm -fr .go-build vendor result*

139
README.md
View File

@@ -1,3 +1,9 @@
> [!TIP]
> Haskell bindings supported!: [Bindings documentaton](https://p2prc.akilan.io/Docs/haskell/P2PRC.html)
> [!NOTE]
> Fixing documentation to latest changes. If you have any questions setting up P2PRC either [create an issue](https://github.com/Akilan1999/p2p-rendering-computation/issues/new/choose) or send me an email (me AT akilan dot io).
> Currently HEAD is always intended to stay on a working state. It is recommended to always use HEAD in your go.mod file.
<h1 align="center">
<br>
@@ -14,24 +20,87 @@
<a href="https://pkg.go.dev/git.sr.ht/~akilan1999/p2p-rendering-computation"><img alt="GoDoc reference example" src="https://img.shields.io/badge/godoc-reference-blue.svg" style="padding:5px;margin:5px;" /></a>
</div>
The main aim of this project was to create a custom peer to peer network. The user acting as the
<!-- The main aim of this project was to create a custom peer to peer network. The user acting as the
client has total flexibility on how to batch the tasks and the user acting as the server has complete
flexibility on tracking the container's usages and killing the containers at any point of time.
flexibility on tracking the container's usages and killing the containers at any point of time. -->
<!-- The objective is to allow users to self host servers in easier
and abstracted manner. The main aim of this project was to create a custom peer to peer network for distributed computing. The user acting as the client has total flexibility on how to batch the tasks to any of nodes in networks. These nodes can anywhere from personal
computers behind NAT to custom hardware for running custom workload. The aim to provide
access to Heterogeneous set of nodes as a singular global list and abstract away the networking details giving the user focus on designing a custom orchestrator based
on the requirements of a user. -->
## Gophers talk
[![IMAGE ALT TEXT](https://i.ytimg.com/vi/ovcZLEhQxWk/hqdefault.jpg)](https://www.youtube.com/watch?v=ovcZLEhQxWk "P2PRC - Gophers monthly talk")
This project aims to simplify self-hosting servers and streamline the creation of distributed systems. The primary focus is to enable users to design custom peer-to-peer networks for distributed computing, offering full flexibility and control while abstracting away the complexities of networking.
## Key Features
Simplified Self-Hosting
Empower users to easily host nodes, whether on personal computers behind NAT, custom hardware, or cloud-based infrastructure.
### Custom Peer-to-Peer Networks
Build a decentralised network tailored to user requirements, enabling distributed task execution without the need for in-depth networking knowledge.
### Heterogeneous Node Support
Harness a diverse array of nodes from personal computers to specialised hardware—seamlessly integrated into a global node list.
### Abstracted Networking Layer
Networking details are completely abstracted, enabling users to focus on developing bespoke task orchestration systems. No more manual configuration of IPs, ports, or connection logic.
### Flexible Task Distribution
Users acting as clients retain full control over how tasks are batched and assigned to nodes, creating endless possibilities for optimising performance based on specific use cases.
## Setup with Nix
- Ensure you have the nix package manager installed.
- Add the following to your .bashrc file.
```bash
alias p2prc='env P2PRC=<path you want the configs to exist in> nix run github:akilan1999/p2p-rendering-computation --'
```
- Check if it's successful
```bash
p2prc -h
```
Read more on command usage: https://p2prc.akilan.io/Docs/#using-basic-commands
## Latest tutorial
[![IMAGE ALT TEXT](https://i.ytimg.com/vi/OMwCpedu5cs/hqdefault.jpg)](https://www.youtube.com/watch?v=OMwCpedu5cs")
<br>
## Table of contents
1. [Introduction](#Introduction)
2. [Installation](#Installation.md)
## Documentation
### We have documented to our best effort the basics of p2prc: [Link](https://p2prc.akilan.io/Docs/)
<!-- ## Table of contents in the current README -->
<!-- 1. [Introduction](#Introduction)
2. [Installation](#extend-your-application-with-p2prc)
3. [Design Architecture](#Design-Architecture)
4. [Implementation](#Implementation)
5. [Find out more](#Find-out-more)
5. [Find out more](#Find-out-more) -->
<br>
<!-- # Table of contents in the Docs folder
1. [Introduction](Docs/Introduction.md)
2. [Installation](Docs/Installation.md)
3. [Abstractions](Docs/Abstractions.md)
3. [Design Architecture](DesignArchtectureIntro.md)
1. [Client Module](ClientArchitecture.md)
2. [P2P Module](P2PArchitecture.md)
3. [Server Module](ServerArchitecture.md)
4. [Implementation](Docs/Implementation.md)
1. [Client Module](Docs/ClientImplementation.md)
2. [P2P Module](Docs/P2PImplementation.md)
3. [Server Module](Docs/ServerImplementation.md)
4. [Config Module](Docs/ConfigImplementation.md)
5. [Cli Module](Docs/CliImplementation.md)
6. [Plugin Module](Docs/PluginImplementation.md)
7. [Language bindings](Docs/Bindings.md)
8. [Domain name mappings](Docs/Bindings.md)
5. Language bindings
1. [Haskell](Docs/haskell/)
5. [Problems](https://github.com/Akilan1999/p2p-rendering-computation/issues) -->
<br>
<!--
## Introduction
This project aims to create a peer to peer (p2p) network, where a user can use the p2p network to act as a client (i.e sending tasks) or the server (i.e executing the tasks). A prototype application will be developed, which comes bundled with a p2p module and possible to execute docker containers or virtual environments across selected nodes.
@@ -39,31 +108,47 @@ This project aims to create a peer to peer (p2p) network, where a user can use t
- Background review on peer to peer network, virtual environments, decentralized rendering tools and tools to batch any sort of tasks.
- Creating p2p network
- Server to create a containerised environment
- The client node to run tasks on Server containerised node
- The client node to run tasks on Server containerised node -->
[Read more on the introduction](Docs/Introduction.md)
<!-- [Read more on the introduction](Docs/Introduction.md) -->
<br>
<!-- <br> -->
## Extend your application with P2PRC
<!-- ## Extend your application with P2PRC
```go
package main
import "github.com/Akilan1999/p2p-rendering-computation/abstractions"
import (
"fmt"
"github.com/Akilan1999/p2p-rendering-computation/abstractions"
)
func main() {
// Initialize with base p2prc config files
err := abstractions.Init("TEST")
if err != nil {
return
}
_, err := abstractions.Init(nil)
if err != nil {
fmt.Println(err)
return
}
// start p2prc
_, err = abstractions.Start()
if err != nil {
return
}
// start p2prc
_, err = abstractions.Start()
if err != nil {
fmt.Println(err)
return
}
// Run server till termination
for {
}
}
``` -->
<!-- ### Export once this is added export P2PRC as environment paths
```
export P2PRC=<PROJECT PATH>
export PATH=<PROJECT PATH>:${PATH}
```
[Read more](Docs/Abstractions.md) ...
@@ -115,12 +200,10 @@ The programming language used for this project was Golang. The reason Go lang wa
[Read more on the Implementation](Docs/Implementation.md)
<br>
<br> -->
## Find out more
As we are working on the open source project p2prc (i.e p2p network designed for computation).If you are interested in participating as a contributor
or just providing feedback on new features to build or even just curious about new features added to the project. We have decided to create a discord group.
[![Support Server](https://discordapp.com/api/guilds/854397492795277322/widget.png?style=banner2)](https://discord.gg/b4nRGTjYqy)
[![Star History Chart](https://api.star-history.com/svg?repos=Akilan1999/p2p-rendering-computation&type=Date)](https://github.com/Gaurav-Gosain)
or just providing feedback on new features to build or even just curious about new features added to the project do file an Issue or send an email to me@akilan.io.
<!-- [![Star History Chart](https://api.star-history.com/svg?repos=Akilan1999/p2p-rendering-computation&type=Date)](https://github.com/Gaurav-Gosain) -->

109
abstractions/base.go Normal file
View File

@@ -0,0 +1,109 @@
package abstractions
import "C"
import (
"github.com/Akilan1999/p2p-rendering-computation/client"
"github.com/Akilan1999/p2p-rendering-computation/client/clientIPTable"
Config "github.com/Akilan1999/p2p-rendering-computation/config"
"github.com/Akilan1999/p2p-rendering-computation/config/generate"
"github.com/Akilan1999/p2p-rendering-computation/p2p"
"github.com/Akilan1999/p2p-rendering-computation/server"
"github.com/Akilan1999/p2p-rendering-computation/server/docker"
"github.com/gin-gonic/gin"
"os"
)
// Init Initialises p2prc
func Init(customConfig interface{}) (config *Config.Config, err error) {
// Get config file path
// Checks P2PRC path initially
// - Get PATH if environment varaible
path, err := Config.GetPathP2PRC("P2PRC")
if err != nil {
return
}
// check if the config file exists
if _, err = os.Stat(path + "config.json"); err != nil {
// Initialize with base p2prc config files
// set the config file with default paths
config, err = generate.SetDefaults("P2PRC", false, customConfig, false)
if err != nil {
return
}
} else {
// If the configs are available then use them over generating new ones.
config, err = Config.ConfigInit(nil, nil)
if err != nil {
return
}
}
return
}
// Start p2prc in a server mode
func Start() (*gin.Engine, error) {
engine, err := server.Server()
if err != nil {
return nil, err
}
return engine, nil
}
// MapPort Creates a reverse proxy connection and maps the appropriate port
func MapPort(port string, domainName string, serverAddress string) (response *client.ResponseMAPPort, err error) {
response, err = client.MAPPort(port, domainName, serverAddress)
return
}
// StartContainer Starts docker container on the remote machine
func StartContainer(IP string) (container *docker.DockerVM, err error) {
container, err = client.StartContainer(IP, 0, false, "", "")
return
}
// RemoveContainer Removes docker container based on the IP address and ID
// provided
func RemoveContainer(IP string, ID string) error {
return client.RemoveContianer(IP, ID)
}
// GetSpecs Get spec information about the remote server
func GetSpecs(IP string) (specs *server.SysInfo, err error) {
specs, err = client.GetSpecs(IP)
return
}
// ViewIPTable View information of nodes in the network
func ViewIPTable() (table *p2p.IpAddresses, err error) {
table, err = p2p.ReadIpTable()
return
}
// UpdateIPTable Force updates IP tables based on new
// new nodes discovered in the network
func UpdateIPTable() (err error) {
return clientIPTable.UpdateIpTableListClient()
}
// AddCustomInformation allows to pass custom information
// through the network to which can be listened on
// all peers in the network to execute a task.
func AddCustomInformation(information string) error {
return clientIPTable.AddCustomInformationToIPTable(information)
}
// AddRootNode Adds root node to the network by using defaults except for
// ip address and port no. Supports only IPV4 as of now.
func AddRootNode(rootIP string, portNo string) error {
var rootNode []p2p.IpAddress
rootNode = append(rootNode, p2p.IpAddress{
Name: "",
Ipv4: rootIP,
ServerPort: portNo,
NAT: false,
})
return generate.GenerateIPTableFile(rootNode)
}

93
abstractions/docs.md Normal file
View File

@@ -0,0 +1,93 @@
<!-- Code generated by gomarkdoc. DO NOT EDIT -->
# abstractions
```go
import "github.com/Akilan1999/p2p-rendering-computation/abstractions"
```
## Index
- [func GetSpecs\(IP string\) \(specs \*server.SysInfo, err error\)](<#GetSpecs>)
- [func Init\(customConfig interface\{\}\) \(config \*Config.Config, err error\)](<#Init>)
- [func MapPort\(port string, domainName string, serverAddress string\) \(entireAddress string, mapPort string, err error\)](<#MapPort>)
- [func RemoveContainer\(IP string, ID string\) error](<#RemoveContainer>)
- [func Start\(\) \(\*gin.Engine, error\)](<#Start>)
- [func StartContainer\(IP string\) \(container \*docker.DockerVM, err error\)](<#StartContainer>)
- [func UpdateIPTable\(\) \(err error\)](<#UpdateIPTable>)
- [func ViewIPTable\(\) \(table \*p2p.IpAddresses, err error\)](<#ViewIPTable>)
<a name="GetSpecs"></a>
## func [GetSpecs](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L73>)
```go
func GetSpecs(IP string) (specs *server.SysInfo, err error)
```
GetSpecs Get spec information about the remote server
<a name="Init"></a>
## func [Init](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L17>)
```go
func Init(customConfig interface{}) (config *Config.Config, err error)
```
Init Initialises p2prc
<a name="MapPort"></a>
## func [MapPort](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L55>)
```go
func MapPort(port string, domainName string, serverAddress string) (entireAddress string, mapPort string, err error)
```
MapPort Creates a reverse proxy connection and maps the appropriate port
<a name="RemoveContainer"></a>
## func [RemoveContainer](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L68>)
```go
func RemoveContainer(IP string, ID string) error
```
RemoveContainer Removes docker container based on the IP address and ID provided
<a name="Start"></a>
## func [Start](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L46>)
```go
func Start() (*gin.Engine, error)
```
Start p2prc in a server mode
<a name="StartContainer"></a>
## func [StartContainer](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L61>)
```go
func StartContainer(IP string) (container *docker.DockerVM, err error)
```
StartContainer Starts docker container on the remote machine
<a name="UpdateIPTable"></a>
## func [UpdateIPTable](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L86>)
```go
func UpdateIPTable() (err error)
```
UpdateIPTable Force updates IP tables based on new new nodes discovered in the network
<a name="ViewIPTable"></a>
## func [ViewIPTable](<https://github.com/Akilan1999/p2p-rendering-computation/blob/master/abstractions/base.go#L79>)
```go
func ViewIPTable() (table *p2p.IpAddresses, err error)
```
ViewIPTable View information of nodes in the network
Generated by [gomarkdoc](<https://github.com/princjef/gomarkdoc>)

View File

@@ -1,27 +0,0 @@
package abstractions
import (
"github.com/Akilan1999/p2p-rendering-computation/config"
"github.com/Akilan1999/p2p-rendering-computation/config/generate"
"github.com/Akilan1999/p2p-rendering-computation/server"
"github.com/gin-gonic/gin"
)
// Init Initialises p2prc
func Init(name string, customConfig interface{}) (config *config.Config, err error) {
// set the config file with default paths
config, err = generate.SetDefaults(name, false, customConfig, false)
if err != nil {
return
}
return
}
// Start p2prc in a server mode
func Start() (*gin.Engine, error) {
engine, err := server.Server()
if err != nil {
return nil, err
}
return engine, nil
}

Some files were not shown because too many files have changed in this diff Show More