me/Akilan-website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

new changes

Browse Source
This commit is contained in:
Akilan Selvacoumar
2025-04-05 11:20:00 +01:00
commit e635774871
255 changed files with 2506 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
content/technicalposts/Custom-static-website-generator.md Normal file
View File

@@ -0,0 +1,42 @@
---
title: "Custom static website generator"
summary: Based on a Website generator built by
hw tech club
date: 2020-02-27
tags: ["Generator","Website"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/technicalposts/website-generator]
weight: 2
---
In this blog post we will talk about the project custom website generator which hw tech club contributors created. Before that a bit of basics of how static website generators work. In the current scenarios websites are either rendered in the front end using React/Vue or in the backend using Jinga , express etc … The flaw in the front end is that websites use a lot of CPU memory from the Client side to load webpages and rendering on the backend does fix this issue but is still relatively more expensive than loading static web pages. A static website generator basically generates static web pages only when content is changed.
### Problems of existing Static website generators
There are many static generators such as Hugo and Jenkyll which are super efficient and do a great job in generating static web pages. The only problem is both are human readable and not Api friendly which made it initially tough for us to interact with it. Ex: Hugo has many variations and with Yaml , Toml and creating an Api to store these states and ensuring to populate according to the template was super painful for us.
### Solution
To build a simple website generator that can take a JSON file of any shape and populate it to the according website template. The only catch is that both the JSON shape and fields that should be populated in the template should be of the same shape. The implementation of this was done using Node js as that was the only language me and my friends were familiar with at that time.
![Describes a high level visualization](https://blogs.hwtech.club/images/Customwebsite.png)
### Process of generating a website
- Send Data via Rest Api
- Populate inside the appropriate template
- Copy the files modified and add it to the project directory which is created based on the project name in the JSON file.
- Respond to the user the website generated link
### Deploying generated websites in IPFS
There are bash scripts added to make it possible to deploy websites generated to IPFS (Interplanetary file system). This is to ensure that your generated website can be deployed on a peer to peer network.
The following project is released on GPLv2 hence it is completely open source and we accept contributions from all developers.
### How to contribute:
1. Create an Issue
2. Create a PR(Pull Request) linking to that issue
3. If we approve it will be merged or we will inform you on the changes we need
We have only covered the high level overview of this project and there are other modules not talked about like the whatsapp bot to interact with this Api.
- [Documentation to the project](https://website-generator.netlify.app)
- [Module of the project we are talking about in this blog](https://github.com/HWTechClub/Custom-Static-Website-Generator/tree/master/website-generator)
- [Repo Link (Main project)](https://github.com/HWTechClub/Custom-Static-Website-Generator)

39
content/technicalposts/IT-infrastrucuture-Beginners.md Normal file
View File

@@ -0,0 +1,39 @@
---
title: "IT infrastructure (Beginners guide)"
summary: Basics of running power and cheap IT infrastructure
date: 2020-05-25
aliases: ["/papermod-how-to-guide"]
tags: ["IT","Beginners"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/technicalposts/beginners]
weight: 2
---
Although the current cost of IT infrastructure is relatively minimal cost these days. We can still dramatically reduce the cost further. A good IT infrastructure is the fact that we rely less on 3rd party companies and they are simple to run. When we mean simple it does mean a fancy UI where everything is drag and drop. Using a terminal can be your best friend and save you a lot of time. Like any skill you need to use it regularly to reap the benefits. It is very important to stay minimal to sustain your IT infrastructure for a longer period of time.
## Always stick to Open Source OS:
If you run a windows server then stop it. Always use an Open source OS over Proprietary. Transition is an open source OS (i.e Ubuntu , Debian) is relatively easy. To start off use a VPS to host your servers (i.e Digital Ocean, Vultr) . The main reason to use a VPS is the fact that you get an static IPV4 address. You pay for the hour rather than the month. If you think something is wrong in your server you can back it up and delete the server. Saves a lot of time on your side.
If you have an inhouse server, You can either get a static IPV4 address or use a DDNS if it is expensive (i.e so that someone can access it from outside your company network). There are many free training courses to learn how to use a linux server and configure it. This means you can be self-reliant and not rely on other companies to configure your server.
### Benefits of Open Source servers:
The main benefit is that there is no licensing cost and you have access to the source code. For Example : The linux community is really big and OS keeps getting updated frequently that means if a new vulnerability is found its most likely fixed really quick. The concept of open source OS is simple if you break it then just back up data and reinstall the OS.
## You should know Git no matter who you are:
Git is a version control system which can be used to store documents or codebase. Using it is the best way to store your company information in a secure way. It recommended you run it on your company server (Recommended to use Gitlab). If you want you can use github ( :) Owned by microsoft) .
1. [Documentation](https://git-scm.com/doc)
2. [Video](https://www.youtube.com/watch?v=2sjqTHE0zok&list=LLhy18QB1hBzZ7MniIW_FJvQ&index=6&t=0s)
## You can avoid office 365:
For your email you can set up your own mail server(i.e https://github.com/maildev/maildev) or use [Migadu](https://www.migadu.com/en/index.html) which is extremely cheap. There are so many alternatives for word , excel etc .. ( i.e LibreOffice , Apache Open Office , Google Suite etc...). If you want to have your own conference on your own server or on outside server use [Jitsi](https://jitsi.org/).
For any tool which is paid there is an open source alternative.
## Building websites:
If you are planning to have a website which is either an E-commerce website or a normal company webpage there is no point of a database like Sql unless your website is really huge. I would recommend using [static website generators](https://dzone.com/articles/6-reasons-why-you-should-go-for-a-static-website) like [Hugo](https://gohugo.io/) and either design your theme or use the ones available online. In most cases you only need to only use static generators for your company page and your basic e-commerce website.
## Deploying websites:
If you use a static website generator you can either deploy it on places like [netlify](https://www.netlify.com/) for free of cost or deploy them on your server.

73
content/technicalposts/Modifying-laplace-keyboard-mouse.md Normal file
View File

@@ -0,0 +1,73 @@
---
title: "Patch: Using remote ScreenShare ,keyboard and mouse using laplace and Barrier"
summary: Modifying the open source project laplace to allow remote keyboard and mouse during screenshare
date: 2021-07-05
aliases: ["/laplace-keyboard-mouse"]
tags: ["WebRTC","laplace"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/laplace-keyboard-mouse]
weight: 2
---
## What is laplace ?
" Laplace is an open-source project to enable screen sharing directly via browser. Made possible using WebRTC for low latency peer-to-peer connections, and WebSocket implemented in golang for WebRTC signaling. "
link: https://github.com/adamyordan/laplace
Link to my branch which has Barrier intergrated with Barrier KVM:
https://github.com/Akilan1999/laplace/tree/keyboard_mouse
Link to the pull request: https://github.com/adamyordan/laplace/pull/8
## Installation
### Installation required to share keyboard and mouse
To do this we ensure that the client either has has a IPV6
address or a public IPV4 address.
We use the use the popular open repository known as [Barrier KVM](https://github.com/debauchee/barrier).
#### What is Barrier kvm?
"Barrier is software that mimics the functionality of a KVM switch, which historically would allow you to use a single keyboard and mouse to control multiple computers by physically turning a dial on the box to switch the machine you're controlling at any given moment. Barrier does this in software, allowing you to tell it which machine to control by moving your mouse to the edge of the screen, or by using a keypress to switch focus to a different system." [Barrier KVM](https://github.com/debauchee/barrier)
#### Barrier KVM build status and links to install
|Platform |Build Status|
| --:|:-- |
|Linux |[![Build Status](https://dev.azure.com/debauchee/Barrier/_apis/build/status/debauchee.barrier?branchName=master&jobName=Linux%20Build)](https://dev.azure.com/debauchee/Barrier/_build/latest?definitionId=1&branchName=master)|
|Mac |[![Build Status](https://dev.azure.com/debauchee/Barrier/_apis/build/status/debauchee.barrier?branchName=master&jobName=Mac%20Build)](https://dev.azure.com/debauchee/Barrier/_build/latest?definitionId=1&branchName=master)|
|Windows Debug |[![Build Status](https://dev.azure.com/debauchee/Barrier/_apis/build/status/debauchee.barrier?branchName=master&jobName=Windows%20Build&configuration=Windows%20Build%20Debug)](https://dev.azure.com/debauchee/Barrier/_build/latest?definitionId=1&branchName=master)|
|Windows Release|[![Build Status](https://dev.azure.com/debauchee/Barrier/_apis/build/status/debauchee.barrier?branchName=master&jobName=Windows%20Build&configuration=Windows%20Build%20Release%20with%20Release%20Installer)](https://dev.azure.com/debauchee/Barrier/_build/latest?definitionId=1&branchName=master)|
|Snap |[![Snap Status](https://build.snapcraft.io/badge/debauchee/barrier.svg)](https://build.snapcraft.io/user/debauchee/barrier)|
### Build from source
```bash
$ git clone https://github.com/adamyordan/laplace.git
$ cd laplace && go build -o laplace main.go
$ export LAPLACE = $PATH
$ ./laplace --help
$ ./laplace -setconfig
$ ./laplace -tls -addrs <ip address>:<port no>
```
Note: ensure you are running none using Sudo
## Limitation
The machine that wants to control the screen needs to either be:
- Same network as the server
- Needs to have an IPV6 address
- Needs to have an public IPV4 address
## Vidoe demo
#### Video demo server side installation
{{< youtube 7c1_jlXJW4Q >}}
#### Video demo client side installation
{{< youtube gF70pqvN3Gk >}}

50
content/technicalposts/P2PRC-1.0.0-Alpha-release.md Normal file
View File

@@ -0,0 +1,50 @@
---
title: "Alpha Release P2PRC v1.0.0"
summary: Alpha release for P2PRC
date: 2021-06-27
aliases: ["/papermod-how-to-guide"]
tags: ["P2PRC","v1.0.0-alpha"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/p2prc-1.0.0-alpha]
weight: 2
---
After 5 months of working on this project. I would be glad to announce that we
have a alpha release. The objective of P2PRC is to create a p2p network where nodes
can share each others computatination power in a virtualized manner (i.e digital
ocean where anyone can share resources). In the alpha release the choice of
virtualization used is docker. The current release will
support docker. It should be noted that from the next release onwards we will
not support the docker daemon. We will try to provide backwards compatability
as much as we can but I have decided to ditch the docker daemon and write
compatable code from containerd.
## Major features
- IPV4 and IPV6 support: All the nodes can listen to both IPV4 and
IPV6 addresses. It can be noted that with IPV6 there are minor issues
detected in certain PC in terms of SSH into a docker container which is
mostly on the server side.
- Hopping mechanism: Since it's a p2p network we have a very simple
implementation that hops though servers and downloads it's IP tables.
In our implementation. This contains about imformtaion of other servers
in the network. This way nodes can automatically learn parts of the
network. The current implementation uses 3 hops.
- Possibility to run any docker container: To satisfy this the server
must already have the docker container and the must ensure that the
SSH username is 'master' and password is 'password' in creation of new
servers. It's not secure but worksout for this version for now.
The test network will be avaliable for use from the beta release
which should come out on the 5th of july 2021. The network will
consist of 1 node which runs on a paid cloud server as I do not
keep my machines on 24/7. The node up will help to connect with
other nodes and the paid cloud server cannot be used to create
and remove containers.
### Link to release and installation instructions:
https://github.com/Akilan1999/p2p-rendering-computation/releases/tag/v1.0.0-alpha

135
content/technicalposts/P2PRC-AST.md Normal file
View File

@@ -0,0 +1,135 @@
---
title: "Extension of P2PRC to create your own modification"
summary: P2PRC as a abstraction
date: 2021-08-31
aliases: ["/P2PRC-Extending"]
tags: ["P2PRC","AST","Extend"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/P2PRC-Extending]
weight: 2
---
# Generate Module
P2PRC is a great layer of abstraction. This means that in many cases it is not an end product but rather
a tool that customized as an end product. An example would be writing your own billing module to monetize
the computation power available. The generate module copies the current with the appropriate git histories
and keeps only the go files which would be useful to edit. To use the generate module the user will need
to have a go compiler present in his computer. Due to the introduction of this module there will 2 releases:
- Regular Release (Consists of only the build binary and cli command cannot access the generate module)
- Developer Release (Consists of important Go files and the cli can access the generate module)
## How does this work ?
### [Struct information](https://github.com/Akilan1999/p2p-rendering-computation/blob/9d69aed8ce0fe5273aaff2828f7d51c3d5ac2ce4/generate/generate.go#L19)
- ###```Generate.go```:
This file creates a local copy of P2PRC from where the CLI was called from.
This go file also does various stuff like instruction of file should be ignored when copying and
which of should not be. Now let's understand this. Below is a sample code which does the following:
```go
//----------------------------------------------------------------
// Action performed:
// - Ensuring main.go file exists
// - Skipping all .go files apart from the ones listed above
// - Skipping .idea/ directory
// - Skipping Makefile file
//----------------------------------------------------------------
Options.Skip = func(src string) (bool, error) {
switch {
case strings.HasSuffix(src, "main.go"):
return false, nil
case strings.HasSuffix(src, ".go"):
return true, nil
case strings.HasSuffix(src, ".idea"):
return true, nil
case strings.HasSuffix(src, "Makefile"):
return true, nil
default:
return false, nil
}
}
// Doing the copy
err = copy.Copy("<P2PRC folder you want to copy from>", "<PATH to the directory>", Options)
```
Unfortunately currently this will have to be manually edited in the ```Generate.go``` file. When using the generate
module the user also creates their own Go module which is the modified version of P2PRC. This means
if the 1 modified package is using another modified package then the appropriate import have to be modified
in the file where the import is called:
Ex:
```go
//Sample Project module name = Test
//Package names:
//- Test/Genius
//- Test/GeGeGenuis
//
// When we call the generate function with the new project with the module name = MicDrop
// The new package name would be:
// - MicDrop/Genius
// - MicDrop/GeGeGenuis
// Test/Genius code depends on the package Test/GeGeGenuis
import (
"Test/GeGeGenuis"
)
// When we create a new module with the copy of the
// existing project we need change:
import (
"MicDrop/GeGeGenuis"
)
```
To do this we have built functions which can modify import names in the Go file provided.
To customize the use case of your generate module you would need to manually add your own
imports which are supposed to be replaced and in which files they are supposed to be replaced
in.
```go
// 1.0 - Test/Genius.go -> GeGeGenuis module
// a is struct of type NewProject
a.FileNameAST = "<path to project to copy from>/Test/Genius.go"
// Get AST information of the file
err := a.GetASTGoFile()
if err != nil {
return err
}
// Change the appropriate Go file
err = a.ChangeImports("Test/GeGeGenuis", "MicDrop/GeGeGenuis")
if err != nil {
return err
}
// Writes the change to the appropriate file
err = a.WriteGoAst()
if err != nil {
return err
}
```
Higher order of execution of ```Generate.go```:
1. Copy entire P2PRC project and ignores files which are not meant to be copied
2. The folder name will be based on the new project name and the module name based on the new
module name provided.
3. Modifies the appropriate imports in the project as instructed in the code.
4. Creates a commit with the new changes in the new project.
- ### ``` modifyGenerate.go```:
This a really simple implementation where we replace the imports
in certain files as instructed from ```generate.go```. To do we create an AST (i.e Abstract Syntax tree)
from new file we want to change the imports in. AST create a tree structure of expression. To change the
import we can just traverse to the appropriate expression and change the value of that expression in
the case of modifying imports. This approach is more simple than using templates.
## Cli command
```
p2prc --gen <project name> --mod <go module name>
```
#### based on PR: https://github.com/Akilan1999/p2p-rendering-computation/pull/69

140
content/technicalposts/P2PRC-Escaping-NAT.md Normal file
View File

@@ -0,0 +1,140 @@
---
title: "Draft for escaping NAT with reverse proxy P2PRC"
summary: Annoying NAT boo boo
date: 2021-10-11
aliases: ["/P2PRC-escaping-NAT"]
tags: ["P2PRC","NAT"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/P2PRC-escaping-NAT]
weight: 2
---
## Abstract
We focus here on escaping NAT for the project P2PRC. The intial
plan is to use a middle server with an IPV4 address to escape NAT.
The future plan would be to use a WebRTC approach to where we
use a middle server to intansiate a direct p2p connection
using UDP sockets with nodes behind NAT.
## Architecture plan
The server behind NAT would use a reverse proxy server with an IPV4 address
to be a possible way to communicate with the outside world.
Ex:
```
NAT
|
Server <--> Reverse Proxy server with <--> Clients
| public IPV4
```
We use the libaray known as [frp](https://github.com/fatedier/frp).
"frp is A fast reverse proxy to help you expose a local server behind a NAT
or firewall to the internet."
The server behind the NAT IPTable would be broadcasted through the reverse proxy.
The mapping would be done automatically though P2PRC because the reverse proxy
could have certain ports already taken.
We are introducting a new concept of detecting reverse proxies over P2PRC.
We call this ``` reverse_proxy_server.json ``` and ``` reverse_proxy_mapping.json```.
The ```reverse_proxy_server.json``` file is incharge of detecting all reverse
proxy servers and the ```reverse_proxy_mapping.json``` is incharge of logging
all the reverse proxy server mappings talking place.
#### reverse_proxy_server.json
```
{
"reverse_proxy_servers": [
{
"Server_name": <server name>,
"Address": <remote address>,
"Latency": <latency>
}, ... n
]
}
```
#### reverse_proxy_mapping.json
```
{
"reverse_proxy_mappings": [
{
"Server_name": <server name>,
"Address": <remote address>,
"Type": <TCP or UDP>,
"blind_port": <port no>
}, ... n
]
}
```
### Port allocation
The function for port allocation would need certain
modifications need to ensure that the port is not only free
on the server but also on the on reverse proxy server.
We are debate on this would occur as it requires
further reading on the inner depths of [frp](https://github.com/fatedier/frp).
The idea scenario would be that the odds a 5 or 6 digit
taken is a low probability.
### How would the ip tables would look
There would not be much change when a client looks
at it. This only difference is that the client
see's the address of the proxy server.
Ex:
```
{
"ip_address": [
{
"ipv4": "<reverse proxy address>",
"ipv6": "",
"latency": 0,
"download": 0,
"upload": 0,
"serverport": "<server port no>"
}
]
}
```
If the server is using a reverse proxy then the server
port no should be decided by P2PRC to ensure that there
is no scenario where the port is already taken.
## New Cli Commands
We will be introducing a new set of Cli commands.
These cli commands would be incharge to starting
the reverse proxy or connecting to the reverse proxy
and various other function.
1. start reverse proxy (need public IPV4)
```
p2prc -s --reverse-proxy
```
This will start the reverse
and autoamtically broadcast
as a reverse proxy as well.
2. connecting to a reverse proxy
```
p2prc -s --connect-reverse-proxy <proxy ip address>
```
This will start p2prc as server
and automatically set the server port
and link it to the reverse proxy and
any connections would go through the
reverse proxy.
3. Listing all proxy servers avaliable
```
p2prc --view-rps or p2prc --view-reverse-proxy-servers
```
4. Listing all reverse proxy connection mapping
```
p2prc --view-rpm or p2prc --view-reverse-proxy-mapping
```

92
content/technicalposts/P2PRC-Plugin-Module-Implementation.md Normal file
View File

@@ -0,0 +1,92 @@
---
title: "P2PRC Plugin module implementation"
summary: Plugin Module implementation P2PRC
date: 2021-07-16
aliases: ["/P2PRC-Plugin"]
tags: ["P2PRC","Plugin"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/P2PRC-Plugin]
weight: 2
---
## Introduction
The plugin module is designed to ensure clients can execute instructions in a declarative manner across different
containers created. This means the user (i.e client) needs to write the instruction only once, and these instructions
can be executed across different nodes in a repetitive manner.
In the scenario of this project Ansibles will be used as the way the users can create these instructions.
The plugin module introduces a new path to the config file known as pluginpath. This path by defaults points to
```${P2PRC}/plugin/deploy```. Any file/folder inside ```plugin/deploy``` is part of the .gitginore. Plugins are
detected by folder names inside the ```plugin/deploy```.
```
plugin
|___ Deploy
|___<plugin name>
|___ site.yml
|___ hosts
|___ description.txt
.
.
.
n: n number of plugins possible
```
## Site File Template
The site file is also known as the Ansible playbook and is incharge of executing
instructions in a declarative manner. The below example specifies how to make one.
```
- hosts: all
tasks:
- name: <task name>
<ansible task>
debug:
msg: <debug message>
```
Read more about ansible tasks: https://docs.ansible.com/ansible/latest/user_guide/playbooks_intro.html#about-playbooks
## Hosts file
hosts file is also known as the inventory file. This file consists of all the information required to connect to other
nodes to execute Ansible instructions. In this project this file needs to be set in a certain configuration because the
go code or binary will populate this file automatically with the appropriate information required to connect to local or
remote containers.
#### Note: Add as exactly specified below
```
all:
vars:
ansible_python_interpreter: /usr/bin/python3 // Path to your python 3 interpreter
main:
hosts:
host1:
// Note: These values will be automatically overwritten
// by the Go functions
ansible_host: 0.0.0.0
ansible_port: 39269
ansible_user: master
ansible_ssh_pass: password
ansible_sudo_pass: password
```
## Description file
This is a simple text file used to describe what the module does.
When the client is looking at various commands via the ClI.
The description is displayed along-side the plugin name.
Ex: When the flag ```--ViewPlugins``` or ```--vp``` is called
```
{
"PluginsDetected": [
{
"FolderName": "<name of the plugin>",
"PluginDescription": "<description of the plugin>"
}
]
}
```
#### Based on PR: https://github.com/Akilan1999/p2p-rendering-computation/pull/50

60
content/technicalposts/P2PRC-automatic-ports.md Normal file
View File

@@ -0,0 +1,60 @@
---
title: "Automatic ports allocation P2PRC"
summary: Allocating ports to plugins P2PRC
date: 2021-10-03
aliases: ["/P2PRC-Automatic-ports"]
tags: ["P2PRC","Ports","TCP","Automatic"]
author: "Akilan Selvacoumar"
draft: false
aliases: [//P2PRC-Automatic-ports]
weight: 2
---
## Automatic port allocations
P2PRC would be in-charge to set to the ports to various TCP ports
opened. Due to this implementation the plugin being executed is
copied to the tmp directory with a unique UUID.
```
Command: ls /tmp
output: Semantic <UUID>_<Plugin Name>
2e6d76c4-0ed1-4b55-9385-79a58d4f0492_p2prc-vscode-browser
7b631e08-62ee-4c1c-a2a4-c05857b9aa7d_p2prc-vscode-browser
```
Once the copy of the plugin is added to the /tmp directory
the site.yml file inside the appropriate yaml is modified
with the appropriate ports assigned to the container.
### Ex:
1. Create container called c1 with an automatic generated TCP port
3313 (external) - 3313 (internal)
2. Assumption of plugin p1 exists. p1 has one server which needs to
be mapped to a free open TCP port in container c1. Below shows
an implementation of a sample site.yml file.
```
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:{{index . 0}}
```
Notice there is the following {{index . 0}}. {{index . 0}} does not belong to
Ansible but rather is a way to mention where to add the external free port
of the container. We use the golang [template library](https://pkg.go.dev/text/template)
to parse and populate the site.yml with the appropriate open ports. An array of ints
which consists of open free ports are sent to the site.yml. 0 in {{index . 0}} refers
to the index in the int array passed on.
After the port is automatically it's ready to run !
```
---
- hosts: all
tasks:
- name: start vscode code server
shell: sh server.sh 0.0.0.0:3313
```
### Sample plugins implemented:
- [VSCode Plugin](https://github.com/Akilan1999/p2prc-vscode-browser)
## based on PR: https://github.com/Akilan1999/p2p-rendering-computation/pull/73

6
content/technicalposts/_index.md Normal file
View File

@@ -0,0 +1,6 @@
---
title: Technical Posts
summary: Fun CS concepts to play around with
description: Contains posts regrading CS topics I love
to play around with.
---

25
content/technicalposts/distributed-computing-local-network.md Normal file
View File

@@ -0,0 +1,25 @@
---
title: "Distributed computing in a local network"
summary: Brainstroming ideas to run distributed tasks locally
date: 2021-01-01
aliases: ["/distributed-local-network"]
tags: ["Distributed","Brainstorming"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/technicalposts/distributed-local-network]
weight: 2
---
mDNS or called multicast DNS is just a DNS over a local network. mDNS is also considered as a Zero configuration technique. It can be used to discover nodes in a local network (ex: Apple’s bonjour).
## Implementation
- [multicast-dns](https://github.com/mafintosh/multicast-dns)
- [dns-discovery](https://github.com/mafintosh/dns-discovery)
- [libp2p-mdns](https://github.com/libp2p/js-libp2p-mdns)
Assuming all machines are running a linux kernel. All nodes in the network can have an SSH server and can broadcast each other's public key. All the nodes can add broadcasted public keys to their authorized_keys file. This is assuming it’s done in a local private network (i.e not a secure solution but an easier way). Now all nodes can SSH into each other.
To render tasks [DrQueue](https://github.com/DrQueue/drqueue) seems to be a promising choice. DrQueue is an open source render farm system. Similar to Farmer-Joe-Render. DrQueue is used in the Visual effects ,science and finance industry. The objective of DrQueue is batch tasks and run as a task management tool for multiple nodes. DrQueue is compatible on Windows, macOS, Linux, Irix and FreeBSD. DrQueue can be used with any renderer that supports a CLI(Command Line Interface). DrQueue auto generated scripts for Blender, Maya, Lightwave, Cinema 4D, Maya, lightwave, Mental Ray, After Effects, Aqsis, 3Delight, Pixie, Snake , Terragen and Nuke. DrQueue makes it possible to talk to master and slave nodes with a custom API which any user can create based on the inbuilt functions.
This is just a thought in my mind of a logical and simple way to run tasks distributed. This is really important as we underestimate the computing power we have if we use our home devices collectively. We could start pushing game developers to start building games for distributed systems. Imagine if [FS2020](https://blogs.hwtech.club/post/fs2020/) could be rendered distributed :).

155
content/technicalposts/riscv.md Normal file
View File

@@ -0,0 +1,155 @@
---
title: "Challenging myself to understand RISC-V"
summary: Basics of what is RISC-V
date: 2021-05-30
aliases: ["/riscv"]
tags: ["RISCV","Open Source"]
author: "Akilan Selvacoumar"
draft: false
aliases: [/technicalposts/riscv"]
weight: 2
---
## Why do we need open standards for CPU Architecture ?
- We want to allow users to see all the parts of the architecture
without any proprietary constraints.
- We want to have the rights to modify and distribute without paying any licensing fees and constraints in sharing.
- With open standards it's easier to build on top of others work and possibly build CPU designs custom made for certain tasks.
## What is RISC-V ?
RISC-V is just an open source ISA (Instruction Set Architecture). An ISA is the software interface for the hardware. A single ISA can
have many hardware implementations. In technical terms as ISA defines set of Instructions and how they behave such as:
- Data types
- Registers
- Addressing modes
- memory models
- Protection levels
- How is I/O are done
- Virtual memory
- Exceptions
Another important factor to understand is that RISC-V is a standard
and not an implementation. This means that the entire ISA is defined in a huge latex file which can be found on Github.
Repo link: https://github.com/riscv/riscv-isa-manual
The RISC-V is a well organized ISA and is divided into various categories and extensions in order to keep it as a modular design.
The RISC-V is maintained the non profit organization called [RISC-V foundation](https://riscv.org/).
### ["RISC-V in contrast was made specifically to be easy to teach while pragmatic enough to actually allow the implementation of high performance microprocessors."](https://medium.com/swlh/risc-v-assembly-for-beginners-387c6cd02c49)
## Few companies working on RISC-V
- [NVIDIA](https://riscv.org/wp-content/uploads/2017/05/Tue1345pm-NVIDIA-Sijstermans.pdf): using RISC-V in it's GPU
- [SiFive](https://www.sifive.com/): Startup that allows you to create your own RISC-V board or use exsisting models they provide with great toolkits around them.
- [Western Digital](https://blog.westerndigital.com/risc-v-swerv-core-open-source/): Focusing on building custom
RISC-V cores.
## Complier support for RISC-V ISA
- [GCC](https://github.com/riscv/riscv-gnu-toolchain)
- [LLVM/Clang](https://github.com/sifive/riscv-llvm)
- [glibc](https://sourceware.org/git/?p=glibc.git)
- [Go](https://github.com/riscvarchive/riscv-go)
- [Rust](https://github.com/riscv-rust/riscv-rust-quickstart)
## How is the RISC-V ISA organized
In this section we look into how RISC-V is organised
### RV N (Extension letter)
- RV stands for RISC-V
- N - Number of bits (Ex: 32 bits)
- Extension letter: This is the Extension for the instruction
sets (I stands for Integer)
### Example Base ISA
RV32I, RV64I, RV128I
### Standard Extensions
- M: Math
- A: Atomic
- F: Floating Point
- D: Double Precision Floating Point
#### - G: General Purpose (Includes, Integer, Math, Atomic, Floating Point and Double Precision Floating Point)
### Linux supports
#### RV64G
This is called as RISC-V 64 bit General Purpose
## Lets talk a bit indepth of RV32I
RV32I is a base ISA and the easiest to understand.
RV32I means it's RISC-V 32 bit Integer ISA.
```
Registers: x0-x31
(x0 is hardwired to 0)
```
Each register is 32 bits which can be called as 1 word.
![](https://raw.githubusercontent.com/Artoriuz/RV32I-SC/master/images/schematic.png)
Fig 1.0 [Simplified schematics of RV32I](https://github.com/Artoriuz/RV32I-SC)
Now we will recall the basics and try to understand
how to read an instruction.
```
add rd,rs1,rs2
```
- add: is the opcode
- rd: is the destination register
- rs1: Source register 1
- rs2: Source register 2
![](/img/rv32i.png)
Fig 1.1 [Assembly instructions for RV32I](https://github.com/jameslzhu/riscv-card/blob/master/riscv-card.pdf)
![](/img/registersrv32i.png)
Fig 1.2 [Registers for RV32I](https://github.com/jameslzhu/riscv-card/blob/master/riscv-card.pdf)
## Interesting research papers
- [Implementing RISC-V System-on-Chip for Acceleration of Convolution Operation and Activation Function Based on FPGA (Field programmable gate arrays)](https://ieeexplore.ieee.org/document/8564810): FGPA are mostly for application specific integrated circuits. An example would be Intel using FGPA to
prototype new chips. The objective of the paper was to design
a RISC-V processor for specific tasks such as Convolution functions and activation functions. The result was that the
RISC-V processor was faster than CPU + co-processor mode and
used lesser than the CPU + GPU mode.
- [Towards deep learning using Tensorflow lite on RISC-V](
https://edge.seas.harvard.edu/files/edge/files/carrv_workshop_submission_2019_camera_ready.pdf
): This paper focuses on ISA extensions customized for
machine learning kernels. The software infrastructure implemented was optimized for neural network execution.
The following was integrated into Tensorflow lite. The
result was that instructions was reduced by 8X.
- [A compiler comparison in the RISC-V ecosystem](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9191411): Comparing LLVM and GCC performance on RISC-V. LLVM and GCC
produce same binary size but both have different execution
times.
## Interesting Open Source projects
- [RISCBoy](https://github.com/Wren6991/RISCBoy): It is an open-source portable games console, designed from scratch.
RISC-V compatible CPU. Has raster graphics pipelines and display controllers. It consists of other infrastructure such as memory
controllers and GPIO ports. It also consists of a CAD design of
the PCB layout.
- [Potato](https://github.com/skordal/potato): The Potato Processor is a simple RISC-V processor written in VHDL for use in FPGAs. It implements the 32-bit integer subset of the RISC-V Specification version 2.0.
- [Vulcan](https://github.com/vmmc2/Vulcan): RISC-V instruction set simulation built for education using flutter.