7 lines
13 KiB
HTML
7 lines
13 KiB
HTML
<!doctype html><html lang=en dir=auto><head><script src="/livereload.js?mindelay=10&v=2&port=1313&path=livereload" data-no-instant defer></script><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1,shrink-to-fit=no"><meta name=robots content="index, follow"><title>Distributed computing in a local network | Akilan</title>
|
||
<meta name=keywords content="Distributed,Brainstorming"><meta name=description content="Brainstroming ideas to run distributed tasks locally"><meta name=author content="Akilan Selvacoumar"><link rel=canonical href=http://localhost:1313/technicalposts/distributed-computing-local-network/><link crossorigin=anonymous href=/assets/css/stylesheet.f49d66caae9ea0fd43f21f29e71a8d3e284517ed770f2aa86fa012953ad3c9ef.css integrity="sha256-9J1myq6eoP1D8h8p5xqNPihFF+13Dyqob6ASlTrTye8=" rel="preload stylesheet" as=style><link rel=icon href=http://localhost:1313/favicon.ico><link rel=icon type=image/png sizes=16x16 href=http://localhost:1313/favicon-16x16.png><link rel=icon type=image/png sizes=32x32 href=http://localhost:1313/favicon-32x32.png><link rel=apple-touch-icon href=http://localhost:1313/apple-touch-icon.png><link rel=mask-icon href=http://localhost:1313/safari-pinned-tab.svg><meta name=theme-color content="#2e2e33"><meta name=msapplication-TileColor content="#2e2e33"><link rel=alternate hreflang=en href=http://localhost:1313/technicalposts/distributed-computing-local-network/><noscript><style>#theme-toggle,.top-link{display:none}</style><style>@media(prefers-color-scheme:dark){:root{--theme:rgb(29, 30, 32);--entry:rgb(46, 46, 51);--primary:rgb(218, 218, 219);--secondary:rgb(155, 156, 157);--tertiary:rgb(65, 66, 68);--content:rgb(196, 196, 197);--code-block-bg:rgb(46, 46, 51);--code-bg:rgb(55, 56, 62);--border:rgb(51, 51, 51)}.list{background:var(--theme)}.list:not(.dark)::-webkit-scrollbar-track{background:0 0}.list:not(.dark)::-webkit-scrollbar-thumb{border-color:var(--theme)}}</style></noscript><meta property="og:url" content="http://localhost:1313/technicalposts/distributed-computing-local-network/"><meta property="og:site_name" content="Akilan"><meta property="og:title" content="Distributed computing in a local network"><meta property="og:description" content="Brainstroming ideas to run distributed tasks locally"><meta property="og:locale" content="en"><meta property="og:type" content="article"><meta property="article:section" content="technicalposts"><meta property="article:published_time" content="2021-01-01T00:00:00+00:00"><meta property="article:modified_time" content="2021-01-01T00:00:00+00:00"><meta property="article:tag" content="Distributed"><meta property="article:tag" content="Brainstorming"><meta name=twitter:card content="summary"><meta name=twitter:title content="Distributed computing in a local network"><meta name=twitter:description content="Brainstroming ideas to run distributed tasks locally"><script type=application/ld+json>{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Technical Posts","item":"http://localhost:1313/technicalposts/"},{"@type":"ListItem","position":2,"name":"Distributed computing in a local network","item":"http://localhost:1313/technicalposts/distributed-computing-local-network/"}]}</script><script type=application/ld+json>{"@context":"https://schema.org","@type":"BlogPosting","headline":"Distributed computing in a local network","name":"Distributed computing in a local network","description":"Brainstroming ideas to run distributed tasks locally","keywords":["Distributed","Brainstorming"],"articleBody":"mDNS or called multicast DNS is just a DNS over a local network. mDNS is also considered as a Zero configuration technique. It can be used to discover nodes in a local network (ex: Apple’s bonjour).\nImplementation multicast-dns dns-discovery libp2p-mdns Assuming all machines are running a linux kernel. All nodes in the network can have an SSH server and can broadcast each other’s public key. All the nodes can add broadcasted public keys to their authorized_keys file. This is assuming it’s done in a local private network (i.e not a secure solution but an easier way). Now all nodes can SSH into each other.\nTo render tasks DrQueue seems to be a promising choice. DrQueue is an open source render farm system. Similar to Farmer-Joe-Render. DrQueue is used in the Visual effects ,science and finance industry. The objective of DrQueue is batch tasks and run as a task management tool for multiple nodes. DrQueue is compatible on Windows, macOS, Linux, Irix and FreeBSD. DrQueue can be used with any renderer that supports a CLI(Command Line Interface). DrQueue auto generated scripts for Blender, Maya, Lightwave, Cinema 4D, Maya, lightwave, Mental Ray, After Effects, Aqsis, 3Delight, Pixie, Snake , Terragen and Nuke. DrQueue makes it possible to talk to master and slave nodes with a custom API which any user can create based on the inbuilt functions.\nThis is just a thought in my mind of a logical and simple way to run tasks distributed. This is really important as we underestimate the computing power we have if we use our home devices collectively. We could start pushing game developers to start building games for distributed systems. Imagine if FS2020 could be rendered distributed :).\n","wordCount":"283","inLanguage":"en","datePublished":"2021-01-01T00:00:00Z","dateModified":"2021-01-01T00:00:00Z","author":{"@type":"Person","name":"Akilan Selvacoumar"},"mainEntityOfPage":{"@type":"WebPage","@id":"http://localhost:1313/technicalposts/distributed-computing-local-network/"},"publisher":{"@type":"Organization","name":"Akilan","logo":{"@type":"ImageObject","url":"http://localhost:1313/favicon.ico"}}}</script></head><body id=top><script>localStorage.getItem("pref-theme")==="dark"?document.body.classList.add("dark"):localStorage.getItem("pref-theme")==="light"?document.body.classList.remove("dark"):window.matchMedia("(prefers-color-scheme: dark)").matches&&document.body.classList.add("dark")</script><header class=header><nav class=nav><div class=logo><a href=http://localhost:1313/ accesskey=h title="Akilan (Alt + H)">Akilan</a><div class=logo-switches><button id=theme-toggle accesskey=t title="(Alt + T)" aria-label="Toggle theme"><svg id="moon" width="24" height="18" viewBox="0 0 24 24" fill="none" stroke="currentcolor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 12.79A9 9 0 1111.21 3 7 7 0 0021 12.79z"/></svg><svg id="sun" width="24" height="18" viewBox="0 0 24 24" fill="none" stroke="currentcolor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="5"/><line x1="12" y1="1" x2="12" y2="3"/><line x1="12" y1="21" x2="12" y2="23"/><line x1="4.22" y1="4.22" x2="5.64" y2="5.64"/><line x1="18.36" y1="18.36" x2="19.78" y2="19.78"/><line x1="1" y1="12" x2="3" y2="12"/><line x1="21" y1="12" x2="23" y2="12"/><line x1="4.22" y1="19.78" x2="5.64" y2="18.36"/><line x1="18.36" y1="5.64" x2="19.78" y2="4.22"/></svg></button></div></div><ul id=menu><li><a href=http://localhost:1313/about/ title=About><span>About</span></a></li><li><a href=http://localhost:1313/resume/ title=Resumé><span>Resumé</span></a></li><li><a href=http://localhost:1313/projects/ title=Projects><span>Projects</span></a></li><li><a href=http://localhost:1313/technicalposts/ title="Technical Posts"><span>Technical Posts</span></a></li><li><a href=http://localhost:1313/generalposts/ title="General Posts"><span>General Posts</span></a></li><li><a href=http://localhost:1313/papers/ title=Papers><span>Papers</span></a></li><li><a href=http://localhost:1313/chat/ title=Chat><span>Chat</span></a></li><li><a href=http://localhost:1313/hiking/ title=Hiking><span>Hiking</span></a></li></ul></nav></header><main class=main><article class=post-single><header class=post-header><div class=breadcrumbs><a href=http://localhost:1313/>Home</a> » <a href=http://localhost:1313/technicalposts/>Technical Posts</a></div><h1 class="post-title entry-hint-parent">Distributed computing in a local network</h1><div class=post-meta><span title='2021-01-01 00:00:00 +0000 UTC'>2021-01-01</span> · 2 min · Akilan Selvacoumar | <a href=https://github.com/Akilan1999/Akilan-website/tree/master/content/technicalposts/distributed-computing-local-network.md rel="noopener noreferrer" target=_blank>Suggest Changes</a></div></header><div class=toc><details><summary accesskey=c title="(Alt + C)"><span class=details>Table of Contents</span></summary><div class=inner><ul><li><a href=#implementation aria-label=Implementation>Implementation</a></li></ul></div></details></div><div class=post-content><p>mDNS or called multicast DNS is just a DNS over a local network. mDNS is also considered as a Zero configuration technique. It can be used to discover nodes in a local network (ex: Apple’s bonjour).</p><h2 id=implementation>Implementation<a hidden class=anchor aria-hidden=true href=#implementation>#</a></h2><ul><li><a href=https://github.com/mafintosh/multicast-dns>multicast-dns</a></li><li><a href=https://github.com/mafintosh/dns-discovery>dns-discovery</a></li><li><a href=https://github.com/libp2p/js-libp2p-mdns>libp2p-mdns</a></li></ul><p>Assuming all machines are running a linux kernel. All nodes in the network can have an SSH server and can broadcast each other’s public key. All the nodes can add broadcasted public keys to their authorized_keys file. This is assuming it’s done in a local private network (i.e not a secure solution but an easier way). Now all nodes can SSH into each other.</p><p>To render tasks <a href=https://github.com/DrQueue/drqueue>DrQueue</a> seems to be a promising choice. DrQueue is an open source render farm system. Similar to Farmer-Joe-Render. DrQueue is used in the Visual effects ,science and finance industry. The objective of DrQueue is batch tasks and run as a task management tool for multiple nodes. DrQueue is compatible on Windows, macOS, Linux, Irix and FreeBSD. DrQueue can be used with any renderer that supports a CLI(Command Line Interface). DrQueue auto generated scripts for Blender, Maya, Lightwave, Cinema 4D, Maya, lightwave, Mental Ray, After Effects, Aqsis, 3Delight, Pixie, Snake , Terragen and Nuke. DrQueue makes it possible to talk to master and slave nodes with a custom API which any user can create based on the inbuilt functions.</p><p>This is just a thought in my mind of a logical and simple way to run tasks distributed. This is really important as we underestimate the computing power we have if we use our home devices collectively. We could start pushing game developers to start building games for distributed systems. Imagine if <a href=https://blogs.hwtech.club/post/fs2020/>FS2020</a> could be rendered distributed :).</p></div><footer class=post-footer><ul class=post-tags><li><a href=http://localhost:1313/tags/distributed/>Distributed</a></li><li><a href=http://localhost:1313/tags/brainstorming/>Brainstorming</a></li></ul><nav class=paginav><a class=prev href=http://localhost:1313/technicalposts/riscv/><span class=title>« Prev</span><br><span>Challenging myself to understand RISC-V</span>
|
||
</a><a class=next href=http://localhost:1313/technicalposts/it-infrastrucuture-beginners/><span class=title>Next »</span><br><span>IT infrastructure (Beginners guide)</span></a></nav></footer></article></main><footer class=footer><span>© 2025 <a href=http://localhost:1313/>Akilan</a></span> ·
|
||
<span>Powered by
|
||
<a href=https://gohugo.io/ rel="noopener noreferrer" target=_blank>Hugo</a> &
|
||
<a href=https://github.com/adityatelange/hugo-PaperMod/ rel=noopener target=_blank>PaperMod</a></span></footer><a href=#top aria-label="go to top" title="Go to Top (Alt + G)" class=top-link id=top-link accesskey=g><svg viewBox="0 0 12 6" fill="currentcolor"><path d="M12 6H0l6-6z"/></svg>
|
||
</a><script>let menu=document.getElementById("menu");menu&&(menu.scrollLeft=localStorage.getItem("menu-scroll-position"),menu.onscroll=function(){localStorage.setItem("menu-scroll-position",menu.scrollLeft)}),document.querySelectorAll('a[href^="#"]').forEach(e=>{e.addEventListener("click",function(e){e.preventDefault();var t=this.getAttribute("href").substr(1);window.matchMedia("(prefers-reduced-motion: reduce)").matches?document.querySelector(`[id='${decodeURIComponent(t)}']`).scrollIntoView():document.querySelector(`[id='${decodeURIComponent(t)}']`).scrollIntoView({behavior:"smooth"}),t==="top"?history.replaceState(null,null," "):history.pushState(null,null,`#${t}`)})})</script><script>var mybutton=document.getElementById("top-link");window.onscroll=function(){document.body.scrollTop>800||document.documentElement.scrollTop>800?(mybutton.style.visibility="visible",mybutton.style.opacity="1"):(mybutton.style.visibility="hidden",mybutton.style.opacity="0")}</script><script>document.getElementById("theme-toggle").addEventListener("click",()=>{document.body.className.includes("dark")?(document.body.classList.remove("dark"),localStorage.setItem("pref-theme","light")):(document.body.classList.add("dark"),localStorage.setItem("pref-theme","dark"))})</script></body></html> |