Hi,
THIS POST HAS NOW BEEN UPDATED WITH THE SOURCE CODE AND BINARIES. MAKE SURE TO CHECK THE README FOR INSTRUCTIONS.
I've been making a program in C# to handle networking rendering for Indigo. It works similar to a P2P program. You set up a "tracker" server on a system you know will always be up. Then you install the client renderers on your remote computers. You have to also be sure to have indigo installed of course.
Then you run the network master controller program on your own system. It connects to the tracker and gets a list of clients (connecting on port 8090). It then connects to each client on port 5200 and gives it commands, such as BEGINRENDER, CANCELRENDER, etc.
The big advantage to my central render controller is that it will upload the indigo XML file to each client, as well as uploading any other files needed - if there are any <path> elements in the XML file (for textures, etc) it also uploads those files to the remote render computer too!
Right now I have it hardcoded to C:\indigo for most everything, and the client and tracker don't have user interfaces, or INI settings yet. But the controller works really good, at least on my local network.
I wonder if anyone would be interested in using this in the long term, the code is definitely messy right now but we could have it open source so people could add features when they want. I think another good feature would be to have a indigo updater built into it so it could upload a whole indigo folder tree when you have a new one on the master system (so it can update all the remote clients with that version).
-niko
Screenshot.
network rendering central control program (C#)
-
nikolatesla20
- Posts: 52
- Joined: Fri Mar 09, 2007 4:44 am
network rendering central control program (C#)
- Attachments
-
- Indigo_Network_Renderer.zip
- (135.59 KiB) Downloaded 310 times
-
- networkrender.JPG (130.61 KiB) Viewed 15825 times
Last edited by nikolatesla20 on Thu Apr 24, 2008 1:57 am, edited 1 time in total.
-
PureSpider
- Posts: 1459
- Joined: Tue Apr 08, 2008 9:37 am
- Location: Karlsruhe, BW, Germany
- Contact:
-
nikolatesla20
- Posts: 52
- Joined: Fri Mar 09, 2007 4:44 am
I'll probably be able to post it in a week or so, once I have added a UI to the tracker and the remote rendering client. And I need to add INI settings ability so you can change the tracker server, etc. Right now I have stuff hardcoded for testing.
The only issue with it is that the master controller connects to the remote servers, and not the other way around - this can be problematic if the remote server is behind a NAT or firewall, because inbound traffic can be blocked.
The tracker runs on port 8090 and the remote servers run on port 5200, so if you are in control of the firewall/NAT you can forward those ports, but otherwise the traffic will get blocked between the master and the remote servers.
Also, I can upload the code probably as well, it should also have a security check done on it to check for buffer overflows, etc.
-niko
The only issue with it is that the master controller connects to the remote servers, and not the other way around - this can be problematic if the remote server is behind a NAT or firewall, because inbound traffic can be blocked.
The tracker runs on port 8090 and the remote servers run on port 5200, so if you are in control of the firewall/NAT you can forward those ports, but otherwise the traffic will get blocked between the master and the remote servers.
Also, I can upload the code probably as well, it should also have a security check done on it to check for buffer overflows, etc.
-niko
-
nikolatesla20
- Posts: 52
- Joined: Fri Mar 09, 2007 4:44 am
dougal, that sounds good. I have lots of experience with SVN so no problem there as well 
Thanks for the offer.
I threw this together pretty fast using a variety of code samples, so yes it's a mess right now, but a few days of cleaning would probably straighten it right up...
As I mentioned before though the only weak point right now is that the master controller connects to the slaves, whereas it may be better to make it the other way around, although this may increase network traffic (because the slaves would have to poll the tracker iteratively to see if a master has connected to the tracker yet.) Come to think of it though having it this way allows multiple masters, but I currently only allow one indigo process on the slaves (they will use auto-threads then anyway)
I was wondering about distributed internet rendering, but as far as internet rendering goes I don't know if such a thing is even possible with Indigo in an efficient manner..
-niko
Thanks for the offer.I threw this together pretty fast using a variety of code samples, so yes it's a mess right now, but a few days of cleaning would probably straighten it right up...
As I mentioned before though the only weak point right now is that the master controller connects to the slaves, whereas it may be better to make it the other way around, although this may increase network traffic (because the slaves would have to poll the tracker iteratively to see if a master has connected to the tracker yet.) Come to think of it though having it this way allows multiple masters, but I currently only allow one indigo process on the slaves (they will use auto-threads then anyway)
I was wondering about distributed internet rendering, but as far as internet rendering goes I don't know if such a thing is even possible with Indigo in an efficient manner..
-niko
I think regardless of whether your master controller connects to slaves, or the slaves connect to the tracker, there's always going to be an issue with having a correctly configured firewall.
Probably having the slaves contact the tracker will be more efficient in terms of configuration because you then have only one firewall to configure (the one on the tracker), rather than having to check every slave.
Also, I'm totally ignorant of how to implement such a thing, but could the tracker configure it's own firewall through UPnP ?
It seems that as long as the tracker can accept connections OK, then the whole system will work?
What does the tracker do exactly?
I imagine that the tracker ought to have queue management too and set the halt_sec or halt_samples on the slaves for each job.
Sorry for the flood of ideas - I love designing systems.
Probably having the slaves contact the tracker will be more efficient in terms of configuration because you then have only one firewall to configure (the one on the tracker), rather than having to check every slave.
Also, I'm totally ignorant of how to implement such a thing, but could the tracker configure it's own firewall through UPnP ?
It seems that as long as the tracker can accept connections OK, then the whole system will work?
What does the tracker do exactly?
I imagine that the tracker ought to have queue management too and set the halt_sec or halt_samples on the slaves for each job.
Sorry for the flood of ideas - I love designing systems.
-
nikolatesla20
- Posts: 52
- Joined: Fri Mar 09, 2007 4:44 am
Thanks Ono, I didn't know about the pack feature. Does the packing work though, during rendering frames that are uploaded to the indigo network master? Because if I render a 1200*1600 pixel image (or so) it's like 21megs of data, so its a lot of data regardless of scene transfer, so like I said I don't know about ability to do internet rendering.
dougal: Very good ideas and comments. The tracker is only there to allow the slaves to "register" their presence, that way the master can connect with the tracker to get a list of slaves. Once it has the list the master connects directly to the slave thru port 5200. Of course we can make it so this port is configurable.
The main reason I thought to use a "tracker" mechanism instead of having the slaves connect directly to the master program is twofold:
1. It allows the tracker to reside on a stable system that won't be moving much, i.e., I can run the master program on my laptop without worrying about IP resolution problems.
2. It makes the code simpler, because the tracker only contains a list of current available slaves, so the master does not have to keep track of that. If a slave drops out, the tracker automatically removes it from the list. Also, if the tracker drops out, each slave will try to auto-reconnect with it every 25 seconds. Once the connection is made, it remains open.
The connection from master to slave is basically all one-shot sockets, they open , do the job of sending data, and then close. They don't stay open. But they are still all TCP.
When you click "render" it tells all the slaves to render at once (currently). The same with cancelling the render. I also do not have any code to handle multiple network renders, in other words, when you upload a scene, it will overwrite the current one. I figured this is OK because for network rendering you really want to upload the scene each time because chances are high that you have made changes anyway since last time.
-niko
dougal: Very good ideas and comments. The tracker is only there to allow the slaves to "register" their presence, that way the master can connect with the tracker to get a list of slaves. Once it has the list the master connects directly to the slave thru port 5200. Of course we can make it so this port is configurable.
The main reason I thought to use a "tracker" mechanism instead of having the slaves connect directly to the master program is twofold:
1. It allows the tracker to reside on a stable system that won't be moving much, i.e., I can run the master program on my laptop without worrying about IP resolution problems.
2. It makes the code simpler, because the tracker only contains a list of current available slaves, so the master does not have to keep track of that. If a slave drops out, the tracker automatically removes it from the list. Also, if the tracker drops out, each slave will try to auto-reconnect with it every 25 seconds. Once the connection is made, it remains open.
The connection from master to slave is basically all one-shot sockets, they open , do the job of sending data, and then close. They don't stay open. But they are still all TCP.
When you click "render" it tells all the slaves to render at once (currently). The same with cancelling the render. I also do not have any code to handle multiple network renders, in other words, when you upload a scene, it will overwrite the current one. I figured this is OK because for network rendering you really want to upload the scene each time because chances are high that you have made changes anyway since last time.
-niko
Wow, my feature request got killed in the forum downtime but now only 1 day later it is reality! 
Well anyway here were my ideas:
- distribute the scene file (especially helpful for large scenes) via a P2P protocol like BitTorrent (I'm sure there is something like LibTorrent for Windows available on the net)
- Maybe take a look at http://burp.boinc.dk/ <-- a similar BOINC project for Indigo would be THE feature to even win over Maxwell and nearly any other commercial renderer! Noone's got a decent renderfarm like that at home... I'm sure you can talk to Janus (Mastermind of BURP) about this, he wants to release the code anyway.
- For this program (besides using Torrents, which might take a while to implement):
* Install as a service, so you can start your program in the background where it tries to connect to a server periodically (like Indigo in Slave mode does right now) even from a restricted account.
* Make sure the slaves get the paths in their files right, Indigo on C:\Indigo on one computer and on D:\Program Files\Indigo can be a real problem! You might have to change the paths after unzipping on the client. Maybe add a notification for the client where Indigo is installed on the master, so it can search for those paths and replace them with the correct ones for the slave computer.
* Internet rendering options: Maybe allow to set a host name (like "myserver.dyndns.com") to connect to on slaves additional to an IP adress. Send image updates only after XXXXX Samples per Pixel, not every X Seconds and compress them really hard.
* Maybe queue options for animations/several renders
Great program! As far as I understand C# also works for our Linux guys...
Well anyway here were my ideas:
- distribute the scene file (especially helpful for large scenes) via a P2P protocol like BitTorrent (I'm sure there is something like LibTorrent for Windows available on the net)
- Maybe take a look at http://burp.boinc.dk/ <-- a similar BOINC project for Indigo would be THE feature to even win over Maxwell and nearly any other commercial renderer! Noone's got a decent renderfarm like that at home... I'm sure you can talk to Janus (Mastermind of BURP) about this, he wants to release the code anyway.
- For this program (besides using Torrents, which might take a while to implement):
* Install as a service, so you can start your program in the background where it tries to connect to a server periodically (like Indigo in Slave mode does right now) even from a restricted account.
* Make sure the slaves get the paths in their files right, Indigo on C:\Indigo on one computer and on D:\Program Files\Indigo can be a real problem! You might have to change the paths after unzipping on the client. Maybe add a notification for the client where Indigo is installed on the master, so it can search for those paths and replace them with the correct ones for the slave computer.
* Internet rendering options: Maybe allow to set a host name (like "myserver.dyndns.com") to connect to on slaves additional to an IP adress. Send image updates only after XXXXX Samples per Pixel, not every X Seconds and compress them really hard.
* Maybe queue options for animations/several renders
Great program! As far as I understand C# also works for our Linux guys...
-
nikolatesla20
- Posts: 52
- Joined: Fri Mar 09, 2007 4:44 am
Ono:
The question I was wondering about is that even though we can pack the scene data and upload it to the client, during actual rendering, does Indigo pack the image data before sending it to the network master? Because during a master / slave render with an image of 1200*1600 or so, Indigo master reports that it is getting a frame that is 21 megs in size. So even though we can optimize by packing the scene, it doesn't really give an advantage if during network rendering, large amounts of data are transferred anyway. (The image data that is transferred from slave to master)
-niko
The question I was wondering about is that even though we can pack the scene data and upload it to the client, during actual rendering, does Indigo pack the image data before sending it to the network master? Because during a master / slave render with an image of 1200*1600 or so, Indigo master reports that it is getting a frame that is 21 megs in size. So even though we can optimize by packing the scene, it doesn't really give an advantage if during network rendering, large amounts of data are transferred anyway. (The image data that is transferred from slave to master)
-niko
Nikola:
The frame transfer is completely different from the scene transfer.
The scene transfer (scene may be packed) happens only once per slave from the master to the slave.
The frame (image buffer) transfer happens periodically, and is a transfer from the slave to the master.
The frame transfer does not use compression currently.
The frame transfer is completely different from the scene transfer.
The scene transfer (scene may be packed) happens only once per slave from the master to the slave.
The frame (image buffer) transfer happens periodically, and is a transfer from the slave to the master.
The frame transfer does not use compression currently.
Who is online
Users browsing this forum: Majestic-12 [Bot] and 71 guests