[REQ] Bucket Render Mode for network rendering

[Zom-B]

Rendering big images with high SS and some light layers generates super big IGI files.
Network rendering is based on the idea to render such a IGI for each slave and transfer it as-it-is to the Master,
who combines the IGI files together.

This approach has some drawbacks:
- Big IGI files are even for LAN based networks quite time consuming to transfer
- Internet based Networks are impossible because of to high transfertime
- Master needs to do a lot of work to merge big IGI files
- Renderfarms with Indigo need realy long to merge final images
- Each Slave needs high amount of RAM to keep IGI in memory



My idea here is simply divide the rendered image into buckets, and do a some kind of region rendering for each slave.
Each bucket has some own SPP value, and Slaves render allways the lowest Bucket after uploading data.
Since the whole scene is in RAM, changin a bucket for the slave should be ultra fast.

The Benefits (example for a 4x4 bucked grid and a 512MB IGI)
- a region rendered IGI can be zipped (fastest algo) to compress all of the black area to near Zero KB so a bucket igi would only consume 32MB!!!
(anyway a method to truly generate a IGI in size of bucket is optimal to reduce RAM usage for holding black area in memory!!)
- reduced IGI size also results in reduced RAM usage for the slave, so even computers with low RAM could help rendering ultra big images!
- transferring such a bucket IGI is even possible for Internet based render networks!
- merging of Data would be way faster for the master, also renderfarms would benefit very much from this!
- The buckets could be changed on the fly (spp for each bucket would need recalculation)
- A simple grid based region rendering could be made. A Grid overlay with info for each bucket (spp & Slave working on it), would be the base to determined manually which slave should give additional computation to a specific hard to compute bucket (SSS Object or Caustics for example). Also perfectly rendered buckets could be canceled out from this roulette (like Sky)
- This could be also applied for light Layers

Drawbacks
- development time
- in the beginning the render would resolve bucket wise, and no good preview of the scene could be possible
To work around this the master should fist render the whole image for X spp to get a base preview.


What do you guys think about this (optional) method?

[StompinTom]

I would just be worried about differing samples/px between all the slaves and how that could be equalized, but otherwise that sounds good!

[PureSpider]

I love the idea!
Implement please!

[Zom-B]

I would just be worried about differing samples/px between all the slaves and how that could be equalized, but otherwise that sounds good!

maybe my explanation wasn't that good. I try again:

imagine a simple 3x3 grid with 9 buckets rendered on 1 master and 2 slaves.
Master starts rendering the whole image for (lets say) 50 spp until joining in Bucket rendering (to allow a basic overview of the image).
The Slaves grab the first and second bucket, rendering each bucket until the bucket (not the whole image!)
reached (lets say) 200spp, then the slave uploades this data and get info from master at the same time which bucket has the fewest spp. The slave starts rendering this bucket until 200 spp are reached....

This game continues until each bucket has 200spp, and starts over again to (lets say) 400spp.

I hope this clears things up