best machine to renderer?

Jambert · Post by **Jambert** » Sun Jan 11, 2009 5:20 am

ZomB

Yes but a q6600 is enough to make test with some indigo experience... I have a professionnal activity and to me doing test on a render farm is realy much expensive, I'll never do it... but I understand what you mean, it's much confortable to buy is own render farm, 10000$ sounds expensive to me

Sukrim · Post by **Sukrim** » Wed Jan 14, 2009 1:18 am

Example:
For each node:
PSU:
http://geizhals.at/eu/a378175.html 300 Watts is far too much already... ~30€
Mainboard:
A good µATX AM2+ Board with 16GB DDR2 RAM support + integrated graphics ~60€
RAM:
4*2GB DDR2-800 @10-15€/GB ~80€ or more
Storage:
http://geizhals.at/eu/a281545.html silent, fast, big enough for Windows/Linux + a few big scenes/renders ~55€
CPU:
Dual/Quad/Triplecore? Depends mostly on your needs I guess - if you take more cores on one CPU it's going to be more expensive, on the other hand you save money for RAM/PSU/Mainboard/Storage. I'd say since it's the most important part of your Renderfarm you invest a bit, let's say ~150€ for a medium clocked Phenom Quadcore or even 200€ for a Phenom II

-->~225€ + CPU (let's say 250€ + CPU so you can have a better PSU or RAM for example) + Cooling stuff + cables --> 500€ max per node.

8 Nodes + Network stuff (Gbit Switch, NAS/Fileserver?...) should be around 10k$ and you have 32 CPUs @ around 2,5GHz or a bit more which would mean you have a ~60 GHz machine with 8GB RAM on each node and a combined power consumption of about 2 Kilowatts or less, silent + fast SSDs and maybe upgradable to some cheap AM2 PhenomII CPUs in 1 year as well if needed.

This is calculated rather conservatively and without OS - so if you use Linux or similar stuff you can get even more powerful hardware, on the other hand it would be maybe even a little less powerful if you have to buy licenses + stuff for each node as well.

How would you other guys on the forum build a renderfarm with 10k$? Different? Intels? Multiprocessor? Come on, show us your dreams!

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 1:42 am

If I were building simple dumb render slaves I would not give them any HDD storage at all, I'd use that budget to get a better processor. Intel quads for me.

I'd set up a fileserver system to support network boot so that all slaves boot identically - also better for making system updates to the slaves - you only have to change the OS image on the fileserver.
(I actually would like to build a network boot system one day, just for fun).

Seeing also that each slave node only needs power for mainboard, ethernet, CPU and RAM. I might fork out for slightly more powerful PSUs (400W ?) and run two slaves on a single PSU. That (if possible) would probably save a bit of expense and energy.

I would use Linux, obviously... this would also allow easier scripting for renderfarm management etc. I would also set it up such that I would never have to log in directly to any slave node (because otherwise you'd have to log into ALL slaves nodes which could be time consuming) - they just boot and run.

If only I had the time, money and need for such a system, this would be a fun thing to try

suvakas · Post by **suvakas** » Wed Jan 14, 2009 1:50 am

dougal2 wrote:I'd set up a fileserver system to support network boot so that all slaves boot identically - also better for making system updates to the slaves - you only have to change the OS image on the fileserver.
(I actually would like to build a network boot system one day, just for fun).

Do you know any link to a resource that explains how to do it ?

Post by **fused** » Wed Jan 14, 2009 1:55 am

suvakas wrote:
dougal2 wrote:I'd set up a fileserver system to support network boot so that all slaves boot identically - also better for making system updates to the slaves - you only have to change the OS image on the fileserver.
(I actually would like to build a network boot system one day, just for fun).
Do you know any link to a resource that explains how to do it ?

interested, too.

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 3:30 am

No details - i haven't attempted it yet!

But google for 'PXE boot linux' gives a starting point.

this looks like a good result:
http://www.kegel.com/linux/pxe.html

I know that for example the Debian installer can be booted over network.

Sukrim · Post by **Sukrim** » Wed Jan 14, 2009 5:16 am

dougal2 wrote:If I were building simple dumb render slaves I would not give them any HDD storage at all, I'd use that budget to get a better processor. Intel quads for me.

Well, you could also use jsut small CompactFlash cards or cheap USB-Sticks to boot the nodes, I was just thinking that maybe several renders starting + reading files over network multiplied by eight would be quite stressful for a NAS/Fileserver. But I see your point here.

dougal2 wrote:I'd set up a fileserver system to support network boot so that all slaves boot identically - also better for making system updates to the slaves - you only have to change the OS image on the fileserver.
(I actually would like to build a network boot system one day, just for fun).

Me too, but since I'd rather boot Windows and also have seperate clients WOL would already do.

dougal2 wrote:Seeing also that each slave node only needs power for mainboard, ethernet, CPU and RAM. I might fork out for slightly more powerful PSUs (400W ?) and run two slaves on a single PSU. That (if possible) would probably save a bit of expense and energy.

I don't know of any PSU that has 2 ATX-Connectors and if there is no easy solution (like adapters for Molex-connectors) I doubt that this could be cheaper than 60€. Interesting approach though!

dougal2 wrote:I would use Linux, obviously... this would also allow easier scripting for renderfarm management etc. I would also set it up such that I would never have to log in directly to any slave node (because otherwise you'd have to log into ALL slaves nodes which could be time consuming) - they just boot and run.

Of course, but this could be done with Windows and custom scripts etc. as well since especially Indigo's network rendering stuff is more "let's use that other computer too!" oriented than really suitable for using it in a render farm.

On network booting http://etherboot.org/wiki/index.php would be a good place to start, also the TechTalk is rather interesting!

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 5:33 am

Etherboot (and derivatives) seem to be GPL replacements for network board ROMs - most modern network interfaces are already compatible with PXE specifications.

I just tried these instructions on my machine here (debian lenny):
http://www.debian-administration.org/articles/478

Seems easy to set up the boot server, but I couldn't get a client machine to get a network address at boot time. If I let the machine boot from it's HD into XP, it did get an address from this machine, but by that time it's too late. Might be a quirk with the network ROM in that machine - or it might be why Etherboot is necessary. I don't know enough about it yet to know why the machine did not PXE boot.

I haven't any other spare machines here to try it with either

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 5:40 am

Sukrim wrote:
I don't know of any PSU that has 2 ATX-Connectors and if there is no easy solution (like adapters for Molex-connectors)

I would get right in there with a pair of wire cutters and soldering iron!

EDIT: It's been done and you can buy ATX splitters apparently!
http://www.overclock.net/faqs/112866-ho ... power.html

Sukrim · Post by **Sukrim** » Wed Jan 14, 2009 7:44 am

Hm, sounds interesting - at current hardware prices a setup like this might even be cheaper than a dual-socket approach! :O

Anyways: Someone here who gets more GHz than approx. 60 (conservatively calculated of course) out of 10k$ than me?

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 9:07 am

My pricing for the q6600 system i specced in that other thread works out at 86.8 USD/GHz (12 machines for 10k$) - and that includes buying cases, monitors, keyboards, mice.

If you omit the cost of monitor (biggest cost), XP licenses, keyboards and cases - that comes down to about 52 USD/GHz (15 machines for 10k$)

sukrim, your figure is 166.66667 USD/Ghz - that seems quite high ?!

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 9:13 am

edit: I can add/multiply but apparently I can't read

If you overclock the q6600's to 3.2GHz you get 39 USD/GHz

dougal2 · Post by **dougal2** » Wed Jan 14, 2009 10:00 am

I decided to add things up properly, each node works out cheaper that I expected. These specs don't include any kind of racking/casing, so not a very practical renderfarm. However I did budget in a little bit for network switch/cables etc:

EDIT: better spreadsheet

Zom-B · Post by **Zom-B** » Wed Jan 14, 2009 4:26 pm

Very interesting math dougal...

I would really go the shared PSU way!!! Saves money + energy + space + heat + noise

Also I would invest into Phenom II on AM3 Boards! you wouldn't have this much bang for your money atm. But in 2010/2011 you can upgrade your renderfarm to some 6/8/16/whatever core AM3 compatible CPU, while your Intel based system is by definition not upgradeable

fused wrote:
suvakas wrote:
dougal2 wrote:I'd set up a fileserver system to support network boot so that all slaves boot identically - also better for making system updates to the slaves - you only have to change the OS image on the fileserver.
(I actually would like to build a network boot system one day, just for fun).
Do you know any link to a resource that explains how to do it ?
interested, too.

link

ave · Post by **ave** » Wed Jan 14, 2009 11:42 pm

Guys, I am really impressed by the numbers and quality of the answers... thanks a lot!

I will start to consider the various options together with our technician and see what comes out...
BTW I think ZomB got it right about the renderfarm option: I think it would be unpratical for daily work. Also, since the company will deal with light design, we want to be fast in trying different solutions as soon as we design them.

best machine to renderer?

wow!

Who is online