New exporter

[zurkdahool]

Hi,

i wrote already i am trying to do a new (mesh)-exporter as a python extension in C++ for blender. Because this is a complete new approach i am starting this new topic instead to post my progress at the 'blendigo topic'...

Until now there is not too much happened: I am still doing some software architecture stuff and figure out the necessary APIs (blender) and libs (e.g. boost) to deal with. My development- and independent build-system are ready and the git repository is working.

At the moment i am fighting with the blender source and the boost libs. Both of them are beasts and together they become a huge dragon I have done also a first POC to see how things are working and how i can access blender objects via c++... well... the main concept is working: I've tested it with one object (~8m. verts, ~8 m. faces and ~16millions tris):

Code: Select all

found bundled python: /home/zurkdahool/blender-2.76-linux-glibc211-x86_64/2.76/python
[IndigoRender 2015-Oct-12 16:27:53] INFO: Attempting to import indigopython module 
[IndigoRender 2015-Oct-12 16:27:53] INFO: indigopython: Version 0.1 pre-alpha imported successfully.
[indigopython.so] Starting export mesh: Suzanne
[indigopython.so] Vertex to parse:8063618
[indigopython.so] Finished.
[indigopython.so] Time elapsed = 2874632 microsec.

Well ... this is without writing to disk and the other stuff (xml, material ect.) but i think the exporter will become faster in some way or another...

Some words about the OS compatibility: The exporter module should work on all platforms in the future (OSX, Linux and Windows) and i always take care of compatibility. Maybe in the future i need some help with MacOSX (compile/test) because at the moment i do not have access to Mac OSX.

There is much to do and some tasks requiring great diligence ... hopefully i have some good news in some weeks. stay tuned!

[Oscar J]

This is awesome.

[h2o2]

I agree this is great !

Can't wait to see where you will end up with this, keep us informed about your progress, i know i'll keep more than an eye on this.

[bacon]

Isn't there a medal of some sort we could give this guy (zurkdahool) ?
He definitely earned one, taking on this task

Looks like better days are coming...

[Headroom]

This is great! I agree and have mentioned it already in another thread. We need some form of reward for people that go above and beyond!


Sent from my iPad using Tapatalk

[zurkdahool]

progress update 1:

"Dear little diary...."
the last week was full of surprises. Much surprises coming from boost::python and my rusty c++ coding experiences and i am also was struggling with igmesh format version 3 a little bit and how to pass all the data in 'a good' way to 'the extension' from within blender via python. I am still figure out components, possible ways and do only some rough coding. At the moment i become closer to export my first mesh (igmesh format 3) from within blender via my own c++/python/boost extension . It is a bit clumsy (the bottleneck is not the harddisk!) ... i have to do a lot (a lot ...a lot.... alot ....)of tuning for the first alpha and have to check the original blendigo exporting times to get a feeling if it's worth to write such a extension (instead of refactoring blendigo)..

First pre-alpha testing (w/wo write on SSD and without tris/quads):

Code: Select all

[IndigoRender 2015-Oct-15 20:51:45] INFO: Attempting to import indigopython module
[IndigoRender 2015-Oct-15 20:51:45] INFO: indigopython: Version 0.1 pre-alpha imported successfully.
############################## start indigopython export ########################################
MaterialStringBytes 0 <using model2>
#################### write 15685382 verts ###########################
#################### write 15685382 normals ###########################
#################### write 0 triangles ###########################
#################### write 0 quads ###########################
[indigopython.so] Time elapsed = 24075079 microsec.

~15 mio. verts/normals result in 360MB on ssd:

Code: Select all

 ls -lh /tmp/alphadebug.igmesh
-rw-r--r-- 1 zurk zurk 360M Okt 15 20:54 /tmp/alphadebug.igmesh

without write to harddisk:

Code: Select all

...
[indigopython.so] Time elapsed = 23349062 microsec.

"a second difference?" ...lessons learned

Also a really big thanks to indigo for exception handling:

Code: Select all

...
Blender quit
############################## start indigo ########### #############################
Executing '/home/zurk/testscene.igs -h 12 -image_save_period 4 -o /tmp/debug -seed 0 -t 8'
Indigo Renderer v3.8.26, Linux 64-bit build.
Licence verified, licence type: Indigo 3.x Full, licensed to 'xxx@xxxx.xxx <xxxx xxxx>'
Scene file path: '/home/zurk/testscene.igs'
WARNING: Mesh 'alpha' had zero triangles/quads, ignoring it.
Num buffer layers: 2

i like software who is ignoring 'me' when something is wrong: WARNING: Mesh 'alpha' had zero triangles/quads, ignoring it.

stay tuned for next update... (more coding means less writing to forum) !


EDIT:
try to export the mesh with blendigo 3:

Code: Select all

....
Info: [Indigo 2015-Oct-15 23:13:00] Mesh Export took: 109.152885 s
....

hopefully 'we can beat them, just for one day' or ... 'for ever and ever'

[zurkdahool]

first full export seems to be working :

~11 Mio. Vertices / ~ 11,6 Mio Quads / Export Time: ~39,5sec (instead of 55 secs with blendigo 3) / File size ~668MB

enough for this week ...

[Headroom]

This looks very promising!
Not only for what you've already achieved in terms of the export speed, but in general how quickly you've done it. Thank you very much for your effort!


Sent from my iPad using Tapatalk

[zurkdahool]

the last week i was thinking about the whole 'UI python scripting' stuff and how to avoid using the 'extension framework' python module which is used by blendigo a lot. i don't want to use the framework because it is (afaik) not well maintained at the moment and there is no good documentation available which makes it hard to extend it.

i decide to code the layout, properties, types ect. in a very clean, easy and basic way without 'fancy class generation code'. this may result in some more code but makes it easy to implement new features in the future - for everyone -
the basic logic is finished and i started with the boring work: coding all needed buttons,properties, list and so
first tests looks good and i still figure out all possible features (blender API):

it is a comparsion between blendigo and the new exporter. Remember: it is just a first impression!

I also checked some other methods of the exporter modul. It seems that the boost::python C++ libs are not really a friend (speed and compiling on other platforms). At the moment i am testing Cython and i looks much better. Anyway, because of the modular concept it doesn't matter which solution i will use at the end

next update will take some time because of holidays.... stay tuned!

[Oscar J]

That looks reeeally slick man!

Couple of things:

I don't think BIH Tri Threshold should be in the filter settings.

Advanced filtering settings should probably be hidden by default, with the old "crisp", "smooth" etc settings visible, but with a "Custom" checkbox visible. The supersampling value should always be visible though, because that can hardly be considered an advanced setting.

Same with advanced rendering settings - that was recently added by blendigo users, and it's a genius move.

Also really curious what those different Clay buttons do?

[zurkdahool]

Thank you for the input Oscar The screenshot is just a 'proof of concept' and was done by me to see if the code is working. The whole layout and where to put what is not yet started The 'clay buttons' was also a test to put elements into existing blender ui parts. Maybe in the future i will implement the buttons to quick 'override' all materials (normal materials/ emit+normal / emit+normal+spec material)....
On the screenshot there is only one 'real' feature which is comming into final version: the 'preset'-subsystem to save/load individual render settings....

[zurkdahool]

uhhh... roundabout three weeks no news?! here are some signs of life!
I am back from holidays and a lot of work is on my desktop... i still coding on the new exporter and do some UI stuff at the moment but at the moment i do not have too much free time for coding. hopefully i have some more news in one or two weeks. stay tuned!

[Oscar J]

Yay!

[Polinalkrimizei]

I'm not staying tuned, I'm excited!!

[Oscar J]

We miss you, zurk