Indigo Binary Mesh (.igmesh) format with source code.

A forum for exporter development discussion.
Deus
Posts: 336
Joined: Sun Feb 04, 2007 3:47 am

Post by Deus » Sat Sep 06, 2008 12:13 pm

Suvakas my friend. a binary char takes 1 byte the corresponding char takes 3.333 bytes in a "text" file. So in practice XML is about 5:1 loss in data size.

User avatar
suvakas
3rd Place Winner
Posts: 2613
Joined: Mon Sep 04, 2006 11:08 pm
Location: Estonia
Contact:

Post by suvakas » Sat Sep 06, 2008 8:19 pm

Gotcha.
Thanks. Yes, I get the point now.
Sounds like a good deal 8)

Deus
Posts: 336
Joined: Sun Feb 04, 2007 3:47 am

Post by Deus » Sun Sep 07, 2008 3:27 pm

Damn I cant do simple multiplication. It should be 2.4 bytes. The approximate formula to calculate this for any number of bits is nbits*0.3 ex 8*0.3 = 2.4 bytes for a byte 4.8 bytes for a 16 bit number etc.

XML tags, line feeds, whitespace etc also adds overhead obviously.

Its dependant on the XML structure and other things but its rougly 5:1 overhead in the general case. The normal way of "fixing" this architecturally in a modern file format while still retaining size and the strength and flexibility of XML is to use zlib or similar to compress the XML stream. Good examples of this in practice is JARs, XPS (Vistas pdf "replacement") and Indigo files. It requires CAREFUL implementation and reading speed and size will still suffer a bit but not as bad as 5:1 size wise.

It's important to realize is that XML is really strong because

A) It can be manually typed in an editor
B) Its easily extended
C) Underlying architecture is easily understood by just reading documents written in the specification and understanding = engineering trust and engineering trust = something that gets used i.e. xml makes people want to use the product or platform

However. For "trivial" things that is well known prior art and that is highly unlikely to change over time XML is NOT the best choice. Examples of this is bitmaps, masses of raw data (audio, geometry)

Onos choice to go for a binary format here is a good choice and I hope I gave some of you a bit more insight into software architecture and data formats.

Im starting to babble here ;) It's important stuff though.

/ D

User avatar
fused
Developer
Posts: 3648
Joined: Fri Sep 22, 2006 7:19 am
Location: Berlin, Germany
3D Software: Cinema 4D

Post by fused » Sun Sep 07, 2008 9:41 pm

Deus wrote:Onos choice to go for a binary format here is a good choice and I hope I gave some of you a bit more insight into software architecture and data formats.
right. my words. :)

i implemented it into cindigo and the results are quite impressive:

Code: Select all

Exported mesh: Buddah, 1085617 tris, 548494 vertices, 543717 uvs. Processing time: 3.125s. Total export time: 3.502s.
igmesh size: 45.6MB
heres another one:

Code: Select all

Exported mesh: Landscape, 3992000 tris, 1996002 vertices, 2000997 uvs. Processing time: 7.333s. Total export time: 8.698s.
igmesh size: 167MB
it writes 4m tris in 1.365 seconds. thats impressive.

will export those to embedded_2 and post the results, too. for coparison.

User avatar
fused
Developer
Posts: 3648
Joined: Fri Sep 22, 2006 7:19 am
Location: Berlin, Germany
3D Software: Cinema 4D

Post by fused » Sun Sep 07, 2008 9:53 pm

results embedded_2:

Code: Select all

Exported mesh: Buddah, 1085617 tris, 548494 vertices, 543717 uvs. Total export time: 21.78s.
igs size(mesh element only): 153MB

Code: Select all

Exported mesh: Landscape, 3992000 tris, 1996002 vertices, 2000997 uvs. Total export time: 60.27s.
igs size(mesh element only): 429MB

User avatar
OnoSendai
Developer
Posts: 6241
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Post by OnoSendai » Sun Sep 07, 2008 10:12 pm

Thanks for posting some results, fused.

User avatar
pixie
Indigo 100
Posts: 2332
Joined: Sat Dec 29, 2007 4:54 am
Location: Away from paradise
3D Software: Cinema 4D
Contact:

Post by pixie » Mon Sep 08, 2008 1:04 am

Fused is the man! :D

User avatar
pixie
Indigo 100
Posts: 2332
Joined: Sat Dec 29, 2007 4:54 am
Location: Away from paradise
3D Software: Cinema 4D
Contact:

Post by pixie » Mon Sep 08, 2008 1:06 am

@ Deus:
I think that models are one of those things that binary makes sense, the scene however makes sense to be in xml, still as there's includes there's already a match made in heaven for those two :)

Deus
Posts: 336
Joined: Sun Feb 04, 2007 3:47 am

Post by Deus » Mon Sep 08, 2008 4:54 am

Ono can you please supply Vec2,Vec3 code too so I can integrate directly into indigoworld? :)

User avatar
OnoSendai
Developer
Posts: 6241
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Post by OnoSendai » Mon Sep 08, 2008 3:30 pm

Deus wrote:Ono can you please supply Vec2,Vec3 code too so I can integrate directly into indigoworld? :)
Nope, just write some conversion functions.

User avatar
Whaat
Developer
Posts: 1827
Joined: Fri Dec 22, 2006 6:15 am
Location: Canada
Contact:

Post by Whaat » Mon Sep 08, 2008 4:41 pm

This may be a stupid question, but I'm tired. :) Can this binary format be created using Ruby code or do I have to use C++? Is there a way to create the mesh files using Ruby? Forgive me...I only briefly looked at the C++ code.

User avatar
OnoSendai
Developer
Posts: 6241
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Post by OnoSendai » Mon Sep 08, 2008 7:34 pm

Yes, you should be able to write the format from ruby just fine.

User avatar
Whaat
Developer
Posts: 1827
Joined: Fri Dec 22, 2006 6:15 am
Location: Canada
Contact:

Post by Whaat » Tue Sep 09, 2008 2:49 pm

OnoSendai wrote:Yes, you should be able to write the format from ruby just fine.
good to know....too bad I don't have a clue how to do it. :(

User avatar
OnoSendai
Developer
Posts: 6241
Joined: Sat May 20, 2006 6:16 pm
Location: Wellington, NZ
Contact:

Post by OnoSendai » Tue Sep 09, 2008 5:35 pm

Whaat wrote:
OnoSendai wrote:Yes, you should be able to write the format from ruby just fine.
good to know....too bad I don't have a clue how to do it. :(
Try array.pack

User avatar
Whaat
Developer
Posts: 1827
Joined: Fri Dec 22, 2006 6:15 am
Location: Canada
Contact:

Post by Whaat » Wed Sep 10, 2008 3:54 am

ok. That helps a bit. Do I need to create an IndigoMesh class in Ruby that is identical to the C++ class? And then, do I have to figure out how to write the information contained in the IndigoMesh object to a file using the same template as the one that you have provided? :?

Post Reply
66 posts

Who is online

Users browsing this forum: No registered users and 6 guests