Cycles vs Mitsuba vs Luxrender – round 2

Edit: here is newer version of this article.

Update: look at post of helder Cunha who actually took some time to explain how correctly setup cycles to get clean result with non-progressive integrator in Cycles. Update2: I finally had some time to test Cunha settings (see update above with link to his post). I adjusted Cycles  with settings provided by Cunha and I just set total AA samples to 11 so render time was close to 12 minutes, for easier comparison with other render.  Most of images in this post were updated with this  Cycles setup. Cycles settings were:

  • nonprogressive integrator with 11 AA samples and 8 diffuse,  4 glossy, 26 transmission samples. AO, SSS and Emision were  set to 1 sample.
  • MIS for HDR envrionament light was ON with 16 samples, and resolution of 128.

End of Updates In 2011 I did free render engines comparison – NOX, Luxrender, Cycles, Mitsuba, Yafaray. Cycles was the slowest  and Mitsuba fastest. Did anything changed in past 2 years? Right now I did 3 test Cycles, Mitsuba 0.4.3, Luxrenderer.  All engines were setup with same material and lightning (HDR image).  Render setup was adjusted for each renderer based on feedback I got from forums of comments belove. (update) For Luxrender i used Metropolis sampler, and paused it at 13min so I’m not sure how many samples there were(/update).  Here are results with similar render times (see bottom of post my my rig specs):

Cycles vs Lux vs Mitsuba

Cycles vs Lux vs Mitsuba

Render settings for Luxrender Cycles and Mitsuba

Render settings for Luxrender Cycles and Mitsuba

Difference between Cycles and Mitsuba in not that big. It is  big step forward compared to year 2011, where cycles gave more noisy renders (back then it was actually worse than Luxrender). But then how much longer will it take to clean up Cycles render? I bumped samples AA samples from 11 to around 24 and result seems comparable to Mitsuba, but render time increased to 27 minutes.

Cycles vs Mitsuba

Cycles vs Mitsuba

About Luxrender, I’m not sure If my setup was correct but it seems slowest – (update) even with Metropolis sampler and QBVH, as some people were suggesting, render time did not improved much. About BVH: SQBVH build time was something like 9 sec, QBVH -around 3 seconds, so in the end they take small % of overall time. Screen with example of material setup for wheels: Material Stup for MgO In Mitsuba to setup metal material I just put chemical symbol – MgO (Magnesium Oxide), and it’s roughness – 0.08 Luxrender support metal nk data too, however loading MgO.nk file caused error, so I set it up with brown color metal. Pylux didn’t worked for material preview – maybe because I was using latest unstable Blender build. Cycles has material setup was similar to Lux – brown glossy material, with 0.08 roughness. Summary:

  • ease of use – for me Mitsuba wins – all I have to do is increase number of samples to get rid on noise. Lux is second. Cycles seems most complex see post of Cunha here. I spent (wasted from my point of view) most time adjusting it
  • render speed  1. Mitsuba;  2. Cycles 3. Lux.  Not sure why lux is so slow, but it is supposed work much better in more complex scenarios. But then Photon Mapping in Mitsuba is very, very fast too – and default settings are ok in most of cases – so it is one click setup render.
  • flexibility of  material setup: 1. Cycles – definitely winner – great integration with blender and nodes are fun to work with.  2. Lux – will support nodes too, and has lot of cool features (displacement, spectral rendering, realistic sun&sky model – same as mitsuba actually; cycles sun&sky is really unrealistic/outdated). 3. Mistuba – no nodes support for now, not as flexible, but easy to use.
  • fun  and workflow speed – definitely Cycles wins with interactive render preview, scene setup is quick and fun. Lux will have this too.   Someone want add support for realtime render preview in Mitsuba? 😉 It is to complex programming for me… For now scene setup in Mutsuba is slow and not fun, because of constant need to render scene preview manually.
  • animation – cycles definitely is the best enough said; lux – seems to be to slow for that (maybe with direct lightning you could render animation in reasonable render time, and there is big hope for quick renders with GPU openCL). Mitsuba would be great for animaion, but exporter is not optimized for that. For each frame it exports whole scene data, so it is not usable for heavy scenes. It could be improved with better exporter  and use of Mitsuba python libraries, but  it is above my programming skills.

Other stuff: And bunch of cycles tests -progressive vs non-progressive sampler, HDR MIS ON and OFF. Lexus_CS_2054-CyclesOnly PS Car model – Lexus CS 2054 form Minority Report – is not mine. You can download it form here. My setup is – AMD 3 core, 2,6GHz, 8GB of ram DDR 2. Is is very old, so on newest hardware, you could probably expect half the time to render those images.

Advertisements

24 thoughts on “Cycles vs Mitsuba vs Luxrender – round 2

  1. Terrific comparison. But i would like to see some other test too like indoor lighting test.
    Good to see mitsuba making a cut here. Thats why i am focusing on it nowadays. I wish the exporter from blender improves and make it more usable.

  2. I Love Mitsuba. It is just something that will have you discovering new things. The tricky part is depth of field, it is hard to focus on an object and you have to have the specific distance of that object you want to focus on.
    But Mitsuba is the one.

  3. I think there is some information on blender guru or blender cookie but there are some improvements that can be made to the cycles rendering times like changing the tile size, running a script to remove all the ‘double sided’ objects, and I also think there is a clamp option setting which will remove a greater number of fire-flies.

    Would be nice to see the same comparison with the tune ups and not just with Cycles.

    regards

    TFS

  4. in LUX you shouldnt use path rendering except if rendering in hybrid mode.
    The default LUX rendering method is bidirectional + metropolis. Bidirectional is sometimes faster that the hybrid gpu assistend path tracing(in interiors). Maybe it would be correct to use that as it is the default render mode.

  5. Another pointless comparison. The progressive integrator in cycles does not compare to the non-progressive integrator as this one allows you to balance the kind and amount of rays that are shot into the scene, AA samples etc. I get cleaner, faster results from the non-progressive integrator than I do from the progressive one with a GPU.

    It’s about time you clear this up, it’s been three years since your original “comparison” and it has been influencing people’s opinion and render choices since (all over forums) and it’s basically an un-educated one.

    You’ve got to un-click that “progressive” button mate.

    • I you read my post, mate, you would see I compared progressive vs nonprogressive – last image. Progressive with MIS OFF was best so it contradicts your expectations.(and mine too actually). Maybe there were some optimizations now, but I do not have time to test cycles every month. I saw many posts saying: ‘If you enabled X you would get better time’ but in the end render quality did not changed much – I tested all possible settings, and here I posted the best ones.
      Anyway I you don’t believe me, I can send you blend file, so you can test it yourself.

  6. So, I had a look at your file and here are some points to take into consideration:

    1. World MIS sampling rate

    Unlike the progressive integrator, the non-progressive one will take all lights into account for each sample and can multi-sample them under your control, and this includes the World. When it is active a new option in the world settings pops up allowing you to choose the number of world samples per AA sample. Change it from 4 to 16 and just this will give you an enormous reduction in noise, specially diffuse noise. I’d also recommend going with 128 for the size of the map, it’s quite an uniform map you’re using anyway.

    2. Per-type ray samples

    Using the non-progressive integrator with the amount of samples for all ray types set to 1 will give you the advantage of point 1 just by itself but you’d be missing out on its biggest strength: it lets you establish an important balance between the amount of rays shot into the scene based on ray type. In your scene for example, transmission samples for the car windows and diffuse samples for the car interior are in dire need of an increase! But by what amount you ask?

    I set the target to be 32AA samples. To reach a balance I started by looking at the diffuse interior by hiding the windows and rendering a region while increasing the diffuse samples until it was mostly clean. After, I un-hid the windows and increased the transmission samples until most of the noise was gone. I did the same for specular, it required the least amount for good quality in key areas.

    After that I rendered the whole scene at 1/16 the resolution to get an estimation of how long a full render would take (x16) and adjusted all the ray samples proportionally until my estimation was around your 24mins for the 32AA samples I targeted. I used your time of 19mins for a 128 sample progressive render to estimate how much faster/slower my machine is by doing the exact same render with the same settings (MacBook White 2007 Core 2 Duo 2.2Ghz 2GB). So:

    for 32 AA Samples
    8 Diffuse, 4 Glossy, 26 Transmission, 1 for everything else
    MIS ON 128 Resolution 16 samples

    All those samples for transmission are quite needed to clear the windows of most noise and they don’t impact render time as much as it looks since these are about 1/10 of all image pixels. In your image getting clean diffuse is quite important, specially for the interiors and right side of the car. Most of the really visible reflection on the scene happens at fresnel angles and the map samples quite nicely so they take the smallest cut of render time. The wheels will survive 😉

    Also, I think i should point out that ray samples are cheaper than full AA samples.

    non-prog 32A 8-4-26 128-16 MacBook

    So to test this just the ray samples to these amounts, change the number of world samples to 16 and let it run on progressive refine mode for the 24mins.

    Notes:
    – If you’re planning on using visible DOF you might want to increase the number of AA samples. Using 24-32 looks good edge-wise for an all-in-focus image. A simple strategy to use more AA samples while keeping your quality and render time almost the same is to multiply the number of samples while dividing the amount of each ray type samples by the same factor. (a factor of two for example, from 32AA 7 4 28 to 64AA 4 2 14). Number wise, you’ll the taking the same number of total samples. Juggling act I’d call it.

    – The fastest render tile size for CPU is 16×16 and its has quite an impact on render time.
    http://www.blenderguru.com/4-easy-ways-to-speed-up-cycles/

    – Setting the minimum light bounces to the same as the maximum disables probabilistic path termination, forcing every bounce to happen and be summed to the result. Having it enabled does help with render time but it might give you a layer of noise on your image that you can’t easily get rid of, not even with more samples. I noticed it’s quite scene dependent, so for some scenes and for when there is a need for complete and utter cleanliness, I would turn this off.

    • Thanks for taking your time to explain how thing supposed to be set up. I doubted there could be as big render quality improvement as you said, because most of the time, when I tweaked things suggested by other people, render did not changed much. I guess this will be my last render engine test, and I will focus more on art 😉

  7. I agree with some comments regards the difficulty of making this kind of direct comparison.
    When I used to work in a related area of computer aided design some people advocated a test result and allowed experts in either system to achieve that result in the minimum time but of course this becomes heavily dependent on the expertise of each tester. Even someone who has used a particular system may show bias in their knowledge or ability to get the best out of that system – understand the strengths and weaknesses of different renders and then testing them fairly is probably an impossible task – that is not to say that people very familiar with two systems will not have a favorite or be justified but thats a considered thing over a long period of time and its still often subjective.

    I was interested in Cycles as a standalone, I already have used the others but personally I do not want to get involved with Blender as good as others say it is I have always ripped my hair out with the interface and that is despite buying books and also being a professional programmer in the 3D arena for over 15 years. Can cycles be downloaded and used without having to download blender or deal with blender?

    I am somewhat arcane here, I am very happy to use scene files and edit them by hand which I feel is often more productive for a programmer than GUIs I will usually write my own programs to generate scripts or xml scenes if necessary so I will tend to focus on flexibility and ability to get to the core of the code without being concerned about the niceties of interface – we are all different and have different ideas about software.

    I am still trying to decide which render to pursue before I start tweaking and developing the source code for my own purpose, at the moment I am leaning towards mitsuba but that might change.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s