Thursday, June 17, 2010

F80EXR Reviewed at DPReview –- Errors in Interpretation

Introduction


It was with much anticipation that I opened this brand new review to see how the best current Fuji long zoom compact fared against a rather impressive number of compact cameras. Of course, as I have mentioned before, pretty much all reviewers so far have screwed the pooch on their tests of EXR technology … no one has used decent images shot in the binned modes so the camera’s real abilities never show up.

Edit: I am pretty critical of the review in the rest of this review review (yes, that’s correct grammar … I’m reviewing their review) … and a user in the Fuji Talk Forum correctly pointed out that all camera owners could say that their cameras’ special modes also did not receive special treatment, so the review was fair.


Wrong.

The EXR technology is unique in that it is effective in reducing the pixel density by half. This brings pixel densities very close to those of the early Fxx series of cameras and I maintain that in these shooting modes, the F70EXR and F80EXR wipe the field at ISO 400 and above. None of the other cameras in this field have that kind of potentially dominating technology. Not one.
Sidebar
Just in case you are new to my blog, I was pretty disappointed with mistakes Fuji made with the F80 … they added higher saturation and unneeded pixels, so the camera tends to show heavy blue channel contamination as soon as the light is even remotely challenging. The F70 does not have that problem and retains better detail to boot, but Fuji “wisely” replaced the F70EXR after only 6 months, no doubt trying to maintain their unbroken record of eliminating their best cameras much too quickly … 
Ok, that was catty. Sorry. But Fuji deserves whatever ridicule they get for replacing great cameras much too soon. It would be a different story if they did a wonderful job with the successors … but that really has not happened. Remember the F50fd? Yuck. Remember the S700? Yuck. 
But all is not actually lost … the F80 is actually quite good when shot properly and when you don’t try to rebalance poor light too much. That always emphasizes the weak blue channel … so just leave it alone.
Ok … back to DPReview … much anticipation yada yada yada … well, it really did not shine. And I know how good it can be, so I was now wondering how they shot it.

As I went through the review, I realized how tricky it is to shoot these cameras well. Another knock for Fuji. The reviewer, one supposes, was too busy to learn how this camera really works. And apparently too busy to inform himself by reading my blog :-)

But that’s not my only gripe with the review. There are omissions and errors in a few places. Let’s go through them.

Page 4 – The F80EXR Page
On If we owned the F80 EXR, we would tape up the shooting mode dial so that it never comes off 'EXR'.
There are limitations with EXR mode. And EXR Auto is evil, the cam is draining the battery at a frightening rate. Further, this cam should *never* be shot at 12mp. P mode does all the tricks with none of the pain.
The most obvious source of confusion in the menu system is a 'dynamic range' setting, which is available in all modes (including 'D-Range Priority' EXR) but is in fact quite unrelated to EXR. 'Dynamic range' is in fact a conventional d-range optimization function, designed to protect highlight detail. Options run from 100%-400%, and when selected, the entire shot is underexposed to retain highlights, then a different tone curve is applied to present the correct final exposure. Confusingly, even when you're shooting in the D-range priority EXR mode, dynamic range optimization is still an option, albeit with an extra 800% level available.
At L size it uses the software implementation common to the S100fs et al. But at M size, DR400 is available at 100 ISO, which tells us that it is binning. And it makes images that are identical to EXR DR but without that mode’s limitations.
It is a shame though that Fujifilm doesn't have the confidence to make the EXR modes available universally. The only way that you can shoot in the excellent DR mode is to switch the shooting dial to 'EXR' first. This is annoying, and a little confusing, since the 'HR' mode for highest resolution actually results in exactly the same output as you'll get from turning the wheel away from 'EXR' to one of the other shooting modes.
EXR DR mode is enabled as soon as you set the camera to M size in P mode. Only at L size do you get the same output as HR Mode in other shooting modes.
We were disappointed with the performance of the F80's 'Pro Focus' mode as well. One of the 'Special Program' (SP) modes on the mode dial, Pro Focus combines two or three frames taken at different focus points together, to deliver the illusion of a smaller depth of field. It only really works where there is some depth of field available (i.e. at the long end of the zoom) and in our testing, it's benefit is marginal, at best.
I agree. But two things: (1) Images are usually unacceptable even if the camera will play ball because of really messy edges. (2) Pro Low Light mode is not mentioned at all, and that is the SP mode that sings and dances.
It is a shame though that (for some reason) the highest available ISO setting in 'SN' mode is ISO 1600. Like SN and DR modes, the ultra-high ISO settings are also achieved by pixel-binning (the highest 12,800 setting outputs a 3Mp file) and it does therefore seem odd that you have to switch out of the EXR setting to access them.
It may seem odd … but once you understand that P mode at M size is binning, then it all falls into place.
The key selling point of the Fuji F80 EXR is undoubtedly its triple-mode sensor. Whilst it is a solid and (generally) reliable compact camera overall, its sensor is genuinely unique, and offers significant improvements in image quality if used correctly.
Ironically, I think that the reviewer generally shot the camera incorrectly.
It is a shame though that its interface isn't better designed, and the EXR modes more sensibly implemented.
I agree with that. My how to shoot EXR sensors article would have been very helpful with respect to settings to avoid like the plague. And especially for the roundup where they were all compared ... shooting the cam at 12mp guaranteed a beating, and it got one.
HR mode 
In most situations, the F80EXR uses its high resolution 'HR' setting in which it uses the full 12Mp resolution output from its sensor much like any other camera. This is also how the camera behaves when it's not placed in EXR mode. When the mode dial is set to EXR and 'auto' is selected, the camera will default to HR mode when it determines that there is enough light for it to offer the best quality result.
HR mode is not the best quality result by any stretch of the imagination. Upsizing a 6mp image beats a 12mp every time.

Summary so far: A lot of errors on page 4 in my opinion. Perhaps it is Fuji’s fault for making the user interface so complex. But I think the reviewer should have informed himself a bit better on this camera.
And the situation doesn’t get much better.

Page 14 – Movie Modes
No discussion of audio. The audio on the F80EXR sucks. Interesting, because it is blindingly obvious when a 10 second clip is shot in the office. Big omission in my book. This is one lucky break for Fujifilm.


Page 17 – Studio Comparison at 400 ISO
At this ISO, high resolution is really starting to hurt the F80EXR. And it really shows with the tree from the Bailey’s label. Shooting the camera at M size and upsizing to 12mp would have made a world of difference. But that’s more work than DPR has ever seemed able to do. They have gone as far as normalizing sharpening settings now and again, but they never normalize resolution, which would help a great deal here I think. It would have allowed shooting at both 6mp and 12mp and fairly comparing the results.

For example, normalizing everything to an 8x10 (3000x2400) and showing us those crops would eliminate all doubt as to which camera to buy for most people. The F80EXR would have to be upsized from 6mp and downsized from 12, most others would be downsized. The 6mp mode of the EXR cam would likely clean all of these others’ clocks at 400 and 1600 ISO. As it is shot, it looks mediocre …

Now Fuji cannot be upset here … their user interface just sucks. And it must never have occurred to them to simply bin everything and upsize when 12mp was needed. What a different THAT would have made to these test results.


Page 18 – Studio Comparison at 1600 ISO
There is no way on God’s green earth that any EXR sensor should be shot in high resolution mode at 1600 ISO. And especially not a 1/2” sensor with 12mp on it. And that’s what was done for the main test on page 18. Of course, the image looks horrid.

The F80EXR is given kudos for managing the high contrast details on this page fairly well in the corner crop. Yet I know for a fact that an upsized P mode image would have done far better. The camera itself would never choose to shoot at this ISO in anything but a binned mode.

The reviewer actually does shoot the camera in EXR SN mode at 1600. But unfortunately he totally blows out the highlights! And that occurs right where the corner crop is, wiping out half the detail he is trying to preserve.

SN mode is also very high contrast because DR100 is the only DR mode available. If you shoot P mode with DR400 at 1600 ISO, the camera appears to handle dynamic range much better … and that helps with shots like these.

He goes on to say:
Whilst images shot in SN mode certainly look better at a pixel level than those shot in the conventional, full-resolution mode, it is fair to say that the improvement is hardly dramatic in high-contrast areas, and subtle to the point of being almost invisible in small prints.
Really? Images are not made up only of high contrast areas. It’s the large swatches of low contrast details that are thoroughly destroyed by HR mode at 1600 ISO, and that is *plainly* visible on small prints.

The rest of the review goes on to use the crappy 12mp images and the F80EXR gets lost in a sea of mediocrity. These cams all suck at 1600 ISO. But shot properly, the F80EXR is considerably better than the rest. We’d never know though, were it up to this review.

Now … before I go on to show how I would interpret the images from this review, I should note that the reviewer and DPReview staff in general would make one very simple argument in their defense: there is no time to explore the special modes of every camera, so they did the best they could. To which I of course reply: of course. But unfortunate. The fact is that this site is where enthusiasts go to get the scoop. But the reviews are a big undertaking, and this case shows that the enthusiast is short-changed in favor of J6P.

Reinterpreting the Images From This Review
So before we get going here, I will state up front that DPReview.com holds the copyright on the images from which I take crops and that I claim fair use of said copyright for educational purposes.

DPReview shot two test images at 1600 ISO … we have the 12mp test image shot in HR mode and we have the totally blown out 6mp image. Now, first we deal with the HR image. How could they have shot this one in HR Mode when HR Mode does not offer 1600 ISO as an option?

image
image

Oops. :-) 

It doesn’t really matter what mode was used, but it just underscores again that this cam was shot like a mom and pop cam.

Anyway, you can clearly see the near-total lack of low contrast detail in the so-called “HR mode” image at the top. And there is some extra detail in the SN image at the bottom, but not as much as might be expected. Why is that you might ask? Well, I’m guessing it’s caused by pushing all the mid and high tones into the shoulder of the histogram … this makes the differences in tones much less visible … i.e. it lowers the contrast in these tones.

image

You can see the data smashed over onto the right hand side … now that’s what I call “exposing to the right” :-) Interestingly, the data on the left is at the edge of the histogram, indicating that the tone curve is also too harsh, since it created a wider tonal curve than can actually fit. That’s a basic flaw with SN mode … DR100 just plain sucks as the tone curve is very harsh. I *never* use SN mode for that very reason. Crappy capture equals crappy image.

This special section of the review needed an image whose exposure matched the exposure of the 12mp image, so this one should have been discarded and reshot. He had to have loaded it into photoshop to use it for the review which means that he had to have seen the blowout, so I fail to see how invalidating his own conclusions served anyone here.

But he could still have saved this section of the review by creating a version of the 12mp image with levels adjusted to match the histograms. I have done just that and I am seeing that the SN mode image crushes the 12mp image.

Here are the two images with their histograms matched.

SN Mode (the blown out image)

image

Original 12mp image adjusted in levels to match
 image

And even at these tiny, tiny sizes, you can see slight differences. The latter image shows color noise so strong in the color and grey wedges that it is actually visible on my TN monitor, which has a brighter than normal white point (just like almost everyone that reads DPReview these days.) Here is a set of crops for you to examine from the 800 pixel version of the above. This is a highly compressed (in size) image that is fairly similar to looking at a 4x6 print. 

image
So … what does the reviewer say about prints this size?
Whilst images shot in SN mode certainly look better at a pixel level than those shot in the conventional, full-resolution mode, it is fair to say that the improvement is hardly dramatic in high-contrast areas, and subtle to the point of being almost invisible in small prints.
Well, I can see differences at 400px wide, and the differences are very clear to me at 800px wide as shown in the above crops. So again, how could they be considered invisible? Here’s how: by examining images that are at least 1 stop apart in exposure and drawing a completely false conclusion.
Once normalized in exposure, as I did above, the differences become quite visible. And here is a list that I can see:

Top crops:
  • Massive noise clearly visible in HR grey scale
  • Fur is softer
  • Coin is much softer
  • Needlepoint is much less detailed (smearing)
  • Lots of noise in the box and color swatches at the bottom
Bottom crops:
  • Same issue in the grey scale
  • Massive blue channel contamination ruins the color swatch, 3rd row from the bottom is destroyed
  • Heavy noise in the Bailey’s bottle and the cup
  • Much more detail in the top left part of the feathers
So … for small prints or the web, will anyone care? Probably not. But this is pretty good light, and already the color blotching is ugly. Imagine a face by candle light. Total destruction.
Moving on to my final points. Returning to the concept of normalizing to 8x10 images (for example.) The reasoning behind that is that people should not be *forced* to stay at 4x6 or web image size. It should be possible with images shot in any light (i.e. at any ISO) to be processed into a decent 8x10 image. So let me now show you crops at that size to see whether the HR mode image has a chance of making a decent 8x10.

image

Now, I only want you to look for three things in of these crop pairs:
  1. Does one side give you more of the feeling of 3-dimensionality?
  2. Does one side show less color noise?
  3. Does one side retain more detail?
The answers, of course, are pretty obvious …
  1. SN mode. Very significant difference that will show up in any print, and especially in an 8x10.
  2. SN Mode. Several areas have significant pollution in the HR mode crops. Very little in SN mode.
  3. SN mode. Many areas in the HR crops are almost devoid of detail. NR has wiped it clean.
So … do you see what I mean? The 12mp image has the advantage here … is is downsized by 25% to be sized to an 8x10 image at 300ppi. The 6mp image is upsized by a small percentage and yet it pounds the 12mp image in every way.

So … Fuji … get your act together. You cam could have legitimately been marketed as a 12mp cam (sensor has 12m photosites) with all images shot binned and upsize in L mode. Better detail, better noise, better every-fricken-thing!

DPReview … come on. This camera behaves much better in P mode at M size. Perhaps it is time that you began normalizing to a specific size (e.g. 8x10) as does DXO Labs so that your conclusions and crops always have a truly common feel to them. You are leaving far too much in the reader’s hands right now, and you cannot easily review a cam at much smaller resolutions because they look like a joke when crops are compared.

This camera *could* have wiped the floor with the others at higher ISOs …

The final comment from the reviewers:
The JZ500's 'big brother' the FinePix F80 EXR is a mixed-bag though, and although it gives superb results in its 'DR' mode, the EXR functionality that sets it apart from its competitors is poorly implemented, and overcomplicated.
As is the review without any effort to normalize the outputs of these cameras to some useful standard.

4 comments:

Lili said...

Is it me or did they get the F200EXR review right and this one so wrong.
if so, HOW?
*Beatsheadondesktop*
At least with the 200 they actually seem to have tried use the cameras as meant....

Tut said...

Different person did this review than the F200EXR.

With the F80EXR review being so inaccurate, I wonder if the Casio FH100 and Samsung HZ35W are actually any good? Looking closely at the FH100 sample gallery on dcresource, I was not as impressed the first time I looked.

Lili said...

The Samsung shots on Flickr look about as good as my former Samsung L85; that is to say quite good indeed at lo ISO.

Barry Fitzgerald said...

Too much NR for the Samsung have a look at the images and even low ISO it's melting away fine details. Go out in low light and it's soundly thumped by many rivals. It's by no means a bad camera I tried one for a few days but the IQ is over-processed and lacks refinement. Nice enough camera in a lot of ways but it's not as good as it should be. AF is also bad in low light failing to lock a lot of times.

Kim's picked up some valid points here on the review. I'm a bit puzzled as to how the reviewer can miss so many thing..he's taken shots with a CX3 (which I am also testing) yet never bothered to turn the NR "off" available in the main menu. The Casio isn't bad really and it's sharp, but how dumb to have raw only up to ISO 200?.. it's a blow torch and candlewax job for lower light levels. 7cm macro..erm isn't even classed as a close up again no mention of this in the review article.

Half the problem is are they testing the full auto modes? (fair enough but that's just a small part of any camera). Or did they bother to try to get the best out of each camera? Ditto on sharpness..did they try adjusting those models that have user selectable sharpening in camera? If not you can't really talk about sharpness!

Did the Fuji get a fair stab at it? I don't think so but then I'm not sure many of the models there got a fair hit at it either..they might have covered many models but their methods leave something to be desired. Pick any model out of that and their review is off most others. The Casio has had some good ones but also a few mixed as well. Ditto on the Samsung which has been criticized for poor jpeg processing and DPR rave about it at pixel level!

It's not an easy choice for users in this range but I'm not sure DPR should be the place to go to decide that.

Strange..very strange