The Labyrinth

Now, this wasn't the issue I had in mind to talk about next, but a user alerted me to a strange artifact in his M8 photos.

I recognized it, because I also saw it in one of my M9 photos, but on my photo it was a tiny patch and only visible at 400% magnification. I hadn't really bothered about it, under the assumption it was photo specific.

This is what it looks like:

800% crop from a red patch from M9 photo

As you can see, it's sort of a maze pattern.

Not immediately realizing what caused it, I started investigating. This user was kind enough to send me the original RAW file, because I only have a handful of M8 photos, and on my M9 photos this problem is hardly a problem and was difficult to spot.

On the M8 photo the artifact was way more obvious: visible at 200%, quite ugly at 300% and it was almost everywhere in the photo.

First I thought the sharpening was set too high, because if you import M8 monochrome DNGs, Lightroom (my version at least) starts at a default of 25% sharpening, which I think is too high for monochrome DNGs. Lowering the sharpening helped, and the pattern then disappears enough to not be distracting at 200%. Adding some masking also helped.

Problem gone, nice DNG at 100%.

But that didn't take away the fact I thought it was weird.

What caused this very specific pattern on parts that should look more equally gray?

So I first started tweaking the algorithm, assuming it got lost on these equal parts... something wrong with determining the direction for the averaging perhaps?

But that didn't help a lot. I could improve it a bit, but not get rid of it.

It was a bit maddening, since none of my attempts seemed to make a big difference, just these tiny changes that were not enough. It felt like I wasn't only trying to fix the maze, but that I was actually walking in it! Then I decided to try a re-size, because that only leaves the green pixels, and then see what kind of pattern that would create, if any.

And there it was: the green pixels were all over the place. And that didn't make sense to me, because I always assumed a regular patch of brightness would give similar values for the green pixels (apart from noise).

I could see the origins of the maze in this pattern... quite obvious interpolating on these green pixels would bring it out in full glory.

Concentrating on this strange difference between the green pixels, I suddenly remembered reading in the DNG documentation about the BayerGreenSplit value, which indicates how closely two green pixels (one on a red row, and one on a blue row) 'track' each others value.

And that was the 'eureka' moment.

Not taking this value seriously leads to the maze artifact, because the software wrongly assumes the value of the second green channel (the green pixels on a blue row) is unique. And although the value is indeed unique, it doesn't necessarily mean the algorithm should treat it as such.

How to implement the BayerGreenSplit value exactly is at this point still a bit unclear, because the documentation doesn't state in what units the value is set. The value can range from 0 to 5000, and the M8/M9 DNGs are set to 500, but 500 of what? Is this the direct difference between the two pixels? Or is it the difference after gamma correcting or exposure correcting? Or is it the minimum allowed divergence at which point you have to start correcting?

Frankly I think Adobe should be more clear on this value, since they use it in ACR. DNG is supposed to be an open standard, so not specifying clearly how certain parameters work, or how they should be interpreted, sort of leaves developers guessing, like I am now...

Experimenting with it in a first rough trial, by pulling the two green channels closer together, I've managed to get rid of the maze completely, so it's definitely the cause and the solution.

The divergence itself seems linked to the red channel: the more red, the more the green channels start to diverge, at least in the M9 conversion. It might be that red light is bleeding into the green pixels or the green pixels on a blue row are more sensitive to red light.

For now I've only concentrated on the M8 and M9, I'll have to analyze further to see what the Canon photos do.

You can mitigate the problem by not sharpening too much and by using the masking option (talking about Lightroom sharpening) so the more equal areas don't get sharpened at all.

Obviously that tip is just a work around, since this pattern needs to go.

This has priority, so I'll try to fix it as soon as possible, and bring out a new release.