Prev: Performance Issue: Canon Lenses or Camera Body Causing Softer Focus
Next: What place/s in the world would you most want to take a selfportrait in ?
From: Alan Lichtenstein on 15 Jul 2010 07:51
Better Info wrote:
> On Wed, 14 Jul 2010 07:30:24 -0400, Alan Lichtenstein <arl(a)erols.com>
> wrote:
>
>
>>Better Info wrote:
>>
>>
>>>On Wed, 14 Jul 2010 09:50:15 +0100, Bruce <docnews2011(a)gmail.com> wrote:
>>>
>>>
>>>
>>>>On Tue, 13 Jul 2010 17:09:06 -0700 (PDT), Rich <rander3127(a)gmail.com>
>>>>wrote:
>>>>
>>>>
>>>>>On Jul 13, 5:34 pm, Alan Lichtenstein <a...(a)erols.com> wrote:
>>>>>
>>>>>
>>>>>>Please explain the pixel-mapping function. Is it to be used for
>>>>>>'preventative maintenance' or just if a problem is identified? The
>>>>>>manual is less than satisfactory. As an Olympus owner, I never used
>>>>>>this feature, because I had and have no problems.
>>>>>
>>>>>If a hot (or dead) pixel appears (a pixel whose response rate differs
>>>>>significantly from its neighbours) you can "map" the image of the
>>>>>adjacent pixel over the bad one, thereby eliminating it from the
>>>>>image. Kind of like how long exposure dark framing is used to
>>>>>eliminate hot pixels. I never had to use it on an Olympus either, but
>>>>>it would have come in handy with a few Nikon's I've seen.
>>>>
>>>>
>>>>It was useful on the Olympus E-20 DSLR which seemed particularly prone
>>>>to hot pixels - more so than any Nikon DSLR, I suspect.
>>>
>>>
>>>EVERY sensor is prone to hot and dead pixels. Just because you don't have
>>>access to mapping them out or know about them doesn't mean that they don't
>>>exists.
>>>
>>>On average, there's at least 2,000 to 15,000 dead or hot pixels on every
>>>sensor of every size by every manufacturer. This has been found by the
>>>"badpixel.lua" script for CHDK cameras. (Through 3 years of sensors from
>>>various providers.). That LUA script required to run before using CHDK's
>>>DNG output format. CHDK has to find all the bad-pixels that the cameras
>>>already knows about before it can convert the RAW sensor data to a DNG
>>>format.
>>>
>>>If your sensor is reporting ZERO bad photosites someone is most certainly
>>>lying to you. EVERY sensor has them, to varying degrees, no matter how much
>>>money that you threw at your camera.
>>>
>>
>>Perhaps you can add something to my question. I have never used the
>>pixel mapping function. The manufacturer suggests that it be used once
>>a year. I cannot detect any 'bad pixels' optically, albeit that is a
>>very crude and likely very imprecise function, but I see no evidence of
>>any in photographs. Consequently, I have never used the function. Are
>>you suggesting that I operate it as per the manufacturer's advice anyway?
>
>
> There's no reason to use it unless you are detecting stuck pixels in all of
> your photos. Stuck (hot or dead) pixels are a dynamic thing. They can
> change over time. Depending on static charges and stray cosmic rays, some
> can become hot or dead for weeks then suddenly clear up on their own.
> (People who fiddle with webcams and take them apart to add optics or remove
> the IR filter find this out all too well, the image can be a noisy mess for
> a week, then it just clears up one day.) Some will show up only when the
> sensor has warmed up. Others only showing when the sensor is first turned
> on, while it's still cold. I wouldn't map any out unless you absolutely
> have to.
>
> The nice thing about CHDK's two-fold method (noise removal on/off option)
> or the DNG saving format (requiring a different file of pixel mappings,
> filename "badpixel.bin") is that you can even edit the noise-removal's bad
> pixels file ("badpixel") by hand by entering their X,Y coordinates. One of
> my Sony superzoom cameras also has a remapping feature (button hidden in
> the battery compartment), but in the 8 year life of that camera so far I've
> never had reason to use it.
>
> Can you find documentation that it will remap the whole sensor and wipe out
> the old internal list of bad pixels? Or just look for new ones and add them
> to a list in memory? If the latter, then you could be losing valuable
> pixels over time if it won't free up ones that have switched from bad to
> good.
>
> I also wouldn't use it after the camera has been used for a long time, the
> number detected would rise with sensor temperature. When doing my own CHDK
> camera tests for shutter speeds from 1 second to 64 seconds (for a special
> build of CHDK that had a long-exposure hot-pixel removal routine, requiring
> 17 different badpixel files, one for each shutter speed), it was surprising
> how many more show up as the sensor gets warmer (not shutter speed
> dependent). And many times, they were not even the same ones from session
> to session. This is probably why this special-build routine was not made
> into a base function of all CHDK builds.
>
> If you'd like to find out just how many bad pixels your camera has, look
> for a little utility called "DeadPixelTest.exe". Or hunt through the CHDK
> documentation for their own version called "show_bad.exe". The CHDK one
> outputs the pixels in a X,Y coordinate list from a dark-frame RAW file. You
> can also define the threshold of just how warm of a pixel you are willing
> to tolerate as bad. Most choose a threshold value of 32, 64, or 128 (on the
> RAW data, of 10 (1024), 12 (2048), or 14 (4096) bits) as being indicative
> of a too warm of a pixel, but this is just a personal preference. One of my
> CHDK cameras has an exceptionally quiet sensor on it (luck of the
> manufacturing draw) so I use a warm-pixel threshold value of only 12 on it.
> Conversely, all of those with a value of a solid zero, 0, in a RAW file
> have to be mapped out too, as being fully dead (cold, instead of warm or
> hot).
>
>
>
I appreciate your response, and thank you for it. Your reply, however,
points up more of my ignorance due to inexperience. If I may, I'd like
to ask a few more questions, some of which I suppose, are pretty basic.

1) Exactly what is CHDK? DNG? Are these programs? If so, how do in
use them?

2) How would the above be integrated for use with my pixel mapping function?

I probably will have more questions after you reply. You gave me quite
a bit of information above, which I'll keep, but a good deal of it is
over my head, right now. So forgive me if I ask questions that will
provide some information which will build up to your reply.

Again, in advance, thank you for your reply and of course, your patience
in making them.
From: Better Info on 15 Jul 2010 17:49
On Thu, 15 Jul 2010 07:51:26 -0400, Alan Lichtenstein <arl(a)erols.com>
wrote:

>
>I appreciate your response, and thank you for it. Your reply, however,
>points up more of my ignorance due to inexperience. If I may, I'd like
>to ask a few more questions, some of which I suppose, are pretty basic.
>
>1) Exactly what is CHDK? DNG? Are these programs? If so, how do in
>use them?

CHDK is a 3rd party program to *greatly* enhance the operation of many
cameras from the Canon Powershot line. See
<http://chdk.wikia.com/wiki/CHDK>

DNG is a RAW file format that is more universally standard. The developers
of CHDK implemented this as one of the 2 available RAW formats from
CHDK-enhanced cameras so most all editors can read the RAW files from these
cameras.

I raised these issues as examples to explain to you how bad pixels are
mapped out in these cameras because all cameras use similar methods, though
not as thorough and precise as CHDK's user-definable methods. Nothing more.

>
>2) How would the above be integrated for use with my pixel mapping function?

You wouldn't. But understanding why and how mapping out bad pixels is done
it might help you understand if you should use that function or not on your
own camera. Though the two little utilities of "DeadPixelTest.exe" and
"show_bad.exe" could be used to determine if you are plagued with bad
pixels or not. But even then, if they are not ruining your images, you
don't notice them, etc. don't worry about it. Because *ALL* cameras have
them to much greater degrees than most are ever aware. Nearly all of them
mapped out before you even get your camera so you'll never even know.
(Correction, CHDK's "show_bad.exe" utility probably wouldn't work with
dark-frame RAW files from your camera, it requires PURE RAW, a no header
info file-structure to do its magic.)

Anyway, bottom line is ... if your images are not being ruined by dead or
hot pixels then don't bother.

Make certain too that it really is a dead-pixel problem. Defects in certain
areas all the time might just be crud stuck on the sensor and it needs a
good physical cleaning. Bad-pixels will usually only be 1 or 2 pixels in
expanse. Rarely, a group of 4 of them might be bad in the same spot.

Doing pixel-mapping as a scheduled maintenance thing as suggested by your
camera maker is nonsense. Even then they are not suggesting an optimal
sensor temperature at which to employ that function. Though, personally,
after about a six-month period of heavy use from when first buying it (a
decent burn-in time period), then after the camera has been sitting idle
for a few days to a week or so, I *might* employ it just once a couple
minutes after turning on the camera to see if I could improve on the
bad-pixel map as it had left from the factory. No more than that during the
life of the camera unless I detected problems in the images. Their
scheduled maintenance advice was written by someone that doesn't really
know how sensors behave in the real world. Many camera manuals are written
by such people.

[Aside: This is why it's so easy to spot the role-playing
pretend-photographer trolls in these groups. The only experience they have
with cameras are the poorly written manuals that they download and read. My
manuals are full of hand-written notes in order to correct what the manual
authors never knew about the cameras. Should I sell or give a camera away
then the next owner will be better educated than those who first buy them.]


From: Alan Lichtenstein on 17 Jul 2010 09:31
Better Info wrote:
> On Thu, 15 Jul 2010 07:51:26 -0400, Alan Lichtenstein <arl(a)erols.com>
> wrote:
>
>
>>I appreciate your response, and thank you for it. Your reply, however,
>>points up more of my ignorance due to inexperience. If I may, I'd like
>>to ask a few more questions, some of which I suppose, are pretty basic.
>>
>>1) Exactly what is CHDK? DNG? Are these programs? If so, how do in
>>use them?
>
>
> CHDK is a 3rd party program to *greatly* enhance the operation of many
> cameras from the Canon Powershot line. See
> <http://chdk.wikia.com/wiki/CHDK>
>
> DNG is a RAW file format that is more universally standard. The developers
> of CHDK implemented this as one of the 2 available RAW formats from
> CHDK-enhanced cameras so most all editors can read the RAW files from these
> cameras.
>
> I raised these issues as examples to explain to you how bad pixels are
> mapped out in these cameras because all cameras use similar methods, though
> not as thorough and precise as CHDK's user-definable methods. Nothing more.
>
>
>>2) How would the above be integrated for use with my pixel mapping function?
>
>
> You wouldn't. But understanding why and how mapping out bad pixels is done
> it might help you understand if you should use that function or not on your
> own camera. Though the two little utilities of "DeadPixelTest.exe" and
> "show_bad.exe" could be used to determine if you are plagued with bad
> pixels or not. But even then, if they are not ruining your images, you
> don't notice them, etc. don't worry about it. Because *ALL* cameras have
> them to much greater degrees than most are ever aware. Nearly all of them
> mapped out before you even get your camera so you'll never even know.
> (Correction, CHDK's "show_bad.exe" utility probably wouldn't work with
> dark-frame RAW files from your camera, it requires PURE RAW, a no header
> info file-structure to do its magic.)
>
> Anyway, bottom line is ... if your images are not being ruined by dead or
> hot pixels then don't bother.
>
> Make certain too that it really is a dead-pixel problem. Defects in certain
> areas all the time might just be crud stuck on the sensor and it needs a
> good physical cleaning. Bad-pixels will usually only be 1 or 2 pixels in
> expanse. Rarely, a group of 4 of them might be bad in the same spot.
>
> Doing pixel-mapping as a scheduled maintenance thing as suggested by your
> camera maker is nonsense. Even then they are not suggesting an optimal
> sensor temperature at which to employ that function. Though, personally,
> after about a six-month period of heavy use from when first buying it (a
> decent burn-in time period), then after the camera has been sitting idle
> for a few days to a week or so, I *might* employ it just once a couple
> minutes after turning on the camera to see if I could improve on the
> bad-pixel map as it had left from the factory. No more than that during the
> life of the camera unless I detected problems in the images. Their
> scheduled maintenance advice was written by someone that doesn't really
> know how sensors behave in the real world. Many camera manuals are written
> by such people.
>
> [Aside: This is why it's so easy to spot the role-playing
> pretend-photographer trolls in these groups. The only experience they have
> with cameras are the poorly written manuals that they download and read. My
> manuals are full of hand-written notes in order to correct what the manual
> authors never knew about the cameras. Should I sell or give a camera away
> then the next owner will be better educated than those who first buy them.]
>
>
Again, thank you for your explanation. One additional question,
however. It seems, based on your explanation that the software you cite
is particular to Canon. Are the extensions you cite part of the
proprietary software, or are they found elsewhere? I would conjecture
that the cited items would not be compatible with my camera, since it is
not a Canon.

You should know that I haven't examined the proprietary software which
came with my camera, as the manuals seem to assert that it is an editing
program, and I presently use another editing program.

I appreciate your comment regarding extending the manufacturer's
manuals. In many instances, not just regarding cameras, I find similar
lapses. So it isn't difficult to conclude that these omissions extend
to cameras. However, unlike you, I currently lack sufficient technical
language to draw the conclusions you have. Until I do, I'm stuck with
what the manufacturer says, except when I get lucky and get the advice
of others, like you and several others in this NG who can give me
answers that essentially 'read between the lines.'
From: Better Info on 17 Jul 2010 18:36
On Sat, 17 Jul 2010 09:31:26 -0400, Alan Lichtenstein <arl(a)erols.com>
wrote:

>
>Again, thank you for your explanation. One additional question,
>however. It seems, based on your explanation that the software you cite
>is particular to Canon. Are the extensions you cite part of the
>proprietary software, or are they found elsewhere? I would conjecture
>that the cited items would not be compatible with my camera, since it is
>not a Canon.

If by "extensions" do you mean the file-naming conventions of CRW and CR2
and their associated file formats? Then yes, those are proprietary to the
Canon line of products.

Now if you're talking about that utility to analyze the pixels of your
sensor, then the program "DeadPixelTest.exe" is universally useful to all
digital photography on all cameras. I will use it one or two times on the
output from any new camera, just to see what's going on with the sensor I
just bought, then again maybe 6 months later to see how things have "burned
in" after extensive use. But I'm not manic about dead or hot pixels. I just
like to know what I'm working with and whether to give kudos to that
manufacturer or if I'll be buying a different brand next time around.

["DeadPixelTest.exe" is unlike the 'show_bad.exe" program which is
proprietary to CHDK RAW file formats, unlike any other RAW formats out
there, being pure RAW, no header information at all (the DNG output does
include a full header). Some interesting information: Better written
software today has no problem reading these pure RAW files. CHDK RAW file
formats have become well known to all software developers due to their
usefulness and popularity. You can even see the non-imaging areas of your
sensor in CHDK RAW files because these RAW files are a full data-dump of
every photosite on the sensor. You'll often see cameras listing widely
different counts for total pixels and imaging pixels. A large percentage of
any sensor is also used for black and white limit detection, color-balance,
and other "behind the scenes" calculations. A small plus is that CHDK RAW
files also give you an extra (approx.) 30-40 more pixels of image
horizontally and vertically that all other cameras discard in their RAW
image data. This is due to the less intelligent RAW to JPG conversions done
in camera. The border pixels require surrounding pixels on all sides of
them for proper interpolation if using fast in-camera methods. If doing the
interpolation on the computer with more advanced interpolation software,
then those border image pixels can be put to good use, giving you about an
extra 0.5� field-of-view in your images (an average based on wide-angle
lenses and pixel density). That's quite a bit when you consider the sun and
moon are about 0.5� in width to the unaided eye, or a view from a 45-55mm
EFL lens.]

>
>You should know that I haven't examined the proprietary software which
>came with my camera, as the manuals seem to assert that it is an editing
>program, and I presently use another editing program.

I'm not sure why that figures into the questions you asked. It was about if
applying pixel-mapping would be beneficial or not. The editing software
that came with your camera has nothing to do with that. I think I've
answered the issues about dead, warm, and hot pixels, and pixel-mapping
about as fully as possible from the knowledge I have about camera firmware,
sensors, and general behaviors of them both.

>
>I appreciate your comment regarding extending the manufacturer's
>manuals. In many instances, not just regarding cameras, I find similar
>lapses. So it isn't difficult to conclude that these omissions extend
>to cameras. However, unlike you, I currently lack sufficient technical
>language to draw the conclusions you have. Until I do, I'm stuck with
>what the manufacturer says, except when I get lucky and get the advice
>of others, like you and several others in this NG who can give me
>answers that essentially 'read between the lines.'
From: Better Info on 20 Jul 2010 03:58
On Tue, 20 Jul 2010 02:09:13 -0500, John Turco <jtur(a)concentric.net> wrote:

>
>
>Only recently, I'd discovered that my Kodak P850's built-in "Calibrate
>Imager" function, is supposed to remap bad pixels (of which, my camera
>has always had a few).
>
>I've never used it; do you believe it's safe to do so?

It's always safe to do so. Whether or not it is adding to an already
mapped-out array of pixels (not freeing up bad ones that have switched from
bad to good), or rebuilding a completely new mapped-out set of pixels, only
your manufacturer knows for certain.

If you have always been annoyed by some unmapped pixels, then go for it.

First  |  Prev  |  Next  |  Last
Pages: 1 2 3 4
Prev: Performance Issue: Canon Lenses or Camera Body Causing Softer Focus
Next: What place/s in the world would you most want to take a selfportrait in ?