How to develop your own raw files by hand, Step 2 - build offset files (after Step 1 - convert raw to tiff)

How to develop your own raw files by hand, Step 2 - build offset files (after Step 1 - convert raw to tiff)

This is a follow-up to

Offset files (or bias files) are files that remove fixed-pattern noise produced by the analog-to-digital converters on the camera sensor. These thousands or millions of converters on - or close to - the CCD or CMOS sensor, do not all produce the same values, when given the same photon count. This offset, or bias, can be corrected for. Offset files contain this sensor pattern. This also reduces the low pixel brightness values in your tiff files (for my camera, around 1012) to mostly 0. From what I've read, it is good to have an offset file for each ISO setting that you plan to use. More on that later.

For the kind of operation we have to perform, we need ImageMagick:

Install this program, when asked, make sure to install ImageMagick in the Windows Path variable, and navigate to its directory, and remove or rename dcraw.exe - it conflicts with the dcraw version installed in my previous post, "step one".

I'm almost sure ImageMagick configures itself in the Windows Path variable. Test this by rebooting your computer, and opening a command line (Start > Command Prompt). Type


and press Enter. You should say: "Error: Invalid argument or not enough arguments" etc. That tells you the command line knows where ImageMagick is installed at.

Now, you'll need a batch file to produce these offset files. For this, we'll use the statistical method called median, which takes the middle value of a sorted list of values. This is useful to find the most relevant value of any pixel, when you have a range of images that may contain different values for the same pixel each time. The more pictures you have, the more likely that the middle values occur when taking a picture.

To get the median values of a range of pictures, open C:\Program Files\Utilities, right-click, and choose New > Text Document, name it


then right-click that file, choose Edit, and type

magick.exe %1 -evaluate-sequence median %2

Save the file.

We can use median.bat to calculate the median of any range of images, wherever that is useful. Photoshop CS2 can't perform this function, I know CS6 can but that may not help many people. Taking the median of an image is more useful than taking the mean (commonly called "average") because the mean is easily thrown off by outliers that occur once in a series. Outliers are unpredictably high or low values, that make the mean, meaningless. Consider the mean, or average, of 1, 2, 3, 4, 10000. The highest value really messes with the average, where the median is simply 3. It is more representative of the series.

Take a series of pictures with your camera, with the fastest shutter speed, a lens cap, and at the ISO value you're likely to use much. The ISO value of a digital camera is actually an amplification factor, applied directly after sensor-readout, before or after conversion by the camera's ADC. It affects the raw file's offset, and so, we need to produce files with this ISO before creating the offset file for that ISO value.

With this series of raw files, taken at the proper ISO value, and no light, we can calculate the median values into an offset file.

To perform this calculation, navigate to your raw images directory (assuming only dark "offset" raw-files are located here), make sure nothing's selected, Shift + right-click on an empty area there, and click Open commmand window here.

Then convert the raw files to tiff files, replace .cr2 with whatever raw extension you have. Type:

dcraw-unprocessed *.cr2

and hit Enter. dcraw will create a bunch of tiff files.

Then, type

median *.tiff isoXXXX.tiff

and hit Enter. ImageMagick will calculate the median of your tiff files, and save them in isoXXXX.tiff (change the filename to the correct iso).

Save this tiff file for future use, you could build a library with an image for each of your camera's ISO values.

That's it! In a later post, we'll subtract this file from actual "light frames", exposures, which will visibly remove patterned noise from your photo.

Comments and questions welcome!

Submitted November 23, 2016 at 10:13PM by TheMerovingian from reddit via /r/astrophotography