It turns out that this is not a simple question to answer. There are plenty of forum posts that suggest one gain/offset or another but the methodology behind those numbers is often opaque.

In this post I'll explore the process I used to determine the “ideal” gain, exposure time, and offset settings for my situation. This process should work with common consumer CMOS cameras being used for DSO astro-photography.

I'll use a ZWO ASI1600MM as an example throughout.

Understanding Gain

What is gain? In simple terms, gain determines how photon (electron) counts are converted into ADU counts in the camera. The ADU counts will eventually become the values for each pixel in the resulting image file.

ADUs have a fixed number of bits with which to represent values. The ASI1600MM has 12 bits or 2^12 = 4096 possible values per pixel. All values are positive numbers, e.g. [0-4096), with 0 being a special value to be avoided (more on this later).

Unity gain is the gain setting at which the number of electrons measured is exactly equal to the value stored in the ADU. For the ASI1600MM, this value is gain 139.

Some CMOS cameras also have a special gain setting where there is a step-change in read noise. This happens when the internal electronics switch into “high gain” mode and can be seen on the manufacturer graphs. For the ASI290MM, this is gain ~70.

Gain offers a trade-off. It's primarily a choice between larger dynamic range (lower gain) vs lower effective read noise (higher gain).

The formulas are:

// The max full well capacity (FWC) for the camera.
// The value is for the ASI1600MM. Change this to match your camera.
full_well_capacity_gain0_e = 20000

// The number of bits the ADU has for each value.
// The value is for the ASI1600MM. Change this to match your camera.
adu_bits = 12

// Calculates the baseline gain e/ADU (gain 0).
// For ASI1600MM the result is 20000 / 4096 = 4.88.
gain0_e_per_adu = full_well_capacity_gain0_e / pow(2, adu_bits)

// Caculates the ADU value for a specific electron count and gain.
//
// `gain` is the camera gain setting (in 0.1db).
// `e` is the photon/electron count from sensor.
adu_value = e / gain0_e_per_adu * pow(10, gain/200)

// Calculates the e/ADU for a specific gain setting (0.1db).
gain_e_per_adu = gain0_e_per_adu / pow(10, gain/200)

These formulas are derived from the Voltage Gain formula:

// Calculates the gain setting (in 0.1 dB).
gain = 20 * log(gain0_e_per_adu / gain_e_per_adu) * 10

Understanding Dynamic Range

Dynamic Range is a measure of the effective number of distinct values that can be represented by the camera's sensor. It's primarily a function of gain. As gain increases, the usable full well capacity of the sensor decreases, since the larger counts will be clipped. This is why higher gain results in lower dynamic range.

In general, broadband (LRGB) and bright targets benefit most from higher dynamic range. Narrowband (Ha, Oiii, Sii) and dimer targets generally need less dynamic range.

The formulas are:

// Calculates the effective full well capacity (electrons) for a specific gain.
// `offset_adu` is the offset setting for the camera. 
full_well_capacity_e = gain_e_per_adu * (pow(2, adu_bits) - offset_adu)

// The dynamic range in decibels.
dynamic_range_db = 20 * log(full_well_capacity_e / read_noise_e)

// The dynamic range in F-stops / bits.
dynamic_range_stops = dynamic_range_db / 6

// The dynamic range in steps / values.
//
// `read_noise_e` is the read noise for the camera at a gain setting (electrons).
dynamic_range_steps = full_well_capacity_e / read_noise_e

Stacking also influences the dynamic range. Additional subs will improve the dynamic range. The formula is:

// Calculate the dynamic range in decibels after stacking/integrating
// sub-exposures.
//
// `num_subs` is the number of sub-exposures.
dynamic_range_db = (20 * log(full_well_capacity_e /
                             (read_noise_e / sqrt(num_subs))))

Understanding Offset

Offset compensates for negative values introduced by read noise.

Offset is a fixed value that is added to the electron count after gain is applied but before the value is stored in the ADU. The offset value is not scaled up by the gain.

The purpose of offset is to prevent pixels from clipping to zero in dark areas of the image (usually background). Clipping reduces the signal to noise ratio and can reduce the accuracy of calibration frames. It can happen when read noise and dark current introduce a negative charge that is greater than the offset.

Read noise is a Gaussian distribution centered on zero. The reported read noise for a given gain setting is the standard deviation (sigma or RMS) of the distribution. Consequently, it is normal for read noise to introduce a negative charge.

For example, a 1.5e read noise (sigma) means that ~50% of the time, the read noise will be between +/- 1.5e. Given millions of pixels, it would not be unreasonable to expect to see negative noise that is 10-20x the read noise.

Offset is also a trade-off. It prevents clipping but it also reduces the usable capacity of the camera ADU because it is a constant value added to all pixels. This reduces the available dynamic range. As a result, care should be taken to not use an offset value which is too large.

// Determines the amount of dynamic range (in steps) that is lost by using
// the specified offset value.
dynamic_range_loss_steps = offset_adu * gain_e_per_adu

Determining the “ideal” Gain

Lets start with the goals. We want:

1. Enough dynamic range so we are not clipping stars too much
2. Exposure times which are long enough to swamp the read noise
3. Exposure times which are not too long (tracking errors, satellites, airplanes, clouds, etc)

Dynamic Range

Dynamic range requirements will vary from target to target and filter to filter. For the purpose of this analysis, let's assume that more dynamic range is always better and not try to define a fixed min/max value.

Exposure Time

For my equipment and location, reasonable exposure times are between 30 seconds and 5 minutes. Anything shorter consumes too much disk space and processing time. Anything longer risks throwing away too many subs. We also need to expose for long enough to swamp to read noise.

The formulas are:

// Caculates the background value in electrons/second.
sky_flux_e_per_sec = ((background_adu - darkframe_adu) * gain_e_per_adu) /
                     exposure_time_sec

// The minimum exposure time
min_exposure_sec = 5 * pow(read_noise_e, 2) / sky_flux_e_per_sec

// The optimal exposure time
optimal_exposure_sec = pow(read_noise_e, 2) /
                       ((pow(1.05, 2) - 1) * sky_flux_e_per_sec)

Optimizing Gain and Exposure time

To simplify things, let's consider 4 gain settings:

1. Gain 0
2. The gain where there is a step-change in the read noise
3. Unity Gain
4. High gain (e.g. 200 or 300)

High gain levels are only needed if your mount can't support long enough exposures (e.g. alt-az), for narrowband filters, or extremely dark skies.

To decide which gain setting to use, I used the following procedure:

1. Start with lowest reasonable gain (to maximize dynamic range)
   1. For LRGB, gain 0 can be a good starting point
   2. For narrowband filers, unity gain can be a good starting point
2. Take a test light exposure and a corresponding dark frame
3. In the light frame, select a region of background with no stars or other objects
   1. Record the median ADU value for this region as background_adu
   2. Record the median ADU value from the same region in the dark frame as darkframe_adu
4. Calculate optimal exposure time using the formulas above
   1. If it's less than your minimum exposure time, use your minimum exposure time. You are done.
   2. If it's between your minimum and maximum exposure time, use it. You are done.
5. Otherwise, repeat steps 2-4 with the next gain setting

Determining “ideal” Offset

The “ideal” offset value depends on the read noise that the camera has at a specific gain setting. The following procedure should be repeated for each gain setting that will be used. See Ideal Gain for how to determine the “best” gain settings.

To determine the “ideal” offset for a specific gain:

1. Set the gain setting to the value you want to test (e.g. unity)
2. Figure out what offset value is used by default at this gain
   • Let's call this the test_offset_adu
   • If you are unsure, set offset to a specific value (e.g. 0)
3. Take 50-100 bias frames
4. Stack the bias frames using Maximum as the algorithm
   • Make sure to disable weighting, normalization, and pixel rejection
5. Use image statistics to find the maximum value in the integrated image in ADU
   • Let's call this max_value_adu
6. Calculate the ideal offset (in electrons)

   ideal_ofset_adu = max_value_adu - test_offset_adu + 1
   

Example calculations for ZWO ASI1600MM

For the ASI1600MM in my light polluted skies, here are the values I got following the procedures above.

• RGB: Gain 0, offset 10, 60s exposures
• Narrowband: Gain 200, offset 100, 300s exposures

Note: I use synthetic luminance rather than a L filter (due to light pollution).

Gain and Exposure time:

Offset:

Example calculations for ZWO ASI290MM

Offset:

References

• Choosing the right gain for Deep Sky imaging with CMOS cameras
• Gain settings for ASI1600MM-Cool
• ZWO graphs for ASI1600MM
• ZWO graphs for ASI290MM
• Gain, Offset, and Bit Depth
• Imaging Electronics 101: Basics of Digital Camera Settings for Improved Imaging Results
• sub exposure tables for ASI 1600 (and maybe QHY163?)
• Dynamic Range
• Swamping the read noise question
• Read Noise in CCD Cameras
• Normal Distribution
• The Astrophotography Manual: A Practical and Scientific Approach to Deep Sky Imaging

I want to share my system for organizing my data and why I think it is better than some of the other approaches you might find on the internet. I will also discuss some of the earlier approaches I took and why they didn't end up working out well.

Directory Structure

Here is the directory structure that I use:

• Capture\
  • {date}\ — session start
    • Calibration\
      • Darks\
      • {filter}\ — flats & flat darks
    • {target}\
      • {filer}\ — lights
• Calibration\ — master darks, bias, etc
• Processed\ — final processed images
  • {target}-{date}\
• Work\ — scratch area for processing
  • {target}\

Why this approach?

I had several goals in mind when designing this approach:

1. Make it easy to keep track of the matching calibration & light frames
2. Make it easy to transfer the data from the latest session to another device
3. Make it easy to perform backups of the important files and ignore the unimportant ones (more on this later)
4. Make it automated; it should not require any manual book keeping (spreadsheets)

When thinking about how to organize the data coming off the telescope, I realized that matching the directory structure to how I use the telescope is really important. It makes the book keeping happen automatically when the frames are captured.

Here's what my typical imaging session looks like:

1. I set up my telescope in my backyard
2. I use it for one or more nights
3. I shoot multiple targets during those nights
4. At one or more points during the process I shoot calibration frames (flats, darks, etc)
5. I put it away

In this scheme, each imaging session is given its own directory, named by the date on which it started. This structure ensures that I can always find the calibration frames which match a set of light frames. I don't need to take duplicate calibration frames per day/target or maintain a separate spreadsheet mapping.

If I make changes to the telescope which don't require new calibration frames, I just continue the current imaging session. If something happens that causes new calibration frames to be needed, I start a new imaging session. For example, changes to the imaging train, rotating the camera, major new dust mote, etc.

By separating all the original/raw frames into their own Capture\ directory, it is easy to ensure they are backed up. The data for a single session can also be easily and quickly transferred from one device to another.

The Processed\ directory is where I store my master files. This includes the final images and the raw integrated masters for each filter (in case I want to reprocess).

The Work\ directory is where I store all the intermediary files generated during processing. e.g., calibrated flats, calibrated lights, registered lights, etc. I create one sub directory for each project I am working on. If I start running low on disk space, I delete these directories. I do not back them up as they can be extremely large and can always be reproduced from the originals.

Sequence Generator Pro (SGP)

I use Sequence Generator Pro to automate image captures. Here's how I configure it to work with this directory structure:

1. At the beginning of the session, set the capture directory to D:\Astro\Capture\{date}\
2. Set the file path pattern to %tn\%fe\%dt_%tm_%ft_%fe_%ct_%el_%su_%04
   • Example: D:\Astro\Capture\2020-05-22\M 51 - Whirlpool Galaxy\Ha\2020-05-24_105724_Light_Ha_NA_300sec_gain139_offset50_0031.<ext>

If I need to start a new session for some reason, I just change the capture directory.

To capture calibration frames, I create a new target in SGP and name it Calibration. I then fill it with the flats, darks, etc events as needed.

Alternatives

One capture directory per day

I tried using the same directory structure as above but with one key difference. I created one sub-directory per imaging date, not per session. In the beginning, this was fine since I never had an imaging session which spanned more than one night.

Things became difficult to manage as I started to image more. With multi-night sessions, the calibration frames were frequently taken on different dates from the lights. Having to constantly remember which dates match with which was too error prone for me.

This is a fine place to start, just be sure to make the switch once you start doing multi-night imaging sessions.

Organize by project

I briefly tried organizing by “project”.

• Projects\
  • {target}-{date}\
    • {filter}\
      • {lights}\
      • {flats}\
      • {flat_darks}\
    • Processed\
    • Work\

There were a few problems with this approach:

1. Sharing calibration frames across multiple projects captured during the same imaging session didn't work cleanly.
   • It requires duplicating the frames or manual bookkeeping, both I want to avoid.
2. Sometimes I had multiple imaging sessions per project. This made keeping track of the calibration frames more difficult.
3. File transfers were slow because the capture directories became bloated after several sessions.
4. Backups were annoying because I had to exclude each of the Work\ sub-directories