Training Deep Models for Denoising

The following items are required for training a deep model for denoising:

• Training data, which can include input and output images (see Denoising Strategies for information about pairing inputs with outputs).

  You should note that you can extract training data as a subset of the original data (see Cropping Datasets).

• A model that supports denoising.

  A selection of untrained models suitable for regression are supplied with the Deep Learning Tool (see Architectures and Editable Parameters for Regression Models).

The following items are optional for training a deep model for denoising:

• An ROI mask(s), for defining the working space for the model (see Applying Masks).
• A visual feedback region for monitoring the progress of training and a checkpoint cache so that you can save a copy of the model at a selected checkpoint (see Enabling Visual Feedback and Checkpoint Caches).

You use different strategies for denoising by pairing different inputs to outputs and to then train a deep model to predict the original without noise. These include pairing a high-noise input with a low-noise output, a high-noise input with a hIgh-noise output, or a source image with noise to a source image with noise.

High-Noise/Low-Noise… In this intuitive approach, a high-noise input image is paired with a low-noise output image to train a model. Data can be acquired experimentally by capturing short-duration and long-duration images or by creating synthetic low-noise images through image filtering (see Smoothing Filters).

High-Noise/High-Noise… This approach takes advantage of the fact that neural networks are not good at reproducing noise. In this case, a high-noise image provides both the input and output.

Noise-to-Noise…Here, a source image with added noise is paired with another version of the source image with added noise. In this case, the signal common to both the input and output images will be extracted with moderate iterations. You should note Dragonfly provides an additive noise filter for adding noise to images (see Additive Noise Filter).

To help evaluate the progress of training deep models, you can designate a 2D rectangular region for visual feedback. With the Visual Feedback option selected, the model’s inference will be displayed in the Training dialog in real time as each epoch is completed (see Enabling Visual Feedback).

In addition, you can create a checkpoint cache so that you can save a copy of the model at a selected checkpoint (see Saving and Loading Model Checkpoints).

The Deep Learning Tool provides a number of architectures — including Autoencoder, Noise2Noise, Noise2Noise_SRResNet, U-Net, and others — that are suitable for denoising tasks (see Architectures and Editable Parameters for Regression Models).

1. Choose Artificial Intelligence > Deep Learning Tool on the menu bar.

   The Deep Learning Tool dialog appears.

2. On the Model Overview panel, click the New button on the top-right.

   The Model Generator dialog appears (see Model Generator and Deep Learning Architectures for additional information about the dialog).

3. Make sure that only Denoising is checked for the Show architectures for options.

   This will filter the available architectures to those recommended for denoising.

4. Choose the required architecture in the Architecture drop-down menu.

   

   Note A description of each architecture is available in the Architecture description box, along with a link for more detailed information.

   Recommendation Your architecture selection should correspond to your chosen denoising strategy — High-Noise/Low-Noise, High-Noise/HIgh-Noise, or Noise-to-Noise (see Denoising Strategies).

5. Choose Regression in the Model type drop-down menu.
6. Enter the required number of inputs in the Input count box.

   

   Note In most cases, denoising strategies require an Input count of 1.

7. Enter a name and description for the new model, as required.
8. Edit the default parameters of the selected architecture, optional.

   Note Refer to Architectures and Editable Parameters for Regression Models for information about the settings available in the Model Generator dialog.

9. Click Generate.

   After processing is complete, a confirmation message appears.

10. Close the Model Generator dialog.
11. Select your new model in the Model list.

    Information about the loaded model appears in the dialog (see Details), while a graph view of the data flow is available on the Model Editing panel (see Model Editing Panel).

12. Continue to the topic Training Denoising Models to learn how to train your new model for denoising.

You can start training a model for denoising after you have prepared your training input(s) and output(s), as well as any required masks (see Prerequisites for Training Denoising Models).

1. Open the Deep Learning Tool, if it is not already onscreen.

   To open the Deep Learning Tool, choose Artificial Intelligence > Deep Learning Tool on the menu bar.

2. Do one of the following, as required:
   • Generate a new model for denoising (see Generating Deep Models for Denoising).
   • Select a model from the Model list that contains the required architecture, number of inputs, and input dimension.
   • Import a model from Keras (see Keras models that your import into Dragonfly's Deep Learning Tool must meet the following requirements:).
3. Click the Go to Editing button to edit the architecture of the selected model, if required (see Model Editing Panel).

   Note In most cases, you should be able to train a denoising model supplied with the Deep Learning Tool as is, without making changes to its architecture.

4. Click the Go to Training button at the bottom of the dialog.

   The Model Training panel appears (see Model Training Panel).

5. Do the following on the Inputs tab for each set of training data that you want to train the model with:
   • Choose your input dataset in the Input drop-down menu.

     Note If your model requires multiple inputs, select the additional input(s), as required.

   • Choose your output dataset in the Output drop-down menu.

     

     Note Click the Edit Calibration button to open the Normalization dialog, in which you can normalize calibrated and uncalibrated datasets prior to training, optional (see Normalizing Data Ranges Prior to Training or Inference)

   • Choose a mask in the Mask drop-down menu, optional (see Applying Masks).

   Note If you are training with multiple training sets, click the Add New button and then choose the required input(s), output, and mask for the additional item(s).

   

6. Do the following, as required.
   • Adjust the data augmentation settings, if required (see Data Augmentation Settings).

     In most cases, you can deselect the Generate additional training data by augmentation when training denoising models.

   • Adjust the validation settings, if required (see Validation Settings).
7. Click the Training Parameters tab and then choose the required settings.

   In most cases, you should increase the Patch size as much as possible. In addition, the MeanSquareError loss function usually provides good results.

   

   See Basic Settings for information about choosing the patch size, stride ration, batch size, epochs number, loss function, and optimization algorithm.

   Note You should monitor the estimated memory ratio when you choose the training parameter settings. The ratio should not exceed 1.00 (see Estimated Memory Ratio).

8. If required, check the Show advanced settings option and then choose the advanced training parameters (see Advanced Settings).

   You should note that this step is optional and that these settings can be adjusted after you have evaluated the initial training results.

9. Click the Train button.

   You can monitor the progress of training in the Training dialog, which is shown below.

   

   During training, the quantities 'loss' and 'val_loss' should decrease. You should continue to train until 'val_loss' stops decreasing. You can also select any of the other available metrics to monitor training progress.

   Note You can also click the List tab and then review the precise values for each epoch.

10. Wait for training the completed. You can also stop training at anytime, if required.
11. Evaluate the results for the training session, recommended (see Evaluating Training Results).
12. Preview the results of applying the trained model to the input dataset or to the source high-noise dataset to evaluate the model, recommended (see Previewing Model Inference).

    Note The measure of good denoising is how well detail is preserved while noise is removed.

13. If the results are not satisfactory, you should consider doing one or more of the following and then retraining the model:
    • Add additional training data.
    • Adjust the data augmentation settings (see Data Augmentation Settings).
    • Adjust the training parameter settings (see Basic Settings and Advanced Settings).

    Note If your results continue to be unsatisfactory, you might consider choosing another architecture.

14. Process the original dataset or similar datasets, as required (see Applying Deep Models).