Custom registration example ¶

This tutorial whill show how to implement a custom registration model and a custom similarity measure.

Contents

Custom registration example

Introduction ¶

Following pytorch conventions all images in mermaid have to be in the format BxCxXxYxZ (BxCxX in 1D and BxCxXxY in 2D) I.e., in 1D, 2D, 3D we are dealing with 3D, 4D, 5D tensors. B is the batchsize, and C are the channels (for example to support color-images or general multi-modal registration scenarios)

Importing modules ¶

We start by importing some necessary modules

The used mermaid modules are as follows:

mermaid.example_generation allows to generate simply synthetic data and some real image pairs to test the registration algorithms
mermaid.module_parameters allows generating mermaid parameter structures which are used to keep track of all the parameters
mermaid.smoother_factory allows to generate various types of smoothers for images and vector fields
mermaid.data_wrappter allows to put tensors into the right data format (for example CPU/GPU via AdaptVal; makes use of the mermaid setting files for configuration)
mermaid.multiscale_optimizer allows single and multi-scale optimization of all the different registration models
mermaid.load_default_settings is a module from which various default settings can be loaded (without the need for providing your own json configuration file; these are written from the mermaid configuration setting files, but one can overwrite them by placing modified copies in .mermaid_settings).

# standard modules for str, torch, and numpy
from __future__ import print_function
from builtins import str

# first do the torch imports
import torch
import numpy as np

# the mermaid modules
import mermaid.example_generation as eg
import mermaid.module_parameters as pars
import mermaid.smoother_factory as SF
from mermaid.data_wrapper import AdaptVal
import mermaid.multiscale_optimizer as MO
import mermaid.load_default_settings as ds

Specifying settings ¶

Before we start we name our own new model, load some default parameters to configure the registration algorithm and settings from where to load data

# general parameters
model_name = 'mySVFNet'

# creating some initial parameters
params = pars.ParameterDict(ds.par_algconf)

# these are the default parameters we loaded
print(params)

Out:

ext =
{
    "image_smoothing": {
        "smooth_images": true,
        "smoother": {
            "gaussian_std": 0.01,
            "type": "gaussian"
        }
    },
    "model": {
        "deformation": {
            "map_low_res_factor": 1.0,
            "name": "lddmm_shooting",
            "use_map": true
        },
        "registration_model": {
            "forward_model": {
                "number_of_time_steps": 10,
                "smoother": {
                    "gaussian_std": 0.15,
                    "type": "gaussian"
                }
            },
            "similarity_measure": {
                "develop_mod": {
                    "smoother": {
                        "gaussian_std": 0.1,
                        "type": "gaussian"
                    }
                },
                "develop_mod_on": false,
                "sigma": 0.1,
                "type": "ssd"
            }
        }
    },
    "optimizer": {
        "multi_scale": {
            "scale_factors": [
                1.0,
                0.5,
                0.25
            ],
            "scale_iterations": [
                10,
                20,
                20
            ],
            "use_multiscale": false
        },
        "name": "lbfgs_ls",
        "single_scale": {
            "nr_of_iterations": 20
        }
    }
}
int =
{}
com =
{}
currentCategoryName = root

Creating some data ¶

We are now ready to create some data. Either synthetic or real, depending on what was specified via the configuration file.

if ds.load_settings_from_file:
    settingFile = 'test_custom_registration_' + model_name + '_settings.json'
    params.load_JSON(settingFile)

if ds.use_real_images:
    I0,I1,spacing = eg.CreateRealExampleImages(ds.dim).create_image_pair()

else:
    szEx = np.tile( 50, ds.dim )         # size of the desired images: (sz)^dim

    params['square_example_images']=({},'Settings for example image generation')
    params['square_example_images']['len_s'] = int(szEx.min()//6)
    params['square_example_images']['len_l'] = int(szEx.max()//4)

    # create a default image size with two sample squares
    I0,I1,spacing= eg.CreateSquares(ds.dim).create_image_pair(szEx,params)

sz = np.array(I0.shape)

assert( len(sz)==ds.dim+2 )

print ('Spacing = ' + str( spacing ) )

# create the source and target image as pyTorch variables
ISource = AdaptVal(torch.from_numpy( I0.copy() ))
ITarget = AdaptVal(torch.from_numpy( I1 ))

# if desired we smooth them a little bit
if ds.smooth_images:
    # smooth both a little bit
    params['image_smoothing'] = ds.par_algconf['image_smoothing']
    cparams = params['image_smoothing']
    s = SF.SmootherFactory(sz[2::], spacing).create_smoother(cparams)
    ISource = s.smooth(ISource)
    ITarget = s.smooth(ITarget)

Out:

Could not open file = test_custom_registration_mySVFNet_settings.json; ignoring request.
Creating new category: root.square_example_images
Creating key = len_s; category = root.square_example_images; value = 8
Creating key = len_l; category = root.square_example_images; value = 12
Spacing = [0.02040816 0.02040816]
Using default value = 0.001 for key = gaussian_std_min of category = root.image_smoothing.smoother

Setting up the optimizer ¶

We instantiate the multi-scale optimizer, which requires knowledge about image size and spacing, as well as if a map will be used for computation (for this example we do not use one).

# this custom registration algorithm does not use a map, so set it to False
use_map = False
# If a map would be used we could compute at a lower resolution internally.
map_low_res_factor = None
# Instantiate the multi-scale optimizer
mo = MO.MultiScaleRegistrationOptimizer(sz,spacing,use_map,map_low_res_factor,params)

Out:

Overwriting key = use_map; category = root.model.deformation; value =  True -> False
Overwriting key = map_low_res_factor; category = root.model.deformation; value =  1.0 -> None
Using default value = False for key = compute_similarity_measure_at_low_res of category = root.model.deformation
Using default value = 0.0001 for key = rel_ftol of category = root.optimizer.single_scale
Using default value = 1 for key = spline_order of category = root.model.registration_model

Specifiying a custom similarity measure ¶

We first implement a custom similarity measure. We just pretend SSD would not already exist and reimplement it.

Since we specify a new similarity measure we derive it from the general similiary measure class as specified in mermaid.similarity_measure_factory.

# we name our own model ``mySSD`` and add it to the parameter object, so that the model will know which similarity measure to use
params['registration_model']['similarity_measure']['type'] = 'mySSD'

import mermaid.similarity_measure_factory as SM

# this implements the similarity measure (I0Source and phi will most often not be used, but are useful in some special cases)
class MySSD(SM.SimilarityMeasure):
    def compute_similarity(self,I0,I1, I0Source=None, phi=None):
        print('Computing my SSD')
        return ((I0 - I1) ** 2).sum() / (0.1**2) * self.volumeElement

Out:

Creating new category: root.registration_model
Creating new category: root.registration_model.similarity_measure
Creating key = type; category = root.registration_model.similarity_measure; value = mySSD

Specifying a registration model ¶

We are now ready to specify the registration model itself. Here we implment a stationary velocity field which directly advects the image (given this velocity field), i.e., no map is used.

We make use of various mermaid modules here

mermaid.registration_networks: This module contains all the existing registration methods as well as the base classes from which we will derive our own model.
mermaid.utils: This module contains various utility functions. E.g., to apply maps, deal with registration parameter dictionaries, etc.
mermaid.image_sampling: This module is used to resample images (for example to higher or lower resolution as needed by a multi-scale approach).
mermaid.rungekutta_integrators: This module contains Runge-Kutta integrators to integrate differential equations/
mermaid.forward_models: This module contains various forward models. For example also an advection model, which we use here.
mermai.regularizer_factory: This module implements various regularizers.

import mermaid.registration_networks as RN
import mermaid.utils as utils
import mermaid.image_sampling as IS
import mermaid.rungekutta_integrators as RK
import mermaid.forward_models as FM
import mermaid.regularizer_factory as RF

# We define our own image-based SVF class
class MySVFNet(RN.RegistrationNetTimeIntegration):
    def __init__(self,sz,spacing,params):
        super(MySVFNet, self).__init__(sz,spacing,params)
        # we create the parameters (here simply a vector field)
        self.v = self.create_registration_parameters()
        # and an integrator to integrate the advection equation
        self.integrator = self.create_integrator()

    def create_registration_parameters(self):
        """
        Creates the vector field over which we optimize
        """
        return utils.create_ND_vector_field_parameter_multiN(self.sz[2::], self.nrOfImages)

    def get_registration_parameters(self):
        """
        Returns the vector field over which we optimize
        """
        return self.v

    def set_registration_parameters(self, p, sz, spacing):
        """
        Allows setting the registration parameters. This is neede for the multi-scale solver to go between the scales.

        :param p: Registration parameter (here the vector field)
        :param sz: size at current scale
        :param spacing: spacing at current scale
        :return: n/a
        """
        self.v.data = p.data
        self.sz = sz
        self.spacing = spacing

    def create_integrator(self):
        """
        Creates an instance of the integrator (here RK4)
        """
        # here is where parameters for the forward model are always stored
        cparams = self.params[('forward_model',{},'settings for the forward model')]
        # we create an advection forward model
        advection = FM.AdvectImage(self.sz, self.spacing)
        # create the parameters to be passed to the integrator as a dictionary (including default parameters that can be passed)
        pars_to_pass = utils.combine_dict({'v': self.v}, self._get_default_dictionary_to_pass_to_integrator())
        # now we create the integrator and return it
        return RK.RK4(advection.f, advection.u, pars_to_pass, cparams)

    def forward(self, I, variables_from_optimizer=None):
        """
        The forward method simply applies the current registration transform (here integrates the advection equation)

        :param I: input image
        :param variable_from_optimizer: additional parameters that can be passed from the optimizer
        :return: returns the warped image I
        """

        # as we derived our class from RegistrationNetTimeIntegration we have access to member variables tFrom and tTo
        # specifying the desired integration time interval
        I1 = self.integrator.solve([I], self.tFrom, self.tTo)
        return I1[0]

    def upsample_registration_parameters(self, desiredSz):
        """
        Upsamples the registration parameters (needed for multi-scale solver).

        :param desiredSz: desired size to which we want to upsample.
        :return: returns the upsampled parameter (the upsampled velocity field) and the corresponding spacing as a tuple
        """
        sampler = IS.ResampleImage()
        vUpsampled,upsampled_spacing=sampler.upsample_image_to_size(self.v,self.spacing,desiredSz,spline_order=1)
        return vUpsampled,upsampled_spacing

    def downsample_registration_parameters(self, desiredSz):
        """
        Downsamples the registration parameters (needed for multi-scale solver).

        :param desiredSz: desired size to which we want to downsample.
        :return: returns the downsampled parameter (the upsampled velocity field) and the corresponding spacing as a tuple
        """
        sampler = IS.ResampleImage()
        vDownsampled,downsampled_spacing=sampler.downsample_image_to_size(self.v,self.spacing,desiredSz,spline_order=1)
        return vDownsampled,downsampled_spacing

Specifying the loss function ¶

Lastly, we write out custom loss function to penalize deformations that are not smooth and image mismatch. As we already defined the similarity measure (which will get called behind the scenes) we only need to deal with implementing the regularization energy here.

We derive this class from RegistrationImageLoss.

class MySVFImageLoss(RN.RegistrationImageLoss):
    def __init__(self,v,sz_sim,spacing_sim,sz_model,spacing_model,params):
        super(MySVFImageLoss, self).__init__(sz_sim,spacing_sim,sz_model,spacing_model,params)
        # the registration parameters
        self.v = v
        # create a new parameter category
        cparams = params[('loss',{},'settings for the loss function')]
        # under this category the regularizer settings will be saved
        self.regularizer = (RF.RegularizerFactory(self.spacing_sim).
                            create_regularizer(cparams))

    def compute_regularization_energy(self, I0_source, variables_from_forward_model=None, variables_from_optimizer=False):
        # here we compute the regularization energy, by simply evaluating the regularizer
        return self.regularizer.compute_regularizer_multiN(self.v)

Multi-scale optimizing our own custom model ¶

We are now ready to optimize over our model. The only thing left is to associate the similarity measure, the registration model, and the loss function with the multi-scale optimizer. Note that we only had to implement upsampling and downsampling methods for the registration parameters. THe entire multi-scale framework now comes for free

# We have now created our own similiary measure (``MySSD``). We know register it with the multi-scale optimizer, so it knows it exists.
mo.add_similarity_measure('mySSD', MySSD)

# We do the same with the registration model and its loss
mo.add_model(model_name,MySVFNet,MySVFImageLoss,use_map=False)
# and set the model name, so we can refer to it
mo.set_model(model_name)

# here we set some visualization options (if to visualize and how frequently)
mo.set_visualization( ds.visualize )
mo.set_visualize_step( ds.visualize_step )

# we let the optimizer know what is the source and what is the target image
mo.set_source_image(ISource)
mo.set_target_image(ITarget)

# we set the scale factors and the number of iterations
mo.set_scale_factors( ds.multi_scale_scale_factors )
mo.set_number_of_iterations_per_scale( ds.multi_scale_iterations_per_scale )

# and while we are at it we also pick a custom optimizer
mo.set_optimizer(torch.optim.Adam)
mo.set_optimizer_params(dict(lr=0.01))

# and now we do the optimization

mo.optimize()

# If desired we can write out the registration parameters. Which keeps track of all parameters that were used.
# These parameters are as follows (and we can see that out own model was used).
#

print(params)

if ds.save_settings_to_file:
    params.write_JSON( 'test_custom_registration_' + model_name + '_settings_clean.json')
    params.write_JSON_comments( 'test_custom_registration_' + model_name + '_settings_comments.json')

../_images/sphx_glr_example_custom_registration_001.png

Out:

Overwriting key = scale_factors; category = root.optimizer.multi_scale; value =  [1.0, 0.5, 0.25] -> [1.0, 0.5, 0.25]
Overwriting key = scale_iterations; category = root.optimizer.multi_scale; value =  [10, 20, 20] -> [10, 20, 20]
Performing multiscale optmization with scales: [1.0, 0.5, 0.25]
Optimizing for scale = 0.25
Overwriting key = use_map; category = root.model.deformation; value =  False -> False
Overwriting key = map_low_res_factor; category = root.model.deformation; value =  None -> None
Using default value = none for key = weight_clipping_type of category = root.optimizer
Using default value = 1.0 for key = weight_clipping_value of category = root.optimizer
Creating new category: root.optimizer.gradient_clipping
Using default value = True for key = clip_display of category = root.optimizer.gradient_clipping
Using default value = False for key = clip_individual_gradient of category = root.optimizer.gradient_clipping
Using default value = 1.0909090909090908 for key = clip_individual_gradient_value of category = root.optimizer.gradient_clipping
Using default value = True for key = clip_shared_gradient of category = root.optimizer.gradient_clipping
Using default value = 1.0 for key = clip_shared_gradient_value of category = root.optimizer.gradient_clipping
Setting learning rate to None
Registering new model mySVFNet
Creating key = type; category = root.model.registration_model; value = mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Creating new category: root.model.registration_model.env
Using default value = False for key = get_momentum_from_external_network of category = root.model.registration_model.env
Using mySVFNet model
Using default value = True for key = use_CFL_clamping of category = root.model.registration_model
Using default value = True for key = use_odeint of category = root.model.registration_model.env
Using default value = False for key = use_ode_tuple of category = root.model.registration_model.env
works in mermaid iter mode
Using default value = 1.0 for key = reg_factor of category = root.model.registration_model.env
Creating new category: root.model.registration_model.loss
Creating new category: root.model.registration_model.loss.regularizer
Using default value = helmholtz for key = type of category = root.model.registration_model.loss.regularizer
Using default value = 0.2 for key = alpha of category = root.model.registration_model.loss.regularizer
Using default value = 1.0 for key = gamma of category = root.model.registration_model.loss.regularizer
MySVFNet()
Registering new similarity measure mySSD in factory
Overwriting key = type; category = root.model.registration_model.similarity_measure; value =  ssd -> mySSD
Overwriting key = rel_ftol; category = root.optimizer.single_scale; value =  0.0001 -> 0.0001
Optimizing for at most 20 iterations
Warning: optimizer name = lbfgs_ls specified, but ignored since optimizer was set explicitly
Using default value = True for key = use_step_size_scheduler of category = root.optimizer
Creating new category: root.optimizer.scheduler
Using default value = True for key = verbose of category = root.optimizer.scheduler
Using default value = 0.5 for key = factor of category = root.optimizer.scheduler
Using default value = 10 for key = patience of category = root.optimizer.scheduler
Using mySSD similarity measure
Computing my SSD
[31m    0-Tot: E=014.1411 | simE=014.1411 | regE=000.0000 | optParE=000.0000 | relF=  n/a    | [0m
[34m    0-Img: E=014.1411 | simE=014.1411 | regE=000.0000 |[0m
Computing my SSD
[31m    1-Tot: E=015.9233 | simE=012.8073 | regE=003.1160 | optParE=000.0000 | relF=000.1053 | [0m
[34m    1-Img: E=015.9233 | simE=012.8073 | regE=003.1160 |[0m
Computing my SSD
[31m    2-Tot: E=012.1585 | simE=011.5857 | regE=000.5728 | optParE=000.0000 | relF=000.2861 | [0m
[34m    2-Img: E=012.1585 | simE=011.5857 | regE=000.5728 |[0m
Computing my SSD
[31m    3-Tot: E=011.5737 | simE=010.3930 | regE=001.1807 | optParE=000.0000 | relF=000.0465 | [0m
[34m    3-Img: E=011.5737 | simE=010.3930 | regE=001.1807 |[0m
Computing my SSD
[31m    4-Tot: E=011.4163 | simE=009.2816 | regE=002.1347 | optParE=000.0000 | relF=000.0127 | [0m
[34m    4-Img: E=011.4163 | simE=009.2816 | regE=002.1347 |[0m
Computing my SSD
[31m    5-Tot: E=010.2479 | simE=008.2459 | regE=002.0020 | optParE=000.0000 | relF=000.1039 | [0m
[34m    5-Img: E=010.2479 | simE=008.2459 | regE=002.0020 |[0m
Computing my SSD
[31m    6-Tot: E=008.8436 | simE=007.2693 | regE=001.5743 | optParE=000.0000 | relF=000.1427 | [0m
[34m    6-Img: E=008.8436 | simE=007.2693 | regE=001.5743 |[0m
Computing my SSD
[31m    7-Tot: E=007.8801 | simE=006.3656 | regE=001.5145 | optParE=000.0000 | relF=000.1085 | [0m
[34m    7-Img: E=007.8801 | simE=006.3656 | regE=001.5145 |[0m
Computing my SSD
[31m    8-Tot: E=007.3935 | simE=005.5518 | regE=001.8417 | optParE=000.0000 | relF=000.0580 | [0m
[34m    8-Img: E=007.3935 | simE=005.5518 | regE=001.8417 |[0m
Computing my SSD
[31m    9-Tot: E=007.0266 | simE=004.8402 | regE=002.1864 | optParE=000.0000 | relF=000.0457 | [0m
[34m    9-Img: E=007.0266 | simE=004.8402 | regE=002.1864 |[0m
Computing my SSD
[31m   10-Tot: E=006.5307 | simE=004.2374 | regE=002.2933 | optParE=000.0000 | relF=000.0658 | [0m
[34m   10-Img: E=006.5307 | simE=004.2374 | regE=002.2933 |[0m
Computing my SSD
[31m   11-Tot: E=005.9588 | simE=003.7432 | regE=002.2156 | optParE=000.0000 | relF=000.0822 | [0m
[34m   11-Img: E=005.9588 | simE=003.7432 | regE=002.2156 |[0m
Computing my SSD
[31m   12-Tot: E=005.4819 | simE=003.3512 | regE=002.1307 | optParE=000.0000 | relF=000.0736 | [0m
[34m   12-Img: E=005.4819 | simE=003.3512 | regE=002.1307 |[0m
Computing my SSD
[31m   13-Tot: E=005.1951 | simE=003.0498 | regE=002.1453 | optParE=000.0000 | relF=000.0463 | [0m
[34m   13-Img: E=005.1951 | simE=003.0498 | regE=002.1453 |[0m
Computing my SSD
[31m   14-Tot: E=005.0620 | simE=002.8254 | regE=002.2366 | optParE=000.0000 | relF=000.0220 | [0m
[34m   14-Img: E=005.0620 | simE=002.8254 | regE=002.2366 |[0m
Computing my SSD
[31m   15-Tot: E=004.9772 | simE=002.6648 | regE=002.3124 | optParE=000.0000 | relF=000.0142 | [0m
[34m   15-Img: E=004.9772 | simE=002.6648 | regE=002.3124 |[0m
Computing my SSD
[31m   16-Tot: E=004.8676 | simE=002.5548 | regE=002.3128 | optParE=000.0000 | relF=000.0187 | [0m
[34m   16-Img: E=004.8676 | simE=002.5548 | regE=002.3128 |[0m
Computing my SSD
[31m   17-Tot: E=004.7350 | simE=002.4835 | regE=002.2515 | optParE=000.0000 | relF=000.0231 | [0m
[34m   17-Img: E=004.7350 | simE=002.4835 | regE=002.2515 |[0m
Computing my SSD
[31m   18-Tot: E=004.6242 | simE=002.4419 | regE=002.1824 | optParE=000.0000 | relF=000.0197 | [0m
[34m   18-Img: E=004.6242 | simE=002.4419 | regE=002.1824 |[0m
Computing my SSD
[31m   19-Tot: E=004.5686 | simE=002.4241 | regE=002.1445 | optParE=000.0000 | relF=000.0100 | [0m
[34m   19-Img: E=004.5686 | simE=002.4241 | regE=002.1445 |[0m
[32m-->Elapsed time 3.02112[s][0m
Before
[ 1  1 25 25]
Optimizing for scale = 0.5
Overwriting key = use_map; category = root.model.deformation; value =  False -> False
Overwriting key = map_low_res_factor; category = root.model.deformation; value =  None -> None
Setting learning rate to None
Registering new model mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Using mySVFNet model
works in mermaid iter mode
MySVFNet()
Registering new similarity measure mySSD in factory
Overwriting key = type; category = root.model.registration_model.similarity_measure; value =  mySSD -> mySSD
Overwriting key = rel_ftol; category = root.optimizer.single_scale; value =  0.0001 -> 0.0001
Explicitly setting the optimization parameters
Optimizing for at most 20 iterations
Warning: optimizer name = lbfgs_ls specified, but ignored since optimizer was set explicitly
Setting model parameters, delayed
Using mySSD similarity measure
Computing my SSD
[31m    0-Tot: E=003.5237 | simE=001.3409 | regE=002.1828 | optParE=000.0000 | relF=  n/a    | [0m
[34m    0-Img: E=003.5237 | simE=001.3409 | regE=002.1828 |[0m
Computing my SSD
[31m    1-Tot: E=033.4499 | simE=001.2269 | regE=032.2231 | optParE=000.0000 | relF=000.8687 | [0m
[34m    1-Img: E=033.4499 | simE=001.2269 | regE=032.2231 |[0m
Computing my SSD
[31m    2-Tot: E=019.2692 | simE=001.2187 | regE=018.0505 | optParE=000.0000 | relF=000.6996 | [0m
[34m    2-Img: E=019.2692 | simE=001.2187 | regE=018.0505 |[0m
Computing my SSD
[31m    3-Tot: E=015.4519 | simE=001.1785 | regE=014.2734 | optParE=000.0000 | relF=000.2320 | [0m
[34m    3-Img: E=015.4519 | simE=001.1785 | regE=014.2734 |[0m
Computing my SSD
[31m    4-Tot: E=014.9793 | simE=001.1430 | regE=013.8363 | optParE=000.0000 | relF=000.0296 | [0m
[34m    4-Img: E=014.9793 | simE=001.1430 | regE=013.8363 |[0m
Computing my SSD
[31m    5-Tot: E=012.8859 | simE=001.1175 | regE=011.7684 | optParE=000.0000 | relF=000.1508 | [0m
[34m    5-Img: E=012.8859 | simE=001.1175 | regE=011.7684 |[0m
Computing my SSD
[31m    6-Tot: E=011.2947 | simE=001.1038 | regE=010.1909 | optParE=000.0000 | relF=000.1294 | [0m
[34m    6-Img: E=011.2947 | simE=001.1038 | regE=010.1909 |[0m
Computing my SSD
[31m    7-Tot: E=010.2561 | simE=001.0815 | regE=009.1746 | optParE=000.0000 | relF=000.0923 | [0m
[34m    7-Img: E=010.2561 | simE=001.0815 | regE=009.1746 |[0m
Computing my SSD
[31m    8-Tot: E=009.2630 | simE=001.0468 | regE=008.2163 | optParE=000.0000 | relF=000.0968 | [0m
[34m    8-Img: E=009.2630 | simE=001.0468 | regE=008.2163 |[0m
Computing my SSD
[31m    9-Tot: E=008.4002 | simE=001.0064 | regE=007.3938 | optParE=000.0000 | relF=000.0918 | [0m
[34m    9-Img: E=008.4002 | simE=001.0064 | regE=007.3938 |[0m
Computing my SSD
[31m   10-Tot: E=007.5602 | simE=000.9652 | regE=006.5950 | optParE=000.0000 | relF=000.0981 | [0m
[34m   10-Img: E=007.5602 | simE=000.9652 | regE=006.5950 |[0m
Computing my SSD
Epoch    11: reducing learning rate of group 0 to 5.0000e-03.
[31m   11-Tot: E=006.7108 | simE=000.9266 | regE=005.7841 | optParE=000.0000 | relF=000.1102 | [0m
[34m   11-Img: E=006.7108 | simE=000.9266 | regE=005.7841 |[0m
Computing my SSD
[31m   12-Tot: E=006.0397 | simE=000.8933 | regE=005.1465 | optParE=000.0000 | relF=000.0953 | [0m
[34m   12-Img: E=006.0397 | simE=000.8933 | regE=005.1465 |[0m
Computing my SSD
[31m   13-Tot: E=005.5323 | simE=000.8796 | regE=004.6527 | optParE=000.0000 | relF=000.0777 | [0m
[34m   13-Img: E=005.5323 | simE=000.8796 | regE=004.6527 |[0m
Computing my SSD
[31m   14-Tot: E=004.7474 | simE=000.8691 | regE=003.8783 | optParE=000.0000 | relF=000.1366 | [0m
[34m   14-Img: E=004.7474 | simE=000.8691 | regE=003.8783 |[0m
Computing my SSD
[31m   15-Tot: E=004.0304 | simE=000.8623 | regE=003.1682 | optParE=000.0000 | relF=000.1425 | [0m
[34m   15-Img: E=004.0304 | simE=000.8623 | regE=003.1682 |[0m
Computing my SSD
[31m   16-Tot: E=003.5740 | simE=000.8595 | regE=002.7145 | optParE=000.0000 | relF=000.0998 | [0m
[34m   16-Img: E=003.5740 | simE=000.8595 | regE=002.7145 |[0m
Computing my SSD
[31m   17-Tot: E=003.3733 | simE=000.8608 | regE=002.5125 | optParE=000.0000 | relF=000.0459 | [0m
[34m   17-Img: E=003.3733 | simE=000.8608 | regE=002.5125 |[0m
Computing my SSD
[31m   18-Tot: E=003.3213 | simE=000.8655 | regE=002.4558 | optParE=000.0000 | relF=000.0120 | [0m
[34m   18-Img: E=003.3213 | simE=000.8655 | regE=002.4558 |[0m
Computing my SSD
[31m   19-Tot: E=003.2988 | simE=000.8729 | regE=002.4259 | optParE=000.0000 | relF=000.0052 | [0m
[34m   19-Img: E=003.2988 | simE=000.8729 | regE=002.4259 |[0m
[32m-->Elapsed time 3.16497[s][0m
Before
[ 1  1 50 50]
Optimizing for scale = 1.0
Overwriting key = use_map; category = root.model.deformation; value =  False -> False
Overwriting key = map_low_res_factor; category = root.model.deformation; value =  None -> None
Setting learning rate to None
Registering new model mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Overwriting key = type; category = root.model.registration_model; value =  mySVFNet -> mySVFNet
Using mySVFNet model
works in mermaid iter mode
MySVFNet()
Registering new similarity measure mySSD in factory
Overwriting key = type; category = root.model.registration_model.similarity_measure; value =  mySSD -> mySSD
Overwriting key = rel_ftol; category = root.optimizer.single_scale; value =  0.0001 -> 0.0001
Explicitly setting the optimization parameters
Optimizing for at most 10 iterations
Warning: optimizer name = lbfgs_ls specified, but ignored since optimizer was set explicitly
Setting model parameters, delayed
Using mySSD similarity measure
Computing my SSD
[31m    0-Tot: E=003.6770 | simE=001.2173 | regE=002.4598 | optParE=000.0000 | relF=  n/a    | [0m
[34m    0-Img: E=003.6770 | simE=001.2173 | regE=002.4598 |[0m
Computing my SSD
[31m    1-Tot: E=602.4175 | simE=001.2471 | regE=601.1705 | optParE=000.0000 | relF=000.9922 | [0m
[34m    1-Img: E=602.4175 | simE=001.2471 | regE=601.1705 |[0m
Computing my SSD
[31m    2-Tot: E=282.9160 | simE=001.2364 | regE=281.6797 | optParE=000.0000 | relF=001.1253 | [0m
[34m    2-Img: E=282.9160 | simE=001.2364 | regE=281.6797 |[0m
Computing my SSD
[31m    3-Tot: E=246.4863 | simE=001.2705 | regE=245.2158 | optParE=000.0000 | relF=000.1472 | [0m
[34m    3-Img: E=246.4863 | simE=001.2705 | regE=245.2158 |[0m
Computing my SSD
[31m    4-Tot: E=239.5429 | simE=001.3098 | regE=238.2331 | optParE=000.0000 | relF=000.0289 | [0m
[34m    4-Img: E=239.5429 | simE=001.3098 | regE=238.2331 |[0m
Computing my SSD
[31m    5-Tot: E=206.4008 | simE=001.3107 | regE=205.0901 | optParE=000.0000 | relF=000.1598 | [0m
[34m    5-Img: E=206.4008 | simE=001.3107 | regE=205.0901 |[0m
Computing my SSD
[31m    6-Tot: E=177.6915 | simE=001.2897 | regE=176.4019 | optParE=000.0000 | relF=000.1607 | [0m
[34m    6-Img: E=177.6915 | simE=001.2897 | regE=176.4019 |[0m
Computing my SSD
[31m    7-Tot: E=145.0963 | simE=001.2648 | regE=143.8315 | optParE=000.0000 | relF=000.2231 | [0m
[34m    7-Img: E=145.0963 | simE=001.2648 | regE=143.8315 |[0m
Computing my SSD
[31m    8-Tot: E=125.5647 | simE=001.2452 | regE=124.3194 | optParE=000.0000 | relF=000.1543 | [0m
[34m    8-Img: E=125.5647 | simE=001.2452 | regE=124.3194 |[0m
Computing my SSD
[31m    9-Tot: E=119.6125 | simE=001.2309 | regE=118.3816 | optParE=000.0000 | relF=000.0493 | [0m
[34m    9-Img: E=119.6125 | simE=001.2309 | regE=118.3816 |[0m
[32m-->Elapsed time 1.72455[s][0m
ext =
{
    "image_smoothing": {
        "smooth_images": true,
        "smoother": {
            "gaussian_std": 0.01,
            "gaussian_std_min": 0.001,
            "type": "gaussian"
        }
    },
    "model": {
        "deformation": {
            "compute_similarity_measure_at_low_res": false,
            "map_low_res_factor": null,
            "name": "lddmm_shooting",
            "use_map": false
        },
        "registration_model": {
            "env": {
                "get_momentum_from_external_network": false,
                "reg_factor": 1.0,
                "use_ode_tuple": false,
                "use_odeint": true
            },
            "forward_model": {
                "number_of_time_steps": 10,
                "smoother": {
                    "gaussian_std": 0.15,
                    "type": "gaussian"
                }
            },
            "loss": {
                "regularizer": {
                    "alpha": 0.2,
                    "gamma": 1.0,
                    "type": "helmholtz"
                }
            },
            "similarity_measure": {
                "develop_mod": {
                    "smoother": {
                        "gaussian_std": 0.1,
                        "type": "gaussian"
                    }
                },
                "develop_mod_on": false,
                "sigma": 0.1,
                "type": "mySSD"
            },
            "spline_order": 1,
            "type": "mySVFNet",
            "use_CFL_clamping": true
        }
    },
    "optimizer": {
        "gradient_clipping": {
            "clip_display": true,
            "clip_individual_gradient": false,
            "clip_individual_gradient_value": 1.0909090909090908,
            "clip_shared_gradient": true,
            "clip_shared_gradient_value": 1.0
        },
        "multi_scale": {
            "scale_factors": [
                1.0,
                0.5,
                0.25
            ],
            "scale_iterations": [
                10,
                20,
                20
            ],
            "use_multiscale": false
        },
        "name": "lbfgs_ls",
        "scheduler": {
            "factor": 0.5,
            "patience": 10,
            "verbose": true
        },
        "single_scale": {
            "nr_of_iterations": 20,
            "rel_ftol": 0.0001
        },
        "use_step_size_scheduler": true,
        "weight_clipping_type": "none",
        "weight_clipping_value": 1.0
    },
    "registration_model": {
        "similarity_measure": {
            "type": "mySSD"
        }
    },
    "square_example_images": {
        "len_l": 12,
        "len_s": 8
    }
}
int =
{
    "image_smoothing": {
        "smoother": {
            "gaussian_std": 0.01,
            "gaussian_std_min": 0.001,
            "type": "gaussian"
        }
    },
    "model": {
        "deformation": {
            "compute_similarity_measure_at_low_res": false,
            "map_low_res_factor": null,
            "use_map": false
        },
        "registration_model": {
            "env": {
                "get_momentum_from_external_network": false,
                "reg_factor": 1.0,
                "use_ode_tuple": false,
                "use_odeint": true
            },
            "forward_model": {
                "number_of_time_steps": 10
            },
            "loss": {
                "regularizer": {
                    "alpha": 0.2,
                    "gamma": 1.0,
                    "type": "helmholtz"
                }
            },
            "similarity_measure": {
                "sigma": 0.1,
                "type": "mySSD"
            },
            "spline_order": 1,
            "type": "mySVFNet",
            "use_CFL_clamping": true
        }
    },
    "optimizer": {
        "gradient_clipping": {
            "clip_display": true,
            "clip_individual_gradient": false,
            "clip_individual_gradient_value": 1.0909090909090908,
            "clip_shared_gradient": true,
            "clip_shared_gradient_value": 1.0
        },
        "multi_scale": {
            "scale_factors": [
                1.0,
                0.5,
                0.25
            ],
            "scale_iterations": [
                10,
                20,
                20
            ]
        },
        "name": "lbfgs_ls",
        "scheduler": {
            "factor": 0.5,
            "patience": 10,
            "verbose": true
        },
        "single_scale": {
            "rel_ftol": 0.0001
        },
        "use_step_size_scheduler": true,
        "weight_clipping_type": "none",
        "weight_clipping_value": 1.0
    },
    "registration_model": {
        "similarity_measure": {
            "type": "mySSD"
        }
    },
    "square_example_images": {
        "len_l": 12,
        "len_s": 8
    }
}
com =
{
    "image_smoothing": {
        "smoother": {
            "gaussian_std": "std for the Gaussian",
            "gaussian_std_min": "minimal allowed std for the Gaussian",
            "type": "type of smoother (diffusion|gaussian|adaptive_gaussian|multiGaussian|adaptive_multiGaussian|gaussianSpatial|adaptiveNet)"
        }
    },
    "model": {
        "__doc__": "general model settings",
        "deformation": {
            "__doc__": "model describing the desired deformation model",
            "compute_similarity_measure_at_low_res": "If set to true map is not upsampled and the entire computations proceeds at low res",
            "map_low_res_factor": "Set to a value in (0,1) if a map-based solution should be computed at a lower internal resolution (image matching is still at full resolution",
            "use_map": "[True|False] either do computations via a map or directly using the image"
        },
        "registration_model": {
            "__doc__": "general settings for the registration model",
            "env": {
                "__doc__": "env settings, typically are specificed by the external package, including the mode for solver or for smoother",
                "get_momentum_from_external_network": "use external network to predict momentum, notice that the momentum network is not built in this package",
                "reg_factor": "regularzation factor",
                "use_ode_tuple": "once use torchdiffeq package, take the tuple input or tensor input",
                "use_odeint": "using torchdiffeq package as the ode solver"
            },
            "forward_model": {
                "__doc__": "settings for the forward model",
                "number_of_time_steps": "Number of time-steps to per unit time-interval integrate the PDE"
            },
            "loss": {
                "__doc__": "settings for the loss function",
                "regularizer": {
                    "__doc__": "Parameters for the regularizer",
                    "alpha": "penalty for 2nd derivative",
                    "gamma": "penalty for magnitude",
                    "type": "type of regularizer (only helmholtz at the moment)"
                }
            },
            "similarity_measure": {
                "sigma": "1/sigma^2 is the weight in front of the similarity measure",
                "type": "type of similarity measure (ssd/ncc)"
            },
            "spline_order": "Spline interpolation order; 1 is linear interpolation (default); 3 is cubic spline",
            "type": "Name of the registration model",
            "use_CFL_clamping": "If the model uses time integration, CFL clamping is used"
        }
    },
    "optimizer": {
        "__doc__": "optimizer settings",
        "gradient_clipping": {
            "__doc__": "clipping settings for the gradient for optimization",
            "clip_display": "If set to True displays if clipping occurred",
            "clip_individual_gradient": "If set to True, the gradient for the individual parameters will be clipped",
            "clip_individual_gradient_value": "Value to which the gradient for the individual parameters is clipped",
            "clip_shared_gradient": "If set to True, the gradient for the shared parameters will be clipped",
            "clip_shared_gradient_value": "Value to which the gradient for the shared parameters is clipped"
        },
        "multi_scale": {
            "__doc__": "multi scale settings",
            "scale_factors": "how images are scaled",
            "scale_iterations": "number of iterations per scale"
        },
        "name": "Optimizer (lbfgs|adam|sgd)",
        "scheduler": {
            "__doc__": "parameters for the ReduceLROnPlateau scheduler",
            "factor": "reduction factor",
            "patience": "how many steps without reduction before LR is changed",
            "verbose": "if True prints out changes in learning rate"
        },
        "single_scale": {
            "rel_ftol": "relative termination tolerance for optimizer"
        },
        "use_step_size_scheduler": "If set to True the step sizes are reduced if no progress is made",
        "weight_clipping_type": "Type of weight clipping that should be used [l1|l2|l1_individual|l2_individual|l1_shared|l2_shared|None]",
        "weight_clipping_value": "Value to which the norm is being clipped"
    },
    "registration_model": {
        "similarity_measure": {}
    },
    "square_example_images": {
        "__doc__": "Settings for example image generation",
        "len_l": "Maximum side-length of square",
        "len_s": "Mimimum side-length of square"
    }
}
currentCategoryName = root

Writing parameter file = test_custom_registration_mySVFNet_settings_clean.json
Writing parameter file = test_custom_registration_mySVFNet_settings_comments.json

Conclusion ¶

Given this example it is possible to set up a custom registration model without too much overhead and without being exposed to the complex mermaid internals.

Total running time of the script: ( 0 minutes 8.350 seconds)

Gallery generated by Sphinx-Gallery