Qui est en ligne ?
Il y a en tout 3 utilisateurs en ligne :: 0 Enregistré, 0 Invisible et 3 Invités Aucun
Le record du nombre d'utilisateurs en ligne est de 325 le Lun 4 Nov 2019 - 23:38
Derniers sujets
Rechercher
Les posteurs les plus actifs du mois
Aucun utilisateur |
Marque-page social
Conservez et partagez l'adresse de Astron'amis sur votre site de social bookmarking
Conservez et partagez l'adresse de Astron'amis sur votre site de social bookmarking
k0 calibration Pix
3 participants
Astron'amis :: Pratique de l'astronomie amateur :: Astrophotographie :: Ciel profond :: Le coin des apprentis
Page 1 sur 1
k0 calibration Pix
Salut à tous,
juste une question, je voulais savoir quelle est la valeur à partir de laquelle la k0 lors de la calibration des brutes montre un soucis (je ne me souviens plus si c'est le masterdark ou la brute qui est en cause...)
Au stage Pix ils disaient que cela devait être autour de 0.9 ou 1.
J'ai des k0=0.2 comme j'en ai à 0.8, 0.9 ou même 1.8 etc...
Merci pour vos précisions, à quoi sert cette attention portée au k0 et dans quel but ? Comment puis je améliorer l'ensemble ?
Valentin
juste une question, je voulais savoir quelle est la valeur à partir de laquelle la k0 lors de la calibration des brutes montre un soucis (je ne me souviens plus si c'est le masterdark ou la brute qui est en cause...)
Au stage Pix ils disaient que cela devait être autour de 0.9 ou 1.
J'ai des k0=0.2 comme j'en ai à 0.8, 0.9 ou même 1.8 etc...
Merci pour vos précisions, à quoi sert cette attention portée au k0 et dans quel but ? Comment puis je améliorer l'ensemble ?
Valentin
Re: k0 calibration Pix
Salut Valentin
Je n'ai pas la réponse mais cela fait remonter le post et ça m'intéresse aussi
A+
Je n'ai pas la réponse mais cela fait remonter le post et ça m'intéresse aussi
A+
Re: k0 calibration Pix
Salut Valentin,
Cela dénote un problème dans la construction des masters dark et/ou Bias.
Plusieurs causes possibles:
Différences en temp, ISO ou exposition entre darks et light
Pas suffisamment de bias ou darks, amenant beaucoup de bruit sur les masters dark / bias
un ou plusieurs dark/bias mauvais
mauvais paramétrage des format explorer pour les DSLR
autres....
Il faudrait des copies d'écran, voire les fichiers pour déterminer ce qui cloche
Patrick
PS: maintenant, la theorie: ( desolé, en anglais)
Let's state the problem first. We may describe an uncalibrated raw light frame with this very simple equation:
I = I0 + D + B
where I is the uncalibrated raw light frame, D is the dark current, and B is the bias pedestal. All terms in this equation are vectors (or matrices) because we are describing a process for a whole image at the pixel level. I0 is the dark-and-bias-subtracted raw light frame (we know that the next calibration step would be flat fielding, but this is unimportant to this discussion). Our goal here is to obtain I0.
As we have stated it above, this is a trivial problem: just subtract good master bias and dark frames from the uncalibrated light frame, and the result will be a good approximation to I0 (the better the masters, the better the approximation). However, the actual problem is more complex because the dark current D is highly time- and temperature-dependent, so a perfect match between the dark current signals in I and D is only a theoretical goal. A better (but still far from perfect) approximation to the actual problem is more like this:
I = I0 + k*D + B
where k is a scaling factor that attempts to account for effective exposure time and temperature differences between the light frame and the master dark frame. The process of finding a good approximation to the scaling factor k is what we call dark frame optimization.
Note that I have said that this is far from a perfect solution. It is wrong mainly because the dark current does not vary linearly with exposure time and temperature for the whole numeric range of the data, so a single multiplicative factor cannot be very accurate, especially when time and temperature differences between I and D are large. Dark optimization, however, is always better than nothing. From our experience, dark optimization is beneficial even if exposure times and temperatures are matched between light and dark frames. For this reason the corresponding parameter is enabled by default in our ImageCalibration tool.
Now that we have described the problem, let's describe our solution. Algorithmically this is known as an optimization problem: find the value of a parameter that minimizes (or maximizes) the value of a function. For example, think in terms of the economical cost of a production process. We have to produce an item A that depends on a factor r, which leads to the problem: find the value of r that minimizes the cost of producing A. In real cases r usually represents a complex set of parameters and constraints, such as environmental factors, availability of prime matters, production systems, etc., so that the most difficult part of the solution often consists of identifying a significant set of parameters or model to define a suitable cost function. This leads to the subject of linear programming.
The dark frame optimization problem is a relatively simple case of function optimization with a single parameter. The first step is to define the cost function that we want to minimize. To explain why and how we have selected a particular function we need some visual analysis. Take a look at the following two images:
One of them is a crop of a bias-subtracted master dark frame, the other is not. Difficult to say which is which, isn't it? The image to the right is a mix of synthetically generated uniform noise and impulsional noise (salt and pepper noise) with 0.5% probability. It has been generated with the NoiseGenerator tool in PixInsight. With this comparison I am trying to show that a master dark frame looks very similar to random noise: essentially, a master dark frame is composed of pixel-to-pixel intensity variations whose spatial distribution is rather uniform, plus a relatively small amount of hot pixels whose distribution and typical values are similar to impulsional noise. Of course, we know that the thermal noise is a fixed pattern characteristic of a given sensor, so it is not random because it is predictable. However, morphologically a master dark frame is virtually indistinguishable from random noise. It is not noise, but it behaves like that, and we are going to take advantage of this simple property to implement a purely numerical solution.
So our cost function is just a noise evaluation function. We already have implemented a powerful multiscale noise evaluation algorithm in several PixInsight tools, such as ImageIntegration, to implement a noise-based image weighting algorithm (Jean-Luc Starck and Fionn Murtagh, Automatic Noise Estimation from the Multiresolution Support, Publications of the Royal Astronomical Society of the Pacific, vol. 110, February 1998, pp. 193-199). In this case, however, we have implemented a simpler and very efficient method (Jean-Luc Starck and Fionn Murtagh, Astronomical Image and Data Analysis, Springer-Verlag, 2002, pp. 37-38) known as k-sigma noise estimation:
- Compute a single-layer wavelet transform of the image. This transform consists of the finest wavelet scale w1 plus a residual cJ, which we simply discard. We use the standard B3 spline wavelet scaling function as a separable low-pass filter.
- Iterate a k-sigma process as follows: At each iteration n > 0, denote by d(n) the subset of pixels in w1 such that |d(n-1)ij| < k*sigma(n-1), where sigma(n-1) is the standard deviation of the current subset of pixels (from the previous iteration). In our implementation we have d(0)=w1, k=3, and this process is iterated until no significant difference is achieved between two consecutive iterations. For dark frame optimization, we iterate up to 1% accuracy and normally 3 or 5 iterations are sufficient.
- The final noise estimate is the standard deviation of the resulting subset of pixels. Note that this is an unscaled noise estimate of Gaussian white noise, since it depends on the wavelet scaling function used. However, since we don't want to compare noise estimates among different images, but only to minimize the noise in a particular image after dark subtraction, an unscaled estimate is perfectly suitable.
Note that the use of a wavelet transform is actually irrelevant in this process; we use it just because it is very fast and we have it readily available on the PixInsight/PCL platform. A simple convolution with a low-pass filter followed by a subtraction from the original image would be equivalent.
This noise evaluation algorithm is robust and 'cannot fail' The rest of our algorithm is a routine to find the minimum of a unimodal function. For the sake of robustness, I haven't implemented anything fancy here: just an adaptation of the classical golden section search algorithm. Perhaps not the fastest alternative, but golden section search is absolutely robust, which is what I want here.
Summarizing, the algorithm can be described at a high level as follows:
- Find a range [k0,k1] of dark scaling factors that bracket the minimum of the noise evaluation function. This is the initial bracketing phase.
- Iterate the golden section search algorithm to find the minimum of the noise evaluation function. At each iteration, compute the calibrated raw frame as:
I0 = I - k*D - B
and evaluate the noise in I0 with the k-sigma iterative method.
- Iterate until the minimum of the noise evaluation function is determined up to a prescribed accuracy (1/1000 fractional accuracy in our implementation).
- The final value of the dark scaling factor is the value of k from the last iteration.
From our tests, this algorithm has proven extremely robust and accurate. However it has two known problems:
- It is very sensitive to bad quality calibration frames. In particular, if the master bias and/or the master dark frames include significant amounts of read noise, the optimization algorithm will find a value of k that overcorrects thermal noise to compensate for the additive random noise component. In extreme cases, this may lead to dark holes as a result of overcorrected hot pixels. However, if the master calibration frames are bad quality ones, the whole image reduction process makes no sense, actually...
- It implicitly assumes that the dark optimization function is constant for the whole range of numerical data values, since it consists of a simple multiplicative factor. This is not the case in practice, especially for high intensity values. As a result of this simplistic optimization model, the algorithm tends to undercorrect hot pixels. This is a very minor issue, however, since hot pixels are easily rejected during integration (because you dither your images, don't you?) and can be removed also with cosmetic correction techniques.
In a future version of the dark optimization algorithm we'll compute different optimization factors for separate intensity ranges in a spline fashion. This should fix the hot pixel undercorrection problem.
Report to moderator Logged
Juan Conejero
PixInsight Development Team
http://pixinsight.com/
Cela dénote un problème dans la construction des masters dark et/ou Bias.
Plusieurs causes possibles:
Différences en temp, ISO ou exposition entre darks et light
Pas suffisamment de bias ou darks, amenant beaucoup de bruit sur les masters dark / bias
un ou plusieurs dark/bias mauvais
mauvais paramétrage des format explorer pour les DSLR
autres....
Il faudrait des copies d'écran, voire les fichiers pour déterminer ce qui cloche
Patrick
PS: maintenant, la theorie: ( desolé, en anglais)
Let's state the problem first. We may describe an uncalibrated raw light frame with this very simple equation:
I = I0 + D + B
where I is the uncalibrated raw light frame, D is the dark current, and B is the bias pedestal. All terms in this equation are vectors (or matrices) because we are describing a process for a whole image at the pixel level. I0 is the dark-and-bias-subtracted raw light frame (we know that the next calibration step would be flat fielding, but this is unimportant to this discussion). Our goal here is to obtain I0.
As we have stated it above, this is a trivial problem: just subtract good master bias and dark frames from the uncalibrated light frame, and the result will be a good approximation to I0 (the better the masters, the better the approximation). However, the actual problem is more complex because the dark current D is highly time- and temperature-dependent, so a perfect match between the dark current signals in I and D is only a theoretical goal. A better (but still far from perfect) approximation to the actual problem is more like this:
I = I0 + k*D + B
where k is a scaling factor that attempts to account for effective exposure time and temperature differences between the light frame and the master dark frame. The process of finding a good approximation to the scaling factor k is what we call dark frame optimization.
Note that I have said that this is far from a perfect solution. It is wrong mainly because the dark current does not vary linearly with exposure time and temperature for the whole numeric range of the data, so a single multiplicative factor cannot be very accurate, especially when time and temperature differences between I and D are large. Dark optimization, however, is always better than nothing. From our experience, dark optimization is beneficial even if exposure times and temperatures are matched between light and dark frames. For this reason the corresponding parameter is enabled by default in our ImageCalibration tool.
Now that we have described the problem, let's describe our solution. Algorithmically this is known as an optimization problem: find the value of a parameter that minimizes (or maximizes) the value of a function. For example, think in terms of the economical cost of a production process. We have to produce an item A that depends on a factor r, which leads to the problem: find the value of r that minimizes the cost of producing A. In real cases r usually represents a complex set of parameters and constraints, such as environmental factors, availability of prime matters, production systems, etc., so that the most difficult part of the solution often consists of identifying a significant set of parameters or model to define a suitable cost function. This leads to the subject of linear programming.
The dark frame optimization problem is a relatively simple case of function optimization with a single parameter. The first step is to define the cost function that we want to minimize. To explain why and how we have selected a particular function we need some visual analysis. Take a look at the following two images:
One of them is a crop of a bias-subtracted master dark frame, the other is not. Difficult to say which is which, isn't it? The image to the right is a mix of synthetically generated uniform noise and impulsional noise (salt and pepper noise) with 0.5% probability. It has been generated with the NoiseGenerator tool in PixInsight. With this comparison I am trying to show that a master dark frame looks very similar to random noise: essentially, a master dark frame is composed of pixel-to-pixel intensity variations whose spatial distribution is rather uniform, plus a relatively small amount of hot pixels whose distribution and typical values are similar to impulsional noise. Of course, we know that the thermal noise is a fixed pattern characteristic of a given sensor, so it is not random because it is predictable. However, morphologically a master dark frame is virtually indistinguishable from random noise. It is not noise, but it behaves like that, and we are going to take advantage of this simple property to implement a purely numerical solution.
So our cost function is just a noise evaluation function. We already have implemented a powerful multiscale noise evaluation algorithm in several PixInsight tools, such as ImageIntegration, to implement a noise-based image weighting algorithm (Jean-Luc Starck and Fionn Murtagh, Automatic Noise Estimation from the Multiresolution Support, Publications of the Royal Astronomical Society of the Pacific, vol. 110, February 1998, pp. 193-199). In this case, however, we have implemented a simpler and very efficient method (Jean-Luc Starck and Fionn Murtagh, Astronomical Image and Data Analysis, Springer-Verlag, 2002, pp. 37-38) known as k-sigma noise estimation:
- Compute a single-layer wavelet transform of the image. This transform consists of the finest wavelet scale w1 plus a residual cJ, which we simply discard. We use the standard B3 spline wavelet scaling function as a separable low-pass filter.
- Iterate a k-sigma process as follows: At each iteration n > 0, denote by d(n) the subset of pixels in w1 such that |d(n-1)ij| < k*sigma(n-1), where sigma(n-1) is the standard deviation of the current subset of pixels (from the previous iteration). In our implementation we have d(0)=w1, k=3, and this process is iterated until no significant difference is achieved between two consecutive iterations. For dark frame optimization, we iterate up to 1% accuracy and normally 3 or 5 iterations are sufficient.
- The final noise estimate is the standard deviation of the resulting subset of pixels. Note that this is an unscaled noise estimate of Gaussian white noise, since it depends on the wavelet scaling function used. However, since we don't want to compare noise estimates among different images, but only to minimize the noise in a particular image after dark subtraction, an unscaled estimate is perfectly suitable.
Note that the use of a wavelet transform is actually irrelevant in this process; we use it just because it is very fast and we have it readily available on the PixInsight/PCL platform. A simple convolution with a low-pass filter followed by a subtraction from the original image would be equivalent.
This noise evaluation algorithm is robust and 'cannot fail' The rest of our algorithm is a routine to find the minimum of a unimodal function. For the sake of robustness, I haven't implemented anything fancy here: just an adaptation of the classical golden section search algorithm. Perhaps not the fastest alternative, but golden section search is absolutely robust, which is what I want here.
Summarizing, the algorithm can be described at a high level as follows:
- Find a range [k0,k1] of dark scaling factors that bracket the minimum of the noise evaluation function. This is the initial bracketing phase.
- Iterate the golden section search algorithm to find the minimum of the noise evaluation function. At each iteration, compute the calibrated raw frame as:
I0 = I - k*D - B
and evaluate the noise in I0 with the k-sigma iterative method.
- Iterate until the minimum of the noise evaluation function is determined up to a prescribed accuracy (1/1000 fractional accuracy in our implementation).
- The final value of the dark scaling factor is the value of k from the last iteration.
From our tests, this algorithm has proven extremely robust and accurate. However it has two known problems:
- It is very sensitive to bad quality calibration frames. In particular, if the master bias and/or the master dark frames include significant amounts of read noise, the optimization algorithm will find a value of k that overcorrects thermal noise to compensate for the additive random noise component. In extreme cases, this may lead to dark holes as a result of overcorrected hot pixels. However, if the master calibration frames are bad quality ones, the whole image reduction process makes no sense, actually...
- It implicitly assumes that the dark optimization function is constant for the whole range of numerical data values, since it consists of a simple multiplicative factor. This is not the case in practice, especially for high intensity values. As a result of this simplistic optimization model, the algorithm tends to undercorrect hot pixels. This is a very minor issue, however, since hot pixels are easily rejected during integration (because you dither your images, don't you?) and can be removed also with cosmetic correction techniques.
In a future version of the dark optimization algorithm we'll compute different optimization factors for separate intensity ranges in a spline fashion. This should fix the hot pixel undercorrection problem.
Report to moderator Logged
Juan Conejero
PixInsight Development Team
http://pixinsight.com/
Patrick- Club : AMAS AIP
Localisation : Allauch
Humeur : Optimiste et obstiné
Re: k0 calibration Pix
Patrick un GRAND merci pour toutes ces précisions qui répondent à mes questions. Je n'ai pas encore tout bien compris mais l'essentiel malgré tout.
C'est vraiment sympa d'avoir répondu avec tant de détails !
C'est vraiment sympa d'avoir répondu avec tant de détails !
Sujets similaires
» Calibration des filtres RVB avec un capteur monochrome
» Problème calibration pi 1.8
» Première série d'image de l'été
» Calibration des écrans
» Calibration Ecran et Pixinsight
» Problème calibration pi 1.8
» Première série d'image de l'été
» Calibration des écrans
» Calibration Ecran et Pixinsight
Astron'amis :: Pratique de l'astronomie amateur :: Astrophotographie :: Ciel profond :: Le coin des apprentis
Page 1 sur 1
Permission de ce forum:
Vous ne pouvez pas répondre aux sujets dans ce forum
|
|
Mar 23 Avr 2024 - 17:36 par FalCT60
» [Résolu] Évolution progressive
Sam 13 Avr 2024 - 17:07 par FalCT60
» M51 500 f/8 + doubleur
Sam 13 Avr 2024 - 16:59 par FalCT60
» "Journées de l'astronomie de Saverne" les 13 et 14 Avril 2024
Lun 11 Mar 2024 - 19:29 par Walcho
» Mosaique avec MaximDL
Lun 22 Jan 2024 - 18:01 par Tellier
» recherche Mewlon 250 Takahashi CRS ou non
Dim 12 Nov 2023 - 11:40 par boss351
» FRA300 : deux nouveaux tests
Dim 22 Oct 2023 - 10:51 par EtaCas
» Test ASKAR FMA135
Sam 21 Oct 2023 - 10:49 par Gérard.pst
» M27 nébuleuse du trognon de pomme
Sam 21 Oct 2023 - 10:47 par Gérard.pst
» NGC6992 à la FRA300
Sam 21 Oct 2023 - 9:56 par laurent*
» Du cousu main - ou presque
Mer 18 Oct 2023 - 20:08 par FalCT60
» Andromède, une histoire d'ZWO
Jeu 14 Sep 2023 - 21:48 par FalCT60
» SOS d'un utilisateur de kstars en détresse
Lun 11 Sep 2023 - 18:02 par FalCT60
» Perséïdes 2023
Lun 14 Aoû 2023 - 12:04 par EtaCas
» Bonjour à tous !
Lun 24 Juil 2023 - 21:29 par EtaCas
» Mon site web dédié à l'astrophoto
Lun 24 Juil 2023 - 21:22 par EtaCas
» Problème de raquette
Ven 7 Juil 2023 - 17:19 par Invité
» DEBUTER EN PHOTOMETRIE
Jeu 8 Juin 2023 - 11:25 par Etienne60
» Vends Caméra QSI 640 WSG8
Dim 21 Mai 2023 - 19:57 par jmd
» Propus dans la brume
Ven 21 Avr 2023 - 17:20 par FalCT60
» "Journées de l'astronomie de Saverne" les 6 et 7 Mai 2023
Jeu 13 Avr 2023 - 0:17 par Walcho
» Mes débuts de réduction d'étoiles
Mar 11 Avr 2023 - 18:02 par FalCT60
» La comète Z'êtes Tous Fadas
Mer 29 Mar 2023 - 18:34 par FalCT60
» Eclipse oct. 2022
Jeu 9 Mar 2023 - 16:43 par Kartouche
» Une portion de Soleil
Mer 8 Mar 2023 - 19:51 par Artimon
» Grosse Protubérance
Ven 3 Mar 2023 - 7:23 par jmd
» FSQ106EDX4, robofocus et diviseur optique skymeca
Mer 22 Fév 2023 - 23:16 par Davebowman
» Oculaire Televue DELOS 10mm
Lun 20 Fév 2023 - 21:05 par Davebowman
» Reducteur de focale Takahashi 0.73X
Ven 17 Fév 2023 - 15:00 par Davebowman
» Monture neuve Vixen GP2 avec trepied
Lun 6 Fév 2023 - 12:19 par Luzog