B Ú‹dØ«ã²@sxddlZddlZddlZddlmZddlZddlmZm Z ddl m Z m Z m Z mZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZmZm Z ej! "e#¡Z$ye de$ƒZ%Wne&k ròe'dƒ‚YnXe%j(Z)de)_*eej+ddhdeej+ddej,eej-ddeej+ddeej-ddej.ej.ej,g e)_/e 0dddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@dAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_d`dadbdcdddedfdgdhdidjdkdldmdndodpdqdrdsdtdudvdwdxdydzd{d|d}d~dd€dd‚dƒd„d…d†d‡dˆd‰dŠd‹dŒddŽddd‘d’d“d”d•d–d—d˜d™dšd›dœddždŸd d¡d¢d£d¤d¥d¦d§d¨d©dªd«d¬d­d®d¯d°d±d²d³d´dµd¶d·d¸d¹dºd»d¼d½d¾d¿dÀg°¡Z1e2e 3e1dÁ¡ƒZ4dÂdÃgiZ5ddÄdÅdÆgZ6dÇdÈ„Z7e8dÉdddfdÊdË„Z9edÌdÍdÝdÓdÔ„ƒZ:edÌdÍdÄdÎdÏdÐdÒdÑddÐdÑdddÎdÑej;jfdÕdÖ„ƒZ?e:fd×dØ„Z@dÞdÙdÚ„ZAdßdÛdÜ„ZBdS)àéN)Úpartial)Ú ndpointerÚ load_libraryé)ÚKernelÚKernel1DÚKernel2DÚMAX_NORMALIZATION)ÚAstropyUserWarning)Úhuman_file_size)Úunits)Úsupport_nddata)Ú CompoundModel)ÚSPECIAL_OPERATORS)Ú Convolution)ÚKernelSizeErrorÚ has_even_axisÚraise_even_kernel_exceptionÚ _convolvezUsz&_next_fast_lengths..)ÚdtyperzNo next fast length for z! found in list of _good_sizes <= rIÚ.)Z scipy.fftÚnpÚarrayÚ ImportErrorÚemptyÚlenZ atleast_1dÚintÚ enumerateÚmaxÚceilÚlog10Ú _good_rangeÚ _good_sizesÚ ValueError)ÚshapeÚnewshapeÚirQÚscaleÚnrN)rRrSÚ_next_fast_lengthsKs"   "riÚCc Csèt|tƒr|jn|}t|dƒr$|j}|}yŠ|dksFtj |¡sF|dk rœtj |¡rrtj||d|dd}| |¡}ntj||d|dd}|dk r°|||dk<ntj||d|dd}Wn0t t fk râ}zt d|ƒ‚Wdd}~XYnX|S)NÚunitrKFT)rUÚcopyÚorderZsubokrzIinput should be a Numpy array or something convertible into a float array) Ú isinstancerrXÚhasattrÚvaluerWÚmaZ is_maskedZfilledÚ TypeErrorrc)ÚinputrUrmÚ nan_treatmentÚmaskÚ fill_valueÚoutputÚerNrNrSÚ_copy_input_if_neededfs"   ryrX)ÚdataçÚ interpolateTFç:Œ0âŽyE>c  Cs¤|tkrtdt›ƒ‚|dkr&tdƒ‚d} |} |} t| td||tjd} t| d| jƒ} t| tddd|d}t|ƒrxt ƒt | t ƒr¨t | t ƒr¨t   d t¡d }d }d }d }| jd kr¼tdƒ‚n(| jdkrÐtdƒ‚n| j|jkrätdƒ‚t | j¡}t |j¡}|d}|dkr*t |d|k¡s*tdƒ‚|d ko@t |  ¡¡}|sN|rŒ| ¡}tj|d |d}|dtksz|rŒtd dt¡ƒ‚|sœ|d kr¸t | ¡}|d kr¸|| |<tj| jtdd}d }| }|dkr>d}|d kr¼tj|d||tdd}| jdkr.| ||d |d |d …<nŒ| jdkrp| ||d |d |d …|d|d|d…f<nJ| ||d |d |d …|d|d|d…|d|d|d…f<n‚ddddœ}||}|d}| jdkrð|d f}n>| jdkr|d f|dff}n|d f|df|dff}tj| ||d}t|||jtj|jtj dd|tj|tj dd||| ƒ |rŒ|sš||}n|rš||9}|rÂ|sÂt | ¡¡rÂt   dt¡|rÒtj||<t| d dƒ}|dk rð||K}t | t ƒrbt | t!ƒrt!|d!}n t | t"ƒr,t"|d!}nt#d"ƒ‚d|_$| j%|_%t | t ƒr^|j%oZ| j%|_%|S| j&d#krœy|j'| dd$St#k r˜| '| ¡SXn|SdS)%a Convolve an array with a kernel. This routine differs from `scipy.ndimage.convolve` because it includes a special treatment for ``NaN`` values. Rather than including ``NaN`` values in the array in the convolution calculation, which causes large ``NaN`` holes in the convolved array, ``NaN`` values are replaced with interpolated values using the kernel as an interpolation function. Parameters ---------- array : `~astropy.nddata.NDData` or array-like The array to convolve. This should be a 1, 2, or 3-dimensional array or a list or a set of nested lists representing a 1, 2, or 3-dimensional array. If an `~astropy.nddata.NDData`, the ``mask`` of the `~astropy.nddata.NDData` will be used as the ``mask`` argument. kernel : `numpy.ndarray` or `~astropy.convolution.Kernel` The convolution kernel. The number of dimensions should match those for the array, and the dimensions should be odd in all directions. If a masked array, the masked values will be replaced by ``fill_value``. boundary : str, optional A flag indicating how to handle boundaries: * `None` Set the ``result`` values to zero where the kernel extends beyond the edge of the array. * 'fill' Set values outside the array boundary to ``fill_value`` (default). * 'wrap' Periodic boundary that wrap to the other side of ``array``. * 'extend' Set values outside the array to the nearest ``array`` value. fill_value : float, optional The value to use outside the array when using ``boundary='fill'`` normalize_kernel : bool, optional Whether to normalize the kernel to have a sum of one. nan_treatment : {'interpolate', 'fill'} interpolate will result in renormalization of the kernel at each position ignoring (pixels that are NaN in the image) in both the image and the kernel. 'fill' will replace the NaN pixels with a fixed numerical value (default zero, see ``fill_value``) prior to convolution Note that if the kernel has a sum equal to zero, NaN interpolation is not possible and will raise an exception. preserve_nan : bool After performing convolution, should pixels that were originally NaN again become NaN? mask : None or ndarray A "mask" array. Shape must match ``array``, and anything that is masked (i.e., not 0/`False`) will be set to NaN for the convolution. If `None`, no masking will be performed unless ``array`` is a masked array. If ``mask`` is not `None` *and* ``array`` is a masked array, a pixel is masked of it is masked in either ``mask`` *or* ``array.mask``. normalization_zero_tol : float, optional The absolute tolerance on whether the kernel is different than zero. If the kernel sums to zero to within this precision, it cannot be normalized. Default is "1e-8". Returns ------- result : `numpy.ndarray` An array with the same dimensions and as the input array, convolved with kernel. The data type depends on the input array type. If array is a floating point type, then the return array keeps the same data type, otherwise the type is ``numpy.float``. Notes ----- For masked arrays, masked values are treated as NaNs. The convolution is always done at ``numpy.float`` precision. z(Invalid boundary option: must be one of )r|rKz1nan_treatment must be one of 'interpolate','fill'rrj)rUrmrtrurvrUNz¢Both array and kernel are Kernel instances, hardwiring the following parameters: boundary='fill', fill_value=0, normalize_Kernel=True, nan_treatment='interpolate'rKrTr|z$cannot convolve 0-dimensional arraysrzBconvolve only supports 1, 2, and 3-dimensional arrays at this timez5array and kernel have differing number of dimensions.rztfor boundary=None all kernel axes must be smaller than array's - use boundary in ['fill', 'extend', 'wrap'] instead.)Zatolgð?zeThe kernel can't be normalized, because its sum is close to zero. The sum of the given kernel is < {})rUrm)rKrMrLF)rvrUrmZconstantÚedgerL)Ú pad_widthÚmodezÓnan_treatment='interpolate', however, NaN values detected post convolution. A contiguous region of NaN values, larger than the kernel size, are present in the input array. Increase the kernel size to avoid this.rk)rXz%Only 1D and 2D Kernels are supported.Úf)rl)(ÚBOUNDARY_OPTIONSrcryÚfloatrWÚnanÚgetattrrUrrrnrÚwarningsÚwarnr ÚndimÚ ExceptionÚNotImplementedErrorrXrdÚallrÚisnanÚsumÚiscloser ÚformatÚzerosÚfullÚpadÚ _convolveNd_cÚctypesÚc_size_trrrrZ_is_boolZ _separableÚkindZastype)rXÚkernelÚboundaryrvrtÚnormalize_kernelruÚ preserve_nanÚnormalization_zero_tolZ n_threadsZ passed_kernelZ passed_arrayZarray_internalZ array_dtypeZkernel_internalZ array_shapeZ kernel_shaperZnan_interpolateZ kernel_sumZkernel_sums_to_zeroZ initially_nanÚresultZ!embed_result_within_padded_regionZarray_to_convolveZnp_pad_mode_dictZ np_pad_modeZ np_pad_widthÚ array_unitZ new_resultrNrNrSÚconvolve’sÐN                 6L               ržc/ Cstt|tƒr"|j}t|tƒr"tdƒ‚|dkr2tdƒ‚t|ddƒ}t|td||tj d}t|tddddd}|j |j kr|td ƒ‚|j }|j }tj |tj d t |¡jtj}|d tjkrÐ|sÐtd t|ƒ›d ƒ‚t |¡t |¡B}|dkrö|||<nd||<t |¡t |¡B}d||<|dkr^| ¡dtkrHtd dt¡ƒ‚| ¡}||}d }nV|rv||ƒ}||}n>| ¡}t |¡|kr¬|dkr¢tdƒ‚nd }|}n||}|dkrèt dt¡| dkrØd} | dkrld} n„|dkr&| dkrt d| ›dt¡nd} | dkrld} nF|dkrZ| r>tdƒ‚d} | rPtdƒ‚d} d}n|dkrltdƒ‚| rˆt |¡t |¡}n t ||¡}| r¢t |ƒ}tj |tj d t |¡jtj}|d tjkrî|sîtd t|ƒ›d ƒ‚g}g}x|t!t"|||ƒƒD]h\}\}} }!||d d}"|t#|"| d|"| d dƒg7}|t#|"|!d|"|!d dƒg7}qWt$|ƒ}t$|ƒ}t %||k¡sÌt &|¡r´tj'||d |}#ntj(||d }#||#|<n|}#t %||k¡søtj(||d }$||$|<n|}$||#ƒ}%|tj) *|$¡ƒ}&|%|&}'|dk}(|(rŒt &|¡sFtj(||d })ntj'||d })d||(|)|<||)ƒ}*|*|&}+||+ƒ},|,j+||)|<nd })t |'¡ ,¡r¨tdƒ‚|'|9}'|dk rÂ|'|K}'| rÌ|'S|(r:tj-ddd ||'ƒ|)}-WdQRXt .|)¡sB| d!krtj |-|)| k<nd!|-|)d"t /|)j¡j0k<n||'ƒ}-|rVtj |-||<| rj|-|j+}.|.S|-j+SdS)#a " Convolve an ndarray with an nd-kernel. Returns a convolved image with ``shape = array.shape``. Assumes kernel is centered. `convolve_fft` is very similar to `convolve` in that it replaces ``NaN`` values in the original image with interpolated values using the kernel as an interpolation function. However, it also includes many additional options specific to the implementation. `convolve_fft` differs from `scipy.signal.fftconvolve` in a few ways: * It can treat ``NaN`` values as zeros or interpolate over them. * ``inf`` values are treated as ``NaN`` * It optionally pads to the nearest faster sizes to improve FFT speed. These sizes are optimized for the numpy and scipy implementations, and ``fftconvolve`` uses them by default as well; when using other external functions (see below), results may vary. * Its only valid ``mode`` is 'same' (i.e., the same shape array is returned) * It lets you use your own fft, e.g., `pyFFTW `_ or `pyFFTW3 `_ , which can lead to performance improvements, depending on your system configuration. pyFFTW3 is threaded, and therefore may yield significant performance benefits on multi-core machines at the cost of greater memory requirements. Specify the ``fftn`` and ``ifftn`` keywords to override the default, which is `numpy.fft.fftn` and `numpy.fft.ifftn`. The `scipy.fft` functions also offer somewhat better performance and a multi-threaded option. Parameters ---------- array : `numpy.ndarray` Array to be convolved with ``kernel``. It can be of any dimensionality, though only 1, 2, and 3d arrays have been tested. kernel : `numpy.ndarray` or `astropy.convolution.Kernel` The convolution kernel. The number of dimensions should match those for the array. The dimensions *do not* have to be odd in all directions, unlike in the non-fft `convolve` function. The kernel will be normalized if ``normalize_kernel`` is set. It is assumed to be centered (i.e., shifts may result if your kernel is asymmetric) boundary : {'fill', 'wrap'}, optional A flag indicating how to handle boundaries: * 'fill': set values outside the array boundary to fill_value (default) * 'wrap': periodic boundary The `None` and 'extend' parameters are not supported for FFT-based convolution fill_value : float, optional The value to use outside the array when using boundary='fill' nan_treatment : {'interpolate', 'fill'} ``interpolate`` will result in renormalization of the kernel at each position ignoring (pixels that are NaN in the image) in both the image and the kernel. ``fill`` will replace the NaN pixels with a fixed numerical value (default zero, see ``fill_value``) prior to convolution. Note that if the kernel has a sum equal to zero, NaN interpolation is not possible and will raise an exception. normalize_kernel : callable or boolean, optional If specified, this is the function to divide kernel by to normalize it. e.g., ``normalize_kernel=np.sum`` means that kernel will be modified to be: ``kernel = kernel / np.sum(kernel)``. If True, defaults to ``normalize_kernel = np.sum``. normalization_zero_tol : float, optional The absolute tolerance on whether the kernel is different than zero. If the kernel sums to zero to within this precision, it cannot be normalized. Default is "1e-8". preserve_nan : bool After performing convolution, should pixels that were originally NaN again become NaN? mask : None or ndarray A "mask" array. Shape must match ``array``, and anything that is masked (i.e., not 0/`False`) will be set to NaN for the convolution. If `None`, no masking will be performed unless ``array`` is a masked array. If ``mask`` is not `None` *and* ``array`` is a masked array, a pixel is masked of it is masked in either ``mask`` *or* ``array.mask``. crop : bool, optional Default on. Return an image of the size of the larger of the input image and the kernel. If the image and kernel are asymmetric in opposite directions, will return the largest image in both directions. For example, if an input image has shape [100,3] but a kernel with shape [6,6] is used, the output will be [100,6]. return_fft : bool, optional Return the ``fft(image)*fft(kernel)`` instead of the convolution (which is ``ifft(fft(image)*fft(kernel))``). Useful for making PSDs. fft_pad : bool, optional Default on. Zero-pad image to the nearest size supporting more efficient execution of the FFT, generally values factorizable into the first 3-5 prime numbers. With ``boundary='wrap'``, this will be disabled. psf_pad : bool, optional Zero-pad image to be at least the sum of the image sizes to avoid edge-wrapping when smoothing. This is enabled by default with ``boundary='fill'``, but it can be overridden with a boolean option. ``boundary='wrap'`` and ``psf_pad=True`` are not compatible. min_wt : float, optional If ignoring ``NaN`` / zeros, force all grid points with a weight less than this value to ``NaN`` (the weight of a grid point with *no* ignored neighbors is 1.0). If ``min_wt`` is zero, then all zero-weight points will be set to zero instead of ``NaN`` (which they would be otherwise, because 1/0 = nan). See the examples below. allow_huge : bool, optional Allow huge arrays in the FFT? If False, will raise an exception if the array or kernel size is >1 GB. fftn : callable, optional The fft function. Can be overridden to use your own ffts, e.g. an fftw3 wrapper or scipy's fftn, ``fft=scipy.fftpack.fftn``. ifftn : callable, optional The inverse fft function. Can be overridden the same way ``fttn``. complex_dtype : complex type, optional Which complex dtype to use. `numpy` has a range of options, from 64 to 256. Raises ------ ValueError: If the array is bigger than 1 GB after padding, will raise this exception unless ``allow_huge`` is True See Also -------- convolve: Convolve is a non-fft version of this code. It is more memory efficient and for small kernels can be faster. Returns ------- default : ndarray ``array`` convolved with ``kernel``. If ``return_fft`` is set, returns ``fft(array) * fft(kernel)``. If crop is not set, returns the image, but with the fft-padded size instead of the input size Notes ----- With ``psf_pad=True`` and a large PSF, the resulting data can become large and consume a lot of memory. See Issue https://github.com/astropy/astropy/pull/4366 and the update in https://github.com/astropy/astropy/pull/11533 for further details. Examples -------- >>> convolve_fft([1, 0, 3], [1, 1, 1]) array([0.33333333, 1.33333333, 1. ]) >>> convolve_fft([1, np.nan, 3], [1, 1, 1]) array([0.5, 2. , 1.5]) >>> convolve_fft([1, 0, 3], [0, 1, 0]) array([ 1.00000000e+00, -3.70074342e-17, 3.00000000e+00]) >>> convolve_fft([1, 2, 3], [1]) array([1., 2., 3.]) >>> convolve_fft([1, np.nan, 3], [0, 1, 0], nan_treatment='interpolate') array([1., 0., 3.]) >>> convolve_fft([1, np.nan, 3], [0, 1, 0], nan_treatment='interpolate', ... min_wt=1e-8) array([ 1., nan, 3.]) >>> convolve_fft([1, np.nan, 3], [1, 1, 1], nan_treatment='interpolate') array([0.5, 2. , 1.5]) >>> convolve_fft([1, np.nan, 3], [1, 1, 1], nan_treatment='interpolate', ... normalize_kernel=True) array([0.5, 2. , 1.5]) >>> import scipy.fft # optional - requires scipy >>> convolve_fft([1, np.nan, 3], [1, 1, 1], nan_treatment='interpolate', ... normalize_kernel=True, ... fftn=scipy.fft.fftn, ifftn=scipy.fft.ifftn) array([0.5, 2. , 1.5]) >>> fft_mp = lambda a: scipy.fft.fftn(a, workers=-1) # use all available cores >>> ifft_mp = lambda a: scipy.fft.ifftn(a, workers=-1) >>> convolve_fft([1, np.nan, 3], [1, 1, 1], nan_treatment='interpolate', ... normalize_kernel=True, fftn=fft_mp, ifftn=ifft_mp) array([0.5, 2. , 1.5]) zDCan't convolve two kernels with convolve_fft. Use convolve instead.)r|rKz1nan_treatment must be one of 'interpolate','fill'rkNrj)rUrmrtrurvrz4Image and kernel must have same number of dimensions)rUrzSize Error: Arrays will be z2. Use allow_huge=True to override this exception.rKTgð?zeThe kernel can't be normalized, because its sum is close to zero. The sum of the given kernel is < {}r|z5Cannot interpolate NaNs with an unnormalizable kernelz¡The convolve_fft version of boundary=None is equivalent to the convolve boundary='fill'. There is no FFT equivalent to convolve's zero-if-kernel-leaves-boundaryFzpsf_pad was set to z., which overrides the boundary='fill' setting.rLz0With boundary='wrap', psf_pad cannot be enabled.z0With boundary='wrap', fft_pad cannot be enabled.rMz@The 'extend' option is not implemented for fft-based convolutionrz2Encountered NaNs in convolve. This is disallowed.Úignore)ÚdivideÚinvalidgr)1rnrrXrrrcr…ryÚcomplexrWr„rˆrdÚproductZint64rUÚitemsizeÚuÚbyteÚGBr rŒÚisinfrr r‰rÚabsr†r‡r rŠÚmaximumrir]ÚzipÚsliceÚtupler‹ÚisfiniteZonesrrOZ ifftshiftÚrealÚanyZerrstateZisscalarZfinfoZeps)/rXr—r˜rvrtr™r›ršruÚcropZ return_fftZfft_padZpsf_padZmin_wtZ allow_hugeÚfftnÚifftnZ complex_dtyperZ arrayshapeZ kernshapeZ array_size_BZ nanmaskarrayZ nanmaskkernelZ kernel_scaleZnormalized_kernelreZ array_size_CZ arrayslicesZ kernslicesÚiiZ newdimsizeZ arraydimsizeZ kerndimsizeÚcenterZbigarrayZ bigkernelZarrayfftZkernfftZfftmultZinterpolate_nanZbigimwtZwtfftZ wtfftmultZwtsmZrifftrœrNrNrSrJ¬s A                             rJcKsTt t |¡¡s| ¡S| ¡}|||fddddœ|—Ž}t |¡}||||<|S)a\ Given a data set containing NaNs, replace the NaNs by interpolating from neighboring data points with a given kernel. Parameters ---------- array : `numpy.ndarray` Array to be convolved with ``kernel``. It can be of any dimensionality, though only 1, 2, and 3d arrays have been tested. kernel : `numpy.ndarray` or `astropy.convolution.Kernel` The convolution kernel. The number of dimensions should match those for the array. The dimensions *do not* have to be odd in all directions, unlike in the non-fft `convolve` function. The kernel will be normalized if ``normalize_kernel`` is set. It is assumed to be centered (i.e., shifts may result if your kernel is asymmetric). The kernel *must be normalizable* (i.e., its sum cannot be zero). convolve : `convolve` or `convolve_fft` One of the two convolution functions defined in this package. Returns ------- newarray : `numpy.ndarray` A copy of the original array with NaN pixels replaced with their interpolated counterparts r|TF)rtr™rš)rWr°rŒrl)rXr—ržÚkwargsZnewarrayZ convolvedrŒrNrNrSÚinterpolate_replace_nansKs   r·cKsX|dkrt dttf|Ž¡}n.|dkrs                +   ) "