spacekit.generator.draw

class spacekit.generator.draw.DrawMosaics(input_path, output_path=None, fname=None, visit=None, pattern='', gen=3, size=(24, 24), crpt=0, subset_name=None, name='DrawMosaics', **log_kws)

Bases: object

Class for generating machine-learning image inputs from drizzled total detection fits files and their associated catalog files. Primarily used for creating multiple images at once (batch) with capability for single images/single dataset also available.

Initializes a DrawMosaics class object.

Parameters:

• input_path (str (path)) – path to dataset subdirectories containing total or filter fits files

• output_path ([type], optional) – where to save the pngs (path), by default None (will create and save in relative dir path named “img” )

• fname (str (path), optional) – csv (dataframe) fname for generating specific list of datasets, by default None

• visit ([type], optional) – name of specific subdirectory (typically ipppssoot or visit name) containing .fits and .ecsv files, by default None

• pattern (str, optional) – glob search pattern (to restrict image generator to look for only certain visit names, e.g. ‘ia*’, by default ‘’

• gen (int, optional) – generator method to use: 0=generate original only; 1=point-segment, 2=gaia; 3=original, point-segment, and gaia (3 separate images), by default 3

• size (tuple, optional) – size to make the figures i.e. figsize=(size,size), by default (24, 24)

• crpt (int, optional) – modifies the input search pattern as well as output png file naming convention (so that a non-corrupt visit of the same name is not overwritten), by default 0

check_format(dname=None)

check_output()

check if a custom output_path is set, otherwise create a subdirectory “img” in the current working directory and set as the output_path attribute.

Returns:

path to subdirectory for saving png images.

Return type:

str (path)

create_image_name(name, dataset, P=0, S=0, G=0, fgroup=None)

Determines which suffix to append to the output png file based on which catalog(s) are used (if any).

Parameters:

• name (str) – visit name

• dataset ([type]) – visit name (used to adjust name if crpt=1)

• P (int, optional) – draw point catalog overlay (if available), by default 0

• S (int, optional) – draw segment catalog overlay (if available), by default 0

• G (int, optional) – draw Gaia catalog overlay (if eDR3 or GSC242 available), by default 0

Returns:

path to png output for this image.

Return type:

str

draw_catalogs(cfile, catalog)

Open and read .escv catalog file associated with the visit (if available) and map the appropriate values and coordinates to draw as an overlay on the original image. Credit: based in part on code by M. Burger

Parameters:

• cfile (str (path)) – path to source catalog file

• catalog (str) – “point”, “segment” or “gaia”

Returns:

table of catalog values and associated flag colors.

Return type:

Pandas dataframe, lists of flag colors

draw_filter_images(dataset)

Generate relative filter drizzle file images

Parameters:

dataset (str) – name of input visit dataset

draw_total_images(dataset, P=0, S=0, G=0)

Primary class method for plotting the data, drawing the catalogs (if any) and saving to local disk as png.

Parameters:

• dataset (str) – visit name (single dataset, for batch jobs this comes from looping over the list stored in self.datasets)

• P (int, optional) – draw point catalog overlay (if available), by default 0

• S (int, optional) – draw segment catalog overlay (if available), by default 0

• G (int, optional) – draw Gaia catalog overlay (eDR3 or GSC242 if available), by default 0

generate_filter_images()

Generate batch of relative filter drizzle file images.

generate_total_images()

Batch image generation method for multiple datasets (and multiple catalog types)

get_datasets()

Locate inputs (fits file directories) to use for drawing the images. Search method used is based on parameters set when the DrawMosaics class object is instantiated. If multiple parameters are passed in, the order of search priority is 1) fname: only look for visits found in the csv file/dataframe (uses load_from_file method); 2) visit only look for subdirectories matching this specific visit name; 3) local_search: glob-based search for any visit subdirectories matching the pattern set in pattern attribute. (if crpt)

Returns:

list of datasets/visits found according to search method used

Return type:

list

get_hapfiles(dataset)

load_from_file()

only look for visits found in the csv file/dataframe.

Returns:

restricted list of inputs (visits) for which images will be drawn

Return type:

list

local_search()

only look for visit names matching a glob-based search pattern.

Returns:

list of inputs (visits) for which images will be drawn

Return type:

list

point_flag_color(x)

determines whether or not to draw a small red (or green) circle on top of the original image data depending on the value found in point source catalog. More info on the values associated with the “flag” color can be found in the Drizzlepac handbook at drizzlepac.readthedocs.io (Drizzlepac.catalog_generation api)

Parameters:

x (int) – value pulled from point catalog file

Returns:

“red”, “green” or None depending on input value

Return type:

str

segment_flag_color(x)

draw a small blue circle on top of the original image data depending on the value found in segment source catalog.

Parameters:

x (int) – value pulled from segment catalog file

Returns:

“blue”, “green” or None depending on input value

Return type:

str

Generates machine-learning image inputs from drizzled total detection fits files and their associated catalog files. Primarily used for creating a batch of multiple images at once, although it can also be used to draw a single image/single dataset.

Opens fits files from a local directory to generate total detection drizzled images as matplotlib figures, aligned to WCS with point/segment/gaia catalog overlays. Figures are stripped down to just the pixel values (no visible axes or legends) and saved to disk as png files. These images can then be used to train an image classification neural network. To draw the images, you need to have at least the *_drz/*_drc.fits file locally.

To draw the catalog overlay(s), you would also need the .ecsv catalog file(s). The fits and catalog files are generated by running Drizzlepac’s Runsinglehap SVM processing script on the raw input files.

Relative Filter Images

This script can also be used to generate relative filter images for use in the image similarity model by passing img_type=’filter’ (default is “total” for total detection images).

Synthetic (artifically corrupted) datasets

This script also has the capability of drawing images using “corrupt” datasets (see spacekit.skopes.hst.svm.corrupt). Setting crpt=1 or c=1 tells the script how to name the image subfolder for that dataset (so that it doesn’t overwrite a regular/non-corrupt dataset with the same name.

• P: draw point catalog references (0=off, 1=on) default is 0

• S: draw segment catalog references (0=off, 1=on) default is 0

• G: draw GAIA catalog references (0=off, 1=on) default is 0

If the above catalog overlays are turned on, the saved images will be saved with the appropriate suffix appended:

• P=1: _point.png

• S=1: _segment.png

• P=1, S=1 (both point and segment): _source.png

• G=1 _gaia.png

Note

For the SVM Alignment Image Classifier (see spacekit.skopes.hst.svm.prep module), 3 images are produced for each visit:

• original image (no catalog overlay)

• source image (both point and segment catalog overlays)

• gaia image (GAIA catalog overlay)

PNG naming convention is based on fits file unless crpt=1:

./input_path/dataset/fname.fits

saves as:

./img_path/dataset/fname.png

Normal SVM data

dataset=ib1f0a

./{input_path}/ib1f0a/hst_11570_0a_wfc3_uvis_total_ib1f0a_drc.fits

saves as:

./{imgdir}/hst_11570_0a_wfc3_uvis_total_ib1f0a/hst_11570_0a_wfc3_uvis_total_ib1f0a.png

Corruption SVM data

dataset=ia0m04_f110w_all_stat

./{input_path}/ia0m04_f110w_all_stoc/hst_11099_04_wfc3_ir_total_ia0m04_drz.fits

saves as:

./{imgdir}/hst_f110w_all_stoc_uvis_total_ib1f0a/hst_f110w_all_stoc_uvis_total_ib1f0a.png

CREDIT: much of the code used for plotting the data from a .fits sci extension (draw_total_images) and drawing the catalog overlays (point_flag_color, segment_flag_color) is derived from work done by Matthew Burger.