spacekit.builder.architect
- class spacekit.builder.architect.Builder(train_data=None, test_data=None, blueprint=None, algorithm=None, model_path=None, name=None, logname='Builder', **log_kws)[source]
Bases:
object
Class for building and training a neural network.
- batch_fit()[source]
Fits cnn using a batch generator of equal positive and negative number of samples, rotating randomly.
- Returns:
Keras training history
- Return type:
tf.keras.model.history
- decay_learning_rate()[source]
Set learning schedule with exponential decay
- Returns:
exponential decay learning rate schedule
- Return type:
keras.optimizers.schedules.ExponentialDecay
- fit(params=None)[source]
Fit a model to the training data.
- Parameters:
params (dictionary, optional) – set custom fit params, by default None
- Returns:
Keras training history object
- Return type:
tf.keras.model.history
- fit_params(batch_size=32, epochs=60, lr=0.0001, decay=[100000, 0.96], early_stopping=None, verbose=2, ensemble=False)[source]
Set custom model fitting parameters as Builder object attributes.
- Parameters:
batch_size (int, optional) – size of each training batch, by default 32
epochs (int, optional) – number of epochs, by default 60
lr (float, optional) – initial learning rate, by default 1e-4
decay (list, optional) – decay_steps, decay_rate, by default [100000, 0.96]
early_stopping (str, optional) – use an early stopping callback, by default None
verbose (int, optional) – set the verbosity level, by default 2
ensemble (bool, optional) – ensemble type network, by default False
- Returns:
spacekit.builder.architect.Builder class object with updated fitting parameters.
- Return type:
self
- load_saved_model(arch=None, compile_params=None, custom_obj={}, extract_to='models', keras_archive=True)[source]
Load saved keras model from local disk (located at the
model_path
attribute) or a pre-trained model from spacekit.skopes.trained_networks (ifmodel_path
attribute is None). Example forcompile_params
:dict(loss="binary_crossentropy",metrics=["accuracy"], optimizer=Adam(learning_rate=optimizers.schedules.ExponentialDecay(lr=1e-4, decay_steps=100000, decay_rate=0.96, staircase=True)))
- Parameters:
arch (str, optional) – select a pre-trained model from the spacekit library (“svm_align”, “jwst_cal”, or “hst_cal”), by default None
compile_params (dict, optional) – Compile the model using kwarg parameters, by default None
custom_obj (dict, optional) – custom objects keyword arguments to be passed into Keras
load_model
, by default {}extract_to (str or path, optional) – directory location to extract into, by default “models”
keras_archive (bool, optional) – for models saved in the newer high-level .keras compressed archive format (recommended), by default True
- Returns:
pre-trained (and/or compiled) functional Keras model.
- Return type:
Keras functional Model object
- model_diagram(model=None, output_path=None, show_shapes=True, show_dtype=False, LR=False, expand_nested=True, show_layer_names=False)[source]
- save_model(weights=True, output_path='.', keras_archive=True, parent_dir='')[source]
The model architecture, and training configuration (including the optimizer, losses, and metrics) are stored in saved_model.pb. The weights are saved in the variables/ directory.
- set_build_params(input_shape=None, output_shape=None, layers=None, kernel_size=None, activation=None, cost_function=None, strides=None, optimizer=None, lr_sched=None, loss=None, metrics=None, input_name=None, output_name=None, name=None, algorithm=None)[source]
Set custom build parameters for a Builder object.
- Parameters:
input_shape (tuple, optional) – shape of the inputs, by default None
output_shape (tuple, optional) – shape of the output, by default None
layers (list, optional) – sizes of hidden (dense) layers, by default None
kernel_size (int, optional) – size of the kernel, by default None
activation (string, optional) – dense layer activation, by default None
cost_function (str, optional) – function to update weights (calculate cost), by default None
strides (int, optional) – number of strides, by default None
optimizer (object, optional) – type of optimizer to use, by default None
lr_sched (bool, optional) – use a learning_rate schedule such as ExponentialDecay, by default None
loss (string, optional) – loss metric to monitor, by default None
metrics (list, optional) – metrics for model to train on, by default None
algorithm (str, optional) – analysis type, used for determining spacekit.analyzer.Compute class e.g. “linreg” for linear regression or “multiclass” for multi-label classification, by default None
- Returns:
spacekit.builder.architect.Builder class object with updated attributes
- Return type:
self
- class spacekit.builder.architect.BuilderEnsemble(X_train=None, y_train=None, X_test=None, y_test=None, params=None, input_name='svm_mixed_inputs', output_name='ensemble_output', **builder_kwargs)[source]
Bases:
Builder
Subclass for building and training an ensemble model (stacked MLP and 3D CNN)
- Parameters:
Builder (class) – spacekit.builder.architect.Builder class object
- batch()[source]
Gives equal number of positive and negative samples rotating randomly The output of the generator must be either - a tuple
(inputs, targets)
- a tuple(inputs, targets, sample_weights)
.This tuple (a single output of the generator) makes a single batch. The last batch of the epoch is commonly smaller than the others, if the size of the dataset is not divisible by the batch size. The generator loops over its data indefinitely. An epoch finishes when
steps_per_epoch
batches have been seen by the model.
- build()[source]
Builds and compiles the ensemble model
- Returns:
spacekit.builder.architect.Builder.BuilderEnsemble object with
model
attribute initialized- Return type:
self
- class spacekit.builder.architect.BuilderMLP(X_train=None, y_train=None, X_test=None, y_test=None, blueprint='mlp', **builder_kwargs)[source]
Bases:
Builder
Subclass for building and training MLP neural networks
- Parameters:
Builder (class) – spacekit.builder.architect.Builder class object
- batch()[source]
Randomly rotates through positive and negative samples indefinitely, generating a single batch tuple of (inputs, targets) or (inputs, targets, sample_weights). If the size of the dataset is not divisible by the batch size, the last batch will be smaller than the others. An epoch finishes once
steps_per_epoch
have been seen by the model.- Yields:
tuple – a single batch tuple of (inputs, targets) or (inputs, targets, sample_weights).
- class spacekit.builder.architect.BuilderCNN3D(X_train=None, y_train=None, X_test=None, y_test=None, blueprint='cnn3d', **builder_kwargs)[source]
Bases:
Builder
Subclass for building and training 3D convolutional neural networks
- Parameters:
Builder (class) – spacekit.builder.architect.Builder class object
- batch()[source]
Gives equal number of positive and negative samples rotating randomly The output of the generator must be either - a tuple
(inputs, targets)
- a tuple(inputs, targets, sample_weights)
.This tuple (a single output of the generator) makes a single batch. The last batch of the epoch is commonly smaller than the others, if the size of the dataset is not divisible by the batch size. The generator loops over its data indefinitely. An epoch finishes when
steps_per_epoch
batches have been seen by the model.
- class spacekit.builder.architect.BuilderCNN2D(X_train=None, y_train=None, X_test=None, y_test=None, blueprint='cnn2d', **builder_kwargs)[source]
Bases:
Builder
Subclass Builder object for 2D Convolutional Neural Networks
- Parameters:
Builder (class) – spacekit.builder.architect.Builder.Builder object
- batch()[source]
Gives equal number of positive and negative samples rotating randomly The output of the generator must be either - a tuple
(inputs, targets)
- a tuple(inputs, targets, sample_weights)
.This tuple (a single output of the generator) makes a single batch. Therefore, all arrays in this tuple must have the same length (equal to the size of this batch). Different batches may have different sizes.
For example, the last batch of the epoch is commonly smaller than the others, if the size of the dataset is not divisible by the batch size. The generator is expected to loop over its data indefinitely. An epoch finishes when
steps_per_epoch
batches have been seen by the model.
- class spacekit.builder.architect.MemoryClassifier(X_train=None, y_train=None, X_test=None, y_test=None, blueprint='hst_mem_clf', test_idx=None, **builder_kwargs)[source]
Bases:
BuilderMLP
Builds an MLP neural network classifier object with pre-tuned params for Calcloud’s Memory Bin classifier
- Parameters:
MultiLayerPerceptron (object) – mlp multi-classification builder object
- class spacekit.builder.architect.MemoryRegressor(X_train=None, y_train=None, X_test=None, y_test=None, blueprint='hst_mem_reg', test_idx=None, **builder_kwargs)[source]
Bases:
BuilderMLP
Builds an MLP neural network regressor object with pre-tuned params for estimating memory allocation (GB) in Calcloud
- Parameters:
MultiLayerPerceptron (object) – mlp linear regression builder object
- class spacekit.builder.architect.WallclockRegressor(X_train=None, y_train=None, X_test=None, y_test=None, blueprint='hst_wall_reg', test_idx=None, **builder_kwargs)[source]
Bases:
BuilderMLP
Builds an MLP neural network regressor object with pre-tuned params for estimating wallclock allocation (minimum execution time in seconds) in Calcloud.
- Parameters:
MultiLayerPerceptron (object) – mlp linear regression builder object