API Reference

ImageTable

The ImageTable class is a specialized version of swat.CASTable that includes extra methods for working with images.

Constructors

ImageTable(name, **table_params)

Specialized CASTable for Image Data

ImageTable.from_table(tbl[, image_col, …])

Create an ImageTable from a CASTable

ImageTable.load_files(conn, path[, casout, …])

Create ImageTable from files in path

Saving

ImageTable.to_files(path)

Save the images in the original format under the specified directory

ImageTable.to_sashdat([path, name])

Save the ImageTable to a sashdat file

Utilities

ImageTable.copy_table([casout])

Create a copy of the ImageTable

ImageTable.show([nimages, ncol, randomize, …])

Display a grid of images

Image Processing

ImageTable.crop([x, y, width, height, inplace])

Crop the images in the ImageTable

ImageTable.resize([width, height, inplace, …])

Resize the images in the ImageTable

ImageTable.as_patches([x, y, width, height, …])

Generate patches from the images in the ImageTable

ImageTable.as_random_patches([random_ratio, …])

Generate random patches from the images in the ImageTable

ImageTable.random_mutations([color_jitter, …])

Generate random mutations from the images in the ImageTable

AudioTable

The AudioTable class is a specialized version of swat.CASTable that includes extra methods for working with audio data.

Constructors

AudioTable(name, **table_params)

CAS Table for Audio

AudioTable.create_audio_table(conn, …[, …])

Creates an Audio table and takes care of all the necessary steps

AudioTable.create_audio_table_speechrecognition(…)

Creates an Audio table and takes care of all the necessary steps

AudioTable.from_audio_sashdat(conn, path[, …])

Create an AudioTable from a sashdat file

AudioTable.load_audio_files(conn[, path, …])

Load audio files from path

Audio Processing

AudioTable.create_audio_feature_table([…])

Extracts audio features from the audio table and create a new CASTable that contains the features.

AudioTable.extract_audio_features(conn, table)

Extracts audio features from the audio files

AudioTable.load_audio_metadata(conn, path, …)

Pre-process and loads the metadata

AudioTable.load_audio_metadata_speechrecognition(…)

Pre-process and loads the metadata

Timeseries

plot_timeseries(tbl, timeid, timeseries[, …])

Create a timeseries line plot from a CASTable or pandas DataFrame

TimeseriesTable

The TimeseriesTable class is a specialized version of swat.CASTable that includes extra methods for working with timeseries.

Constructors

TimeseriesTable(name[, timeid, groupby_var, …])

Table for preprocessing timeseries

TimeseriesTable.from_table(tbl[, columns, …])

Create an TimeseriesTable from a CASTable

TimeseriesTable.from_pandas(conn, pandas_df)

Create an TimeseriesTable from a pandas DataFrame or Series

TimeseriesTable.from_localfile(conn, path[, …])

Create an TimeseriesTable from a file on the client side.

TimeseriesTable.from_serverfile(conn, path)

Create an TimeseriesTable from a file on the server side

Utilities

TimeseriesTable.timeseries_formatting(…[, …])

Format the TimeseriesTable

TimeseriesTable.timeseries_accumlation([…])

Accumulate the TimeseriesTable into regular consecutive intervals

TimeseriesTable.prepare_subsequences(…[, …])

Prepare the subsequences that will be pass into RNN

TimeseriesTable.timeseries_partition([…])

Split the dataset into training, validation and testing set

Layers

Layer([name, config, src_layers])

Base class for all layers

InputLayer([n_channels, width, height, …])

Input layer

BN([name, act, fcmp_act, src_layers])

Batch normalization layer

ChannelShuffle([name, n_groups, scale, …])

Channel Shuffle layer

Concat([name, act, fcmp_act, src_layers])

Concat layer Concatenate source layers’ outputs along the last dimension

Conv1d(n_filters[, width, stride, name, …])

Convolution layer in 1D

Conv2d(n_filters[, width, height, stride, …])

Convolution layer in 2D

Conv2DTranspose(n_filters[, height, width, …])

Transpose Convolution layer in 2D

Dense(n[, name, act, fcmp_act, init, std, …])

Fully connected layer

Detection([name, act, detection_model_type, …])

Detection layer

EmbeddingLoss([name, margin, src_layers])

EmbeddingLoss layer

FastRCNN([name, act, class_number, …])

FastRCNN layer

FCMPLayer(width, height, depth, n_weights, …)

FCMP layer

GlobalAveragePooling2D([name, src_layers])

Global Average Pooling layer in 2D

GroupConv2d(n_filters[, n_groups, width, …])

Group Convolution layer in 2D

Keypoints([name, act, fcmp_act, init, std, …])

Keypoints layer

LayerNormalization([name, act, epsilon, …])

Layer normalization layer

MultiHeadAttention(n, n_attn_heads[, name, …])

Multi-head attention layer from “Attention is All You Need” (Vaswani et al., NIPS 2017)

Pooling([width, height, stride, name, …])

Pooling layer

Proj(embedding_size, alphabet_size[, name, …])

Projection layer

Recurrent(n[, name, act, fcmp_act, init, …])

RNN layer

RegionProposal(anchor_ratio, anchor_scale[, …])

Region proposal layer

Res([name, act, fcmp_act, src_layers])

Residual layer Element-wise addition

Reshape([name, act, fcmp_act, width, …])

Reshape layer

ROIPooling([name, act, output_height, …])

ROIPooling layer

Scale([name, act, fcmp_act, src_layers])

Scale layer

Segmentation([name, act, error, …])

Segmentation layer

Split(n_destination_layers[, name, src_layers])

Split layer

Survival([name, src_layers])

Survival layer

OutputLayer([name, act, fcmp_act, fcmp_err, …])

Output layer

Model

The Model class is a specialized version of dlpy.Network that adds training, evaluation, tuning, and feature analysis routines.

Constructors

Model(conn[, inputs, outputs, model_table, …])

Model.from_table(input_model_table[, …])

Create a Model object from CAS table that defines a deep learning model

Model.from_keras_model(conn, keras_model[, …])

Generate a model object from a Keras model object

Model.from_caffe_model(conn, input_network_file)

Generate a model object from a Caffe model proto file (e.g. *.prototxt), and convert the weights (e.g. *.caffemodel) to a SAS capable file (e.g. *.caffemodel.h5).

Model.from_onnx_model(conn, onnx_model[, …])

Generate a Model object from ONNX model.

Model.from_sashdat(conn, path[, …])

Generate a model object using the model information in the sashdat file

Model.load(path[, display_note])

Load the deep learning model architecture from existing table

Model Setup

Model.change_labels(label_file, id_column, …)

Overrides the labels already in the model

Model.set_weights(weight_tbl)

Assign weights to the Model object

Model.load_weights(path[, labels, …])

Load the weights from a data file specified by ‘path’

Model.load_weights_from_caffe(path[, …])

Load the model weights from a HDF5 file

Model.load_weights_from_keras(path[, …])

Load the model weights from a HDF5 file

Model.load_weights_from_table(path)

Load the weights from a file

Model.set_weights_attr(attr_tbl[, clear])

Attach the weights attribute to the model weights

Model.load_weights_attr(path)

Load the weights attribute form a sashdat file

Training

Model.fit(data[, inputs, target, …])

Fitting a deep learning model.

Model.fit_and_visualize(data[, inputs, …])

Fitting a deep learning model while visulizing the fit and loss at each iteration.

Model.tune(data[, inputs, target])

Tunes hyper parameters for the deep learning model.

Model.plot_training_history([items, …])

Display the training iteration history.

Inference, Evaluation, and Analysis

Model.predict(data[, text_parms, layer_out, …])

Evaluate the deep learning model on a specified validation data set

Model.forecast([test_table, horizon, …])

Make forecasts based on deep learning models trained on TimeseriesTable.

Model.evaluate(data[, text_parms, …])

Evaluate the deep learning model on a specified validation data set

Model.evaluate_object_detection(…[, …])

Evaluate the deep learning model on a specified validation data set.

Model.plot_evaluate_res([cas_table, …])

Plot the bar chart of the classification predictions

Model.get_feature_maps(data[, label, idx, …])

Extract the feature maps for a single image

Model.get_features(data, dense_layer[, target])

Extract linear features for a data table from the layer specified by dense_layer

Model.heat_map_analysis([data, mask_width, …])

Conduct a heat map analysis on table of images

Model.plot_heat_map([idx, alpha])

Display the heat maps analysis results

Saving

Model.save_to_astore([path, layers])

Save the model to an astore object, and write it into a file.

Model.save_to_table(path)

Save the model as SAS dataset

Model.deploy(path[, output_format, …])

Deploy the deep learning model to a data file

Architecture Information

Model.count_params()

Count the total number of parameters in the model

Model.print_summary()

Display a table that summarizes the model architecture

Model.plot_network()

Display a graph that summarizes the model architecture.

Model.get_model_info()

Return the information about the model table

Solvers

Solver([learning_rate, …])

Solver object

NatGradSolver([approximation_type, …])

Natural gradient solver object

LBFGSolver(m, max_line_search_iters, …[, …])

LBFG solver object

AdamSolver([beta1, beta2, learning_rate, …])

Adam solver object

MomentumSolver([momentum, learning_rate, …])

Momentum solver object

VanillaSolver([learning_rate, …])

Vanilla solver object

Optimizer

Optimizer([algorithm, mini_batch_size, …])

Optimizer object

Learning Rate Scheduler

FCMPLR(conn, fcmp_learning_rate[, …])

FCMP learning rate scheduler.

FixedLR([learning_rate])

Fixed learning rate scheduler

StepLR([learning_rate, gamma, step_size])

Step learning rate scheduler The learning rate is reduced by a factor(gamma) at certain intervals(step_size) Example: # reduce learning rate every 2 epochs lr_scheduler = StepLR(learning_rate=0.0001, gamma=0.1, step_size=2) solver = MomentumSolver(lr_scheduler = lr_scheduler, clip_grad_max = 100, clip_grad_min = -100)

MultiStepLR(learning_rate, gamma, steps)

Multiple steps learning rate scheduler The initial learning rate is decayed by gamma once the number of epoch reaches one of the steps. Example: # reduce learning rate by 0.1 at 20th, 50th, 80th epochs lr_scheduler = MultiStepLR(learning_rate=0.0001, gamma=0.1, steps=[20, 50, 80]) solver = MomentumSolver(lr_scheduler = lr_scheduler, clip_grad_max = 100, clip_grad_min = -100).

PolynomialLR(learning_rate, power)

Polynomial learning rate scheduler Applies a polynomial decay to the learning rate calculated by: lr = initial_lr * (1 −iter / maxiter ) ^ power

ReduceLROnPlateau(conn, learning_rate[, …])

Reduce learning rate on plateau learning rate scheduler Reduce learning rate when loss has stopped improving for a certain number of epochs(patience). Example: lr_scheduler = ReduceLROnPlateau(conn=sess, cool_down_iters=2, gamma=0.1, learning_rate=0.01, patience=3) solver = MomentumSolver(lr_scheduler = lr_scheduler, clip_grad_max = 100, clip_grad_min = -100).

CyclicLR(conn, data, batch_size, factor, …)

Cyclic learning rate scheduler The policy cycles the learning rate between two boundaries[learning_rate, max_lr] with a constant frequency which can be adjusted by factor. The learning rate changes after every batch. batch_size and data are necessary to determine how many batches an epoch requires. Example: lr_scheduler = CyclicLR(conn=sess, data=my_images, max_lr=0.01, batch_size=1, factor=2, learning_rate=0.0001) solver = MomentumSolver(lr_scheduler = lr_scheduler, clip_grad_max = 100, clip_grad_min = -100).

Parameters

DataSpec(type_, layer[, data, data_layer, …])

Data spec parameters.

DataSpecNumNomOpts(length[, token_size])

Data spec numeric nominal parameters.

Sequence([input_length, target_length, …])

Sequence parameters object

TextParms([init_input_embeddings, …])

Text parameters object

Gpu([devices, use_tensor_rt, precision, …])

Gpu parameters object.

Metrics

accuracy_score(y_true, y_pred[, castable, …])

Computes the classification accuracy score.

confusion_matrix(y_true, y_pred[, castable, …])

Computes the confusion matrix of a classification task.

plot_roc(y_true, y_score, pos_label[, …])

Plot the receiver operating characteristic (ROC) curve for binary classification tasks.

plot_precision_recall(y_true, y_score, pos_label)

Plot the precision recall(PR) curve for binary classification tasks.

roc_auc_score(y_true, y_score, pos_label[, …])

Compute the area under the receiver operating characteristic (ROC) curve for binary classification tasks.

average_precision_score(y_true, y_score, …)

Compute the average precision score for binary classification tasks.

f1_score(y_true, y_pred, pos_label[, …])

Compute the f1 score of the binary classification task. f1 score is defined as

explained_variance_score(y_true, y_pred[, …])

Compute the explained variance score for a regression task.

mean_absolute_error(y_true, y_pred[, …])

Compute the mean absolute error of a regression task.

mean_squared_error(y_true, y_pred[, …])

Compute the mean squared error of a regression task.

mean_squared_log_error(y_true, y_pred[, …])

Compute the mean squared logarithmic error of the regression tasks.

r2_score(y_true, y_pred[, castable, id_vars])

Compute the R^2 (coefficient of determination) regression score.

Feature Maps

Constructor

FeatureMaps(conn, feature_maps_tbl[, structure])

Feature Maps object

Utilities

FeatureMaps.display(layer_id[, filter_id])

Display the feature maps

Sequential Model

Constructor

Sequential(conn[, layers, model_table])

Model for sequentially building of deep learning models

Utilities

Sequential.add(layer)

Add layer(s) to model

Sequential.pop([loc])

Delete layer(s) from model and return it

Sequential.switch(loc1, loc2)

Switch the order of two layers in the model.

Sequential.compile()

Convert the layer objects into CAS action parameters

Residual Networks

ResBlock([kernel_sizes, n_filters, strides, …])

Base class for the residual blocks.

ResBlock.compile(src_layer[, block_num])

Convert the block structure into DLPy layer definitions.

ResBlockBN([kernel_sizes, n_filters, …])

Residual block for Residual Network with batch normalization.

ResBlockBN.compile(src_layer[, block_num])

Convert the block structure into DLPy layer definitions.

ResBlock_Caffe([kernel_sizes, n_filters, …])

Residual block for Residual Network with batch normalization.

ResBlock_Caffe.compile(src_layer[, block_num])

Compile the block structure into DLPy layer definitions.

Bidirectional(n[, n_blocks, rnn_type, …])

Bidirectional RNN layers

Bidirectional.compile([block_num])

Convert the options into DLPy layer definition.

Pre-Built Models for Computer Vision Tasks

Image Classification

LeNet5(conn[, model_table, n_classes, …])

Generates a deep learning model with the LeNet5 architecture.

VGG11(conn[, model_table, n_classes, …])

Generates a deep learning model with the VGG11 architecture.

VGG13(conn[, model_table, n_classes, …])

Generates a deep learning model with the VGG13 architecture.

VGG16(conn[, model_table, n_classes, …])

Generates a deep learning model with the VGG16 architecture.

VGG19(conn[, model_table, n_classes, …])

Generates a deep learning model with the VGG19 architecture.

ResNet18_SAS(conn[, model_table, …])

Generates a deep learning model with the ResNet18 architecture.

ResNet18_Caffe(conn[, model_table, …])

Generates a deep learning model with the ResNet18 architecture with convolution shortcut.

ResNet34_SAS(conn[, model_table, n_classes, …])

Generates a deep learning model with the ResNet34 architecture.

ResNet34_Caffe(conn[, model_table, …])

Generates a deep learning model with the ResNet34 architecture with convolution shortcut.

ResNet50_SAS(conn[, model_table, n_classes, …])

Generates a deep learning model with the ResNet50 architecture.

ResNet50_Caffe(conn[, model_table, …])

Generates a deep learning model with the ResNet50 architecture with convolution shortcut.

ResNet101_SAS(conn[, model_table, …])

Generates a deep learning model with the ResNet101 architecture.

ResNet101_Caffe(conn[, model_table, …])

Generates a deep learning model with the ResNet101 architecture with convolution shortcut.

ResNet152_SAS(conn[, model_table, …])

Generates a deep learning model with the SAS ResNet152 architecture.

ResNet152_Caffe(conn[, model_table, …])

Generates a deep learning model with the ResNet152 architecture with convolution shortcut

ResNet_Wide(conn[, model_table, …])

Generate a deep learning model with Wide ResNet architecture.

DenseNet(conn[, model_table, n_classes, …])

Generates a deep learning model with the DenseNet architecture.

DenseNet121(conn[, model_table, n_classes, …])

Generates a deep learning model with the DenseNet121 architecture.

Darknet(conn[, model_table, n_classes, act, …])

Generate a deep learning model with the Darknet architecture.

Darknet_Reference(conn[, model_table, …])

Generates a deep learning model with the Darknet_Reference architecture.

InceptionV3(conn[, model_table, n_classes, …])

Generates a deep learning model with the Inceptionv3 architecture with batch normalization layers.

MobileNetV1(conn[, model_table, n_classes, …])

Generates a deep learning model with the MobileNetV1 architecture.

MobileNetV2(conn[, model_table, n_classes, …])

Generates a deep learning model with the MobileNetV2 architecture.

ShuffleNetV1(conn[, model_table, n_classes, …])

Generates a deep learning model with the ShuffleNetV1 architecture.

Object Detection

YoloV1(conn[, model_table, n_channels, …])

Generates a deep learning model with the Yolo V1 architecture.

YoloV2(conn, anchors[, model_table, …])

Generates a deep learning model with the Yolov2 architecture.

YoloV2_MultiSize(conn, anchors[, …])

Generates a deep learning model with the Yolov2 architecture.

Tiny_YoloV1(conn[, model_table, n_channels, …])

Generates a deep learning model with the Tiny Yolov1 architecture.

Tiny_YoloV2(conn, anchors[, model_table, …])

Generate a deep learning model with the Tiny Yolov2 architecture.

Faster_RCNN(conn[, model_table, n_channels, …])

Generates a deep learning model with the faster RCNN architecture.

Segmentation

UNet(conn[, model_table, n_classes, …])

Generates a deep learning model with the U-Net architecture.

Image Captioning

ImageCaptioning(conn[, model_name, …])

Builds an RNN to be used for image captioning

create_captioning_table(conn, image_table, …)

Builds CASTable with all necessary info to train an image captioning model

create_captions_table(conn, captions_file[, …])

Generate CASTable of captions and filenames

create_embeddings_from_object_detection(…)

Builds CASTable with objects detected in images as numeric data

display_predicted_image_captions(conn, …)

Shows caption prediction for random images

get_image_features(conn, model, image_table, …)

Generate CASTable of image features

reshape_caption_columns(conn, table[, …])

Reshapes table so there is only one caption per row of the table

Image Embedding

Image Embedding Table

ImageEmbeddingTable(name, **table_params)

Specialized CASTable for Image Embedding Data.

ImageEmbeddingTable.load_files(conn, path[, …])

Create ImageEmbeddingTable from files in path

ImageEmbeddingTable.show([n_image_pairs, …])

Display a grid of images for ImageEmbeddingTable

ImageEmbeddingTable.label_freq

Summarize the distribution of different image pairs in the ImageEmbeddingTable

Embedding Model

EmbeddingModel(conn[, inputs, outputs, …])

EmbeddingModel.build_embedding_model(branch)

Build an embedding model based on a given model branch and model type

EmbeddingModel.fit_embedding_model(optimizer)

Fitting a deep learning model for embedding learning.

EmbeddingModel.deploy_embedding_model(path)

Deploy the deep learning model to a data file

Pre-Built Models for NLP Tasks

TextClassification(conn[, model_table, …])

Generates a text classification model

TextGeneration(conn[, model_table, neurons, …])

Generates a text generation model.

SequenceLabeling(conn[, model_table, …])

Generates a sequence labeling model.

SpeechRecognition(conn[, model_table, …])

Generates a speech recognition model.

Speech

Constructor

Speech(conn, data_path[, local_path, …])

Class to do speech recognition using SAS Viya.

Utilities

Speech.load_acoustic_model(acoustic_model_path)

Load the RNN acoustic model.

Speech.load_language_model(language_model_path)

Load the N-gram language model.

Speech.transcribe(audio_path[, …])

Transcribe the audio file into text.

Speech Utilities

read_audio(path)

Read the audio from path into a wave_read object.

check_framerate(params, framerate)

Check if the input audio has the desired framerate (sampling rate).

check_sampwidth(params, sampwidth)

Check if the input audio has the desired sampwidth (byte width).

convert_framerate(fragment, width, …)

Convert framerate (sampling rate) of the input fragment.

convert_sampwidth(fragment, sampwidth_in, …)

Convert the sampwidth (byte width) of the input fragment between 1-, 2-, 3-, 4-byte formats.

calculate_segment_nframes(path, segment_len)

Calculate the number of frames of every segment split from the audio input.

segment_audio(path, local_path, …)

Segment the audio into pieces shorter than segment_len.

clean_audio(listing_path_local, …)

Remove the temporary listing file and the temporary audio files.

check_stereo(params)

Check if the input audio has 2 channels (stereo).

convert_stereo_to_mono(fragment, width)

Convert stereo fragment to mono.

play_one_audio_file(local_audio_file)

Play a local audio file using soundfile and sounddevice.

display_spectrogram_for_one_audio_file(…)

Display spectrogram for a local audio file using soundfile.

display_raw_data_for_one_audio_file(…)

Display raw data for a local audio file using soundfile.

convert_one_audio_file(local_audio_file, …)

Convert a local audio file into a wav format that only contains 1 channel with 16 bits and 16K HZ.

convert_audio_files(local_audio_path[, recurse])

Convert audio files under a local path into wave files that only contains 1 channel with 16 bits and 16K HZ.

convert_one_audio_file_to_specgram(…)

Convert a local audio file into a png format with spectrogram.

convert_audio_files_to_specgrams(…[, recurse])

Convert audio files under a local path into the images (PNG) that contain spectrogram.

Splitting Utilities

two_way_split(tbl[, test_rate, stratify, …])

Split image data into training and testing sets

three_way_split(tbl[, valid_rate, …])

Split image data into training and testing sets