Workflows

The SWAT package includes the ability to call CAS actions and process the output in various ways. These range from simple (calling CAS actions as Python methods and getting a dictionary of results back) to complex (invoking CAS actions in multiple sessions and handling server responses from them directly). The various workflows are described in the following sections to give you an idea of how the classes in SWAT interact with each other.

The Easy Way

In the most basic form of calling CAS actions, the actions are called directly on the connection object or a CASTable object. When you load a CAS action set (using builtins.loadactionset), the CAS actions in that action set are added to the connection object as Python methods. For example, when the simple action set is loaded, the CAS connection object is extended with methods for summary, freq, crosstab, etc. These can then be called on the CAS connection object directly.

In addition, any CASTable objects that are registered with that connection also get methods for those actions. When you call a CAS action on a CASTable object, the table= (or name= and caslib= in some cases) parameter get populated automatically with that table object before it is sent to the CAS server.

Here is a simple example. We are reading a CSV file using the CAS.read_csv() method. This loads the CSV data into a CAS table and returns a CASTable object. You can then use many of the standard pandas.DataFrame methods such as pandas.DataFrame.head() to interact with the CAS table on the server.

In [1]: import swat

In [2]: conn = swat.CAS(host, port, userid, password)

In [3]: tbl = conn.read_csv('https://raw.githubusercontent.com/'
   ...:                     'sassoftware/sas-viya-programming/master/data/cars.csv')
   ...: 

Note: Cloud Analytic Services made the uploaded file available as table TMPJEHQ5PCS in caslib CASUSER(castest).

Note: The table TMPJEHQ5PCS has been created in caslib CASUSER(castest) from binary data uploaded to Cloud Analytic Services.
In [4]: tbl.head() Out[4]: Selected Rows from Table TMPJEHQ5PCS Make Model Type ... Weight Wheelbase Length 0 Acura MDX SUV ... 4451.0 106.0 189.0 1 Acura RSX Type S 2dr Sedan ... 2778.0 101.0 172.0 2 Acura TSX 4dr Sedan ... 3230.0 105.0 183.0 3 Acura TL 4dr Sedan ... 3575.0 108.0 186.0 4 Acura 3.5 RL 4dr Sedan ... 3880.0 115.0 197.0 [5 rows x 15 columns]

You can call CAS actions directly on the CAS connection object and pass in the table argument, or you can call the CAS action directly on the CASTable object from CAS.read_csv().

In [5]: conn.summary(table=tbl)
Out[5]: 
[Summary]

 Descriptive Statistics for TMPJEHQ5PCS
 
         Column      Min       Max    ...              ProbT  Skewness   Kurtosis
 0         MSRP  10280.0  192465.0    ...      4.160412e-127  2.798099  13.879206
 1      Invoice   9875.0  173560.0    ...      2.684398e-128  2.834740  13.946164
 2   EngineSize      1.3       8.3    ...      3.133745e-209  0.708152   0.541944
 3    Cylinders      3.0      12.0    ...      1.515569e-251  0.592785   0.440378
 4   Horsepower     73.0     500.0    ...      4.185344e-216  0.930331   1.552159
 5     MPG_City     10.0      60.0    ...      1.866284e-257  2.782072  15.791147
 6  MPG_Highway     12.0      66.0    ...      1.665621e-292  1.252395   6.045611
 7       Weight   1850.0    7190.0    ...      5.812547e-294  0.891824   1.688789
 8    Wheelbase     89.0     144.0    ...       0.000000e+00  0.962287   2.133649
 9       Length    143.0     238.0    ...       0.000000e+00  0.181977   0.614725
 
 [10 rows x 17 columns]

+ Elapsed: 0.0171s, user: 0.015s, sys: 0.005s, mem: 5.36mb

In [6]: tbl.summary()
Out[6]: 
[Summary]

 Descriptive Statistics for TMPJEHQ5PCS
 
         Column      Min       Max    ...              ProbT  Skewness   Kurtosis
 0         MSRP  10280.0  192465.0    ...      4.160412e-127  2.798099  13.879206
 1      Invoice   9875.0  173560.0    ...      2.684398e-128  2.834740  13.946164
 2   EngineSize      1.3       8.3    ...      3.133745e-209  0.708152   0.541944
 3    Cylinders      3.0      12.0    ...      1.515569e-251  0.592785   0.440378
 4   Horsepower     73.0     500.0    ...      4.185344e-216  0.930331   1.552159
 5     MPG_City     10.0      60.0    ...      1.866284e-257  2.782072  15.791147
 6  MPG_Highway     12.0      66.0    ...      1.665621e-292  1.252395   6.045611
 7       Weight   1850.0    7190.0    ...      5.812547e-294  0.891824   1.688789
 8    Wheelbase     89.0     144.0    ...       0.000000e+00  0.962287   2.133649
 9       Length    143.0     238.0    ...       0.000000e+00  0.181977   0.614725
 
 [10 rows x 17 columns]

+ Elapsed: 0.0156s, user: 0.011s, sys: 0.007s, mem: 5.34mb

In [7]: conn.close()

The output from the CAS action methods is always a CASResults object. This is simply an ordered Python dictionary with a few extra attributes and methods added. You can see the keys printed in the output above (surround by square brackets). This output only contains a single key: Summary.

Here is a diagram showing the process of calling and action and consuming the responses into the CASResults object.

_images/easyway-workflow.png

Using Response and Result Callbacks

The next workflow is to use callback functions to handle either the responses from the CAS server, or the individual result keys in the responses. You still use the CAS action methods on the CAS connection object, but this time you add either a responsefunc= or resultfunc= function argument.

The result callback function takes five arguments: key, value, response, connection, and userdata. Those are the result key and value, the response the result belongs to, the connection that the result belongs to, and an arbitrary user data structure.

The response callback just takes three arguments for the response, connection, and user data structure.

The result callback and response callback are called for each result and response, respectively. Keep in mind that you can only specify one or the other (a response callback would override a result callback). If you want to keep any sort of state information between calls, you can store it in the userdata argument and return it. The returned value will get passed in as the userdata argument on the next call.

Here is an example demonstrating both styles of callbacks.

In [8]: import swat

In [9]: conn = swat.CAS(host, port, userid, password)

In [10]: tbl = conn.read_csv('https://raw.githubusercontent.com/'
   ....:                     'sassoftware/sas-viya-programming/master/data/cars.csv')
   ....: 

Note: Cloud Analytic Services made the uploaded file available as table TMP22PUYQOE in caslib CASUSER(castest).

Note: The table TMP22PUYQOE has been created in caslib CASUSER(castest) from binary data uploaded to Cloud Analytic Services.
In [11]: tbl.head() Out[11]: Selected Rows from Table TMP22PUYQOE Make Model Type ... Weight Wheelbase Length 0 Acura MDX SUV ... 4451.0 106.0 189.0 1 Acura RSX Type S 2dr Sedan ... 2778.0 101.0 172.0 2 Acura TSX 4dr Sedan ... 3230.0 105.0 183.0 3 Acura TL 4dr Sedan ... 3575.0 108.0 186.0 4 Acura 3.5 RL 4dr Sedan ... 3880.0 115.0 197.0 [5 rows x 15 columns] In [12]: def result_cb(key, value, response, connection, userdata): ....: print('>>> RESULT', key, value) ....: return userdata ....: In [13]: tbl.summary(resultfunc=result_cb) >>> RESULT Summary Descriptive Statistics for TMP22PUYQOE Column Min Max ... ProbT Skewness Kurtosis 0 MSRP 10280.0 192465.0 ... 4.160412e-127 2.798099 13.879206 1 Invoice 9875.0 173560.0 ... 2.684398e-128 2.834740 13.946164 2 EngineSize 1.3 8.3 ... 3.133745e-209 0.708152 0.541944 3 Cylinders 3.0 12.0 ... 1.515569e-251 0.592785 0.440378 4 Horsepower 73.0 500.0 ... 4.185344e-216 0.930331 1.552159 5 MPG_City 10.0 60.0 ... 1.866284e-257 2.782072 15.791147 6 MPG_Highway 12.0 66.0 ... 1.665621e-292 1.252395 6.045611 7 Weight 1850.0 7190.0 ... 5.812547e-294 0.891824 1.688789 8 Wheelbase 89.0 144.0 ... 0.000000e+00 0.962287 2.133649 9 Length 143.0 238.0 ... 0.000000e+00 0.181977 0.614725 [10 rows x 17 columns] In [14]: def response_cb(response, connection, userdata): ....: for k, v in response: ....: print('>>> RESPONSE', k, v) ....: return userdata ....: In [15]: tbl.summary(responsefunc=response_cb) >>> RESPONSE Summary Descriptive Statistics for TMP22PUYQOE Column Min Max ... ProbT Skewness Kurtosis 0 MSRP 10280.0 192465.0 ... 4.160412e-127 2.798099 13.879206 1 Invoice 9875.0 173560.0 ... 2.684398e-128 2.834740 13.946164 2 EngineSize 1.3 8.3 ... 3.133745e-209 0.708152 0.541944 3 Cylinders 3.0 12.0 ... 1.515569e-251 0.592785 0.440378 4 Horsepower 73.0 500.0 ... 4.185344e-216 0.930331 1.552159 5 MPG_City 10.0 60.0 ... 1.866284e-257 2.782072 15.791147 6 MPG_Highway 12.0 66.0 ... 1.665621e-292 1.252395 6.045611 7 Weight 1850.0 7190.0 ... 5.812547e-294 0.891824 1.688789 8 Wheelbase 89.0 144.0 ... 0.000000e+00 0.962287 2.133649 9 Length 143.0 238.0 ... 0.000000e+00 0.181977 0.614725 [10 rows x 17 columns] In [16]: conn.close()

Here is the flow diagram for using callbacks. Note that only the result or response callback is called. It will not call both.

_images/callbacks-workflow.png

Handling Multiple Actions Simultaneously

The final workflow is the most hands-on, but also offers the most flexibility. For most cases, you can use the callback form to handle any size of data coming back from the CAS server. However, in that mode, you can only run one action at a time. If you want to run simultaneous actions and handle the responses as they come back from either connection, you need to use the CAS.invoke() method.

The CAS.invoke() method calls a CAS action, but doesn’t automatically retrieve the responses. You must iterate over the connection object to get the responses. To iterate over the responses for multiple connections, you can use the getnext() function. Using this technique, you can take advantage of running multiple CAS actions simultaneously while still just running in a single thread on the client.

In [17]: import swat

In [18]: conn1 = swat.CAS(host, port, userid, password)

In [19]: conn2 = swat.CAS(host, port, userid, password)

In [20]: tbl1 = conn1.read_csv('https://raw.githubusercontent.com/'
   ....:                       'sassoftware/sas-viya-programming/master/data/class.csv')
   ....: 

Note: Cloud Analytic Services made the uploaded file available as table TMP60O8TG1O in caslib CASUSER(castest).

Note: The table TMP60O8TG1O has been created in caslib CASUSER(castest) from binary data uploaded to Cloud Analytic Services.
In [21]: tbl1.head() Out[21]: Selected Rows from Table TMP60O8TG1O Name Sex Age Height Weight 0 Alfred M 14.0 69.0 112.5 1 Alice F 13.0 56.5 84.0 2 Barbara F 13.0 65.3 98.0 3 Carol F 14.0 62.8 102.5 4 Henry M 14.0 63.5 102.5 In [22]: tbl2 = conn2.read_csv('https://raw.githubusercontent.com/' ....: 'sassoftware/sas-viya-programming/master/data/cars.csv') ....:
Note: Cloud Analytic Services made the uploaded file available as table TMPOFC0_PVX in caslib CASUSER(castest).

Note: The table TMPOFC0_PVX has been created in caslib CASUSER(castest) from binary data uploaded to Cloud Analytic Services.
In [23]: tbl2.head() Out[23]: Selected Rows from Table TMPOFC0_PVX Make Model Type ... Weight Wheelbase Length 0 Acura MDX SUV ... 4451.0 106.0 189.0 1 Acura RSX Type S 2dr Sedan ... 2778.0 101.0 172.0 2 Acura TSX 4dr Sedan ... 3230.0 105.0 183.0 3 Acura TL 4dr Sedan ... 3575.0 108.0 186.0 4 Acura 3.5 RL 4dr Sedan ... 3880.0 115.0 197.0 [5 rows x 15 columns]

Now that we have the tables loaded, we can invoke the summary action on each one and retrieve the responses from both connections.

In [24]: tbl1.invoke('summary');

In [25]: tbl2.invoke('summary');

In [26]: for resp, conn in swat.getnext(conn1, conn2):
   ....:     for k, v in resp:
   ....:         print('>>> RESULT', k, v)
   ....: 
>>> RESULT Summary Descriptive Statistics for TMP60O8TG1O

   Column   Min    Max    ...            ProbT  Skewness  Kurtosis
0     Age  11.0   16.0    ...     8.049260e-19  0.063612 -1.110926
1  Height  51.3   72.0    ...     3.202374e-21 -0.259670 -0.138969
2  Weight  50.5  150.0    ...     2.045197e-13  0.183351  0.683365

[3 rows x 17 columns]
>>> RESULT Summary Descriptive Statistics for TMPOFC0_PVX

        Column      Min       Max    ...              ProbT  Skewness   Kurtosis
0         MSRP  10280.0  192465.0    ...      4.160412e-127  2.798099  13.879206
1      Invoice   9875.0  173560.0    ...      2.684398e-128  2.834740  13.946164
2   EngineSize      1.3       8.3    ...      3.133745e-209  0.708152   0.541944
3    Cylinders      3.0      12.0    ...      1.515569e-251  0.592785   0.440378
4   Horsepower     73.0     500.0    ...      4.185344e-216  0.930331   1.552159
5     MPG_City     10.0      60.0    ...      1.866284e-257  2.782072  15.791147
6  MPG_Highway     12.0      66.0    ...      1.665621e-292  1.252395   6.045611
7       Weight   1850.0    7190.0    ...      5.812547e-294  0.891824   1.688789
8    Wheelbase     89.0     144.0    ...       0.000000e+00  0.962287   2.133649
9       Length    143.0     238.0    ...       0.000000e+00  0.181977   0.614725

[10 rows x 17 columns]

In [27]: conn1.close()

In [28]: conn2.close()

The flow diagram for handling multiple connections simultaneously is shown below.

_images/simultaneous-workflow.png