APSDeveloperInstallation.md 30 KB
Newer Older
hammonds's avatar
hammonds committed
1
## APS Data Management System Developer Installation
2

dmadmin's avatar
dmadmin committed
3
The APS Data Management system makes use of many tools (Java, Python, Postgresql, MongoDB, ZeroMQ, etc.).  The Management System itself is built on top of these tools.  While it is possible to install and user the underlying tools using more conventional means (e.g. RPM  installation on Linux) scripts are provided that installs these tools from either source or binary builds and installs & configures them specifically for use with the Data Management System.  These scripts can be found in a git repository at:
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

[https://git.aps.anl.gov/DM/dm-support.git](https://git.aps.anl.gov/DM/dm-support.git)

while the code that makes up the Data Management System itself can be found in a repository at:

[https://git.aps.anl.gov/DM/dm.git](https://git.aps.anl.gov/DM/dm.git)

### Installation for development
An example of setting up the Data Management system for a developer is described here.

 - Create a directory (referred here as DM\_INSTALL\_DIR

> mkdir -p /local/Data Management

 - Change directory into DM\_INSTALL\_DIR

> cd /local/Data Management

 - Install a copy of the code from each of these git repositories in DM\_INSTALL\_DIR.  This can be done in a variety of ways (3rd an 4th should be the most common)
     - Grab a zip file from the APS Gitlab website (from URLs above) and unzip the file.
     - Clone the repositories directly into DM\_INSTALL\_DIR (basically like cloning a forked repo shown below)
     - Fork the repository following the fork link in the top right of the project page and then clone the repository as shown below.  The example shown clones the dm-support repository into a directory __support__ and the dm repository into a directory __dev__.  In each case the clone is pulled from the user _USERNAME_'s fork of the repository.
     - Create a branch of the repository from the project web page and then clone the repository from the branch (similar to clown shown below)

> git clone https://git.aps.anl.gov/_USERNAME_/dm-support.git __support__
>
> git clone https://git.aps.anl.gov/_USERNAME_/dm.git __dev__

 - Change directory into the __support__ directory

> cd support

dmadmin's avatar
dmadmin committed
36
 - Install & build all of the components needed to build the development system running the script _install\_support\_all.sh_ in the _sbin_ directory.
hammonds's avatar
hammonds committed
37
     - During this install/build you will need to provide two passwords for the administration of the __Payara__ application server.  These passwords are for the _master_ (for administration of the keystore) and _admin_ (for administration of the application server properties) user accounts.
38
39
40
41
42
43
44
     - Note that a number of the installed applications/libraries are built during the process so it is common that this process will possibly take a couple of hours to complete, but this is a one time installation process, although individual components can then be updated separately later.
     - There is a configuration build_env.sh file which allows changing things like which version of each package will be installed.  This is executed at the beginning of each script that will be run by install_support_all.sh. At any time, the current version of these tools may change to adapt for a new provided feature or to just ensure that new builds use the latest possible version of a tool to avoid a stale environment which falls far behind the current version of each tool.

> ./sbin/install/_support/_all.sh

 - Change directory to the root of the Data Management components

45
46
 - Note some configuration can be changed before processing the as discussed below.  There are two files **dm_dev.deploy.conf** and **dm.deploy.conf** which define some environment variables used in the scripts used to install & configure.  For the test deployment, **dm_dev.deploy.conf** is used.

47
48
49
50
51
52
> cd ../dev

 - Execute the dm/_deploy/_test/_system.sh file in the sbin directory
     - Like installing the support tools, this script builds and installs several components of the DM system so it will take some time to complete.
     - This deploy process will prompt for user input at several points in the process.
         - passwords for several accounts
hammonds's avatar
hammonds committed
53
              - __postgres__ admin account - This will be used to manage the postgres itself.  Each developer can set this to a unique value.
hammonds's avatar
hammonds committed
54
55
              - __dm__ db management account - This will be for managing the 'dm' database in postgresql.  Each developer can set this to a unique value.
              - __dm__ system account - This is user __dm__in the Data Management system.  This user has administrative privilege in the Data Management system.  This is a user in the 'dm' user table.  Each developer can set this to a unique value.
hammonds's avatar
hammonds committed
56
57
58
59
              - __dmadmin__ LDAP password - This password provides the Data Management software access to the APS/ANL LDAP system to gather reference to that database.  This is a password to an external system and and is therefore a pre-existing password that developers will need to get from the Data Management system administrator.
             
              - __dmadmin__ BSS login password.  This is a password to allow the Data Management system access to the APS Beamline Scheduling system.  This is a password to an external system and and is therefore a pre-existing password that developers will need to get from the Data Management system administrator.
              - __dmadmin__ ESAF DB password.  This is a password to allow the Data Management system access to the ESAF system.  This is a password to an external system and and is therefore a pre-existing password that developers will need to get from the Data Management system administrator.
60
         - Scripts in the Data Management system need a location for the data storage directory.  Files will be moved to/from this directory.            
61
62
63
64
65
66
67
68

For initial test purposes, it is necessary to shortcut some parts of the service, such as using LDAP and Linux services to manage permissions and access control lists on the files.  To do this edit the following files in the top level etc directory:
 * dm.aps-db-web-service.conf 
    - Comment out the entry for principalAuthenticator2 which uses the LDAP authenticator
 * dm.cat-web-service.conf
    - Comment out the entry for principalAuthenticator2 which uses the LDAP authenticator
 * dm.daq-web-service.conf
    - Comment out the entry for principalAuthenticator2 which uses the LDAP authenticator
69
70
 * dm.proc-web-service.conf
    - Comment out the entry for principalAuthenticator2 which uses the LDAP authenticator
71
 * dm.ds-web-service.conf
72
73
74
75
76
   - Comment out the entry for principalAuthenticator2 which uses the LDAP authenticator
   - comment out the two lines for platformUtility which use LinuxUtility and LdapLinuxPlatformUtility
   - Add a new platformUtility line in place of the other two
    - platformUtility=dm.common.utility.noopPlatformUtility.NoopPlatformUtility()
   - Change value for `manageStoragePermissions` in ExpermentManager section to False	   
77

78
### Removing Test system
hammonds's avatar
hammonds committed
79
80
Often in the development of Data Management system components it will be necessary to remove/reload components of the system.  The script _dm/_remove/_test/_test/_system.sh_ in the sbin directory of the 'dm' repository (/local/DataManagement/dev/sbin from the directory describe above) issues commands to clear out database & configurations to allow creating a clean installation of the system.

hammonds's avatar
hammonds committed
81
### Overview of the system & tools
82
The installed development system has a few tools for managing the system.  This section describes some of the available tools and process ideas for the system.  The next section will describe some steps to walk through final setup and use.
83
 - A web portal which should now be up and running at the URL https://localhost:8181/dm.  This portal is powered by a Payara application server which has its own setup page at https://localhost:4848 (once configured above, you may not need to do much with the Payara config page).  
hammonds's avatar
hammonds committed
84
 - A PyQt app installed dm-station-gui which can be used to setup/monitor experiment definition, file transfers and data workflows.  
85
86
87
88
89
90
91
 - A set of command-line scripts for manipulating the system.  THese commands are made accessible by sourcing the file DM_INSTALL_DIR/etc/dm.setup.sh (Note there are some definitions that are blank in the default version of this file). 
 - There are also a couple of underlying databases holding the data.  
     - A postgresql database which holds standard data such as user info, beamline/station definitions, experiments, access info linking users to experiments and data. 
     - A mongo database, which allows a bit more flexibility.  This stores info on workflows and file information.

To start with the Data Management (DM) System is configured with one user __dm__ which is a management account, the third account listed above.  One of the first items to handle is to create accounts that will be associated with managing the beamline setup and some (possibly the same accounts) that will be associated with experiments.  In practice, the DM system is loosely linked to the list of users in the APS Proposal/ESAF system.  Accounts on the ESAF system are coordinated with a list of users on the DM system.  This is done by using the dm-update-users-from-aps-db.  This will require a configuration file (find a good place to put the file).   One other possibility is to create users manually from the supplied web portal. Note that, in the ESAF system, the user name is the badge number of the individual, while in the DM system a 'd' is prepended to the badge number for the user name.

92
Once users have been added to the system, the DM web portal can be used to associate users with a beamline or with experiments that are created.  The __dm__ user can be used to log into the web portal and from the _Experiment Stations_ tab new stations can be added or existing stations, such as the test station, can be edited and station managers can be added.  To create experiments, station managers can log into the system and add/manage experiments for that station.  From the test installation the user can manually create experiments & add users to the experiment.  In practice, at the APS, when a user adds an experiment they are provided with a list of experiments from the proposal system and the list of users is populated from the (Proposal/ESAF ??) info.  Note that it is also possible to add/modify experiments either through the dm-station-gui or through the command line interface with commands such as dm-add-experiment or dm-update-experiment.
93
94
95

After defining an experiment, it is possible to then manage tasks such as file transfers (daq or upload) or workflows & processing jobs.  These tasks can be done using either the dm-station-gui or by the command line interface.  

hammonds's avatar
hammonds committed
96
'daq' transfers monitor selected directories for files from a live data acquisition process from the collected location to a 'storage' location.  'upload' transfers copy any existing files from the collected location to the 'storage' location.  As file are transfered, they are placed into a storage directory with subdirectories for the _(station name)/(storage root path)/(experiment name)_.
97
98
99
100
101
102
103
104
105

DM workflows define a sequence of commands that would operate on data sets to:

 - Stage data
 - Move the data to a particular location such as a transfer between globus endpoints
 - Process data for using reduction/analysis algorithms
 - Add results to files that are tracked by Data Management

Each step in a workflow can define inputs and outputs which can then be used in subsequent steps. 
106
107
108
109
110
111
112
113
114
115
116

### Restarting the test system
If needed the test system can be restarted running a couple of startup commands.  Change directory the DM install directory and then

 * dm/etc/init.d/dm-test-services restart
 * dm/etc/init.d/dm-ds-services restart
 
This may be necessary if, for instance, the system has been rebooted.  These commands restart several services in the install directory.  If you have modified something in only one of these services you may be able to restart that service.  For instance if only the data storage web service needs to be rebooted then you can run

 * dm/etc/init.d/dm-ds-webservice restart

hammonds's avatar
hammonds committed
117
### Testing the system
118

hammonds's avatar
hammonds committed
119
As mentioned earlier, after the initial install we have one user __dm__ which is intended to be for the overall system.  We now need to set up a user for administration of a beamline and start some steps to use the system.
120
121
122
123
124
125

You should at this point have a directory installed which has both the _Data Manangement_ and _support_ software installed.  After doing the installs described above there should be a number of other directories as well such as etc, log and var.  We are now going to walk through changes needed in the etc directory which will allow us to interact with the system.
 1.  source the file _etc/dm.setup.sh_.  This defines a number of environment variables and modifies the path to include, in particular, a number of commands beginning with __dm-__ which interact with the underlying system to add/modify users, experiments, upload and daq (both to move files) and workflows and processes (to define & monitor processing of the collected data).
      - source etc/dm.setup.sh
 2. add a user __dmtest__ to the system which will assume the role of manage what is going on in the system.  
     - dm-add-user --username dmtest --first-name DM --last-name Test --password dmtest
126
 3. add a system role to the created user __dmtest__ to make this a manager of the station TEST which is already defined in the system.  You will be asked to provide username & password.  Use username __dm__ system account and the password given during setup above.
127
128
129
130
131
132
133
134
      - dm-add-user-system-role --role Manager --station TEST --username dmtest
 4. create a file, _etc/.dmtest.system.login_, in the same directory as the dm.setup.sh). This will contain the username & password.
      - dmtest|dmtest      (example contents)
 5. Edit the  file _etc/dm.setup.sh_, the one from step 1, to modify the line DM\_LOGIN\_FILE to point at the file created in step 4.
      - DM\_LOGIN\_FILE=/home/dmadmin/etc/.dmtest.system.login   (modified in file)
 6. Re-source the setup file from step 1.
      - source etc/dm.setup.sh

hammonds's avatar
hammonds committed
135
At this point we will are more in a position to start using the system. As a first test we will add a few test users to the system and then run the command dm-test-upload which will
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
  * create a new experiment
     * attach a list of users to the experiment
     * define a location where data exists
     * defines a path to store the data in the storage system
  * starts an upload which copies data from the original location to the specified directory on the storage system
  
To accomplish this we use the following

To add 3 users

```
dm-add-user --username jprofessor --last-name Professor --first-name John
dm-add-user --username gpostdoc --last-name Postdoc --first-name George
dm-add-user --username jgradstudent --last-name Gradstudent --first-name Jane
```

To add an experiment, define the users, and kick off an upload:

```
dm-test-upload --experiment=e1 --data-directory=/home/dmadmin/testData --dest-directory=MyFirstExperiment --users=jprofessor,gpostdoc,jgradstudent
```

This should provide output like the following

```
EXPERIMENT INFO

id=23 name=e1 experimentTypeId=1 experimentStationId=1 startDate=2019-11-07 16:04:30.919828-05:00 

UPLOAD INFO
id=ec513c1d-45a3-414f-8c56-50a9d4d6dbdd experimentName=e1 dataDirectory=/home/dmadmin/testData status=pending nProcessedFiles=0 nProcessingErrors=0 nFiles=0 startTime=1573160671.17 startTimestamp=2019/11/07 16:04:31 EST 
```
This command will
 * Create an experiment named `e1`with
   - The three experimenters `jprofessor`, `gpostdoc` & `jgradstudent`
   - The data that is being collected will be found at `/home/dmadmin/testData`
172
   - Any data/files found in `/home/dmadmin/testData` will be found in a directory `TEST/e1/MyFirstExperiment` of the storage location defined for the Data Storage service.  NOTE: if the directory `/home/dmadmin/testData` does not exist, then the upload process will fail.
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
 	

Output like the following

```
We trust you have received the usual lecture from the local System
```

likely means that one of the config files did not disable the principalAuthenticator2, LinuxUtility or LdapLinuxPlatformUtility as described at the end of the installation section of this document.

We can now look at the results of what we have done in a number of ways:

The commands `dm-list-users` and `dm-get-experiment --experiment=e1 --display-keys=ALL --display-format=pprint` will give

```
id=1 username=dm firstName=System lastName=Account 
id=2 username=dmtest firstName=DM lastName=Test 
id=3 username=jprofessor firstName=John lastName=Professor 
id=4 username=gpostdoc firstName=George lastName=Postdoc 
id=5 username=jgradstudent firstName=Jane lastName=Gradstudent 
```

and

```
{ u'experimentStation': { u'description': u'Test Station',
                          u'id': 1,
                          u'name': u'TEST'},
  u'experimentStationId': 1,
  u'experimentType': { u'description': u'Experiment type used for testing',
                       u'id': 1,
                       u'name': u'TEST'},
  u'experimentTypeId': 1,
  u'experimentUsernameList': [u'gpostdoc', u'jgradstudent', u'jprofessor'],
  u'id': 23,
  u'name': u'e1',
  u'startDate': u'2019-11-07 16:04:30.919828-05:00',
  u'storageDirectory': u'/home/dmadmin/storage/TEST/e1',
  u'storageHost': u'localhost',
  u'storageUrl': u'extrepid://localhost/home/dmadmin/storage/TEST/e1'}
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
```

Next step will add a workflow and then execute this workflow.  This workflow is an example pulled from the comments in the file workflowProcApi.py (owner name has been changed to match user dmtest). It creates a minimal version of a workflow that grabs the md5sum of a given file.  The workflow is defined by the following 

```
            {
                'name'        : 'example-01',
                'owner'       : 'dmtest',
                'stages'      : {
                    '01-START'  : {
                        'command' : '/bin/date +%Y%m%d%H%M%S',
                        'outputVariableRegexList' : ['(?P<timeStamp>.*)']
                    },
                    '02-MKDIR'  : {
                        'command' : '/bin/mkdir -p /tmp/workflow.$timeStamp'
                    },
                    '03-ECHO'   : {
                        'command' : '/bin/echo "START JOB ID: $id" > /tmp/workflow.$timeStamp/$id.out'
                    },
                    '04-MD5SUM' : {
                        'command' : '/bin/md5sum $filePath | cut -f1 -d" "',
                        'outputVariableRegexList' : ['(?P<md5Sum>.*)']
                    },
                    '05-ECHO'   : {
                        'command' : 'echo "FILE $filePath MD5 SUM: $md5Sum" >> /tmp/workflow.$timeStamp/$id.out'
                    },
                    '06-DONE'   : {
                        'command' : '/bin/echo "STOP JOB ID: $id" >> /tmp/workflow.$timeStamp/$id.out'
                    },
                },
                'description' : 'Workflow Example 01'
            }
```

This workflow can be added to the system with the command:

 > dm-upsert-workflow --py-spec=sampleWorkflow

and will yield a result like:

```
id=5de938931d9a2030403a7dd0 name=example-02 owner=dmtest 
```

hammonds's avatar
hammonds committed
257
This workflow can be executed by the command:
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391

>   dm-start-processing-job --workflow-name=example-02 --workflow-owner=dmtest filePath:/home/dmadmin/testData/myData

This will have a result like:

```
id=2f004219-0694-4955-af05-b29b48ce4c0a owner=dmtest status=pending startTime=1575566109.86 startTimestamp=2019/12/05 12:15:09 EST
```

More information can be found with `dm-get-processing-job` like:

 > dm-get-processing-job --id=2f004219-0694-4955-af05-b29b48ce4c0a --display-keys=ALL --display-format=pprint
 
which returns

```json
{ u'endTime': 1575566111.014859,
  u'endTimestamp': u'2019/12/05 12:15:11 EST',
  u'filePath': u'/home/dmadmin/testData/myData',
  u'id': u'2f004219-0694-4955-af05-b29b48ce4c0a',
  u'md5Sum': u'bac0be486ddc69992ab4e01eeade0b92',
  u'nFiles': 1,
  u'owner': u'dmtest',
  u'runTime': 1.1574599742889404,
  u'stage': u'06-DONE',
  u'startTime': 1575566109.857399,
  u'startTimestamp': u'2019/12/05 12:15:09 EST',
  u'status': u'done',
  u'timeStamp': u'20191205121510',
  u'workflow': { u'description': u'Workflow Example 01',
                 u'id': u'5de938931d9a2030403a7dd0',
                 u'name': u'example-02',
                 u'owner': u'dmtest',
                 u'stages': { u'01-START': { u'childProcesses': { u'0': { u'childProcessNumber': 0,
                                                                          u'command': u'/bin/date +%Y%m%d%H%M%S',
                                                                          u'endTime': 1575566110.898553,
                                                                          u'exitStatus': 0,
                                                                          u'runTime': 0.007671833038330078,
                                                                          u'stageId': u'01-START',
                                                                          u'startTime': 1575566110.890881,
                                                                          u'status': u'done',
                                                                          u'stdErr': u'',
                                                                          u'stdOut': u'20191205121510\n',
                                                                          u'submitTime': 1575566110.859169,
                                                                          u'workingDir': None}},
                                             u'command': u'/bin/date +%Y%m%d%H%M%S',
                                             u'nCompletedChildProcesses': 1,
                                             u'nQueuedChildProcesses': 0,
                                             u'nRunningChildProcesses': 0,
                                             u'outputVariableRegexList': [ u'(?P<timeStamp>.*)']},
                              u'02-MKDIR': { u'childProcesses': { u'1': { u'childProcessNumber': 1,
                                                                          u'command': u'/bin/mkdir -p /tmp/workflow.20191205121510',
                                                                          u'endTime': 1575566110.942735,
                                                                          u'exitStatus': 0,
                                                                          u'runTime': 0.0035638809204101562,
                                                                          u'stageId': u'02-MKDIR',
                                                                          u'startTime': 1575566110.939171,
                                                                          u'status': u'done',
                                                                          u'stdErr': u'',
                                                                          u'stdOut': u'',
                                                                          u'submitTime': 1575566110.925104,
                                                                          u'workingDir': None}},
                                             u'command': u'/bin/mkdir -p /tmp/workflow.$timeStamp',
                                             u'nCompletedChildProcesses': 1,
                                             u'nQueuedChildProcesses': 0,
                                             u'nRunningChildProcesses': 0},
                              u'03-ECHO': { u'childProcesses': { u'2': { u'childProcessNumber': 2,
                                                                         u'command': u'/bin/echo "START JOB ID: 2f004219-0694-4955-af05-b29b48ce4c0a" > /tmp/workflow.20191205121510/2f004219-0694-4955-af05-b29b48ce4c0a.out',
                                                                         u'endTime': 1575566110.972364,
                                                                         u'exitStatus': 0,
                                                                         u'runTime': 0.003882884979248047,
                                                                         u'stageId': u'03-ECHO',
                                                                         u'startTime': 1575566110.968481,
                                                                         u'status': u'done',
                                                                         u'stdErr': u'',
                                                                         u'stdOut': u'',
                                                                         u'submitTime': 1575566110.960305,
                                                                         u'workingDir': None}},
                                            u'command': u'/bin/echo "START JOB ID: $id" > /tmp/workflow.$timeStamp/$id.out',
                                            u'nCompletedChildProcesses': 1,
                                            u'nQueuedChildProcesses': 0,
                                            u'nRunningChildProcesses': 0},
                              u'04-MD5SUM': { u'childProcesses': { u'3': { u'childProcessNumber': 3,
                                                                           u'command': u'/bin/md5sum /home/dmadmin/testData/myData | cut -f1 -d" "',
                                                                           u'endTime': 1575566110.985139,
                                                                           u'exitStatus': 0,
                                                                           u'runTime': 0.0030689239501953125,
                                                                           u'stageId': u'04-MD5SUM',
                                                                           u'startTime': 1575566110.98207,
                                                                           u'status': u'done',
                                                                           u'stdErr': u'',
                                                                           u'stdOut': u'bac0be486ddc69992ab4e01eeade0b92\n',
                                                                           u'submitTime': 1575566110.973093,
                                                                           u'workingDir': None}},
                                              u'command': u'/bin/md5sum $filePath | cut -f1 -d" "',
                                              u'nCompletedChildProcesses': 1,
                                              u'nQueuedChildProcesses': 0,
                                              u'nRunningChildProcesses': 0,
                                              u'outputVariableRegexList': [ u'(?P<md5Sum>.*)']},
                              u'05-ECHO': { u'childProcesses': { u'4': { u'childProcessNumber': 4,
                                                                         u'command': u'echo "FILE /home/dmadmin/testData/myData MD5 SUM: bac0be486ddc69992ab4e01eeade0b92" >> /tmp/workflow.20191205121510/2f004219-0694-4955-af05-b29b48ce4c0a.out',
                                                                         u'endTime': 1575566110.997652,
                                                                         u'exitStatus': 0,
                                                                         u'runTime': 0.0005791187286376953,
                                                                         u'stageId': u'05-ECHO',
                                                                         u'startTime': 1575566110.997073,
                                                                         u'status': u'done',
                                                                         u'stdErr': u'',
                                                                         u'stdOut': u'',
                                                                         u'submitTime': 1575566110.987421,
                                                                         u'workingDir': None}},
                                            u'command': u'echo "FILE $filePath MD5 SUM: $md5Sum" >> /tmp/workflow.$timeStamp/$id.out',
                                            u'nCompletedChildProcesses': 1,
                                            u'nQueuedChildProcesses': 0,
                                            u'nRunningChildProcesses': 0},
                              u'06-DONE': { u'childProcesses': { u'5': { u'childProcessNumber': 5,
                                                                         u'command': u'/bin/echo "STOP JOB ID: 2f004219-0694-4955-af05-b29b48ce4c0a" >> /tmp/workflow.20191205121510/2f004219-0694-4955-af05-b29b48ce4c0a.out',
                                                                         u'endTime': 1575566111.011913,
                                                                         u'exitStatus': 0,
                                                                         u'runTime': 0.001583099365234375,
                                                                         u'stageId': u'06-DONE',
                                                                         u'startTime': 1575566111.01033,
                                                                         u'status': u'done',
                                                                         u'stdErr': u'',
                                                                         u'stdOut': u'',
                                                                         u'submitTime': 1575566111.002148,
                                                                         u'workingDir': None}},
                                            u'command': u'/bin/echo "STOP JOB ID: $id" >> /tmp/workflow.$timeStamp/$id.out',
                                            u'nCompletedChildProcesses': 1,
                                            u'nQueuedChildProcesses': 0,
                                            u'nRunningChildProcesses': 0}}}}
```

Note that the md5 sum of the file `/home/dmadmin/testData/myData` is listed in the `stdOut` of stage `04-MD5SUM` and is used in the command in stage `05-ECHO` which in creates a temp file in /tmp.