|
|
##System Usage
|
|
|
|
|
|
In this section we illustrate typical system usage via DM command line interfaces.
|
|
|
|
|
|
Setup environment by sourcing setup file.
|
|
|
|
|
|
truffle1> # After sourcing setup file, all DM commands will be in your path
|
|
|
truffle1> source /opt/dm/etc/dm.setup.sh
|
|
|
|
|
|
|
|
|
Create DM experiment, and upload data to the GPFS DDN storage:
|
|
|
|
|
|
truffle1> # Experiment name must be unique and start with a letter.
|
|
|
truffle1> dm-5idd-upload --experiment=s5test-01 --data-directory=/data/2017-1/bedzyk
|
|
|
EXPERIMENT INFO
|
|
|
id=284 name=s5test-01
|
|
|
...
|
|
|
|
|
|
|
|
|
Check status of your uploads:
|
|
|
|
|
|
truffle1> # Use dm-list-uploads to get status of all uploads
|
|
|
truffle1> dm-list-uploads
|
|
|
id=50392f24-e7e9-4122-b94d-d7662c6c0ab9 experimentName=s5test-01
|
|
|
...
|
|
|
|
|
|
|
|
|
Start monitoring directory for new files (DAQ mode):
|
|
|
|
|
|
truffle1> # Any files already present in the specified data directory will be ignored.
|
|
|
truffle1> # The --dest-directory option indicates directory path relative to experiment root directory in storage
|
|
|
truffle1> dm-5idd-daq --data-directory=/opt/dm/test --experiment=s5test-01 --dest-directory=daqdata --duration=1h
|
|
|
id=ac392f24-e7e9-3712-b94d-d7662c6c0ab9 experimentName=s5test-01
|
|
|
...
|
|
|
|
|
|
|
|
|
Check status of your DAQs:
|
|
|
|
|
|
truffle1> # Use dm-list-daqs to get status of all DAQs
|
|
|
truffle1> dm-list-daqs
|
|
|
id=ac392f24-e7e9-3712-b94d-d7662c6c0ab9 experimentName=s5test-01
|
|
|
...
|
|
|
|
|
|
|
|
|
Retrieve list of your cataloged file metadata, etc. Alternative way of viewing metadata catalog is via Mongo Express application (https://truffle1.dnd.aps.anl.gov:8182/db/dm).
|
|
|
|
|
|
truffle1> dm-list-experiment-files --experiment=s5test-01
|
|
|
id=591f359171913f50b4df5f6d fileName=
|
|
|
...
|
|
|
|
|
|
|
|
|
Check that you can see your files using Globus Online. Note that experiment files can be read only by those DM usernames that have been assigned a role within a given experiment.
|
|
|
|
|
|
Using your browser, open https://www.globus.org/
|
|
|
|
|
|
Sign in with your Globus username
|
|
|
|
|
|
Click on "Transfer Files"
|
|
|
|
|
|
Chose "aps#xstor" endpoint and click "Continue" in order to authenticate.
|
|
|
|
|
|
Authenticate with your DM username (typically "d<badge number>"). Use the same password that you use to access APS Beamline User Portal.
|
|
|
|
|
|
Navigate to experiment folder ("/gdata/dm/<experiment type>/<experiment name>", for the above example "/gdata/dm/5IDD/s5-test01".
|
|
|
|
|
|
System Administration
|
|
|
|
|
|
DM services are controlled using standard RHEL daemon scripts located under the /opt/dm/production/etc/init.d directory. Relevant scripts are:
|
|
|
|
|
|
dm-daq-web-service (DAQ web service)
|
|
|
dm-cat-web-service (Cataloging web service)
|
|
|
dm-proc-web-service (Processing web service)
|
|
|
dm-mongodb (MongoDB server)
|
|
|
dm-mongo-express (Mongo Express web application)
|
|
|
dm-nginx (Nginx server)
|
|
|
dm-daq-services (controls all DAQ services)
|
|
|
|
|
|
Shown below is an example of DAQ Web Service restart:
|
|
|
|
|
|
truffle1> # Control scripts can be found in /opt/dm/production/etc/init.d directory.
|
|
|
truffle1> # Usage:
|
|
|
truffle1> /opt/dm/production/etc/init.d/dm-daq-web-service
|
|
|
Usage: /opt/dm/production/etc/init.d/dm-daq-web-service {start|stop|status|restart}
|
|
|
|
|
|
truffle1> # Restart DAQ Web Service
|
|
|
truffle1> /opt/dm/production/etc/init.d/dm-daq-web-service restart
|
|
|
Stopping dm.DaqWebService daemon: [ OK ]
|
|
|
Starting dm.DaqWebService daemon: [ OK ]
|
|
|
truffle1> /opt/dm/production/etc/init.d/dm-daq-web-service status
|
|
|
dm.DaqWebService (pid 29314) is running...
|
|
|
|
|
|
Deployment Details
|
|
|
|
|
|
Station Name: 5IDD
|
|
|
DM Software Version: 2.3.0 (Release Notes)
|
|
|
Deployment Machine: truffle1.dnd.aps.anl.gov
|
|
|
Deployed Services:
|
|
|
Data Acquisition Service (port 33336)
|
|
|
Cataloging Service (port 44436)
|
|
|
Processing Service (ports 55536, 55537)
|
|
|
MongoDB Server (localhost, port 27017)
|
|
|
Mongo Express Application (localhost, port 18182)
|
|
|
Nginx Server (port 8182)
|
|
|
DM Admin Unix Account: dmadmin
|
|
|
DM Beamline System Account: user5idd
|
|
|
Deployment Directory: /opt/dm
|
|
|
Support Software Directory: /opt/dm/support
|
|
|
System Configuration Files: /opt/dm/etc
|
|
|
Service Log Directories:
|
|
|
DM Backend Services: /opt/dm/var/log
|
|
|
MongoDB Directory: /opt/dm/support/opt/mongodb/linux-x86_64/data
|
|
|
User Environment Setup File: /opt/dm/etc/dm.setup.sh
|
|
|
Service Control Scripts: /opt/dm/production/etc/init.d
|
|
|
DM Storage Service: catdtn01.aps.anl.gov:22236
|
|
|
DM APS DB Service: catdtn01.aps.anl.gov:11236
|
|
|
DM Web Portal: https://catdtn01.aps.anl.gov:8181/dm
|
|
|
Mongo Express: https://truffle1.dnd.aps.anl.gov:8182/db/dm
|
|
|
Globus Endpoint: aps#data
|
|
|
Globus Folder: /gdata/dm
|
|
|
|