Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • DM/dm-docs
  • hammonds/dm-docs
  • hparraga/dm-docs
3 results
Show changes
Showing
with 0 additions and 478 deletions
.. automodule:: dm.cat_web_service.api
.. currentmodule:: dm.cat_web_service.api
CatApiFactory
-------------
.. autoclass:: dm.cat_web_service.api.catApiFactory.CatApiFactory()
:show-inheritance:
:members: getDatasetCatApi, getFileCatApi, getServiceMonitorCatApi
DatasetCatApi
-------------
.. autoclass:: dm.cat_web_service.api.datasetCatApi.DatasetCatApi()
:show-inheritance:
:members: __init__, addExperimentDataset, updateExperimentDataset, updateDatasetById, getDatasets, getExperimentDatasets, getDatasetById, getExperimentDataset, getExperimentDatasetFiles
FileCatApi
----------
.. autoclass:: dm.cat_web_service.api.fileCatApi.FileCatApi()
:show-inheritance:
:members: __init__, addExperimentFile, updateExperimentFile, updateExperimentFileById, updateExperimentFiles, updateExperimentFilesAsync, getAsyncUpdateStatus, getExperimentFiles, getExperimentFileById, getExperimentFile, deleteExperimentFile, deleteExperimentFileById, getExperimentFileCollectionStats, renameExperimentFileCollection, deleteExperimentFileCollection
ServiceMonitorCatApi
--------------------
.. autoclass:: dm.cat_web_service.api.serviceMonitorCatApi.ServiceMonitorCatApi()
:show-inheritance:
:members: __init__, getServiceStatus, clearServiceStatus
.. automodule:: dm.common.api
.. currentmodule:: dm.common.api
DmApiFactory
--------------
.. autoclass:: dm.common.api.dmApiFactory.DmApiFactory()
:show-inheritance:
:members: USERNAME_KEY, PASSWORD_KEY, PASSWORD_FILE_KEY, LOGIN_FILE_KEY, HOST_KEY, PORT_KEY, PROTOCOL_KEY, setDefaults, setConfigFile, getConfiguration
.. automodule:: dm.daq_web_service.api
.. currentmodule:: dm.daq_web_service.api
DaqApiFactory
-------------
.. autoclass:: dm.daq_web_service.api.daqApiFactory.DaqApiFactory()
:show-inheritance:
:members: getExperimentDaqApi, getServiceMonitorDaqApi
ExperimentDaqApi
-----------------
.. autoclass:: dm.daq_web_service.api.experimentDaqApi.ExperimentDaqApi()
:show-inheritance:
:members: __init__, startDaq, stopDaq, listDaqs, getDaqInfo, clearDaq, upload, stopUpload, listUploads, getUploadInfo, clearUpload, listProcessingPlugins
ServiceMonitorDaqApi
--------------------
.. autoclass:: dm.daq_web_service.api.serviceMonitorDaqApi.ServiceMonitorDaqApi()
:show-inheritance:
:members: __init__, getServiceStatus, clearServiceStatus
.. automodule:: dm.ds_web_service.api
.. currentmodule:: dm.ds_web_service.api
DsApiFactory
-------------
.. autoclass:: dm.ds_web_service.api.dsApiFactory.DsApiFactory()
:show-inheritance:
:members: getUserDsApi, getAuthDsApi, getExperimentDsApi, getFileDsApi, getEsafDsApi, getServiceMonitorDsApi
AuthDsApi
---------
.. autoclass:: dm.ds_web_service.api.authDsApi.AuthDsApi()
:show-inheritance:
:members: __init__, addSession, checkSession, getAuthorizationPrincipal
EsafDsApi
---------
.. autoclass:: dm.ds_web_service.api.esafDsApi.EsafDsApi()
:show-inheritance:
:members: __init__, getSectorEsafsByYear, getEsafById
FileDsApi
---------
.. autoclass:: dm.ds_web_service.api.fileDsApi.FileDsApi()
:show-inheritance:
:members: __init__, processFile, compressFile, decompressFile, deleteFile, statFile, processDirectory, download
ExperimentDsApi
---------------
.. autoclass:: dm.ds_web_service.api.experimentDsApi.ExperimentDsApi()
:show-inheritance:
:members: __init__, getExperimentTypes, getExperiments, getExperimentsByStation, getExperimentByName, getExperimentById, addExperiment, updateExperiment, deleteExperimentById, deleteExperimentByName
ServiceMonitorDsApi
--------------------
.. autoclass:: dm.ds_web_service.api.serviceMonitorDsApi.ServiceMonitorDsApi()
:show-inheritance:
:members: __init__, getServiceStatus, clearServiceStatus
UserDsApi
---------
.. autoclass:: dm.ds_web_service.api.userDsApi.UserDsApi()
:show-inheritance:
:members: __init__, getUsers, getUserById, getUserByUsername, addUserSystemRole, deleteUserSystemRole, addUserExperimentRole, deleteUserExperimentRole
.. automodule:: dm.proc_web_service.api
.. currentmodule:: dm.proc_web_service.api
ProcApiFactory
--------------
.. autoclass:: dm.proc_web_service.api.procApiFactory.ProcApiFactory()
:show-inheritance:
:members: getServiceMonitorProcApi, getWorkflowProcApi
ServiceMonitorProcApi
---------------------
.. autoclass:: dm.proc_web_service.api.serviceMonitorProcApi.ServiceMonitorProcApi()
:show-inheritance:
:members: __init__, getServiceStatus, clearServiceStatus
WorkflowProcApi
---------------
.. autoclass:: dm.proc_web_service.api.workflowProcApi.WorkflowProcApi()
:show-inheritance:
:members: __init__, listWorkflows, addWorkflow, getWorkflowById, getWorkflowByName, updateWorkflow, deleteWorkflowByName, listProcessingJobs, startProcessingJob, getProcessingJobById
.. APS Data Management documentation master file, created by
sphinx-quickstart on Thu Feb 23 09:20:39 2017.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to APS Data Management's documentation!
===============================================
The `dm` package contains python APIs for accessing Data Management services.
.. toctree::
:maxdepth: 4
:caption: Contents:
dm.common.api
dm.aps_db_web_service.api
dm.cat_web_service.api
dm.daq_web_service.api
dm.ds_web_service.api
dm.proc_web_service.api
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
* login screen
- should not appear if user has $DM_LOGIN_FILE set, and login is valid
* list of experiments screen
- view/modify details of an experiment
- button for adding new experiment
* view/modify experiment screen
- display list of users associated with experiment
- display ongoing DAQs/uploads
- start new DAQ/upload
- add users, either manually or associated with a given proposal
- remove users
* add new experiment dialog
- add users, either manually or associated with a given proposal
- start DAQ/upload immediately
* list of DAQs screen
- view DAQ details
- start new DAQ
- stop DAQ
* list of uploads screen
- view details
- start new
- cancel running upload
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = DMUserGuide
SOURCEDIR = source
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
# -*- coding: utf-8 -*-
#
# APS Data Management documentation build configuration file, created by
# sphinx-quickstart on Thu Feb 23 09:20:39 2017.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc',
'sphinx.ext.coverage',
'sphinx.ext.ifconfig',
'sphinx.ext.viewcode']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Users Guide - APS Data Management'
copyright = u'2015-2019, Argonne National Laboratory'
author = u'APS/SDM'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = "dev"
# The full version, including alpha/beta/rc tags.
release = "dev"
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
#html_theme = 'alabaster'
html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# -- Options for HTMLHelp output ------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'APSDataManagementdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'UsersGuide.tex', u'Users Guide - APS Data Management Documentation',
u'APS/SDM', 'manual'),
]
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'usersguide', u'APS Data Management Documentation',
[author], 1)
]
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'Users Guide', u'APS Data Management Documentation',
author, 'UsersGuide', 'One line description of project.',
'Miscellaneous'),
]
dm-station-gui
==============
One of the methods to manage processes in the APS Data Management System is the
application **dm-station-gui**. This application is a PyQt application which gives
access to add/modify/control items such as experiments, file transfer to storage
location ('daq' and 'uploads'), workflows and processing jobs. An example of
this application is shown in the figure below.
.. image:: images/dm-station-gui-experiments.png
:scale: 50 %
Experiments
-----------
**dm-station-gui** opens showing a tab that lists experiments which have been added
to the DM System. At the beamline, these experiments generally will correspond
to an accepted proposal in the APS Proposal system. Experiments in the DM
System define an entity that ties sets of managed data together. When an
experiment is selected by double clicking or by clicking and then clicking the
*Use Selected* button, the contents of the Experiment tab changes to give
details about that experiment. Much of this information is pulled from the
proposal/ESAF databases. This is shown in the image below.
.. image:: images/dm-station-gui-experiments-detail.png
:scale: 50 %
This view of the data enable a couple of key features of the DM System. The
ability to associate files with the Management system, and the ability to set
which users have access to the system and therefore which data they can access.
Selecting users that will have access to experiment data is done by clicking the
*Modify Users* button below the user list and then selecting users in the right
list & pressing the arrow button between the lists to add users or selecting
users from the left list and clicking the arrow button to delete users from the
list. Click the _Save_ button at the bottom to accept the changes and go back
to the Experiment detail or click *Back* button in the upper left to exit back
to Experiment detail without saving. This view is shown below.
.. image:: images/dm-station-gui-experiments-user-management.png
:scale: 50 %
Getting Files into Data Management
----------------------------------
The overall purpose of this system is to get data into the system and providing
access control to that data. This means linking the data files to the users in
an experiment. For each beam station there is storage location defined for that
station. On the **Experiments** tab there are a couple of items relevant to
getting the data onto that storage location. These items are shown in the image
below. Files transferred onto the storage location will go into:
*STORAGE_LOCATION/(storage_root_path)/(experiment_name)*. Files to go into this
directory are specified in the entry "Data Directory or Single File Path". Note
that if the data directory is specified, only the files will go into the
storage location, not a new subdirectory with the transferred files. Any
subdirectories will be copied as will the contents.
Once the the storage location and source are defined the transfer can be started
in one of two ways:
- A monitor can be placed on the directory and any new files/subdirectories in the directory will be transferred if you select *Start DAQ*
- A one time copy will happen and all files/subdirectories currently in the directry will be copied if you select *Start Upload*
When a tranfer is started, the display will switch to either the **DAQs** or
**Uploads** tab. The tab will show the status of all transfers that have been
started since the last restart of the system. Status of a particular transfer is
shown by color
- green: Done success
- yellow: running
- red: Done with an errors
An example of this is shown below.
.. image:: images/dm-station-gui-uploads.png
:scale: 50 %
Clicking on a particular transfer, on **DAQs** or **Uploads** will switch the
view to show detail of that transfer including errors.
.. image:: images/dm-station-gui-uploads-detail.png
:scale: 50 %
user_guide/source/images/dm-station-gui-experiments-detail.png

84.1 KiB

user_guide/source/images/dm-station-gui-experiments.png

147 KiB

user_guide/source/images/dm-station-gui-uploads-detail.png

149 KiB

user_guide/source/images/dm-station-gui-uploads.png

38.6 KiB

.. User guide for the APS Data Management System
Users Guide for the APS Data Management System
==============================================
.. toctree::
:maxdepth: 4
:caption: contents
dm-station-gui
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
\ No newline at end of file