Release 1.0 (15/1/2017) ============================= - Introduced concept of experiment station and redesigned authorization mechanisms to allow beamline managers to manage their stations; all APIs and CLIs now conform to the new authorization scheme - Modified get-experiments utility to allow retrieving list of experiments for a given station Release 0.15 (11/1/2016) ============================= - Resolved issue with incorrect accounting of processing errors for DAQs - Improved DAQ processing algorithm to avoid resource starvation between simultaneous DAQs and uploads - Enhanced monitoring status information for both DAQs and uploads Release 0.14 (10/14/2016) ============================= - Introduced new framework and utilities for synchronizing users with APS DB - Resolved several issues with special characters in file names for gridftp transfer plugin Release 0.13 (05/27/2016) ============================= - Added SFTP file system observer agent - Enhanced MongoDB plugin with file md5 sum calculation Release 0.12 (05/06/2016) ============================= - Developed processing for HDF5 metadata in Mongo cataloging plugin - Modified catalog API and service interfaces to use file collections on a per-experiment basis Release 0.11 (04/29/2016) ============================= - Resolved issue with upload command for directories containing large number of files - Implemented enhanced upload processing algorithm to avoid resource starvation between simultaneous DAQs and uploads - Added new polling file system observer agent as option for monitoring directories - Reworked catalog API and corresponding MongoDB interfaces to use unique experiment file paths, rather than file names Release 0.10 (03/11/2016) ============================= - Added dm-list-daqs and dm-list-uploads commands - Resolved issue with newly created directories treated as files for real-time data acquisitions Release 0.9 (02/25/2016) ============================= - Developed directory processing mode for uploads; in this mode file transfer plugins transfer entire directories as opposed to individual files - Added dm-get-processing-plugins command - Resolved working directory issue that may occur with simutaneous uploads Release 0.8 (01/26/2016) ============================= - Enhanced upload/daq performance and functionality (hidden files are not processed; for uploads system can detect files that had been processed already; improved handling and reporting of processing errors) - Source file checksum is calculated for rsync/gridftp plugins by default - Added dm-stop-upload command - Resolved globus online user authorization delay issue Release 0.7 (12/08/2015) ============================= - Introduced framework and user interfaces for tracking progress of file uploads and data acquisitions in DAQ service - Added ability to monitor multiple directories for the same experiment simultaneously (required changes to DAQ service REST interfaces) - Enhanced start/stop DAQ and upload commands to use DM_FILE_SERVER_URL environment variable - Added user interfaces and utilities that enable experiment data download from machines that have SSH access to the storage host Release 0.6 (11/6/2015) ============================= - Added file system observer agent interface for DAQ service - Implemented FTP file system observer for DAQ service - Added interfaces for deleting user experiment role in DS service - Introduced java REST API framework, and specific experiment DS service API - Web Portal notifies DS service about experiment user modifications Release 0.5 (10/08/2015) ============================= - Implemented Single Sign-On solution for backend services - Enabled user authentication via login file - Added file stat (with checksum) interface in DS web service - After adding user role to experiment via command line, user is also added to experiment group (if one exists) - Added rsync file transfer plugin with checksum and delete Release 0.4 (09/21/2015) ============================= - Number of minor modifications made in preparation for test deployment at beamlines Release 0.3 (07/22/2015) ============================= - Developed initial version of Catalogging Web Service based on MongoDB - Developed sample processing plugins: file metadata catalog, SDDS processing, SGE job submission Release 0.2 (06/30/2015) ============================= - Implemented storage permission management and user group management - Developed common file processing service plugin framework Release 0.1 (04/21/2015) ============================= - Functional web portal (user, experiment, and policy pages) - Developed web service and its API/CLI frameworks - Developed initial version of Data Storage Web Service - Developed initial version of Data Acquisition Web Service; - DAQ service can monitor file system on a detector node and subsequently transfer data to storage