Newer
Older
=============================
- Introduced concept of experiment station and redesigned authorization
mechanisms to allow beamline managers to manage their stations; all
APIs and CLIs now conform to the new authorization scheme
- Modified get-experiments utility to allow retrieving list of experiments
for a given station
- Cleaned up web portal by removing unused views, and enabled station
management functionality
- GPFD DDN (extrepid) has replaced xstor as the main APS storage
- CLI changes:
- add-experiment command requires station name (can be set from env.
variable); experiment type can be specified using type name
- get-experiments command requires station name for beamline managers (can
be set from env. variable)
- start-experiment command is now optional
=============================
- Resolved issue with incorrect accounting of processing errors for DAQs
- Improved DAQ processing algorithm to avoid resource starvation between
simultaneous DAQs and uploads
- Enhanced monitoring status information for both DAQs and uploads
=============================
- Introduced new framework and utilities for synchronizing users with
APS DB
- Resolved several issues with special characters in file names for
gridftp transfer plugin
=============================
- Added SFTP file system observer agent
Release 0.12 (05/06/2016)
=============================
- Developed processing for HDF5 metadata in Mongo cataloging plugin
- Modified catalog API and service interfaces to use file collections on
a per-experiment basis
Release 0.11 (04/29/2016)
=============================
- Resolved issue with upload command for directories containing large
number of files
- Implemented enhanced upload processing algorithm to avoid resource
starvation between simultaneous DAQs and uploads
- Added new polling file system observer agent as option for monitoring
directories
- Reworked catalog API and corresponding MongoDB interfaces to use unique
experiment file paths, rather than file names
=============================
- Added dm-list-daqs and dm-list-uploads commands
- Resolved issue with newly created directories treated as files for
real-time data acquisitions
sveseli
committed
Release 0.9 (02/25/2016)
=============================
- Developed directory processing mode for uploads; in this mode file transfer
plugins transfer entire directories as opposed to individual files
- Added dm-get-processing-plugins command
- Resolved working directory issue that may occur with simutaneous uploads
Release 0.8 (01/26/2016)
=============================
- Enhanced upload/daq performance and functionality (hidden files are not
processed; for uploads system can detect files that had been processed
already; improved handling and reporting of processing errors)
- Source file checksum is calculated for rsync/gridftp plugins by default
- Added dm-stop-upload command
- Resolved globus online user authorization delay issue
- Introduced framework and user interfaces for tracking progress of file
uploads and data acquisitions in DAQ service
- Added ability to monitor multiple directories for the same experiment
simultaneously (required changes to DAQ service REST interfaces)
- Enhanced start/stop DAQ and upload commands to use DM_FILE_SERVER_URL
environment variable
- Added user interfaces and utilities that enable experiment data download
from machines that have SSH access to the storage host
=============================
- Added file system observer agent interface for DAQ service
- Implemented FTP file system observer for DAQ service
- Added interfaces for deleting user experiment role in DS service
- Introduced java REST API framework, and specific experiment DS service API
- Web Portal notifies DS service about experiment user modifications
- Implemented Single Sign-On solution for backend services
- Enabled user authentication via login file
- Added file stat (with checksum) interface in DS web service
- After adding user role to experiment via command line, user is also
added to experiment group (if one exists)
- Added rsync file transfer plugin with checksum and delete
Release 0.4 (09/21/2015)
=============================
- Number of minor modifications made in preparation for test deployment at
beamlines
Release 0.3 (07/22/2015)
=============================
- Developed initial version of Catalogging Web Service based on MongoDB
- Developed sample processing plugins: file metadata catalog, SDDS processing,
SGE job submission
Release 0.2 (06/30/2015)
=============================
- Implemented storage permission management and user group management
- Developed common file processing service plugin framework
Release 0.1 (04/21/2015)
=============================
- Functional web portal (user, experiment, and policy pages)
- Developed web service and its API/CLI frameworks
- Developed initial version of Data Storage Web Service
- Developed initial version of Data Acquisition Web Service;
- DAQ service can monitor file system on a detector node and subsequently
transfer data to storage