Commit 49402938 authored by hammonds's avatar hammonds
Browse files

Merge branch 'python23conversion' into 'master'

Python23conversion

See merge request DM/data-management!1
parents 36e5b838 6cc21951
......@@ -36,4 +36,44 @@ Once logged in, the user will land on a page with a number of directories. File
All experimenters on APS Proposal or ESAF should have created an account through the [APS Web Portal](https://beam.aps.anl.gov/pls/apsweb/usercheckin.start_page). If the user does not have a badge number, they will need to [register with the APS User Office](https://beam.aps.anl.gov/pls/apsweb/ufr_main_pkg.usr_start_page). To verify or update the Web Password, users can try to log into the web portal above. Like other web accounts, it is possible to update an unknown/expired password by clicking a link on the portal web page. For this you will need to provide answers to security questions that were entered at the time the account was set up.
![](images/globus-verify-account.png)
\ No newline at end of file
![](images/globus-verify-account.png)
## Globus Connect Personal
This following section will discuss how to enable Globus Connect Personal. Globus Connect Personal is a service that allows individual users to access and transfer data that is available at the APS on site. It does this by creating an endpoint at at the user's personal machine. Then, the user can transfer data from the endpoint at the APS storage system to the endpoint at their own computer. There are two ways to install it on a personal machine: from the home page and from the file manager. The two methods are explained below.
##### Home Page
On Globus' home page, there are several drop down options in the top right corner. The first drop down is labeled "I Want To.." One of the following options is "Enable Globus on my System."
![](images/globus-home-page.png)
Clicking on that option will lead to the Globus Connect page. Globus Connect can allow users to establish an endpoint at either a personal machine, Globus Connect Personal, or a multi-user computing system, Globus Connect Server. Here, we want to focus on a single user. Scrolling down the page will reveal the two Globus Connect options.
![](images/globus-connect-page.png)
Select the "Get Globus Connect Personal" hyperlink to lead to the Globus Connect Personal installation page. Based on your operating system, select the corresponding hyperlink.
![](images/install-globus-connect.png)
Globus has its own documentation on the installation process for each operating system. Following the steps on that page will allow you to establish an endpoint at your own machine and begin transferring files. It also has detailed instructions for how to remove Globus Connect Personal.
##### File Manager
Globus Connect Personal can also be managed from the file manager page that was discussed before. Once a user logs in, the home page looks as such.
![](images/globus-file-manager.png)
Users can search for specific collections of data by using the search bar shown above. Clicking on the search bar will cause the interface to change to the following layout:
![](images/globus-recent-tab.png)
As visible here, Globus Connect Personal is available through the "Recent" tab under the search bar. If it is not visible from that tab, navigate to the "More Options" tab.
![](images/globus-more-options.png)
Click on the "Install Globus Connect Personal" button. This will lead you to a page with the option to download Globus Connect Personal. The first option is for a Mac download. However, if you have another operating system, click the hyperlink below for Windows and Linux installations. There is also a hyperlink above that will lead you to a detailed explanation about Globus Connect Personal.
![](images/globus-fm-download.png)
#!/bin/sh
cd `dirname $0`/.. && DM_SBIN_DIR=`pwd`
$DM_SBIN_DIR/dm_system_test_daq.sh /export/dm/test
$DM_SBIN_DIR/dm_system_test_daq.sh /home/dm/test
#!/bin/sh
cd `dirname $0`/.. && DM_SBIN_DIR=`pwd`
$DM_SBIN_DIR/dm_system_test_upload.sh /export/dm/test
$DM_SBIN_DIR/dm_system_test_upload.sh /home/dm/test
#!/bin/sh
cd `dirname $0`/.. && DM_SBIN_DIR=`pwd`
$DM_SBIN_DIR/dm_system_test_daq.sh /home/dm_idb/test
#!/bin/sh
cd `dirname $0`/.. && DM_SBIN_DIR=`pwd`
$DM_SBIN_DIR/dm_system_test_upload.sh /home/dm_idb/test
#!/bin/sh
#
# Script used for syncing APS remote user accounts
# Usage:
#
# $0
#
CURRENT_DIR=`pwd`
MY_DIR=`dirname $0` && cd $MY_DIR && MY_DIR=`pwd`
cd $CURRENT_DIR
DM_ROOT_DIR=$MY_DIR/..
DM_ENV_FILE=${DM_ROOT_DIR}/setup.sh
if [ ! -f ${DM_ENV_FILE} ]; then
echo "Environment file ${DM_ENV_FILE} does not exist."
exit 2
fi
. ${DM_ENV_FILE} > /dev/null
DM_CONFIG_FILE=$DM_ROOT_DIR/../etc/dm_remote_access.aps-remote-user-update-utilities.conf
# Simply run update command after sourcing setup file
dm-update-remote-users-from-aps-db --config-file=$DM_CONFIG_FILE --quiet
......@@ -181,7 +181,6 @@ class DaqCli(ApsBeamlineCli):
experimenters = proposal.get('experimenters', [])
if not description:
description = '%s (Proposal id: %s)' % (proposal['title'], proposalId)
description = unicode(description).encode('ascii', 'xmlcharrefreplace')
users = self.getUsers()
pis = []
......
......@@ -35,17 +35,10 @@ class DeleteExperimentCli(ApsBeamlineCli):
def getForceFlag(self):
return self.options.force
def runCommand(self):
self.parseArgs(usage="""
dm-delete-experiment --id=EXPERIMENTID|--experiment=EXPERIMENTNAME
[-f|--force]
Description:
Delete experiment from the DM database, remove experiment data from
storage, and delete file collection from the catalog.
""")
self.checkArgs()
def deleteExperiment(self):
result = ""
dsExperimentApi = ExperimentDsApi(self.loginUsername, self.loginPassword, self.dsServiceUrl)
catFileApi = FileCatApi(self.loginUsername, self.loginPassword, self.catServiceUrl)
......@@ -66,25 +59,43 @@ Description:
print(' 1) Experiment %s will be deleted from the DM DB.' % (experimentName))
print(' 2) All experiment files will be removed from storage.')
print(' 3) Experiment file catalog will be destroyed.\n')
proceed = self.confirm('Proceed (yes|no)?', 'no', ['y', 'yes'])
if not proceed:
print('Experiment %s will not be deleted.' % experimentName)
return
result += 'Experiment %s will not be deleted.\n' % experimentName
return result
# Delete experiment
experiment = dsExperimentApi.deleteExperimentByName(experimentName)
print('')
print('DELETED EXPERIMENT INFO')
print(experiment.getDisplayString(self.getDisplayKeys(), self.getDisplayFormat()))
result += '\n'
result += 'DELETED EXPERIMENT INFO\n'
result += experiment.getDisplayString(self.getDisplayKeys(), self.getDisplayFormat())
result += '\n'
try:
fileCollection = catFileApi.deleteExperimentFileCollection(experimentName)
print('')
print('DELETED FILE COLLECTION INFO')
print(fileCollection.getDisplayString(self.getDisplayKeys(), self.getDisplayFormat()))
result += '\n'
result += 'DELETED FILE COLLECTION INFO\n'
result += fileCollection.getDisplayString(self.getDisplayKeys(), self.getDisplayFormat())
result += '\n'
except ObjectNotFound as ex:
# ok, we do not need to delete anything
pass
return result
def runCommand(self):
self.parseArgs(usage="""
dm-delete-experiment --id=EXPERIMENTID|--experiment=EXPERIMENTNAME
[-f|--force]
Description:
Delete experiment from the DM database, remove experiment data from
storage, and delete file collection from the catalog.
""")
self.checkArgs()
result = self.deleteExperiment()
print(result)
#######################################################################
......
......@@ -174,7 +174,6 @@ class UploadCli(ApsBeamlineCli):
experimenters = proposal.get('experimenters', [])
if not description:
description = '%s (Proposal id: %s)' % (proposal['title'], proposalId)
description = unicode(description).encode('ascii', 'xmlcharrefreplace')
users = self.getUsers()
pis = []
......
#!/usr/bin/env python
from PyQt4.QtGui import QGridLayout, QApplication, QPushButton, QWidget, QFont, QLabel, QTableWidget, \
QTableWidgetItem, QColor, QAbstractItemView, QPalette, QComboBox, QHBoxLayout, QMessageBox
from PyQt4.QtCore import Qt, QElapsedTimer
try:
from PyQt5.QtWidgets import QGridLayout, QApplication, QPushButton, \
QLabel, QTableWidget, QTableWidgetItem, QAbstractItemView, \
QComboBox, QHBoxLayout, QMessageBox, QWidget
from PyQt5.QtGui import QFont, QColor, QPalette
from PyQt5.QtCore import Qt, QElapsedTimer
except ImportError:
from PyQt4.QtGui import QGridLayout, QApplication, QPushButton, QWidget, \
QFont, QLabel, QTableWidget, QTableWidgetItem, QColor, \
QAbstractItemView, QPalette, QComboBox, QHBoxLayout, QMessageBox
from PyQt4.QtCore import Qt, QElapsedTimer
from dm.aps_beamline_tools.gui.apiFactory import ApiFactory
from .objects.userInfo import UserInfo
from datetime import date, datetime
......
#!/usr/bin/env python
from PyQt4.QtGui import QAction
from PyQt4.QtGui import QCursor
from PyQt4.QtGui import QGridLayout, QSpacerItem, QSizePolicy, QPushButton, QWidget, \
QFont, QLabel, QTableWidget, QTableWidgetItem, QColor, QAbstractItemView, QHBoxLayout, QPalette, QMessageBox
from PyQt4.QtCore import Qt
try:
from PyQt5.QtWidgets import QAction
from PyQt5.QtWidgets import QGridLayout, QSpacerItem, QSizePolicy, QPushButton, QWidget, \
QLabel, QTableWidget, QTableWidgetItem, QAbstractItemView, QHBoxLayout, QMessageBox
from PyQt5.QtGui import QColor, QFont, QCursor, QPalette
from PyQt5.QtCore import Qt
except ImportError:
from PyQt4.QtGui import QAction
from PyQt4.QtGui import QCursor
from PyQt4.QtGui import QGridLayout, QSpacerItem, QSizePolicy, QPushButton, QWidget, \
QFont, QLabel, QTableWidget, QTableWidgetItem, QColor, QAbstractItemView, QHBoxLayout, QPalette, QMessageBox
from PyQt4.QtCore import Qt
from dm.aps_beamline_tools.gui.apiFactory import ApiFactory
from .subclasses import customSelectionModel, customStyledDelegate
......@@ -190,14 +197,14 @@ class DaqsTab(QWidget):
# Stops the daq that is selected
def stopDaq(self):
id = self.daqTable.item(self.daqTable.currentRow(), 0).data(Qt.UserRole).toString()
id = self.daqTable.item(self.daqTable.currentRow(), 0).data(Qt.UserRole)
allInfo = self.experimentDaqApi.getDaqInfo(id)
self.experimentDaqApi.stopDaq(allInfo['experimentName'], allInfo['dataDirectory'])
self.updateList()
# Clears the history of the daq from the DB and refreshes the table
def clearDaq(self):
id = self.daqTable.item(self.daqTable.currentRow(), 0).data(Qt.UserRole).toString()
id = self.daqTable.item(self.daqTable.currentRow(), 0).data(Qt.UserRole)
self.experimentDaqApi.clearDaq(id)
self.parent.refreshTables()
self.daqTable.clearSelection()
......@@ -235,7 +242,7 @@ class DaqsTab(QWidget):
def daqDetails(self):
self.detailsTable.setSortingEnabled(False)
id = self.daqTable.item(self.daqTable.currentRow(), 0).data(Qt.UserRole).toString()
id = self.daqTable.item(self.daqTable.currentRow(), 0).data(Qt.UserRole)
self.daqTable.clearFocus()
allInfo = self.experimentDaqApi.getDaqInfo(id)
......
......@@ -6,9 +6,15 @@ from __future__ import print_function
import argparse
import sys
import os
from PyQt4.QtGui import QMainWindow, QStackedLayout, QApplication, QWidget, QTabWidget, QSplashScreen, QPixmap, QMenu, \
QAction, QCursor, QMessageBox, QAbstractButton, QIcon
from PyQt4.QtCore import Qt, QTimer
try:
from PyQt5.QtWidgets import QMainWindow, QStackedLayout, QApplication, QWidget, QTabWidget, QSplashScreen, QMenu, \
QAction, QMessageBox, QAbstractButton
from PyQt5.QtGui import QPixmap, QCursor, QIcon
from PyQt5.QtCore import Qt, QTimer
except ImportError:
from PyQt4.QtGui import QMainWindow, QStackedLayout, QApplication, QWidget, QTabWidget, QSplashScreen, QPixmap, QMenu, \
QAction, QCursor, QMessageBox, QAbstractButton, QIcon
from PyQt4.QtCore import Qt, QTimer
from time import sleep, time
import smtplib
from email.mime.text import MIMEText
......@@ -100,20 +106,21 @@ class DmStationUi(QMainWindow):
self.currentProposals = self.experimentPropApi.listProposals(runName=currentRun, beamlineName=self.beamlineName)
except DmException:
ESAF_FAIL_MODE = "standalone"
self.logger.warn("Failed to connect to APS BSS system. Continuing in ESAF only mode.")
self.logger.warning("Failed to connect to APS BSS system. Continuing in ESAF only mode.")
self.onlyEsaf = True
self.beamlineName = None
esafFail = True
self.logger.debug("ESAF Sector %s" % self.esafSector)
if self.esafSector is not None:
try:
esafApi = apiFactory.getEsafApsDbApi()
esafApi.listEsafs(self.esafSector, datetime.today().year)
esafFail = False
except DmException:
self.logger.warn("Failed to connect to APS ESAF system. Continuing " + ESAF_FAIL_MODE + " mode.")
self.logger.warning("Failed to connect to APS ESAF system. Continuing " + ESAF_FAIL_MODE + " mode.")
else:
self.logger.warn("No ESAF sector specified. Continuing in " + ESAF_FAIL_MODE + " mode.")
self.logger.warning("No ESAF sector specified. Continuing in " + ESAF_FAIL_MODE + " mode.")
if esafFail:
self.onlyEsaf = False
......@@ -339,7 +346,7 @@ class DmStationUi(QMainWindow):
def copyAction(self, row, column, table):
clipboard = ""
index = table.model().index(row, column)
clipboard += str(index.data().toString())
clipboard += str(index.data())
sysclip = QApplication.clipboard()
sysclip.setText(clipboard)
......
import sys
import unittest
from PyQt4.QtGui import QApplication, QColor, QListView, QTableView
from PyQt4.QtTest import QTest
from PyQt4.QtCore import Qt, QPoint, QAbstractTableModel, QTimer
from . import dmStationUi
try:
from PyQt5.QtWidgets import QApplication, QListView, QTableView
from PyQt5.QtGui import QColor
from PyQt5.QtTest import QTest
from PyQt5.QtCore import Qt, QPoint, QAbstractTableModel, QTimer
except ImportError:
from PyQt4.QtGui import QApplication, QColor, QListView, QTableView
from PyQt4.QtTest import QTest
from PyQt4.QtCore import Qt, QPoint, QAbstractTableModel, QTimer
from dm.aps_beamline_tools.gui import dmStationUi
from datetime import datetime
import os
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment