2-IOC-deployment.md

IOC Deployment and Troubleshooting

2 Deploy a new IOC

2.1 Clone xxx

$ cd ${IOC_DIR}
$ git clone https://git.aps.anl.gov/practical_beamline_controls_training/session_2/xxx.git

output

Cloning into 'xxx'...
remote: Enumerating objects: 222, done.
remote: Counting objects: 100% (222/222), done.
remote: Compressing objects: 100% (141/141), done.
remote: Total 222 (delta 65), reused 222 (delta 65), pack-reused 0
Receiving objects: 100% (222/222), 177.75 KiB | 292.00 KiB/s, done.
Resolving deltas: 100% (65/65), done.

2.2 Confirm the directories exist

$ cd ${IOC_DIR}/xxx
$ tree -d .

output

.
├── configure
├── docs
├── iocBoot
│   └── iocxxx
│       ├── autosave
│       ├── examples
│       │   └── detectors
│       ├── softioc
│       │   └── commands
│       └── substitutions
└── xxxApp
    ├── Db
    ├── op
    │   ├── adl
    │   ├── bob
    │   │   └── autoconvert
    │   ├── burt
    │   ├── edl
    │   │   └── autoconvert
    │   ├── opi
    │   │   └── autoconvert
    │   ├── python
    │   └── ui
    │       └── autoconvert
    └── src

25 directories

2.3 Choose a new IOC prefix

A desired prefix can be specified as an argument to newPrefix.sh

$ ./newPrefix.sh kmp

output

New IOC name   = kmp
New IOC prefix = kmp:
Creating a custom caQtDM screen for kmp
Creating a custom MEDM screen for kmp
Creating override.iocsh to override the IOC prefix

If no prefix is specified, one will be chosen at random.

$ ./newPrefix.sh

output

New IOC name   = aoi
New IOC prefix = aoi:
Creating a custom caQtDM screen for aoi
Creating a custom MEDM screen for aoi
Creating override.iocsh to override the IOC prefix

2.4 Build the IOC (Attempt #1)

$ make

output

configure/CONFIG:18: *** EPICS_BASE must be set in a configure/RELEASE file.  Stop.

2.5 Resolve build error #1

Modify configure/RELEASE so that EPICS_BASE is defined.

$ gedit configure/RELEASE &

solution

EPICS_BASE doesn't need to be defined explicitly; The synApps configure/RELEASE file defines the version of base used to build it, so only the SUPPORT path needs to be corrected.

$ git diff
diff --git a/configure/RELEASE b/configure/RELEASE
index 474fd69..e7aba0e 100644
--- a/configure/RELEASE
+++ b/configure/RELEASE
@@ -26,7 +26,7 @@
 ##############################################################################
 
 # Define the path to the synApps support directory and uncomment.
-SUPPORT=/corvette/home/epics/devel
+SUPPORT=/APSshare/epics/synApps_6_2_1/support
 
 # Specify the standard synApps configuration.
 -include $(SUPPORT)/configure/RELEASE
$

2.6 Build the IOC (attempt #2)

If following the build succeeds, skip to step 2.9

$ make

output

If you're logged into the bcda VMs, it is likely you got the following error.

<snip>
/bin/ld: cannot find -lnet
/bin/ld: cannot find -lpcap
collect2: error: ld returned 1 exit status
make[3]: *** [xxx] Error 1
make[3]: Leaving directory `/home/beams/USERNAME/PET-S2/epics/synApps_6_2_1/ioc/xxx/xxxApp/src/O.rhel7-x86_64'
make[2]: *** [install.rhel7-x86_64] Error 2
make[2]: Leaving directory `/home/beams/USERNAME/PET-S2/epics/synApps_6_2_1/ioc/xxx/xxxApp/src'
make[1]: *** [src.install] Error 2
make[1]: Leaving directory `/home/beams/USERNAME/PET-S2/epics/synApps_6_2_1/ioc/xxx/xxxApp'
make: *** [xxxApp.install] Error 2

2.7 Resolve build error #2

Tell the build system the net (and pcap) libraries aren't installed.

$ echo "LINUX_NET_INSTALLED = NO" > configure/CONFIG_SITE.local

2.8 Build the IOC

We do a clean build at this point to ensure the fix for the previous failed build takes effect. The value that was set in in CONFIG_SITE.local causes some software to be exluded from the IOC. Without the make distclean the excluded software can be partially included in the IOC, which results in build problems.

$ make distclean
$ make

2.9 Run the IOC

Run the IOC.

$ cd ${IOC_DIR}/xxx/iocBoot/iocxxx
$ ./softioc/xxx.sh run

2.10 Resolve the runtime errors

Errors that occur when commands in an IOC's startup script are executed are usually critical.

2.10.1 Identify critical runtime errors

Scroll back until the you see the first error message. Resolving the first error message can often eliminate many other error messages that appear unrelated.

2.10.2 Stop the IOC

Exit the IOC from the IOC shell:

kmp> exit

Note: typing ctrl+c will also stop the IOC.

2.10.3 Resolve critcial runtime errors

Resolve the problem that is causing the first error message, which accurately describes the problem.

hint

The envPaths file is generated by the build system and is affected by the Makefile in the IOC's startup directory.

solution

Edit the Makefile and change linux-x86_64 to the EPICS_HOST_ARCH that was set in step 1.1.2.

$ gedit Makefile &

Git can show you the change you made to the file.

$ git diff Makefile 
diff --git a/iocBoot/iocxxx/Makefile b/iocBoot/iocxxx/Makefile
index fa87a45..1e2f07a 100644
--- a/iocBoot/iocxxx/Makefile
+++ b/iocBoot/iocxxx/Makefile
@@ -7,10 +7,11 @@ include $(TOP)/configure/CONFIG
 #ARCH = windows-x64
 #ARCH = cygwin-x86
 #ARCH = linux-x86
-ARCH = linux-x86_64
+#ARCH = linux-x86_64
 #ARCH = vxWorks-ppc32
 #ARCH = vxWorks694-ppc32
 #ARCH = vxWorks-ppc603
+ARCH = $(EPICS_HOST_ARCH)
 
 #TARGETS = cdCommands
 TARGETS = envPaths

Once the Makefile is corrected, issue a 'make' in the current working directory, ${IOC_DIR}/xxx/iocBoot/iocxxx, to create the envPaths file.

$ make
perl -CSD /APSshare/epics/base-7.0.4.1/bin/rhel8-x86_64/convertRelease.pl -t /home/beams/USERNAME/PET-S2/epics/synApps_6_2_1/ioc/xxx envPaths

2.10.4 Run the IOC

Start the IOC and confirm the error is resolved.

$ ./softioc/xxx.sh run

There will be autosave errors due to missing .sav files. Let the IOC run for at least 30 seconds and it will create the .sav files. Restart the IOC and the autosave errors will be gone.

Note: at this point the IOC is usable.

2.11 Resolve iocInit errors

Errors that occur during iocInit are not always critical and can sometimes be ignored, but it is good to eliminate them if possible.

2.11.1 Identify iocInit errors

These findUioAddr errors occur at iocInit:

findUioAddr: Failed to open '/sys/class/uio/uio0/name'
findUioAddr: Failed to open '/sys/class/uio/uio0/name'
findUioAddr: Failed to open '/sys/class/uio/uio0/name'

The errors come from softGlueZynq, which isn't needed in general purpose Linux IOCs.

2.11.2 Stop the IOC

Exit the IOC from the IOC shell:

kmp> exit

Note: typing ctrl+c will also stop the IOC.

2.11.3 Resolve iocInit errors

Remove softGlueZynq by doing the following:

$ cd ${IOC_DIR}/xxx
$ echo "undefine SOFTGLUEZYNQ" > configure/RELEASE.local
$ make distclean
$ make

Note: we do a distclean before rebuilding because support is being removed from the IOC.

2.11.4 Run the IOC

Start the IOC and confirm the error is resolved.

$ cd ${IOC_DIR}/xxx/iocBoot/iocxxx
$ ./softioc/xxx.sh run

2.12 Stop the IOC

Exit the IOC from the IOC shell:

kmp> exit

Note: typing ctrl+c will also stop the IOC.

3 Add simulated motors to the IOC