Checklist: Process cannot or takes a long time to start

Checklist: Process cannot or takes a long time to start

Time estimate: 45 minutes

Please go through all the points in the following checklist. If necessary, confirm with IT Department any information you cannot verify before contacting your local COPA-DATA Representative.
Checklist usage: #. [Quick hints] (point refers specific module/product) Point description.
Terminology:
Service Engine folder = RT folder + Data folder (*).
      RT folder = Service Engine files created by Engineering Studio.
      Data folder (computer name) = Files created by Service Engine during operation (alarms, events, archives, etc.).

(*) Data folder is not necessarily a subfolder of Service Engine folder. Its location can be amended in the project properties.
  1. Screenshot and followed instructions of error/warning messages shown.
  2. Process/Application ran and was closed correctly before.
  3. No external conditions or incidents caused the process to close inadvertently (e.g. power outage, hardware failure, 3rd party software misbehaviour, etc.).
  4. Other non-related applications can start and execute as expected.
  5. [SIC-Reduced] [Task Manager > Performance] Operating System has free hardware resources to start the process.
  6. Hardware (e.g. specific network adapter, etc.) and Software pre-requisites are fulfilled as per documentation.
    7. DirectX Diagnostic (DxDiag) report provides a summary of system information.
    This is automatically collected by System Information Collector (SIC) tool and can be found in node: Console > DXDIAG > dxd.txt.

    The list of the installed software in can be found in SIC in System Info > Programs > Win32/Win64 node.
    All Service Engine drivers documentation include a Requirements chapter that contains information on the requirements that are necessary to use the driver.
  7. [Task Manager > Details] Process executable is not being executed by other Operating System user or running in the background.
  8. [SIC-Reduced] [Command Prompt: netstat -a -o | findstr :50777] Administration Service (zenAdminSrv.exe ) is running and listening on TCP Port 50777 (default). Traffic is not restricted.
    9. Windows Resource Monitor shows live CPU, Memory, Disk and Network information about running processes in an user friendly interface.
    Processes with network activity are listed and can be filtered for best visualization.
    Some of the most relevant information displayed in Network is the active TCP connections (remote/local port and address) and Firewall status.
    COPA-DATA Services listening ports configuration can be found in zenon Startup Tool:
    Application > Options > Listening ports (Administration Service).
  9. [SIC-Reduced] [Command Prompt: netstat -a -o | findstr :50780] Logging Service (zenLogSrv.exe) is running and listening on the TCP Port 50780 (default). Traffic is not restricted.
  10. zenon was re-registered successfully. Component is started from zenon Startup Tool.
  11. Searched and followed error instructions in LOGs related with the process-executable and zenAdminSrv.exe.
    12. All zenon components write in the background messages to joint log file(s) saved in '%programdata%\COPA-DATA\LOG' folder.
    These files can be read and configured with Diagnosis Viewer.

    You can modify the logging level of a component by clicking 'Client Configuration' in 'Settings' menu and after select the respective process name.
    Attention: Set the logging parameters wisely as the time-frame of the LOGs decrease when increasing the detail of the logs written.

    You can also find LOG files in SIC in 'Platform > Versions > Version XXXX > Directories > LogDirectory' folder.
    Service Engine writes log messages in regard zenrt32.exe stability in case of errors or warnings.
    You can find these entries in Diagnosis Viewer by filtering the column Function name = 'CZenonrtApp::MonitoringThread'.

    Some of the most relevant performance entries are:
    - Memory: “Total available memory fell below warning/error level to [xxx]%”
    - CPU: Total CPU-Load exceeded warning/error level to [xxx]%”
    - Service Engine not responding for XXX ms: "100ms Timer has not cycled for [xxx]ms”
    - Handles: "(%) “Handles exceeded error level to [xxx]”.

    In case Service Engine closes unexpectedly, zenon Drivers write in the logs "RPC error!" as they can no longer communicate with zenrt32.exe.
  12. Since last normal execution of the process, there were not modifications in the system in terms of:
    1. Hardware components.
    2. Process configuration, e.g. project/driver settings, etc..
    3. Operating System:
      1. No updates installed.
      2. No fresh install, uninstall/removal of other software.
      3. No changes in the Local/Group or other IT policies.
    4. Cyber security software configuration.
  13. Cyber security software logs do not report actions to prevent the execution of the process. Anti-virus excludes the process and dependent folders from scanner.
  14. Disabled zenon API, re-registered zenon and restarted component.
    15. zenon API can be switched off by editing the respective sections in zenon configuration file (zenon6.ini) located in %CD_SYSTEM% folder:
    [VBA]
    [VSTA]
    [ADDINS]
    EIN=0
    ON=0
    ON=0
  15. (Service Engine) The problem also occurs when the project RT folder has been recreated.
    1. Renamed (backup) the original RT folder.
    2. Re-created (and remote transported) all Service Engine files.
    3. Restarted Service Engine.
  16. (Service Engine) The problem also occurs when the project Data folder has been recreated.
    1. Renamed (backup) the original Data folder.
    2. Restarted Service Engine.
  17. There are not network issues that can cause problems establishing the connections required by the process or affect process stability.
    18. High network latency, malformed packages and other unwanted conditions may cause unexpected behaviours on processes relying on communications.
  18. Searched in Knowledge Base if a previous issue has been addressed.
  19. [SIC-Full] Collected process crash full dump file in case the component closes unexpectedly and produces an unhandled exception.
    20. Crash full dumps can be generated by adding a [DEBUG] section in zenon6.ini:
    [DEBUG]
    MINIDUMPTYPE=10

    Future full dumps will be saved in the ‘%programdata%\COPA-DATA\zenonXXX’ folder.

If the problem persists after completing this checklist and followed corrective actions, please contact your local COPA-DATA Representative providing a SIC-Full Report from the target computer(s) and the result of this checklist – please include any additional information or comments related with the points addressed you find relevant.

System Information Collector is a standalone COPA-DATA application that collects relevant data about the Operating System and zenon Software Platform for troubleshooting purposes. SIC is installed with zenon and can be started from zenon Startup Tool (Tools).