Monitoring of Tritium release at PTC.

Monitoring of Tritium release at PTC. Scope of the project From more than 20 projects supported by Equipment Manufacturing Support group this is one of the simplest. What is nice about it is that elegant programming technique was used to obtain necessary reliability. Radioactive gas Tritium is used here at PTC for the manufacturing of Minitrons (nutron generators). Because of the personnel safety concerns and environmental regulation the release of the gas into environment should be monitored 24/7. In the event of excessive release personnel should be notified and event should be reported. The existing system was using autonomous TAM100D meters directly connected to the audio/visual alarms to address safety concerns. To collect data every meter was connected to the designated computer via RS232 interface. However, data collection was implemented as a Windows GUI application and requires operators at the area to login every time computer will reboot. Also special role account was needed to be maintained to allow data collection access to the network resources. Access to the real time graphs was provided via local Web server running as an application on each computer. Monthly and annual Tritium release reporting was done manually by processing individual data log files from each monitoring machine in Excel. To improve Tritium release monitoring it was decided to implement data collection as a Windows service that will record data into database, create lightweight GUI application to display real time graphs at each monitoring station, and implement unified Web interface to provide remote access to the data and reporting capabilities. To provide smooth transition the new data collection should support legacy data format, so we can keep using existing reporting routine while the new one will be in development and testing. Support of legacy data also will provide some level of redundancy. System overview System components and data flow are shown on the Figure 1. Each monitored lab has one or more designated computers used to collect data and provide local display of real time data for the operators in the room. One instance of data collection application and one instance of the UI display per TAM100D unit connected is running these computers. Data collection component communicates with one Tritium monitor units and push data into the SQL database and into the log file (for legacy compatibility). It starts automatically when the computer boots up (using Windows service). UI component displays data from single data collection process and starts automatically when any user login into system (using Windows startup option).

Legacy data log file is also used to communicate data from measurement service to the UI display. TAM100D Lab #1 UI PC Service Service TAM100D File server (legacy) Shared directory 1 Shared directory 2 Application server Lab #2 UI SQL TAM100D Service PC Web engine IE, Firefox etc. IE, Firefox etc. RS232 SQL via TCPIP User PC User PC File via MS network Figure 1. Tritium monitoring system design. Data received every 5 6 seconds (minimum measurement time provided by TAM100D). Every point is saved into legacy data file to keep compatibility. To optimize database usage average, minimum, and maximum values for minute are recorded in the SQL. Common view of the local display available in each lab is shown on Figure 2. Two green rectangles in the upper part of the window indicate that tritium concentration is below safe limit and TAM100D unit works properly and provides up to date information. If concentration is going above certain limit or there is an issue related to measurement device, communication, or data collection is detected one of these rectangles will change color as shown Figure 3.

Figure 2. Local UI application shows normal performance Figure 3. Local UI application shows communication error (no update for more than 1 min)

Figure 4. Web interface in normal conditions. Simple Web interface was implemented to provide access to the status of each device (shown on Figure 4). Similar to the local UI mechanism will display red error status in case there were no new data from the measurement unit for more than 2 min (see Figure 5). As you can see there is a device of different type (background monitor) is also listed on the page. This is detector we use to monitor level of background (natural) radiation in the test facility to prevent false failure because of background change when detector supposes to be tested with natural background as a radiation source. This is good example of how by putting in place proper system design we are able to reuse the same components for different but yet similar needs.

Figure 5. No data update for more than 2 min forces error to be reported. User can use Web interface to view real time or historical data (Figure 6). Historical data can be displayed for 24 hours or 30 days period. When 30 days period is selected average for an hour as well as minimum and maximum value for an hour are used to generate graph. This is done to minimize data transfer between client and server.

Figure 6. Access to real time and historical data provided via Web interface. Technical details (real geek stuff) Why Windows service Running program as a Windows service has a few advantages:

1. System can be run unattended (it does not require user to be logged in). Service will be started as soon as computer will boot up. In Windows it is even possible to run GUI application as a service as long it does not require any interaction from the user. 2. System became insensitive to the multiple user login/logout process is still running in the background when the process started under the user account will be terminated when user logout. 3. Windows has built in mechanism to monitor services health status. Any service can be configured to restart automatically in case it will be terminated unexpectedly. 4. Service can be setup to use built in Network Service security account that gives it access to the certain network/domain resources without need to maintain special user role account. This assumes that if person has permission to setup service on the particular computer he has rights to access resources available for the Network Service account. Windows service in C#/.Net Microsoft has a special class ServiceBase designed to simplify creation of the Windows service application. It has a few drawbacks: 1. Service name is hardcoded and can t be easily changed. Well, you may think what s a big deal? Let say I need run two copies of my service with different parameters. Why I may want to do this? There are may be a few reasons: a. Stability if one of two services crashes (sure nobody safe from the bugs) the second one will still running b. Configuration changes sometimes it easier read configuration once at the service start rather implement dynamic configuration reload. In this case if I need to change configuration and everything runs in one service I will need stop everything. c. Scalability If you need one more function that already implemented in your service you just register another instance and start it. 2. Debugging process for the service application is a little bit too complicated you need to setup application as a service on the development computer, then after it started you need attached debugger. You should restart service every time you recompile the project. 3. It is not easy to troubleshoot service installation after deployment because you can t easily see user interface for the service. Service vs. application in Windows Main difference of Windows service compare to the regular application is that service main() function service does not supposed to execute any functional code. It should just allow service to expose five control functions (Start, Stop, Pause, Resume, and Restart) to the Windows service control and keep running until service is going to be terminated. The idea is if we can make main() function to recognize if it was started from the windows service control or from the command line (or Visual Studio development environment) we can make our program run as a service or as a regular application with minimum difference in the code. If this idea will work we will have a nice side effect. During deployment process we will be able quickly run application

with configuration using command line interface and once confirmed it works fine start the service. This will simplify troubleshooting during deployment. Presented idea can be illustrated with the following C like pseudo code: int m_argc; char* m_argv[]; int main( int argc, char* argv[] ) if(!isservice() ) return Run( argc, argv ); StoreParams( argc, argv, l_argc, l_argv ); ExposeServiceControl(); WaitForExitSignal(); int Run( int argc, char* argv[] ) // program core logic will be here void OnStart() Run( m_argc, m_argv ); void OnStop() SetExitSignal(); Implementation Below are just few key elements of the implementation. Details can be obtained by contacting Igor Kozin. After complete implementation our main program file will be looking like this: class Program : PtcProcess static void Main( string[] args ) Program prg = new Program( args ); prg.runprocess(); // Core program logic is located in this routine protected override void MainRoutine() PtcErrorLog.Message( "Started Main routine" ); while(!checkexit() &&!IsExitFlag())

Thread.Sleep(1000); Console.WriteLine("0. Working...", DateTime.Now); PtcErrorLog.Message("Working..."); PtcErrorLog.Message( "Ending loop" ); By complexity level it is similar to the code you will get from the Visual Studio wizard when choosing Windows service project. You can compare for yourself: // Code created by Visual Studio wizard static class Program static void Main() ServiceBase[] ServicesToRun; ServicesToRun = new ServiceBase[] new Service1() ; ServiceBase.Run( ServicesToRun ); public class Service1 : ServiceBase public Service1() InitializeComponent(); protected override void OnStart( string[] args ) protected override void OnStop() Below are key routines that are implemented in the PtcProcess class that is inherited from ServiceBase and implements logic we need for dual run. Even though class PtcProcess is made abstract it has only one abstract method MainRoutine(). This is the method that supposes to be a substitution of the regular main() function in the program file. It is declared abstract only to clearly indicate that it needed to be implemented when creating application.

Core logic that supports switch between application and service execution is hidden in the RunProcess() method: public void RunProcess( ) this.m_isservice = IsService(); if( this.m_isservice ) // Uses ServiceBase.Run method when application runs as //.service. It will create new thread with MainRoutine as // an entry point System.ServiceProcess.ServiceBase.Run( this ); else // Directly calls MainRoutine // when started as regular application if( System.Environment.UserInteractive ) this.mainroutine(); else Error( "Can't start in application mode ); To recognize if the program started as service or application we will be using WMI Windows service. Main idea is to compare current process ID (PID) with the list of PIDs obtained from the Windows Service management. If one of the PIDs match current process it means we are running as a service. This also allows retrieve actual service name assigned during service registration. public static bool IsService() m_service = false; // UserInteractive may return true when the service is // configured with "Allow service interact with desktop" // option enabled. However, application will never report // UserIneractive as FALSE if(!system.environment.userinteractive ) m_isservice = true; else // PID of our program process int procid = System.Diagnostics.Process.GetCurrentProcess().Id; uint processid = 0; try string qry = "SELECT PROCESSID, NAME FROM WIN32_SERVICE"; ManagementObjectSearcher searcher = new ManagementObjectSearcher( qry ); foreach( ManagementObject mngntobj in searcher.get() ) processid = (uint)mngntobj["processid"]; if( processid == procid ) m_isservice = true; m_servicename = (string)mngntobj["name"];

catch( Exception ex ) PtcErrorLog.Error( ex, "When trying get service name." ); return m_isservice; To be able use ServiceName property we will need to hide corresponding property of the ServiceBase class: public new string ServiceName get if( IsService() ) if( m_servicename.length==0 ) return base.servicename; return m_servicename; return Path.GetFileNameWithoutExtension( Application.ExecutablePath ); set So, it will return default MS service name in case it do not have access to the WMI information, and just an executable name when run from command line or from the Visual Studio for debugging. This allows use of this property to be safe irrelevant of the how was application started. // Special exception hangler should be set inside the thread // So, we make this routine to serve as a main thread routine protected void MainRoutineWrapper() PtcErrorLog.SetExceptionHandlers(); MainRoutine(); // Implements logic that is executed on the service Start command protected override void OnStart( string[] args ) // Lookup all services to get current one); if( IsService() ) PtcErrorLog.OnTermination += new SafeEvent.EventHandler( OnAbnormalTerminate );

// This property will be used to identify any messages // automatically placed by ServiceBase into the Windows log EventLog.Source = this.servicename; // Copy service parameters into the member variable, // so they can be used late in MainRoutine if( args.length> 0 ) m_cmdln = new string[args.length]; args.copyto(m_cmdln, 0); PtcErrorLog.Init(); ThreadStart MainThreadStart = new ThreadStart( MainRoutineWrapper ); m_mainthread = new Thread( MainThreadStart ); m_mainthread.start(); How we use it On one computer using Windows SC command line utility we have setup two services that start the same exe file (from the same location) but with different parameters. Using the same executable simplifies updates. Below you can see examples of log files (file name is automatically generated based on the service name). Service ptctritmonsrvroom creates log file ptctritmonsrvroom.log with following information: 2014/09/15, 17:09:01. =============[ Service ptctritmonsrvroom. PID=2828 ]======= 2014/09/15, 17:09:01. Main module: PtcMonitorService.exe v.1.2.1.0 2014/09/15, 17:09:01. Message: Open DB to the server sqlsrv1.princeton.nam.slb.com with SSPI ON 2014/09/15, 17:09:01. Message: Station 4, communication COM1 2014/09/15, 17:09:01. Message: Open port COM1 When the service ptctritmonsrvstack creates different ptctritmonsrvstack.log log file: 2014/09/15, 17:09:01. =============[ Service ptctritmonsrvstack. PID=6996 ]======= 2014/09/15, 17:09:01. Main module: PtcMonitorService.exe v.1.2.1.0 2014/09/15, 17:09:04. Message: Open DB to the server sqlsrv1.princeton.nam.slb.com with SSPI ON 2014/09/15, 17:09:04. Message: Station 5, communication COM2 2014/09/15, 17:09:04. Message: Open port COM2