MB Foster White Paper January 2002 Data Access Using ODBC Drivers from MBFoster
Table Of Contents Data Access with ODBCLink/SE Background...1 ODBCLink/SE...1 Data Source Creation...2 Configuring the MS-IIS Server...4 Build A Basic Web Page...4 Registry Settings for Internet Information Server 4.0 Using ODBCLink/SE...4 Starting the Listener...5 Populate a Web Page with Data from a HP e3000 Image/SQL Server...5 Advanced ODBC access with MBF-UDALink...6 Data Source Creation...6 Figure 5a Data source creation...6 Host Based Security...7 Figure 6 Host Based Security - User ID...7 Figure 7 Host Based Security - Databases...7 Figure 8 Host Based Security - User Tables...7 3 Tier Client Server Connection Using IIS 4.0...8 Registry Settings for MBF-UDALink...8 Registry Settings for Internet Information Server 4.0 Using MBF-UDALink...8 Connection Pooling...9 Multi-Threading...9 Monitoring...9 How MBF-Console Works...9 MB Foster Associates Limited 2002 i
Background This paper explores the use of ODBCLink/SE to webify HP e3000 servers. The use of ODBC technology is pervasive. Many web servers in use today are NT Servers running one of a variety of the WEB development tool kits. This paper will examine the use of the Microsoft Internet Information Server, in conjunction with ODBCLink/SE to access data stored in an IMAGE/SQL database on an HP e3000. ODBCLink/SE is developed by MB Foster exclusively for Hewlett Packard. It is a subset of the data access functionality of MBF-UDALink. MB Foster has been shipping data access, delivery and migration solutions for the HP e3000 since 1985 and ODBC technology since 1994. Using WEB to HP e3000 ODBC WEB BROWSER NT Server HP e3000 Figure 1 ODBC Access ODBCLink/SE ODBCLink/SE is an ODBC Driver that enables Microsoft Windows based applications and tools to access IMAGE/SQL on the HP e3000 and ALLBASE/SQL on the HP e3000 and HP9000, in a client-server environment. Application developers and end-users can take advantage of the desktop applications, their graphical user interfaces (GUI) and processing power, while relying on the security, integrity and database management capabilities of ALLBASE/SQL and IMAGE/SQL. ODBCLink/SE runs under MPE/iX 6.0 or higher on the HP e3000 server as well as under HP-UX Versions 10.x and 11 on HP 9000 servers. ODBCLink/SE runs under Microsoft Windows 98/NT/2000 or WindowsXP on the client. ODBCLink/SE can be used in two ways: either by direct calls from a Windows program (Excel, Impromptu, etc.) or through an ODBC-compliant application such as Microsoft Access, Visual Basic, etc. MB Foster Associates Limited 2002 1
Client software is available for both ODBC 2.0 and ODBC 3.0 compliant applications. There are slight differences between the two versions especially in the area of connection pooling which is a useful technology for use in a web environment. 7 Steps for Setting up a web server with ODBCLink/SE Create the data source on the NT server Configure the ODBCLink/SE server on the HP e3000 Configure the MS-IIS server to use the data source Start the listener (or make it permanent by adding it to SYSStart) Design Considerations Build a basic web page Populate the web page with data from the HP e3000 Image/SQL server 1. Data Source Creation From the control panel settings on your PC screen, double click the ODBC Data Sources icon. ODBC Data Sources.lnk Note: If the ODBC Icon is missing, follow the instructions in the ODBCLink/SE manual to install the client software on the NT server. Before you can create a new data source you will have to know what you want to call the data source, name, group and account of the database and the name or TCP/IP address of the server. In addition you will need a logon for the server and some idea of the kind of transaction management and server isolation level your application needs. Using the MB Foster ODBCLink/SE driver, a Data Source is created. Figure 2a Data Source Creation MB Foster Associates Limited 2002 2
The logon information must be supplied database access depends on the logon identification. Figure 2b Data Source Creation After the Data Source is created, your data is accessible in a variety of desktop tools or in this case the MS-Internet Information Server (MS-IIS) can be used. MB Foster DataExpress for Client/Server provides access to other data such as KSAM or MPE files. Figure 2c Data Source Creation MB Foster Associates Limited 2002 3
2. Configuring the MS-IIS Server Figure 3 Active Server Page Code The Active Server Page contains the data source connection used to access the data with ODBCLink/SE using ActiveX Data Objects (ADO). 3. Design Considerations 1. First, remember the speed of the web page will depend on the amount of time spent getting the records back to the user. When designing the web page start by story boarding it. Look at the data requirements from the following aspects: 2. Security Who should be allowed to use these web pages? How will you validate the user? Is there any data that needs security? Should the server put any pages into Secure Sockets Mode or is it OK for everyone to read the data? 3. Performance How many records need to be read to satisfy the request? How many records will be selected by the largest query? How will the selection be done? Do any of the queries span multiple databases? Does your web server allow connection pooling? 4. Simplicity How will we avoid supplying more data than the user wants? Is there a need to require selection criteria to reduce the information retrieved? Registry Settings for Internet Information Server 4.0 Using ODBCLink/SE HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W3SVC\Parameters PoolIDCConnections to 1 PoolIDCConnectionsTimeOut to 30 (seconds) AllowKeepAlives to 1 UploadReadAhead to 48000 (KB) HKEY LOCAL_MACHINES\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters MaxPoolThreads to 1 (Threads) PoolThreadLimit to 40 (Meg) AcceptExOutstanding to 100 (Sockets Waiting) MB Foster Associates Limited 2002 4
Note: Setting the MaxPoolThreads to 1 will avoid data corruption. 4. Starting the Listener To start the listener on the HP e3000 data base server, enter: STREAM ODBCJOB.ODBCSE.SYS PM and NM capability is required to start the listener job. Therefore it is normal for the job to be started by MANAGER.SYS. You should start the listener as root user-id preferably in your system startup file. When the server is started it makes a call to SQLVER and puts the output into a file called ODBCVER in the same directory that the server is running from. For this reason, upon startup, it may take 20 or more seconds for the program to return. The ODBCVER file is used to determine the DBMS version for a SQLGetInfo call. If the ODBCVER file is deleted the SQLGetInfo call for DBMS version will return UNKNOWN. It is possible to incorporate the process to start the listener in the SYSSTART stream process making in a permanent connection. 5. Build A Basic Web Page Proceed to use a desktop Web Page builder product to create a basic web page. The example in Figure 3 was built using Microsoft Visual InterDev 6.0. 6. Populate a Web Page with Data from a HP e3000 Image/SQL Server Figure 4 - Image/SQL Server using an ODBCLink/SE connection MB Foster Associates Limited 2002 5
Advanced ODBC access using MBF-UDALink MBF-UDALink provides data access to today's popular web server applications using a thin client model. In addition to Image/SQL and AllBase/SQL the technology provides access to KSAM, MPE files including self describing files, Robelle s SuprTool, and Cognos PowerHouse. The architecture is such that multiple data bases on the same server can be accessed and joined through a single connection with the result set passed to the client requesting the data. Users can move the responsibility for data access and updates to the Internet using today's popular multithreaded web server applications such as MTS and IIS. Using MBF-UDALink in a thin client environment for fast and reliable multi-threaded access to host based data. MBF-UDALink provides additional security features to assist in securing information at a user level with field value security. Data Source Creation ODBC Data Sources.lnk Data sources are created using the ODBC Data Sources icon found in the Control Panel. Using the MBF-UDALink driver, a data source is created and associated to a User ID. Figure 5a Data source creation MB Foster Associates Limited 2002 6
Host Based Security Host based security ensures that you retain control of data integrity and access controls. Developers of corporate Intranets can provide secure and reliable access to the HP e3000 data through browser clients. User ID's are added in the Host Configuration Utility of MBF-UDALink Figure 6 Host Based Security - User ID Databases are defined in the Host Configuration Utility. Note: the ability to defined access to Image/SQL, AllBase/SQL and direct to Image, to files defined by MBF-UDALink Views and File Definitions (FDs) as well as Cognos PDL Dictionaries and Oracle databases. Figure 7 Host Based Security - Databases The Host Configuration Utility of MBF-UDALink is used to control User access down to the table level. Figure 8 Host Based Security - User Tables MB Foster Associates Limited 2002 7
Three Tier Client Server Connection Using IIS 4.0 3 Tier Client-Server Web Browser/ Application IIS/ColdFusion/ MTS (Business Rules) (Driver Mgr.) (Driver) Data Data Access Software Client Server Data Server Figure 9 Three-Tier Client Server Connection The connection is completed using the MBF-UDALink driver as set up in the Host Configuration Utility. The following Registry Setting have been made to optimize the connection. All values are given as examples. Registry Settings for MBF-UDALink HKEY_LOCAL_MACHINE\Software\ODBC\odbc.ini\!YOURDSN! MaxStmt to 48 CommandTimeout to 300000 (5min) LongTimeout to 300000 (5min) Registry Settings for Internet Information Server 4.0 Using MBF-UDALink HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W3SVC\Parameters PoolIDCConnections to 1 PoolIDCConnectionsTimeOut to 30 (seconds) AllowKeepAlives to 1 UploadReadAhead to 48000 (KB) MB Foster Associates Limited 2002 8
HKEY LOCAL_MACHINES\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters MaxPoolThreads to 20 (Threads) PoolThreadLimit to 40 (Meg) AcceptExOutstanding to 100 (Sockets Waiting) Connection Pooling Connection Pooling is entirely implemented within the ODBC 3 Driver Manager. It can be used with either an ODBC 3 or ODBC 2 data source and works with both the MBF-UDALink and ODBCLink/SE drivers. The driver Manager intercepts the SQLDriverConnect and SQLDisconnect calls. After the application disconnects, the Manager will keep the connection open for a given length of time to enable further connections if required. Connecting to a DataSource typically is one of the longest operations in a set of SQL transactions (it usually takes several seconds). Thererfore, the time to get data is significantly reduced, if you can keep the connection open. For example, to read a small table typically takes.1 to.25 of a second. With connection pooling, it is possible to get 10 web pages to read a table in one second. Without connection pooling it could take as long as 40 sec to read the 10 tables. Multi-Threading Multi-threading is a programming model that allows an application to have multiple threads of execution. Multithreading allows an application to have concurrency and parallelism on multiprocessor systems. A thread safe function is one that may be safely called by multiple threads at the same time. If the function accesses shared data or resources, a mutex or some other form of synchronization regulates this access. A mutex is a mutual exclusion synchronization primitive. Mutexes provide threads with the ability to regulate or serialize access to process shared data and resources. When a thread locks a mutex, other threads trying to lock the mutex block until the owning thread unlocks the mutex. The ODBC 3 spec does NOT require that an ODBC 3 driver be thread safe, however to implement certain optional functions available in ODBC 3.5 e.g. enlistment in distributed transactions, you must be thread safe. The UDALink.com driver supports thread affinity. This means that more than one thread can work on a given statement. e.g. thread one can open a connection and allocate a statement, thread two can execute and fetch data on the statement and thread three can issue a freestmt and free connect. Monitoring MBF-Console is a monitor tool for MBF-UDALink and ODBCLink/SE, that allows you to identify and manage your ODBC users. It features the following:?? Monitor resource utilization of ODBC connections to a specific HP e3000 or HP 9000 computer?? Kill runaway or inactive server processes?? Start or stop logging of a specific ODBC connection and then view activity from the host logfile, including all SQL sent from the client How MBF-Console Works The listener process creates server processes as needed to handle incoming connection requests. On MPE; these processes run as children of the listener process and therefore are not visible in SHOWJOB. Normally, the system administrator has no way of knowing the identity and resource utilization of ODBC users on the system. That is the purpose of MBF-Console. MBF-Console creates an ODBC connection to the listener process and to a server. For this it needs a valid data-source configuration (through the ODBC Control Panel). Once connected, it is able to obtain MB Foster Associates Limited 2002 9
information about other server processes by reading the statistics file. The statistics file is a shared file on the host that is updated by active server processes at a maximum interval of 1 minute. MBF-Console does not interrupt the server processes; rather, it produces statistical information when ODBC requests come in from the client. MBF-Console displays information for all ODBCLink/SE or MBF-UDALink server processes on the specific host, even when multiple listeners are running. If information is required from several hosts, multiple copies of MBF-Console may be run one for each host to be monitored. The Listener Window The listener window contains information about ODBCLink/SE or MBF-UDALink listener processes running on your system. It is a graphic representation of active listener jobs. To view detailed process information in the Connections Window, double-click anywhere on the table, or use the Window, Connections command. The following statistics are available in the Listeners Windows: Port Number Time Process Started Version Number of Job Number of Users Number of CPU Seconds The Connections Window The connection window lists all ODBCLink/SE or MBF-UDALink connections to your system from all listener processes, including Console itself. The following information is presented for each connection: Port Number Pin Number Time Process Started IP Address of Process Login ID of Process CPU Seconds Used Number and Rate of Messages Received Number and Rate of Messages Sent Figure 10a Listener and Connection Windows in MBF Console using an MBF-UDALink connection MB Foster Associates Limited 2002 10
Figure 10b Listener and Connection Windows in MBF Console using an ODBCLink/SE connection. To obtain more detailed information about a specific connection, double-click on it. This will open the Server Window. Figure 11 Server Window in MBF Console MB Foster Associates Limited 2002 11
In addition to repeating what is in the connections window, the following statistics are available in the Server Windows:? ODBCLOG Setting? Client Version? Statistics for Time Period? Time of Last Server Activity Reported? Bytes Received? Bytes Sent? Number of Select Statements? Number of Records Fetched? Number of Update Statements? Number of Delete Statements Conclusion Figure 12 ODBCLOG Generated from MBF-Console The Start logging command is the same as setting the ODBCLINK_LOG environment variable on the host. MBF Console will display the Listeners window and the Connections window. Data access is critical in today's business environment. MB Foster provides the ability to access HP e3000 data from a web environment in addition to transforming the data into information, using tools such as ODBCLink/SE or MBF-UDALink. For More Information Contact MB Foster at http://www.mbfoster.com or by email at info@mbfoster.com. Fax requests to MB Foster at (613) 448-2588 or call (613) 448-2333 or toll free at 1-800-ANSWERS that s 1-800-267-9377. MB Foster Associates Limited 2002 12