Mdus v4.4 Database Infrmatin
Database Infrmatin Mdus v4.4 Database Infrmatin Latest revisin: September 2007 The cntent f this manual is fr infrmatinal use nly and is subject t change withut ntice. Neither Vircm nr anyne else wh has been invlved in the creatin r prductin f this manual assumes any respnsibility r liability fr any errrs r inaccuracies that may ccur in this manual, nr fr any lss f anticipated prfit r benefits, resulting frm the use f this manual. This manual is prtected by cpyright laws and internatinal treaties. Yur right t cpy this manual is limited by cpyright law and the terms f yur sftware license agreement. As the sftware licensee, yu may make a reasnable number f cpies r printuts, prvided they are fr yur wn use. Making unauthrized cpies, adaptatins, cmpilatins r derivative wrks fr any type f distributin is prhibited and cnstitutes a punishable vilatin f the law. Any references t names f actual cmpanies, prducts, peple and/r data used in screenshts are fictitius and are in n way intended t represent any real individual, cmpany, prduct, event and/r data unless therwise nted. VOP, Vircm Online Platfrm, VOP Mail, Mdus, MdusMail, MdusGate, Mdus Sieve, SCA and VOP Radius are trademarks f Vircm, Inc. Nrman Virus Cntrl and NVC are trademarks f Nrman Data Defense Inc. Windws, Windws NT, Windws 2000, Windws Server 2003, IIS, Internet Infrmatin Server and Data Access Cmpnents are either registered trademarks r trademarks f Micrsft Crpratin in the United States and/r ther cuntries. Platypus, RODOPI, Emerald, EcBuilder, Lgisense and Wrldgrup are trademarks f their respective wners. All ther prducts r services mentined in this dcument are identified by the trademarks r service marks f their respective cmpanies r rganizatins. McAfee and NetScan are registered trademarks f Netwrk Assciates, Inc. and/r its affiliates in the US and/r ther cuntries. 2006 Netwrks Assciates Technlgy, Inc. All Rights Reserved. Prtins f this sftware are based, in part, n ImageMagick, Cpyright 1999-2006, ImageMagick Studi LLC. MdusMail is based n the Prfessinal Internet Mail Services prduct licensed frm the University f Edinburgh. Certain algrithms used in parts f this sftware are derived frm the RSA Data Security, Inc. MD5 Message-Digest Algrithm. Cpyright 1995-2007 Vircm, Inc. Vircm Inc., 460 Ste-Catherine W, Suite 600, Mntreal, QC, Canada, H3B 1A7 Fr mre infrmatin, cntact Custmer Supprt at +1 514.845.8474; Sales at +1.514.845.1666, Ext. 1 r visit ur website at www.vircm.cm. 2
Database Infrmatin Table f Cntents Intrductin...4 Database Frmats...5 Creating a New Database...6 SQL Server... 6 PstgreSQL... 6 MS SQL Express... 6 Running a Database Script...7 SQL Server... 7 PstgreSQL... 7 MS SQL Express... 7 Creating an ODBC Cnnectin...8 SQL Server / SQL Express... 8 PstgreSQL... 8 MS Access... 10 SQL Server...17 Extended (MdusMail nly)... 17 SQL Server...17 PstgreSQL...17 Security and Maintenance...18 SQL Server Security Enhancements... 18 Maintaining a SQL Server Database... 20 Full Recvery Mdel...20 Simple Recvery Mdel...20 Bulk-Lgged Recvery Mdel...21 Changing Recvery Mdels...21 Optimizing PstgreSQL fr Mdus... 22 Checkpint Segments...22 Max_fsm_pages...23 Creating and Cnfiguring the Quarantine Database...10 SQL Server / SQL Express... 10 PstgreSQL... 10 MS Access... 11 Mail Client Quarantine Access... 11 Creating and Cnfiguring the Mnitring Database...11 SQL Server... 11 PstgreSQL... 12 MS Access... 13 Creating and Cnfiguring the Sieve Database...13 SQL Server / SQL Express... 13 PstgreSQL... 13 MS Access... 14 Creating and Cnfiguring the Audit Database...14 SQL Server / SQL Express... 14 PstgreSQL... 15 Creating and Cnfiguring the Extended Database...15 SQL Server... 15 PstgreSQL... 16 Upgrading the Databases...16 Quarantine... 16 SQL Server... 16 PstgreSQL... 16 Audit... 17 3
Database Infrmatin Intrductin This dcument prvides infrmatin necessary t create and cnfigure the varius databases required fr use by Mdus. It als prvides the scripts fr upgrading Mdus. This dcument des nt replace the Mdus Administratin Guides but serves t cmpliment the infrmatin. Please cntinue t refer t the Administratin Guides fr detailed infrmatin cncerning yur Mdus prduct. 4
Database Infrmatin Database Frmats The fllwing table prvides the recmmended database frmats fr Mdus: Database MS SQL2000+ MS SLQ Express PstgreSQL MS Access & MSDE Quarantine Mnitring Audit Sieve Extended MdusMail nly MS Access and MSDE have a size limitatin f 2GB These are nt recmmended fr large envirnments but are acceptable in small envirnments Clients shuld cnsider using anther database prgram as these have limitatins and will nt supprt future features SQL Express has a size limitatin f 4GB It can be used in medium-sized envirnments This nly applies t the Audit DB as it can grw fairly large MySQL can be used fr the Quarantine database but is nt recmmended Additinal infrmatin can be fund in the fllwing KB article: http://www.vircm.cm/kbase/default.asp?id=1328&lang=1&sid= 5
Database Infrmatin Creating a New Database The fllwing prvides infrmatin fr creating a new database: SQL Server Open the SQL Enterprise Manager Expand the tree and g t Databases Click n Actin > New Database In Database Prperties, name the database and, at Cllatin Name, use the drpdwn menu t select Latin1_General_CI-AS Click n OK This ensures that the database remains case-insensitive PstgreSQL Open the pgadmin III Expand the tree and g t Databases Right-click n Databases and select New Database Click n OK When creating a Quarantine and Sieve databases, select SQL_ASCII fr the encding Fr all ther databases, select UNICODE fr the encding MS SQL Express Open the SQL Server Management Studi Right-click Databases and select New Database At the General panel and enter the name f the new database Click n Optins and use the drp-dwn menu t select Latin1_General_CI-AS as the Cllatin Name Click n OK 6
Database Infrmatin Running a Database Script The fllwing prvides basic infrmatin fr running database scripts: SQL Server Start the SQL Query Analyzer Select the apprpriate database Click n File Open and g t the lcatin f the scripts ( Vircm\Mdus<Mail r Gate>\DBStructures\SQL Server) Click n t run the scripts PstgreSQL Open the pgadmin III Expand the tree and select the apprpriate database Click n Tls > Query Tl Click n File > Open and g t the lcatin f the scripts ( Vircm\Mdus<Mail r Gate>\DBStructures\PstgreSQL) Click n t run the scripts MS SQL Express Open the SQL Server Management Studi Click n File > Open and g t the lcatin f the scripts ( Vircm\Mdus<Mail r Gate>\DBStructures\SQL Server Select the database Click n!execute t run the scripts 7
Database Infrmatin Creating an ODBC Cnnectin The fllwing prvides infrmatin fr creating an ODBC cnnectin: On the server, g t Administrative Tls > Data Surces (ODBC) Click n System DSN Click n Add Select the apprpriate driver (SQL Server, PstgreSQL r MS Access) and click Finish SQL Server / SQL Express Enter a name fr the data surce name Optinally, enter a descriptin f the data surce Select the SQL server frm the drp-dwn menu and click Next Select the SQL Server authenticatin methd and enter the lgin ID and passwrd Click n Next Change the default database t whichever database is being cnfigured (e.g. Quarantine, Sieve, etc.) Click n Next and then Finish PstgreSQL Enter a name fr the data surce name Optinally, enter a descriptin f the data surce Enter the name f the database being cnfigured (e.g. Quarantine, Sieve, etc.) Enter the name r IP address f the PstgreSQL server Enter the user name and passwrd Click n Datasurce and enter the fllwing infrmatin 8
Database Infrmatin G t Page 2 and set the fllwing parameters Click n OK Click n Save 9
Database Infrmatin MS Access Enter the data surce name and descriptin Click n Select t chse the database being cnfigured and click n OK Click n OK Exit the ODBC Data Surce Administratr panel Creating and Cnfiguring the Quarantine Database The fllwing prvides infrmatin fr creating the Quarantine database and cnfiguring it in Mdus: SQL Server / SQL Express Create a new database Run the fllwing script, lcated in \Vircm\mdus<Mail r Gate>\DBStructures\SQL Server\Quarantine mssql_quarantine_imap.sql Create the ODBC cnnectin In the Mdus Cnsle, g t System Prperties Quarantine Database Enter yur ODBC infrmatin and click Apply Stp and start all Mdus services PstgreSQL Create a new database Remember t select SQL_ASCII fr the encding Run the fllwing script, lcated in \Vircm\mdus<Mail r Gate>\DBStructures\PstgreSQL\Quarantine pg_quarantine_imap.sql Create the ODBC cnnectin In the Mdus Cnsle, g t System Prperties Quarantine Database Enter yur ODBC infrmatin and click Apply Stp and start all Mdus services 10
Database Infrmatin MS Access The database is created autmatically when Mdus is installed In the Mdus Cnsle, g t System Prperties Quarantine Database Select Use Native MDB Database Click n Apply Stp and start all Mdus services Mail Client Quarantine Access If yu are using SQL Server r PstgreSQL fr the Quarantine database, users can be given access t their quarantine cntents frm their mail clients: Ensure that mailbxes are using an IMAP cnnectin t the mail server In the Mdus Cnsle, g t System Prperties Services Select IMAP4S Click Settings IMAP4 Quarantine Access shuld be checked In the users mail client, a Quarantine flder will appear in the flder list alng with a Spam subflder Only spam messages will appear in the Spam subflder (nt viruses r frbidden attachments) The cntents f the Spam flder are synchrnized with WebMail & WebQuarantine Creating and Cnfiguring the Mnitring Database The fllwing prvides infrmatin fr creating the Mnitring database and cnfiguring it in Mdus: SQL Server Create a database Open a Cmmand Prmpt n the SQL server G t Vircm\Mdus<Mail r Gate>\DBStructures\SQL Server\Mnitring Run CreateMnitringSQL.exe with the fllwing parameters: createmnitringsql <server> <database> <username> <passwrd> Create the ODBC cnnectin 11
Database Infrmatin In the Mdus Cnsle, g t System Prperties Mnitring Database Select Use ODBC Database and enter yur ODBC infrmatin Click Apply Stp and start all Mdus services PstgreSQL If yu answered yes t install PstgreSQL during the initial Mdus installatin, the Mnitring database was autmatically created: In the Mdus Cnsle, g t System Prperties Mnitring Database Select Use Native Pstgres Database and enter the apprpriate infrmatin Click n Apply Stp and start all Mdus services If yu answered n t install PstgreSQL during the initial Mdus installatin ensure that versin 8.0 r higher is installed: The necessary files can be fund in the \Vircm\Mdus<Mail r Gate> directry: mduspgmn.dll Mnitr_pg.sql Create the Mdus tables in yur PstgreSQL database and link Mdus t it: Extract the mduspgmn.dll file t \PstgreSQL\8.0\lib Frm a Cmmand Prmpt, g t \Vircm\Mdus<Mail r Gate> and type the fllwing cmmands: SET PGPASSWORD=<<yur superuser passwrd>> <enter> \Pstgresql\8.0\bin\createdb -h lcalhst -p 5432 -U pstgres -q -E SQL_ASCII -T template1 -O pstgres Mdus <enter> \Pstgresql\8.0\bin\psql -h lcalhst -p 5432 -U pstgres -q -d Mdus -f Mnitr_pg.sql <enter> SET PGPASSWORD= <enter> In the Mdus Cnsle, g t System Prperties Mnitring Database Select Use Native Pstgres Database and enter the apprpriate infrmatin Click n Apply Stp and start all Mdus services 12
Database Infrmatin MS Access This database is created autmatically during the Mdus installatin In the Mdus Cnsle, g t System Prperties Mnitring Database Select Use Legacy Access Database Click n Apply Stp and start all Mdus services Creating and Cnfiguring the Sieve Database The fllwing prvides infrmatin fr creating the Sieve database and cnfiguring it in Mdus: SQL Server / SQL Express Create a new database Run the fllwing script, lcated in \Vircm\mdus<Mail r Gate>\DBStructures\SQL Server\SieveStre mssql_sievestre_new.sql Create the ODBC cnnectin In the Mdus Cnsle, g t Spam Prperties Sieve Database Enter yur ODBC infrmatin and click Apply Stp and start all Mdus services PstgreSQL Create a new database Remember t select SQL_ASCII fr the encding Run the fllwing script, lcated in \Vircm\mdus<Mail r Gate>\DBStructures\PstgreSQL\SieveStre pg_sievestre_new.sql Create the ODBC cnnectin In the Mdus Cnsle, g t Spam Prperties Sieve Database Enter yur ODBC infrmatin and click Apply Stp and start all Mdus services 13
Database Infrmatin MS Access The database is created autmatically when Mdus is installed In the Mdus Cnsle, g t Spam Prperties Sieve Database Select Use Native MDB Database Click n Apply Stp and start all Mdus services Creating and Cnfiguring the Audit Database The fllwing prvides infrmatin fr creating the Message Audit database and cnfiguring it in Mdus: SQL Server / SQL Express Create a new database Run the fllwing scripts, in this rder, lcated in \Vircm\mdus<Mail r Gate>\DBStructures\SQL Server\Audit Audit_tables.sql Audit_prcs.sql In the Mdus Cnsle, g t the System Prperties Message Audit Database Select Micrsft SQL Server Click n Apply Enter the required infrmatin In the Cnsle, g t Lgs Prperties Message Audit Click n Apply Enable Audit Lgging and System-Wide Lgging Stp and start all Mdus services Frm a Cmmand Prmpt, type iisreset <enter> t restart the IIS service Reminder: if the SQL Server is installed n the same machine as Mdus and the Message Audit is enabled, yu will likely experience perfrmance prblems. 14
Database Infrmatin PstgreSQL Create a new database Remember t select UNICODE fr the encding Run the fllwing scripts, in this rder, lcated in \ Vircm\mdus<Mail r Gate>\DBStructures\PstgreSQL\Audit Audit_tables.sql Audit_prcs.sql In the Mdus Cnsle, g t the System Prperties Message Audit Database Select PstgreSQL Click n Apply Enter the required infrmatin In the Cnsle, g t Lgs Prperties Message Audit Click n Apply Enable Audit Lgging and System-Wide Lgging Stp and start all Mdus services Frm a Cmmand Prmpt, type iisreset <enter> t restart the IIS service Creating and Cnfiguring the Extended Database The fllwing prvides infrmatin fr creating the Extended database and cnfiguring it in Mdus. The Extended database is nly available fr MdusMail. SQL Server The fllwing steps must be executed in this rder: Create the database Run the fllwing script, lcated in Vircm\MdusMail\DBStructures\SQL Server\ExtendedDB mssql_extendeddb_new.sql Migrate the mailbx settings frm the registry t the database using the SyncUserRegDB.exe tl Select Cnvert existing registry mailbxes (this is nt selected by default) Create an ODBC cnnectin In the Mdus Cnsle, g t Auth Prperties ODBC Database Ensure Use Extended Database is checked Enter yur ODBC infrmatin and click Apply Stp and start all Mdus services 15
Database Infrmatin PstgreSQL The fllwing steps must be executed in this rder: Create the database Run the fllwing script, lcated in \Vircm\MdusMail\DBStructures\PstgreSQL\ExtendedDB pg_extendeddb_new.sql Migrate the mailbx settings frm the registry t the database using the SyncUserRegDB.exe tl Select Cnvert existing registry mailbxes (this is nt selected by default) Create an ODBC cnnectin In the Mdus Cnsle, g t Auth Prperties ODBC Database Ensure Use Extended Database is checked Enter yur ODBC infrmatin and click Apply Stp and start all Mdus services Upgrading the Databases The upgrade scripts can be run fr whichever versin f Mdus is being upgraded. Vircm prvides scripts that are run autmatically with every upgrade. These scripts are run manually because they take very lng t cmplete. It is, therefre, recmmended that yu run these scripts during ff peak times and fr every upgrade. Quarantine Befre running the manual script, stp the MODUSADM service. SQL Server Lcatin: \Vircm\Mdus<Mail r Gate>\DBStructures\SQL Server\Quarantine Run mssql_quarantine_manual_upgrade.sql PstgreSQL Lcatin: \Vircm\Mdus<Mail r Gate>\DBStructures\PstgreSQL\Quarantine Run pg_quarantine_manual_upgrade.sql 16
Database Infrmatin Audit SQL Server Lcatin: \Vircm\Mdus<Mail r Gate>\DBStructures\SQL Server\Audit Stp all Mdus services Run audit_upgrade.sql Run audit_prcs.sql Start all Mdus services Run audit_upgrade_data.sql Extended (MdusMail nly) Stp all services befre running the upgrade. Run the SyncUserRegDB.exe tl (lcated with the scripts) and use all defaults. Start the services upn cmpletin. SQL Server Lcatin: \Vircm\Mdus<Mail r Gate>\DBStructures\SQL Server\ExtendedDB Run mssql_extendeddb_upgrade.sql PstgreSQL Lcatin: \Vircm\Mdus<Mail r Gate>\DBStructures\PstgreSQL\ExtendedDB Run pg_extendeddb_upgrade.sql 17
Database Infrmatin Security and Maintenance The fllwing prvides basic infrmatin abut the security and maintenance f databases. These measures are ptinal but Vircm recmmends that yu maintain database standards. SQL Server Security Enhancements T imprve security, the database user shuld have minimum privileges: N server administratr rle Figure 1: SQL Lgin Server Rles Access granted t the new database nly Figure 2: SQL Lgin Database Access 18
Database Infrmatin Only db_datareader and db_datawriter permissins fr the new database Figure 3: SQL Database User Rle N permissin t create, drp r alter tables Permissin t execute all DB stred prcedures Figure 4: SQL Database User Permissins 19
Database Infrmatin Maintaining a SQL Server Database The fllwing prvides basic infrmatin fr maintaining SQL databases. SQL Server allws fr the recvery f databases. The recvery mdels available fr SQL Server databases each establish the type f backup and restre task that is permitted as well as the acceptable limit fr data lss. There are three recvery mdels: Full, Simple and Bulk lgged. Full Recvery Mdel Uses database and transactin lg backups which prvide the ptin fr full r differential restres Allws yu t precisely recver the database t a specific time which greatly reduces the lss f data All peratins are lgged and can easily be recvered Prvides the mst flexibility and is the mst cmmnly used ptin Use this methd if: yur data is critical and yu cannt affrd t lse any f it yu require recvery t a pint in time yu use replicatin and require the ptin t synchrnize all f yur databases t a particular pint in time yur standard transactin incrprates bulk lgged activities Regular backups are required. It is suggested that a full backup be perfrmed daily and transactin lg backups be perfrmed hurly. Simple Recvery Mdel Only restres data frm the mst recent full r differential backup There are n transactin lg backups The cntents fr the lg are truncated whenever a checkpint is issued fr the database Use this methd if: yur data is nt critical and des nt change ften data is cllected frm varius surces and can be easily reprduced disk space is limited fr lgging transactin (the issue with disk space shuld be addressed with a pssible hardware upgrade) 20
Database Infrmatin Bulk-Lgged Recvery Mdel Prvides prtectin withut affecting system perfrmance Sme peratins are minimally lgged and are nt fully recverable Damaged file recvery requires yu t manually reslve the issue fr peratins that are nt cmpletely lgged Only allws fr database restres t the end f a transactin lg backup (where it cntains bulk changes) Use this methd if: data is critical but s is system perfrmance bulk peratins are perfrmed during ff-hurs and d nt impede regular prcessing recvery t a pint in time is required This ptin is nt recmmended fr use with MdusMail. Changing Recvery Mdels In SQL Server, pen Enterprise Manager Right click n the database name and select Prperties Select the Optins tab At Recvery Mdel use the pull-dwn menu t select the mdel Click n OK Figure 5: Recvery Mdel Settings 21
Database Infrmatin The recmmended minimum settings fr Mdus databases are: Audit: Simple Quarantine: Simple Mnitring: Simple Sieve: Full Extended: Full Extended and sieve databases are mre imprtant in that they cntain the user settings and sme cmputed values used by the services. As such, a full backup and transactin lg backups will reduce user interventin in the event f a recvery. Optimizing PstgreSQL fr Mdus Checkpint Segments Mdus data written t the PstgreSQL database is cached in a lg file until enugh data is accumulated t warrant a disk-write. Ideally, the lg size shuld be larger than the default setting as this allws Mdus t perfrm faster (if PstgreSQL writes t the disk t ften, Mdus perfrmance slws). The checkpint_segments cnfiguratin setting determines when PstgreSQL can write data t the disk. PstgreSQL has tw settings fr the checkpints. The first, checkpint_segments, is based n the amunt f data mdified. The secnd, checkpint_timeut, is a timeut in secnds. If there are n checkpints perfrmed in the past X secnds, PstgreSQL will perfrm ne. A segment represents a certain amunt f data mdified. During the PstgreSQL installatin, this value defaults t 3 s that when 3 segments f data are accumulated, PstgreSQL writes t the disk. Vircm recmmends changing the checkpint_segments value t 10 fr ptimum Mdus perfrmance. Keep the default value fr checkpint_timeut. T change the checkpint_segments setting: G t PstgreSQL > Cnfiguratin files and select edit pstgresql.cnf Find the line checkpint_segments and change the value t 10 Remve the # at the start f the line t uncmment the line Save the file Stp and start the POSTGRESQL service n the server Stp and start all Mdus services 22
Database Infrmatin Max_fsm_pages Whenever a database is mdified, PstgreSQL allcates new space in pages f 8K. The ld page is still there and cannt be re-used until there is a vacuum cmmand called. Once every hur, Mdus issues the cmmand t be able t re-use the ld pages. Re-using an ld page will nt increase the size f the file n the disk. The setting max_fsm_pages indicates the maximum number f pages t mark as re-usable when the vacuum cmmand is used. Prblems ccur when there are mre than 20,000 pages that are mdified per hur. Setting the cnfiguratin t 1,000,000 pages shuld re-use all f the ld pages even if the vacuum peratin takes many hurs t cmplete. The dwnside is that it uses mre memry. PstgreSQL will use abut 5MG f additinal memry when the setting is changed frm 20,000 pages. Frm the Start menu, g t PstgreSQL > Cnfiguratin Files > Edit pstgresql.cnf Lcate the line #max_fsm_pages = 20000 and mdify it t max_fsm_pages = 1000000 Remve the # s that the line is n lnger a cmment Stp and start the PstgreSQL service Stp and start all Mdus services Fr mre infrmatin, cnsult the Knwledge Base article Inf: PstgreSQL Mnitring Database Grws T Large. 23