for QSS/OASIS Presented by Bill Genske Session 135

Transcription

1 Quintessential School Systems Session 135 SQL Server Administration for QSS/OASIS Presented by Bill Genske Gary Jackson Copyright Quintessential School Systems, 2011 All Rights Reserved 867 American Street --- Second Floor --- San Carlos, CA /

2

3 SQL Server Administration for QSS/OASIS QSS Users Group 27th Annual Conference, February 28 - March 2, 2011 Session 135 Bill Genske & Gary Jackson, QSS

4 Scope of Presentation We will cover some of the common SQL database server administration tasks. SQL database server platforms covered Microsoft SQL Server referred to as SQL Server Examples will be shown for SQL Server 2005 but 2000 and 2008 have similar concepts / methods. PostgreSQL Server QSS Requires a minimum version of 8.2 Examples will be shown using 8.3.x Available versions 8.2.x, 8.3.x, 8.4.x, 9.0.x Very technical so ask questions at anytime.

5 Common SQL Database Server Administration Functions Starting / Stopping services Maintaining Users / Roles Managing Disk Space Data compaction Managing Log files Backup and Recovery Managing SQL Server Jobs

6 How do we do this? Proprietary tools specific do each database platform. SQL Server Management Studio (2005 and 2008) Enterprise Manager (2000) PostgreSQL Various PostgreSQL commands Linux Commands

7 Who s your DBA? SQL Server SQL Server Authentication sa - default SQL Server Administrator qssdba - QSS Database Administrator if given the sysadmin role Windows Authentication Any user given the sysadmin role PostgreSQL Linux users postgres (usually has no password set) qssdba

8 QSS Databases Version L finance hrspay pque strsmf sysctl Version H GLDSYS, ACTRCV, BDEVOL, CONVRT, ECOMRC, FASSET, RECON, STRSYS PERPAY, BENMGT, PPHIST, PYHIST, RETSYS, STTSYS PQUE STRSMF SYSCTL, MENU, MENCTL, QDACTL, QSSPRE

9 Starting / Stopping SQL Server Services All required services should be configured to start automatically when server boots. Managed locally on SQL Server SQL Server Configuration Manager Show you if service is running, started automatically. Can stop and start SQL Server Surface Area Configuration Show you if service is running Lets you configure automatic startup Can stop and start

10 SQL Server Configuration Manager

11 SQL Server Surface Area Configuration

12 PostgreSQL Automatic Startup Automatic start is setup via Linux Use distribution specific GUI tool to manage servers run levels linux-demo:~ # ll /etc/init.d/postgresql -rwxr-xr-x 1 root root 6330 Dec /etc/init.d/postgresql linux-demo:~ # ll /etc/init.d/rc?.d/???postgresql lrwxrwxrwx 1 root root 13 Feb 15 09:56 /etc/init.d/rc3.d/k03postgresql ->../postgresql lrwxrwxrwx 1 root root 13 Sep /etc/init.d/rc3.d/s07postgresql ->../postgresql lrwxrwxrwx 1 root root 13 Feb 15 09:56 /etc/init.d/rc5.d/k03postgresql ->../postgresql lrwxrwxrwx 1 root root 13 Sep /etc/init.d/rc5.d/s07postgresql ->../postgresql

13 Starting / Stopping PostgreSQL Using init script Done from root user linux-demo:~ # /etc/init.d/postgresql Usage: /etc/init.d/postgresql {start stop status try-restart restart force-reload reload probe} linux-demo:~ # /etc/init.d/postgresql status Checking for PostgreSQL: Using PostgreSQL command Done from postgres user linux-demo:~ # su - postgres postgres@linux-demo:~> pg_ctl --help pg_ctl is a utility to start, stop, restart, reload configuration files, report the status of a PostgreSQL server, or signal a PostgreSQL process. running Usage: pg_ctl start [-w] [-t SECS] [-D DATADIR] [-s] [-l FILENAME] [-o "OPTIONS"] pg_ctl stop [-W] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE] pg_ctl restart [-w] [-t SECS] [-D DATADIR] [-s] [-m SHUTDOWN-MODE] [-o "OPTIONS"] pg_ctl reload [-D DATADIR] [-s] pg_ctl status [-D DATADIR]

14 qssdba Login qssdba - database administrator create/alter/drop databases create/alter/drop tables in databases read/write access to all data in all databases Minimum roles SQL Server: bulkadmin, dbcreator, public PostgreSQL: superuser Created using SQL Server: Management Studio PostgreSQL: createuser command

15 qssdbr, qssdbu Logins qssdbr read only access to all data in all databases qssdbu read/write access to all data in all databases SQL Scripts for creating qssdbr / qssdbu available on version L Linux server SQL Server: /opt/qss/install/sql/qssdbx-ms.sql PostgreSQL: /opt/qss/install/sql/qssdbx-pg.sql Needs to be re-run after any database changes which create tables

16 SQL Server Users / Roles

17 Database capacities TurboIMAGE databases had pre-defined capacities SQL Databases have no capacity limitation, but are limited by file system and the database platform. SQL Server Maximum database size: 524,258 terabytes Maximum data file size: 12 terabytes Maximum log file size: 2 terabytes PostgreSQL Maximum database size: unlimited Maximum table size 32 terabytes Maximum log file size: file system limits

18 SQL Server Disk Files SQL Server databases reside in one or more disk file. An initial size, and increment size is used for data and log files when database is created. Current size is viewable from database properties. Additional disk files can be added at anytime.

19 SQL Server Database Size

20 PostgreSQL Disk Usage Data files reside in the data directory specified when the service is started or if omitted the default is used. Files names consist of numbers in numbered directories. df -h commands shows available disk space on the server du -sh postgresdatadir shows space used by all of PostgresSQL s data files.

21 Available Disk Space on Linux file system Put PostgreSQL into it s own file system, possibly even it s own volume group Allows adding more space later if needed Allows an easy check via df command to see how much is used and available qssdba@linux-demo:~> df -h /var/lib/pgsql/data Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg01-postgres 252G 6.0G 234G 3% /var/lib/pgsql

22 PostgreSQL data directory disk space summary base is where the database files (tables, indexes, etc) reside Log files are in pg_log directory Configuration files are here too. du -sh /var/lib/pgsql/data/* 3.6G /var/lib/pgsql/data/base 812K /var/lib/pgsql/data/global 380K /var/lib/pgsql/data/pg_clog 4.0K /var/lib/pgsql/data/pg_hba.conf 4.0K /var/lib/pgsql/data/pg_hba.conf.default 4.0K /var/lib/pgsql/data/pg_ident.conf 434M /var/lib/pgsql/data/pg_log 28K /var/lib/pgsql/data/pg_multixact 192K /var/lib/pgsql/data/pg_subtrans 4.0K /var/lib/pgsql/data/pg_tblspc 4.0K /var/lib/pgsql/data/pg_twophase 4.0K /var/lib/pgsql/data/pg_version 129M /var/lib/pgsql/data/pg_xlog 20K /var/lib/pgsql/data/postgresql.conf 4.0K /var/lib/pgsql/data/postmaster.opts 4.0K /var/lib/pgsql/data/postmaster.pid

23 Sizes of PostgreSQL databases and tables To see the size of a database qssdba@linux-demo:~> psql finance finance=# SELECT pg_size_pretty(pg_database_size('finance')) As dbsize; dbsize MB To see the size of a table s data, or table s data plus indexes finance=# pg_size_pretty(pg_relation_size('public.account')) AS tablesize, finance=# pg_size_pretty(pg_total_relation_size('public.account')) As totalsize; tablesize totalsize MB 76 MB

24 Data Compaction Compacts database files by removing no longer used pages May take a while depending on size of database SQL Server Referred to as shrink Database Properties can set Auto Shrink to True to have it automatic. Manually done from the database s task list or using SQL command DBCC SHRINKDATABASE ( finance ) PostgreSQL Manually done using SQL command vacuum PostgreSQL command vacuumdb is a wrapper for above

25 Manually Shrink SQL Server database

26 PostgreSQL vacuumdb Help man vacuumdb VACUUMDB(1) PostgreSQL Client Applications NAME vacuumdb - garbage-collect and analyze a PostgreSQL database VACUUMDB(1) SYNOPSIS vacuumdb [ connection-option... ] [ [ --full ] [ -f ] ] [ [ --verbose ] [ -v ] ] [ [ --analyze ] [ -z ] ] [ --table -t table [ ( column [,...] ) ] ] [ dbname ] vacuumdb [ connection-options... ] [ [ --all ] [ -a ] ] [ [ --full ] [ -f ] ] [ [ --verbose ] [ -v ] ] [ [ --analyze ] [ -z ] ] DESCRIPTION vacuumdb is a utility for cleaning a PostgreSQL database. vacuumdb will also generate internal statistics used by the PostgreSQL query optimizer. vacuumdb is a wrapper around the SQL command VACUUM [vacuum(7)]. There is no effective difference between vacuuming databases via this utility and via other methods for accessing the server. OPTIONS vacuumdb accepts the following command-line arguments: -a --all Vacuum all databases. [-d] dbname Manual page vacuumdb(1) line 1

27 SQL Server Logs

28 Configure SQL Server Error Logs

29 View SQL Server Logs

30 PostgreSQL Server Logs What is logged, log file naming, and more logging options are controlled in postgresql.conf Listing log files ll data/pg_log -rw postgres postgres :59 postgresql _ log -rw postgres postgres :59 postgresql _ log -rw postgres postgres :42 postgresql _ log -rw postgres postgres :13 postgresql _ log -rw postgres postgres :59 postgresql _ log -rw postgres postgres :00 postgresql _ log -rw postgres postgres :00 postgresql _ log -rw postgres postgres :59 postgresql _ log -rw postgres postgres :59 postgresql _ log -rw postgres postgres :59 postgresql _ log Old files are not deleted but can easily be done via a crontab for postgres user and the command find data/pg_log -name postgresql*.log -mtime +30 -exec rm {} \;

31 View PostgreSQL Server Log Use any Linux command for viewing text files such as less less data/pg_log/postgresql _ log :25:06 PST hrsweb qssdba LOG: could not receive data from client: Co nnection reset by peer :25:06 PST hrsweb qssdba LOG: unexpected EOF on client connection :25:06 PST hrsweb qssdba LOG: could not receive data from client: Co nnection reset by peer :25:06 PST hrsweb qssdba LOG: unexpected EOF on client connection :49:08 PST hrsweb qssdba LOG: unexpected EOF on client connection :49:08 PST hrsweb qssdba LOG: unexpected EOF on client connection :49:08 PST sysctl qssdba LOG: unexpected EOF on client connection :49:08 PST hrspay qssdba LOG: unexpected EOF on client connection :54:38 PST hrsweb qssdba LOG: unexpected EOF on client connection :54:38 PST hrsweb qssdba LOG: unexpected EOF on client connection :54:38 PST sysctl qssdba LOG: unexpected EOF on client connection :54:38 PST hrspay qssdba LOG: unexpected EOF on client connection :56:31 PST hrsweb qssdba LOG: unexpected EOF on client connection :57:36 PST hrspay qssdba LOG: unexpected EOF on client connection data/pg_log/postgresql _ log lines 1-14/446 2%

32 Backup and Recovery Critical maintenance task Disaster recovery Site loss, off-site backups, entire server backed up Data recovery On-site backup of database files Solid Backup Plan How frequently to backup How long can you be down for both backup and recovery process What is acceptable data loss Backups are monitored to ensure running Recovery methods are tested before needed.

33 Database Backup / Restore methods QSS DBX Utilities dbxunld, dbxld - for entire database dbxget, dbxadd - for table Database Platform Proprietary tools Backup of entire server

34 SQL Server Backup/Restore Tasks

35 SQL Server Backup

36 SQL Server New Database

37 SQL Server Restore

38 PostgreSQL Backup pg_dump, pg_dumpall QSS Automated Task module pgbackup Examples backup finance database example pg_dump -Fc finance > finance Fc option is the most flexible format and is compressed by default

39 PostgreSQL Restore pg_restore Examples (assumes previous pg_dump -Fc) Restore entire database replacing existing database dropdb finance createdb -E SQL_ASCII finance pg_restore -d finance finance Restore entire database to a new database name createdb -E SQL_ASCII finance pg_restore -d finance finance

40 PostgreSQL Restore - examples Examples (continued) Restore one table replacing existing table Option 1. drops the table and restores data and table definition but not indexes pg_restore -d finance -t bde_budget -c -v finance Option 2. restores data only leaving existing structure in place psql finance -c "DELETE from bde_budget;" pg_restore -d finance -t bde_budget -a finance Option 3. restore specific entries from dump list file pg_restore -l finance grep bde_budget > finance.restore.list vi finance.restore.list #remove lines not associated with desired table pg_restore -d finance -L finance.restore.list -c -v finance

41 QSS Automated Task Module - pgbackup Automated method to backup PostgreSQL databases Configuration file /etc/opt/qss/qat.d/pgbackup Has naming conventions for 3 types of backups Daily, weekly, monthly Definition for end of week, end of month is configurable Based on naming conventions files can be saved until manually deleted or overwritten

42 Managing SQL Server Jobs Automating maintenance tasks such as backup SQL Server agent Allows you to setup jobs SQL commands based on a schedule Job Activity Monitor lets you review results PostgreSQL can automate tasks via cron

43 SQL Server Agent