Neutron Monitor Database ADRESKLOKSTRAAT 12 A - 2600 BERCHEMTEL03.230.88.10FAX03.303.02.59BTWBE 886.370.360MAILINFO@SPUTNIKWEB.BEWEBWWW.SPUTNIKWEB.BE
Introduction In this document the work of Sputnik Web (BIRA) related to the NMDB project is described. Main chapters: - Choice of the technology to be used - Benchmarking the MySQL setup - How to create a replication node - Current database structure In attachment please find the dump of the database created by Sputnik Web for BIRA.
Choice of the technology to be used Important requirements for the prototype of the distributed database were: - Cross platform (Lunix and Windows) - Easy to install for the different international teams - Reasonably priced - Robust and dependable We hereby provide an overview of the different technologies used to develop this Multi-master database and of our findings and conclusions. PostgreSQL PostgreSQL initially does not support a Multi-master database. In order to be able to use PostgreSQL in a Multi-master database we therefore had to include third-party applications/extensions. We first evaluated the use of the PGCluster extension. The application was compiled and manually extensively configured. Developing a replication node proved to be extremely complex. Keeping in mind the varying technical experience of the international teams, the conclusion had to be drawn that the PGCluster is not the suited solution. Moreover, the extension only operates in a Unix based environment. The same conclusions were drawn for the second extension which was evaluated. PGPool is another PostgreSQL cluster/load balancing system that allows multiple master systems but only operates in a Unix Based environment. Again not the right solution. In a third instance CyberCluster was evaluated. In comparison to the previous two systems, CyberCluster has its advantages. For example, the system operates both in a Unix and a Windows environment. Unfortunately, the system is hardly documented. This is a serious disadvantage, as it severely complicates its implementation. The system does have a support system (http://www. postgresql.at/english/support_postgresql_e.html). However, this service is expensive. MySQL MySQL version 5 is the first MySQL version to support a multiple master database. Therefore the system still has to overcome some stumbling blocks. MySQL is supported both in a Unix and in a Window environment. It is open source and easy to install. Furthermore, developing a replication node within this system doesn t cause any problems. Problems might arise when multiple masters simultaneously write in one and the same table (primary key synchronization). This possible stumbling block can be overcome by assuring that the writing permissions of a specific master are restricted to a specific table in the database. If every Neutron Monitor has its own table in which only he is permitted to write, these possible problems with primary key synchronization will not arise. This solution of restricted rights will have to be implemented in order to be able to use this system.
Conclusion Our preference goes out to MySQL because the system does not imply the use of third party software and because developing the database based on MySQL is fairly uncomplicated. Benchmarking the MySQL setup The configuration of the whole system is setup to be multiple master. Each node is a master node, meaning that each node can read, add, change or delete data in the database. These nodes can be connected to each other in many different ways, but the best way is to implement the system with a primary node. On this node new connections can be setup to other nodes (replications), changes can be made to the structure of the database,. Every node will be connected to this master node and will synchronize with this node (as a fake master-slave configuration). The system will stay manageable in this way. One neutron monitor station could be chosen to oversee the system on a daily / operational basis. This setup makes sure that users of the system can always read data from and write data to a local database node. As described above we have to make sure that the content of a table will only be changed by one master. Therefore different users have to be created on one node: a user with read access to all tables, users with write access to specific tables,. This has to be done locally. If the mirroring is setup this way, the chance of losing data or data to get corrupted is slim to none. To test this MySQL setup we installed 2 nodes of a MySQL database on 2 separate machines. After running the system for more than 2 months the results of the tests were very positive. Whenever changes were made to the structure (tables, fields, ) of the database on 1 node, the changes were immediately and automatically implemented on node 2. Also adding, changing and deleting data on one node had an immediate effect on the other node. The setup never failed and never was data lost or corrupted. Offcourse these tests were done on 2 machines located in Belgium. This means the system hasn t been benchmarked on a greater scale. The results of those tests would be very interesting. Until now we never noticed any delay in the communication between the nodes.
How to create a replication node Legend Next conventions will be used throughout the explanation. Primary master server: Master1 Second master server: Master2 Database: bira-sql Install MySQL The installation method of MySQL depends on the chosen distribution. There are a lot of tutorials and documentation you can find on the internet to complete the installation procedure. Change some settings of MySQL It is important that you change the administrator passwords after installing MySQL. Change the passwords of root@localhost as well as root@%. You also have to make sure that the MySQL server can be reached from the localhost as well as from any network interface. This can be done by editing mu.cnf in /etc/mysql/my.cnf (on debian Linux) as follows: Change bind-addres localhost to #bind-addres localhost (comment). Restart the MySQL process. Prepare the replication The necessary users needed for the replication of bira-sql have to be created on both Master1 and Master2. In that way Master2 can connect to Master1 to create a copy of bira-sql. For example: Create a new user repsrv_2 on the primary master server. GRANT REPLICATION SLAVE ON *.* TO repsrv_2 @ % IDENTIFIED BY password ; FLUSH PRIVILEGES; Create a new user repsrv_1 on the second master server. GRANT REPLICATION SLAVE ON *.* TO repsrv_1 @ % IDENTIFIED BY password ; FLUSH PRIVILEGES;
Then edit my.cnf on both servers (Master1 and Master2). Next parameters should be changed or added (section [mysqld]) on Master1. server-id = 1 replicate-same-server-id = 0 auto-increment-increment = 2 auto-increment-offset = 1 master-host = Master2 master-user = repsrv_1 master-password = password master-connect-retry = 60 replicate-do-db = bira-sql log-bin = /var/log/mysql/mysql-bin.log binlog-do-db = exampledb relay-log = /var/lib/mysql/slave-relay.log relay-log-index = /var/lib/mysql/slave-relay-log.index expire_logs_days = 14 max_binlog_size = 500M Next parameters should be changed or added (section [mysqld]) on Master2. server-id = 2 replicate-same-server-id = 0 auto-increment-increment = 2 auto-increment-offset = 1 master-host = Master1 master-user = repsrv_2 master-password = password master-connect-retry = 60 replicate-do-db = bira-sql log-bin = /var/log/mysql/mysql-bin.log binlog-do-db = exampledb relay-log = /var/lib/mysql/slave-relay.log relay-log-index = /var/lib/mysql/slave-relay-log.index expire_logs_days = 14 max_binlog_size = 500M Restart the MySQL process on both servers.
Flush the master database Log on to the MySQL shell on Master1. mysql u root p Execute next query: USE bira-sql; FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS; You will get output similar to: mysql> SHOW MASTER STATUS; File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000087 98 bira-sql 1 row in set (0.00 sec) Keep this output by copying and pasting it in a file. Leave this MySQL shell open, open a new MySQL shell and execute next commando in the new shell: cd /tmp mysqldump u root p opt bira-sql > bira-sql.sql scp bira-sql.sql root@slave:/tmp Go back to the first shell and remove the LOCK: UNLOCK TABLES;
Import bira-sql on Slave Log on to a MySQL shell on the second master server and execute next commando: mysqladmin u root p stop-slave mysql u root p bira-sql < /tmp/ bira-sql.sql Get the replication status on the second master server: mysql u root p USE bira-sql; FLUSH TABLES WITH READ LOCK; SHOW MASTER STATUS; You will get output similar to: mysql> SHOW MASTER STATUS; File Position Binlog_Do_DB Binlog_Ignore_DB mysql-bin.000061 98 bira-sql 1 row in set (0.00 sec) Keep this output by copying and pasting it in a file. Activate replication On the second master server: CHANGE MASTER TO MASTER_HOST= Master1, MASTER_USER= repsrv_2, MASTER_ PASSWORD= password, MASTER_LOG_FILE= mysql-bin.000087, MASTER_LOG_POS=98; START SLAVE; You will find all paramaters needed in the file where you kept the output from SHOW MASTER STATUS. Repeat this on the primary master server: mysql u root p STOP SLAVE; CHANGE MASTER TO MASTER_HOST= Slave, MASTER_USER= repsrv_1, MASTER_ PASSWORD= password, MASTER_LOG_FILE= mysql-bin.000061, MASTER_LOG_POS=98; START SLAVE; The replication should be up and running. Check /var/log/syslog for errors that might have occured. Create the users needed on Slave.
Current database structure Each Neutron Monitor Station has a separate table in the current database for storing the data and for storing the specifications of the station. Data table Field id DateTime uncorrected corrected_for_pressure corrected_for_efficiency pressure version quality_flag comment created_at updated_at Type int(10) (auto_increment) datetime int(1) tinyint(1) text datetime datetime Specifications table Field id pressure pressure_coefficient longitude latitude altitude cutoff Type int(10) (auto_increment) Neutron Monitor Stations included This is a list of the stations included in the current database. aatb apatity athn bira erv_erv3 esoi jung kerg_tera kiel lmks mosc oulu rome ubern yakutsk