THE ARCHIVAL SECTOR IN DW2.0 By W H Inmon

Size: px
Start display at page:

Download "THE ARCHIVAL SECTOR IN DW2.0 By W H Inmon"

Transcription

1 The fourth sector of the DW2.0 environment is the archival sector. Fig arch.1 shows the architectural positioning of the archival sector. Fig arch.1 The archival sector All data that flows into the archival sector comes from the near line sector. Fig arch.2 shows the source of the data. Fig arch.2 The source of data for the archival sector is the near line sector The reason why data is placed in the archival sector is that the probability of access has dropped significantly. Fig arch.3 shows that data whose probability of access approaches zero is placed in the archival sector. Fig arch.3 The probability of access of archival data is very low In many cases data is archived for legal reasons. The probability of access is actually very near zero. Yet the data still needs to be saved. Fig arch.4 shows the archiving of data for the purpose of satisfying legal requirements.

2 Fig arch.4 Often times data is archived for legal reasons, not for reasons of probability of access From a philosophical standpoint, if the corporation has taken the trouble of capturing and electronically structuring data, then throwing the data away seems like a poor choice. If the data ever has to be reconstructed, then once it is thrown away, it is either impossible to reconstruct the data or very expensive and troublesome do such a reconstruction. Therefore, if there is a need for ever accessing the data, then it usually is not destroyed. One of the reasons why archival data is held indefinitely is that storing archival data is an inexpensive thing to do. For that reason archival data is almost never stored on disk storage, as seen in Fig arch.5. Fig arch.5 Archived data is almost never stored on disk storage The essence of archival data is the storage of data for a long time 10 years, 20 years, and beyond. Fig arch.6 shows that archival data is meant to be kept for long periods of time. Fig arch.6 Archived data is stored for a long time As such all data in the archival environment is related to time. Fig arch.7 shows that data in the archival environment is organized by time, usually by years.

3 Fig arch.7 All data inside the archival environment is related to time Because there is a lot of data in the archival environment and because the data is organized primarily by time, metadata becomes very important. It is through metadata that the different types of data are located. Fig arch.8 shows the importance of metadata. Md Fig arch.8 Metadata is a very important component of the archival sector The importance of metadata is such that without metadata the archival environment becomes a one way street, as seen in Fig arch.9.

4 one way Md Fig arch.9 Without metadata, the archival sector becomes a one way street Once the metadata is in place, the archival environment can be searched in a reasonably efficient manner. But without metadata, entire files may have to be scanned, which is a huge waste of resources. From the standpoint of data structure, the records in the archival sector can take many different forms. Some of the possibilities of the form of record that can be taken are that records can be split, written as is, combined. Fig arch.10 shows some of the possibilities for structuring records in the archival sector. Fig arch.10 The records in the archival sector can be copies of records, can be records that have been split, or can be any number of other record types. In addition to metadata being important as a guide to the contents to the archival sector, indexes are important as well. Metadata describes the types of data that are found in the archival sector, while indexes describe the contents. Fig arch.11 shows the indexes that can be created for the archival environment.

5 Fig arch.11 Passive indexes for the archival sector are as important as they are for the near line sector In most cases the archival sector has a separate processor that manages the data found in the sector. And in most cases the machine is kept idle most of the time. A good usage of the machine resources is to create indexes in anticipation of the future usage of archival data. These indexes can be called passive indexes, for they are created not based on any known information requirement, but are based on future unknown requirements. Once the passive indexes are created and the metadata infrastructure is created, the archival environment can be accessed with a reasonable amount of efficiency. The metadata that is created needs to be stored as an actual part of the archival sector itself. It needs to be stored in the actual data set itself. The reason for storing metadata as part of the actual data is so that over time the data and the metadata won t become separated. Fig arch.12 shows that metadata is part of the archival sector and is stored with the data itself. Md Fig arch.12 The metadata needs to be stored as a close and integral part of the archival sector

6 The practice of storing metadata with data is to ensure that over time the metadata will not become lost. If the metadata is ever lost, then the worth of the archival data is much less. Fig arch.13 illustrates this fact. Fig arch.13 If the metadata ever becomes lost to the archival sector, then using the archival becomes very difficult. Access to the archival sector occurs in a pattern that can be described as a sequentially random pattern, as seen in Fig arch.14 When access archival data, it is normal to access the first record in a random manner, followed by a number of records that are sequentially accessed after the first record is found.

7 When activities are run against the archival sector, those activities tend to be large, as seen in Fig arch.15. Fig arch.15 When transactions are run against the archival sector, they tend to be large transactions When data is inserted in the archival sector, it is inserted in the form of snapshots, as seen in Fig arch.16. Fig arch.16 When data is inserted into the archival sector, it is inserted in the form of snapshots But suppose an erroneous unit of data happens to be found in the archival sector. At most, the erroneous data may be deleted. Then a correcting snapshot is entered into the archival sector. Fig arch.17 shows this process. Fig arch.17 If an error is found in archival data, it is not corrected or removed, Instead, a correcting snapshot is entered

8 And of course, on occasion whole sections of data can be pulled out of the archival sector. Once pulled out they can be placed anywhere in DW2.0 in the interactive sector, in the integrated sector, or in the near line sector. Fig arch.18 shows this placement. Fig arch.18 Once it has been decided to pull data out of the archival sector, the data can be placed anywhere - the interactive sector, the integrated sector, or the near line sector.

10 Ways to Not Get Caught Hacking On Your Mac

10 Ways to Not Get Caught Hacking On Your Mac 10 Ways to Not Get Caught Hacking On Your Mac Three18 is a Comprehensive Technology Solutions Provider Apple Certified Partner Microsoft Gold Partner Symantec Security Solutions Partner Novell and RedHat

More information

Hadoop Architecture. Part 1

Hadoop Architecture. Part 1 Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,

More information

PARALLEL PROCESSING AND THE DATA WAREHOUSE

PARALLEL PROCESSING AND THE DATA WAREHOUSE PARALLEL PROCESSING AND THE DATA WAREHOUSE BY W. H. Inmon One of the essences of the data warehouse environment is the accumulation of and the management of large amounts of data. Indeed, it is said that

More information

DATABASE MANAGEMENT SYSTEMS

DATABASE MANAGEMENT SYSTEMS CHAPTER DATABASE MANAGEMENT SYSTEMS This chapter reintroduces the term database in a more technical sense than it has been used up to now. Data is one of the most valuable assets held by most organizations.

More information

Data Memo. BY: Associate Director John B. Horrigan (202-419-4500) RE: USE OF CLOUD COMPUTING APPLICATIONS AND SERVICES September 2008

Data Memo. BY: Associate Director John B. Horrigan (202-419-4500) RE: USE OF CLOUD COMPUTING APPLICATIONS AND SERVICES September 2008 Data Memo BY: Associate Director John B. Horrigan (202-419-4500) RE: USE OF CLOUD COMPUTING APPLICATIONS AND SERVICES September 2008 Cloud computing takes hold as 69% of all internet users have either

More information

The Future of Information

The Future of Information Place image here The Future of Information Management Bill Tolson Iron Mountain In the face of an ongoing electronic information explosion, information management becomes an imperative. Organizations need

More information

Original-page small file oriented EXT3 file storage system

Original-page small file oriented EXT3 file storage system Original-page small file oriented EXT3 file storage system Zhang Weizhe, Hui He, Zhang Qizhen School of Computer Science and Technology, Harbin Institute of Technology, Harbin E-mail: wzzhang@hit.edu.cn

More information

Institute for Advanced Study Shelby White and Leon Levy Archives Center

Institute for Advanced Study Shelby White and Leon Levy Archives Center Institute for Advanced Study Shelby White and Leon Levy Archives Center Managing Electronic Records - Recommendations for Institute Staff File Management: Guidelines & Policies Which files are considered

More information

Laserfiche Volumes: Introduction and Best Practices

Laserfiche Volumes: Introduction and Best Practices Laserfiche Volumes: Introduction and Best Practices White Paper November 2005 The information contained in this document represents the current view of Compulink Management Center, Inc on the issues discussed

More information

Welcome to the unit of Hadoop Fundamentals on Hadoop architecture. I will begin with a terminology review and then cover the major components

Welcome to the unit of Hadoop Fundamentals on Hadoop architecture. I will begin with a terminology review and then cover the major components Welcome to the unit of Hadoop Fundamentals on Hadoop architecture. I will begin with a terminology review and then cover the major components of Hadoop. We will see what types of nodes can exist in a Hadoop

More information

DESIGN AND IMPLEMENTATION OF A SECURE MULTI-CLOUD DATA STORAGE USING ENCRYPTION

DESIGN AND IMPLEMENTATION OF A SECURE MULTI-CLOUD DATA STORAGE USING ENCRYPTION DESIGN AND IMPLEMENTATION OF A SECURE MULTI-CLOUD DATA STORAGE USING ENCRYPTION Prof. M. Ben Swarup Professor, Department of CSE Vignan's Institute of Information Technology Visakhapatnam-49, India Chukkala

More information

Secure information storage

Secure information storage Secure information storage in a private cloud built upon local network resources Student Project for Large Installation Administration Master of Science in System and Network Engineering Universiteit van

More information

The Key Elements of Digital Asset Management

The Key Elements of Digital Asset Management The Key Elements of Digital Asset Management The last decade has seen an enormous growth in the amount of digital content, stored on both public and private computer systems. This content ranges from professionally

More information

A block based storage model for remote online backups in a trust no one environment

A block based storage model for remote online backups in a trust no one environment A block based storage model for remote online backups in a trust no one environment http://www.duplicati.com/ Kenneth Skovhede (author, kenneth@duplicati.com) René Stach (editor, rene@duplicati.com) Abstract

More information

Machine Architecture and Number Systems. Major Computer Components. Schematic Diagram of a Computer. The CPU. The Bus. Main Memory.

Machine Architecture and Number Systems. Major Computer Components. Schematic Diagram of a Computer. The CPU. The Bus. Main Memory. 1 Topics Machine Architecture and Number Systems Major Computer Components Bits, Bytes, and Words The Decimal Number System The Binary Number System Converting from Decimal to Binary Major Computer Components

More information

LOADING DATA INTO THE WAREHOUSE BY W. H. Inmon

LOADING DATA INTO THE WAREHOUSE BY W. H. Inmon LOADING DATA INTO THE WAREHOUSE BY W. H. Inmon The data warehouse is loaded with data that has been transformed coming from operational systems. The data coming from the operational systems undergoes a

More information

Recommendations for Performance Benchmarking

Recommendations for Performance Benchmarking Recommendations for Performance Benchmarking Shikhar Puri Abstract Performance benchmarking of applications is increasingly becoming essential before deployment. This paper covers recommendations and best

More information

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes

More information

Users are Complaining that the System is Slow What Should I Do Now? Part 1

Users are Complaining that the System is Slow What Should I Do Now? Part 1 Users are Complaining that the System is Slow What Should I Do Now? Part 1 Jeffry A. Schwartz July 15, 2014 SQLRx Seminar jeffrys@isi85.com Overview Most of you have had to deal with vague user complaints

More information

SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011

SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011 SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications Jürgen Primsch, SAP AG July 2011 Why In-Memory? Information at the Speed of Thought Imagine access to business data,

More information

Chapter 7: Termination Detection

Chapter 7: Termination Detection Chapter 7: Termination Detection Ajay Kshemkalyani and Mukesh Singhal Distributed Computing: Principles, Algorithms, and Systems Cambridge University Press A. Kshemkalyani and M. Singhal (Distributed Computing)

More information

CSE 120 Principles of Operating Systems

CSE 120 Principles of Operating Systems CSE 120 Principles of Operating Systems Fall 2004 Lecture 13: FFS, LFS, RAID Geoffrey M. Voelker Overview We ve looked at disks and file systems generically Now we re going to look at some example file

More information

Digital Forensics Tutorials Acquiring an Image with FTK Imager

Digital Forensics Tutorials Acquiring an Image with FTK Imager Digital Forensics Tutorials Acquiring an Image with FTK Imager Explanation Section Digital Forensics Definition The use of scientifically derived and proven methods toward the preservation, collection,

More information

A Deduplication-based Data Archiving System

A Deduplication-based Data Archiving System 2012 International Conference on Image, Vision and Computing (ICIVC 2012) IPCSIT vol. 50 (2012) (2012) IACSIT Press, Singapore DOI: 10.7763/IPCSIT.2012.V50.20 A Deduplication-based Data Archiving System

More information

6. Storage and File Structures

6. Storage and File Structures ECS-165A WQ 11 110 6. Storage and File Structures Goals Understand the basic concepts underlying different storage media, buffer management, files structures, and organization of records in files. Contents

More information

IDERA WHITEPAPER. The paper will cover the following ten areas: Monitoring Management. WRITTEN BY Greg Robidoux

IDERA WHITEPAPER. The paper will cover the following ten areas: Monitoring Management. WRITTEN BY Greg Robidoux WRITTEN BY Greg Robidoux Top SQL Server Backup Mistakes and How to Avoid Them INTRODUCTION Backing up SQL Server databases is one of the most important tasks DBAs perform in their SQL Server environments

More information

Acrobat for Healthcare Professionals

Acrobat for Healthcare Professionals Acrobat for Healthcare Professionals Mark Middleton Business Development Manager mmiddlet@adobe.com Rick Borstein Business Development Manager borstein@adobe.com What we ll cover... Slides PDF Background

More information

Template 4: Description of Archiving System

Template 4: Description of Archiving System Template 4: Description of Archiving System 1: Institutional Arrangements 2: Methods and Data Documentation 3: Description of QA/QC Procedures 4: Description of Archiving System 5: Key Category Analysis

More information

Dynamic Publisher Manual of Features

Dynamic Publisher Manual of Features Dynamic Publisher Manual of Features Introduction to Dynamic Publisher Dynamic Publisher is an extension module for Dynamic Knowledgebase that enables you to take atlases developed and published in Dynamic

More information

DOCUMENT MANAGEMENT. Evo2: YOUR FLEXIBLE FRIEND Evo3: SEEK AND YE SHALL FIND

DOCUMENT MANAGEMENT. Evo2: YOUR FLEXIBLE FRIEND Evo3: SEEK AND YE SHALL FIND DOCUMENT MANAGEMENT Evo2: YOUR FLEXIBLE FRIEND Evo3: SEEK AND YE SHALL FIND Merlin created DocuStor to provide low-cost management of scanned documents. There are many Electronic Document Management products

More information

In-Memory Databases MemSQL

In-Memory Databases MemSQL IT4BI - Université Libre de Bruxelles In-Memory Databases MemSQL Gabby Nikolova Thao Ha Contents I. In-memory Databases...4 1. Concept:...4 2. Indexing:...4 a. b. c. d. AVL Tree:...4 B-Tree and B+ Tree:...5

More information

iservdb The database closest to you IDEAS Institute

iservdb The database closest to you IDEAS Institute iservdb The database closest to you IDEAS Institute 1 Overview 2 Long-term Anticipation iservdb is a relational database SQL compliance and a general purpose database Data is reliable and consistency iservdb

More information

This exam contains 13 pages (including this cover page) and 18 questions. Check to see if any pages are missing.

This exam contains 13 pages (including this cover page) and 18 questions. Check to see if any pages are missing. Big Data Processing 2013-2014 Q2 April 7, 2014 (Resit) Lecturer: Claudia Hauff Time Limit: 180 Minutes Name: Answer the questions in the spaces provided on this exam. If you run out of room for an answer,

More information

RAID HARDWARE. On board SATA RAID controller. RAID drive caddy (hot swappable) SATA RAID controller card. Anne Watson 1

RAID HARDWARE. On board SATA RAID controller. RAID drive caddy (hot swappable) SATA RAID controller card. Anne Watson 1 RAID HARDWARE On board SATA RAID controller SATA RAID controller card RAID drive caddy (hot swappable) Anne Watson 1 RAID The word redundant means an unnecessary repetition. The word array means a lineup.

More information

Discovery Technology Group

Discovery Technology Group Discovery Technology Group E-mail Retention: Readiness Survey E-mail now represents the largest source of new documents and records generated within a company, and the most troublesome from a retention

More information

Unit 4.3 - Storage Structures 1. Storage Structures. Unit 4.3

Unit 4.3 - Storage Structures 1. Storage Structures. Unit 4.3 Storage Structures Unit 4.3 Unit 4.3 - Storage Structures 1 The Physical Store Storage Capacity Medium Transfer Rate Seek Time Main Memory 800 MB/s 500 MB Instant Hard Drive 10 MB/s 120 GB 10 ms CD-ROM

More information

Understanding Disk Storage in Tivoli Storage Manager

Understanding Disk Storage in Tivoli Storage Manager Understanding Disk Storage in Tivoli Storage Manager Dave Cannon Tivoli Storage Manager Architect Oxford University TSM Symposium September 2005 Disclaimer Unless otherwise noted, functions and behavior

More information

BridgeWays Management Pack for VMware ESX

BridgeWays Management Pack for VMware ESX Bridgeways White Paper: Management Pack for VMware ESX BridgeWays Management Pack for VMware ESX Ensuring smooth virtual operations while maximizing your ROI. Published: July 2009 For the latest information,

More information

Best Practices for Architecting Storage in Virtualized Environments

Best Practices for Architecting Storage in Virtualized Environments Best Practices for Architecting Storage in Virtualized Environments Leverage Advances in Storage Technology to Accelerate Performance, Simplify Management, and Save Money in Your Virtual Server Environment

More information

SPARC Enterprise s Approach to Virtualization and Its Contribution to ICT Society

SPARC Enterprise s Approach to Virtualization and Its Contribution to ICT Society SPARC Enterprise s Approach to Virtualization and Its Contribution to ICT Society Masaru Nukada Akio Satori In recent years, it has become common practice to implement virtualization technology across

More information

Availability and Disaster Recovery: Basic Principles

Availability and Disaster Recovery: Basic Principles Availability and Disaster Recovery: Basic Principles by Chuck Petch, WVS Senior Technical Writer At first glance availability and recovery may seem like opposites. Availability involves designing computer

More information

Catalog Software Solves Copy Chaos

Catalog Software Solves Copy Chaos Catalog Software Solves Copy Chaos Author : David Floyer Categories : Economic Models, Software-led Infrastructure, Updates Date : November 17, 2014 One of the greatest operational challenges in modern

More information

How To Manage A Virtual Data Center In A Country With Limited Space

How To Manage A Virtual Data Center In A Country With Limited Space 3. Technology Technology UKAI This article introduces our research into the UKAI* 1 storage system, which enables flexible control over the actual data locations of virtual disk images of virtual machines

More information

RAID. RAID 0 No redundancy ( AID?) Just stripe data over multiple disks But it does improve performance. Chapter 6 Storage and Other I/O Topics 29

RAID. RAID 0 No redundancy ( AID?) Just stripe data over multiple disks But it does improve performance. Chapter 6 Storage and Other I/O Topics 29 RAID Redundant Array of Inexpensive (Independent) Disks Use multiple smaller disks (c.f. one large disk) Parallelism improves performance Plus extra disk(s) for redundant data storage Provides fault tolerant

More information

Recordkeeping for Good Governance Toolkit. GUIDELINE 14: Digital Recordkeeping Choosing the Best Strategy

Recordkeeping for Good Governance Toolkit. GUIDELINE 14: Digital Recordkeeping Choosing the Best Strategy Recordkeeping for Good Governance Toolkit GUIDELINE 14: Digital Recordkeeping Choosing the Best Strategy i The original version of this guideline was prepared by the Pacific Regional Branch of the International

More information

Azure VM Performance Considerations Running SQL Server

Azure VM Performance Considerations Running SQL Server Azure VM Performance Considerations Running SQL Server Your company logo here Vinod Kumar M @vinodk_sql http://blogs.extremeexperts.com Session Objectives And Takeaways Session Objective(s): Learn the

More information

Parallel Programming Map-Reduce. Needless to Say, We Need Machine Learning for Big Data

Parallel Programming Map-Reduce. Needless to Say, We Need Machine Learning for Big Data Case Study 2: Document Retrieval Parallel Programming Map-Reduce Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Carlos Guestrin January 31 st, 2013 Carlos Guestrin

More information

Physical Design. Meeting the needs of the users is the gold standard against which we measure our success in creating a database.

Physical Design. Meeting the needs of the users is the gold standard against which we measure our success in creating a database. Physical Design Physical Database Design (Defined): Process of producing a description of the implementation of the database on secondary storage; it describes the base relations, file organizations, and

More information

Distributed File Systems

Distributed File Systems Distributed File Systems Paul Krzyzanowski Rutgers University October 28, 2012 1 Introduction The classic network file systems we examined, NFS, CIFS, AFS, Coda, were designed as client-server applications.

More information

ICOM 6005 Database Management Systems Design. Dr. Manuel Rodríguez Martínez Electrical and Computer Engineering Department Lecture 2 August 23, 2001

ICOM 6005 Database Management Systems Design. Dr. Manuel Rodríguez Martínez Electrical and Computer Engineering Department Lecture 2 August 23, 2001 ICOM 6005 Database Management Systems Design Dr. Manuel Rodríguez Martínez Electrical and Computer Engineering Department Lecture 2 August 23, 2001 Readings Read Chapter 1 of text book ICOM 6005 Dr. Manuel

More information

Glossary of Records Management Terms

Glossary of Records Management Terms Glossary of Records Management Terms Active record: A record referenced often in the conduct of current departmental business. Administrative record: Records documenting the day to day operation and administration

More information

A Survey on Data Integrity of Cloud Storage in Cloud Computing

A Survey on Data Integrity of Cloud Storage in Cloud Computing A Survey on Data Integrity of Cloud Storage in Cloud Computing A B S T R A C T Mr.Vitthal Raut, Prof. Suhasini Itkar Department Computer Engineering, PES Modern College of Engineering, Pune, India. raut.vitthal@gmail.com,

More information

Image Gateway for Apeos 2.0

Image Gateway for Apeos 2.0 IGA2.0 Image Gateway for Apeos 2.0 Business Process Solutions Business Process Optimisation Versatility of the multi-function device (MFD) is an often-touted benefit by all the major office equipment manufacturers,

More information

HowTo: Logging, reporting, log-analysis and log server setup Version 2007nx Release 3. Log server version 2.0

HowTo: Logging, reporting, log-analysis and log server setup Version 2007nx Release 3. Log server version 2.0 Log server version 2.0 Contents 1 Setting up the log server for the appliance... 4 1.1 Registering the log server on the appliance... 4 1.2 Entering the Syslog server to the appliance... 6 2 Log server...

More information

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl Big Data Processing, 2014/15 Lecture 5: GFS & HDFS!! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl 1 Course content Introduction Data streams 1 & 2 The MapReduce paradigm Looking behind

More information

INFORMATION SECURITY GUIDELINES

INFORMATION SECURITY GUIDELINES INFORMATION SECURITY GUIDELINES TABLE OF CONTENTS: Scope of Document 1 Data Definition Guidelines (Appendix 1).2 Data Protection Guidelines (Appendix 2).3 Protection of Electronic or Machine- Readable

More information

EMC VNX2 Deduplication and Compression

EMC VNX2 Deduplication and Compression White Paper VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, & VNX8000 Maximizing effective capacity utilization Abstract This white paper discusses the capacity optimization technologies delivered in the

More information

Addressing the Abandoned Email Archive Risk

Addressing the Abandoned Email Archive Risk Addressing the Abandoned Email Archive Risk WHITE PAPER Table of Contents 3 Executive Summary 3 Introduction 4 What are Email Archives? 4 EOL, Obsolete, and Abandoned Email Archives 5 The Risks and Costs

More information

Guideline for stresstest Page 1 of 6. Stress test

Guideline for stresstest Page 1 of 6. Stress test Guideline for stresstest Page 1 of 6 Stress test Objective: Show unacceptable problems with high parallel load. Crash, wrong processing, slow processing. Test Procedure: Run test cases with maximum number

More information

Backup. Contents. 1 Storage, the base of a backup system. 2 Selection, extraction and manipulation of data. 3 Managing the backup process.

Backup. Contents. 1 Storage, the base of a backup system. 2 Selection, extraction and manipulation of data. 3 Managing the backup process. Backup In information technology, a backup or the process of backing up refer to making copies of data so that these additional copies may be used to restore the original after a data loss event. These

More information

Google File System. Web and scalability

Google File System. Web and scalability Google File System Web and scalability The web: - How big is the Web right now? No one knows. - Number of pages that are crawled: o 100,000 pages in 1994 o 8 million pages in 2005 - Crawlable pages might

More information

Index Terms Cloud Storage Services, data integrity, dependable distributed storage, data dynamics, Cloud Computing.

Index Terms Cloud Storage Services, data integrity, dependable distributed storage, data dynamics, Cloud Computing. Volume 3, Issue 5, May 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Privacy - Preserving

More information

Storage Switzerland White Paper Storage Infrastructures for Big Data Workflows

Storage Switzerland White Paper Storage Infrastructures for Big Data Workflows Storage Switzerland White Paper Storage Infrastructures for Big Data Workflows Sponsored by: Prepared by: Eric Slack, Sr. Analyst May 2012 Storage Infrastructures for Big Data Workflows Introduction Big

More information

DATA MIGRATION IN ARCHIVES OF SERBIA AND MONTENEGRO CONCEPT AND EXAMPLE

DATA MIGRATION IN ARCHIVES OF SERBIA AND MONTENEGRO CONCEPT AND EXAMPLE Преглед НЦД 5 (2004), 83 88 (Archives of Serbia and Montenegro) DATA MIGRATION IN ARCHIVES OF SERBIA AND MONTENEGRO CONCEPT AND EXAMPLE Abstract. We present organization of database used to store metadata

More information

Best Practices for Virtualised SharePoint

Best Practices for Virtualised SharePoint Best Practices for Virtualised SharePoint Brendan Law Blaw@td.com.au @FlamerNZ Flamer.co.nz/spag/ Nathan Mercer Nathan.Mercer@microsoft.com @NathanM blogs.technet.com/nmercer/ Agenda Why Virtualise? Hardware

More information

Backup and Recovery 1

Backup and Recovery 1 Backup and Recovery What is a Backup? Backup is an additional copy of data that can be used for restore and recovery purposes. The Backup copy is used when the primary copy is lost or corrupted. This Backup

More information

ZNetLive Malware Monitoring

ZNetLive Malware Monitoring Introduction The criminal ways of distributing malware or malicious software online have gone through a change in past years. In place of using USB drives, attachments or disks to distribute viruses, hackers

More information

Considerations for Management of Laboratory Data

Considerations for Management of Laboratory Data Considerations for Management of Laboratory Data 2003 Scientific Computing & Instrumentation LIMS Guide, November 2003 Michael H Elliott Drowning in a sea of data? Nervous about 21 CFR Part 11? Worried

More information

The Hadoop Distributed File System

The Hadoop Distributed File System The Hadoop Distributed File System The Hadoop Distributed File System, Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler, Yahoo, 2010 Agenda Topic 1: Introduction Topic 2: Architecture

More information

Automated file management with IBM Active Cloud Engine

Automated file management with IBM Active Cloud Engine Automated file management with IBM Active Cloud Engine Redefining what it means to deliver the right data to the right place at the right time Highlights Enable ubiquitous access to files from across the

More information

Worldwide Managed Services for. 402 Amherst Street, Suite 300 Nashua, NH 03063, USA. Phone: 603 879 9022 e mail: holland@sciinc.com www.sciinc.

Worldwide Managed Services for. 402 Amherst Street, Suite 300 Nashua, NH 03063, USA. Phone: 603 879 9022 e mail: holland@sciinc.com www.sciinc. Worldwide Managed Services for OpenVMS and Rdb Software Concepts International, LLC 402 Amherst Street, Suite 300 Nashua, NH 03063, USA Phone: 603 879 9022 e mail: holland@sciinc.com www.sciinc.com About

More information

an introduction to networked storage

an introduction to networked storage an introduction to networked storage How networked storage can simplify your data management The key differences between SAN, DAS, and NAS The business benefits of networked storage Introduction Historical

More information

Configuring Apache Derby for Performance and Durability Olav Sandstå

Configuring Apache Derby for Performance and Durability Olav Sandstå Configuring Apache Derby for Performance and Durability Olav Sandstå Database Technology Group Sun Microsystems Trondheim, Norway Overview Background > Transactions, Failure Classes, Derby Architecture

More information

MONITORING PERFORMANCE IN WINDOWS 7

MONITORING PERFORMANCE IN WINDOWS 7 MONITORING PERFORMANCE IN WINDOWS 7 Performance Monitor In this demo we will take a look at how we can use the Performance Monitor to capture information about our machine performance. We can access Performance

More information

EMC DATA DOMAIN DATA INVULNERABILITY ARCHITECTURE: ENHANCING DATA INTEGRITY AND RECOVERABILITY

EMC DATA DOMAIN DATA INVULNERABILITY ARCHITECTURE: ENHANCING DATA INTEGRITY AND RECOVERABILITY White Paper EMC DATA DOMAIN DATA INVULNERABILITY ARCHITECTURE: ENHANCING DATA INTEGRITY AND RECOVERABILITY A Detailed Review Abstract No single mechanism is sufficient to ensure data integrity in a storage

More information

DATA WAREHOUSE/BIG DATA AN ARCHITECTURAL APPROACH

DATA WAREHOUSE/BIG DATA AN ARCHITECTURAL APPROACH DATA WAREHOUSE/BIG DATA AN ARCHITECTURAL APPROACH By W H Inmon and Deborah Arline First there was data warehouse. Then came Big Data. Some of the proponents of Big Data have made the proclamation When

More information

Operating Systems. Virtual Memory

Operating Systems. Virtual Memory Operating Systems Virtual Memory Virtual Memory Topics. Memory Hierarchy. Why Virtual Memory. Virtual Memory Issues. Virtual Memory Solutions. Locality of Reference. Virtual Memory with Segmentation. Page

More information

Oracle Total Recall with Oracle Database 11g Release 2

Oracle Total Recall with Oracle Database 11g Release 2 An Oracle White Paper September 2009 Oracle Total Recall with Oracle Database 11g Release 2 Introduction: Total Recall = Total History... 1 Managing Historical Data: Current Approaches... 2 Application

More information

BACKUP STRATEGY AND DISASTER RECOVERY POLICY STATEMENT

BACKUP STRATEGY AND DISASTER RECOVERY POLICY STATEMENT TADCASTER GRAMMAR SCHOOL Toulston, Tadcaster, North Yorkshire. LS24 9NB BACKUP STRATEGY AND DISASTER RECOVERY POLICY STATEMENT Written by Steve South November 2003 Discussed with ICT Strategy Group January

More information

How To Manage Records And Information Management In Alberta

How To Manage Records And Information Management In Alberta 8. RECORDS AND INFORMATION MANAGEMENT Overview This chapter is intended to help public bodies understand how good records and information management practices assist in the effective administration of

More information

Efficient database auditing

Efficient database auditing Topicus Fincare Efficient database auditing And entity reversion Dennis Windhouwer Supervised by: Pim van den Broek, Jasper Laagland and Johan te Winkel 9 April 2014 SUMMARY Topicus wants their current

More information

Using Speccy to Report on Your Computer Components

Using Speccy to Report on Your Computer Components Using Speccy to Report on Your Computer Components Ronald Ross rross@spkaa.com Today we re going to have a look at what I consider to be a useful Windows reporting utility. I ll be talking about Speccy,

More information

Rethinking Cybersecurity. Rethinking Cybersecurity. Page 1

Rethinking Cybersecurity. Rethinking Cybersecurity. Page 1 Rethinking Cybersecurity Page 1 Engineering Control, Freedom & Privacy: TABLE OF CONTENTS It s not about the networks, it s about the data. It s the data that s valuable, and it s the data that s risky.

More information

A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS

A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS A STUDY ON HADOOP ARCHITECTURE FOR BIG DATA ANALYTICS Dr. Ananthi Sheshasayee 1, J V N Lakshmi 2 1 Head Department of Computer Science & Research, Quaid-E-Millath Govt College for Women, Chennai, (India)

More information

Data Classification Technical Assessment

Data Classification Technical Assessment Data Classification Update: February 13th, 2015 Statement of Confidentiality This Confidential Information is being provided to Customer ABC as a deliverable of this consulting engagement. The sole purpose

More information

Chapter 2: OS Overview

Chapter 2: OS Overview Chapter 2: OS Overview CmSc 335 Operating Systems 1. Operating system objectives and functions Operating systems control and support the usage of computer systems. a. usage users of a computer system:

More information

Cloud Computing with Azure PaaS for Educational Institutions

Cloud Computing with Azure PaaS for Educational Institutions International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 4, Number 2 (2014), pp. 139-144 International Research Publications House http://www. irphouse.com /ijict.htm Cloud

More information

Cosmos. Big Data and Big Challenges. Pat Helland July 2011

Cosmos. Big Data and Big Challenges. Pat Helland July 2011 Cosmos Big Data and Big Challenges Pat Helland July 2011 1 Outline Introduction Cosmos Overview The Structured s Project Some Other Exciting Projects Conclusion 2 What Is COSMOS? Petabyte Store and Computation

More information

Assignment 1 Briefing Paper on the Pratt Archives Digitization Projects

Assignment 1 Briefing Paper on the Pratt Archives Digitization Projects Twila Rios Digital Preservation Spring 2012 Assignment 1 Briefing Paper on the Pratt Archives Digitization Projects The Pratt library digitization efforts actually encompass more than one project, including

More information

Internet Grocery Stores What does the future look like? By: Matthew Rousu

Internet Grocery Stores What does the future look like? By: Matthew Rousu Internet Grocery Stores What does the future look like? By: Matthew Rousu In the past several years, there has been an explosion of Internet companies on the retail market. Internet grocery stores exist,

More information

Database Normalization. Mohua Sarkar, Ph.D Software Engineer California Pacific Medical Center 415-600-7003 sarkarm@sutterhealth.

Database Normalization. Mohua Sarkar, Ph.D Software Engineer California Pacific Medical Center 415-600-7003 sarkarm@sutterhealth. Database Normalization Mohua Sarkar, Ph.D Software Engineer California Pacific Medical Center 415-600-7003 sarkarm@sutterhealth.org Definition A database is an organized collection of data whose content

More information

Overview of Storage and Indexing

Overview of Storage and Indexing Overview of Storage and Indexing Chapter 8 How index-learning turns no student pale Yet holds the eel of science by the tail. -- Alexander Pope (1688-1744) Database Management Systems 3ed, R. Ramakrishnan

More information

Alternatives to Big Backup

Alternatives to Big Backup Alternatives to Big Backup Life Cycle Management, Object- Based Storage, and Self- Protecting Storage Systems Presented by: Chris Robertson Solution Architect Cambridge Computer Copyright 2010-2011, Cambridge

More information

The Classical Architecture. Storage 1 / 36

The Classical Architecture. Storage 1 / 36 1 / 36 The Problem Application Data? Filesystem Logical Drive Physical Drive 2 / 36 Requirements There are different classes of requirements: Data Independence application is shielded from physical storage

More information

A Performance Study of Load Balancing Strategies for Approximate String Matching on an MPI Heterogeneous System Environment

A Performance Study of Load Balancing Strategies for Approximate String Matching on an MPI Heterogeneous System Environment A Performance Study of Load Balancing Strategies for Approximate String Matching on an MPI Heterogeneous System Environment Panagiotis D. Michailidis and Konstantinos G. Margaritis Parallel and Distributed

More information

Benchmarking Hadoop & HBase on Violin

Benchmarking Hadoop & HBase on Violin Technical White Paper Report Technical Report Benchmarking Hadoop & HBase on Violin Harnessing Big Data Analytics at the Speed of Memory Version 1.0 Abstract The purpose of benchmarking is to show advantages

More information

Why You Need Email Archiving

Why You Need Email Archiving Why You Need Email Archiving www.exclaimer.com Table of Contents Introduction...2 The IT Administrator...3 The Email User...5 The Team Leader...6 The Senior Manager/Business Owner...7 Conclusion...8-1

More information

2) What is the structure of an organization? Explain how IT support at different organizational levels.

2) What is the structure of an organization? Explain how IT support at different organizational levels. (PGDIT 01) Paper - I : BASICS OF INFORMATION TECHNOLOGY 1) What is an information technology? Why you need to know about IT. 2) What is the structure of an organization? Explain how IT support at different

More information

Addressing Legal Discovery & Compliance Requirements

Addressing Legal Discovery & Compliance Requirements Addressing Legal Discovery & Compliance Requirements A Comparison of and Archiving In today s digital landscape, the legal, regulatory and business requirements for email archiving continue to grow in

More information

Migrating Legacy Healthcare Data to the Cloud

Migrating Legacy Healthcare Data to the Cloud Business White Paper Migrating Legacy Healthcare Data to the Cloud Four key considerations for healthcare providers Page 2 of 8 Migrating Legacy Healthcare Data to the Cloud Table of Contents Page 2 Record

More information