Resilient Dynamic Programming

Size: px
Start display at page:

Download "Resilient Dynamic Programming"

Transcription

1 Resilient Dynamic Programming Irene Finocchi, Saverio Caminiti, and Emanuele Fusco Dipartimento di Informatica, Sapienza Università di Roma via Salaria, Rome, Italy. {finocchi, caminiti, Kickoff AlgoDEEP Bertinoro, Italia. April (task C.1.1)

2 Outline 1 Introduction 2 A resilient framework for dynamic programming 3 Testing and experimental validation

3 Memories and faults Why should we care about memory faults in algorithm design? Memory faults happen: a large cluster of computers with a few gigabytes per node can experience one bit error every few minutes [Sah06]. Memory faults are harmful: undetected memory faults cause data corruption to spread; (potentially safety critical, e.g., avionics). Hardware solutions may be inadequate: fault-tolerant memory chips does not guarantee complete fault coverage; (expensive system halt upon detection of uncorrectable errors interruptions of service) [JNW08].

4 From liars to data corruption Algorithmic research related to memory errors has focused mainly on sorting and searching problems: late 70 s: Rényi [Rén94] and Ulam [Ula77]: twenty questions game against a liar, handling noise in binary search. Yao and Yao [YY85], and then [AU91, LM99, LMP97]: destructive faults in fault-tolerant sorting networks, comparison gates can destroy one of the input values.... [FI04] sorting in the faulty RAM model.

5 Faulty memories: an adversarial model Memory in a faulty-ram of word-size w is divided in three classes: a large unreliable memory: an adaptive adversary of unlimited computational power can modify up to δ memory words; O(1) safe memory words: the adversary can read but not modify this memory; O(1) private memory words: the adversary cannot even read this memory.

6 Local dependency dynamic programming edit distance Let e i,j be the edit distance between the prefix up to the i-th symbol of the input string X and the prefix up to the j-th symbol of the input string Y. e i,j := { ei 1,j 1 if i, j > 0 and x j = y i 1 + min {e i 1,j, e i,j 1, e i 1,j 1 } if i, j > 0 and x j y i (e 0,j = j, e i,0 = i.)

7 Correctness requirements Correctness of sorting and searching required only on uncorrupted values. In our setting, such a relaxed definition of correctness does not seem to be natural.

8 Correctness requirements Correctness of sorting and searching required only on uncorrupted values. In our setting, such a relaxed definition of correctness does not seem to be natural. We seek algorithms that correctly compute the edit distance between the two input strings, in spite of memory faults.

9 Tools Majority. Table decomposition. Fingerprinting.

10 Majority A variable can be made resilient by making 2δ + 1 copies. As at most δ of them can be altered by the adversary, the majority value is the correct value. The majority value can be read in time O(δ) and space O(1) [BM91].

11 Table decomposition The DP table is split in blocks of size δ δ. The boundaries of each block are written reliably in the faulty memory. δ 2 values result in roughly 5δ 2 memory words.

12 Fingerprinting A fingerprint for a column is computed as: ϕ k = v 1 v 2... v δ mod p where p is a prime number uniformly chosen at randomly in interval [n c 1, n c ] (where c is an appropriate constant).

13 Fingerprinting A fingerprint for a column is computed as: ϕ k = v 1 v 2... v δ mod p where p is a prime number uniformly chosen at randomly in interval [n c 1, n c ] (where c is an appropriate constant). Using logical shifts and Horner s rule, each fingerprint can be incrementally computed while generating the values v h : for h = 1 to δ do ϕ = ((ϕ 2 w ) + v h ) mod p end for

14 Block computation B i 1,j 1 B i 1,j B i,j 1 B i,j The first column of a block is computed reading reliably all values it depends from. ϕ 1

15 Block computation B i 1,j 1 B i 1,j B i,j 1 B i,j While computing the first column, fingerprint ϕ 1 is also computed. ϕ 1

16 Block computation B i 1,j 1 B i 1,j B i,j 1 B i,j While computing the first column, fingerprint ϕ 1 is also computed. ϕ 1

17 Block computation B i 1,j 1 B i 1,j B i,j 1 Bi,j While computing column k + 1, we produce two fingerprints, ϕ k+1 and ϕ k. ϕ k ϕ k ϕk+1

18 Block computation B i 1,j 1 B i 1,j B i,j 1 Bi,j Fingerprint ϕ k is then compared with ϕ k (i.e., the fingerprint produced while computing column k). ϕ k ϕ k ϕk+1

19 Block computation B i 1,j 1 B i 1,j B i,j 1 Bi,j If ϕ k ϕ k, the block is recomputed from scratch. ϕ k ϕ k ϕk+1

20 As a result we have: Theorem The edit distance between two strings of length n and m, with n m, can be correctly computed, with high probability, in: O(nm + αδ 2 ) time; O(nm) space, when δ is polynomial in n.

21 Generalizing Theorem A d-dimensional local dependency dynamic programming table M of size n d can be correctly computed, with high probability, in: O(n d + αδ d ) time; O(n d + nδ) space, when the actual number α δ of memory faults occurring during the computation is polynomial in n. (Edit distance, longest common subsequence, sequence alignment,...)

22 faultylib We are developing a library to test program behavior in presence of memory faults. Plugging in the library should be very easy: existing C/C++ code should require minimal changes to be tested with our library. Implementation of different (and meaningful) adversaries should be easy....

23 faultylib: usage FaultyUInt M[n+1u][m+1u]; // An n+1 X m+1 matrix of // faulty unsigned int... for (unsigned int i = 1; i <= n; i++) { for (unsigned int j = 1; j <= m; j++) { M[i][j] = min(1 + min(m[i-1][j], M[i][j-1]), M[i-1][j-1] + ((x[i-1]==y[j-1])? 0 : 1)); } }...

24 faultylib: faulty types implementation template <typename T> class Faulty : public FaultyBase {... private: T _val; T read() const { FaultyMM::getInstance()->faultBeforeRead(&_val, sizeof(t), context); return _val; } void write(t v) { _val = v; FaultyMM::getInstance()->faultAfterWrite(&_val, sizeof(t), context); } }... typedef Faulty<unsigned int> FaultyUInt;

25 faultylib: overriding operators... //Assignment operator template <typename Targ> Faulty & operator=(const Targ & v) { write((t)v); return *this; }... //OR template <typename Targ> bool operator (const Targ & v) const { return (read() (T)v); } }...

26 faultylib: adversaries implementation class REDAdversary : public Adversary {... virtual void faultafterwrite(void * location, size_t s, Context * cnt) { if ((cnt!= NULL) && (cnt->tag == EDMATRIX_TAG)) { MatrixContext * m = (MatrixContext *)cnt; unsigned int * i = (unsigned int *)location; if (m->getindex(0) == 3) if (m->getindex(1) == 7) *i = *i +3; } }...

27 Thanks! Thank you for your attention!

28 References [AU91] [BM91] [FI04] S. Assaf and E. Upfal. Fault tolerant sorting networks. SIAM J. Discrete Math., 4(4): , R. S. Boyer and J. S. Moore. Mjrty: A fast majority vote algorithm. In Automated Reasoning: Essays in Honor of Woody Bledsoe, pages , Irene Finocchi and Giuseppe F. Italiano. Sorting and searching in the presence of memory faults (without redundancy). In László Babai, editor, STOC, pages ACM, [JNW08] B. L. Jacob, S. W. Ng, and D. T. Wang. Memory Systems: Cache, DRAM, Disk. [LM99] Morgan Kaufmann, F. T. Leighton and Y. Ma. Tight bounds on the size of fault-tolerant merging and sorting networks with destructive faults. SIAM J. Comput., 29(1): , [LMP97] F. T. Leighton, Y. Ma, and C. G. Plaxton. Breaking the θ(n log 2 n) barrier for sorting with faults. J. Comput. Syst. Sci., 54(2): , [Rén94] [Sah06] [Ula77] [YY85] A. Rény. A diary on information theory. J. Wiley and Sons, Original publication: Napló az információelméletröl, Gondolat, Budapest, G. K. Saha. Software based fault tolerance: a survey. Ubiquity, 7(25), S. M. Ulam. Adventures of a mathematician. Charles Scribner s Sons, New York, A. C. Yao and F. F. Yao. On fault-tolerant networks for sorting. SIAM J. Comput., 14(1): , 1985.

Chapter 13 File and Database Systems

Chapter 13 File and Database Systems Chapter 13 File and Database Systems Outline 13.1 Introduction 13.2 Data Hierarchy 13.3 Files 13.4 File Systems 13.4.1 Directories 13.4. Metadata 13.4. Mounting 13.5 File Organization 13.6 File Allocation

More information

Chapter 13 File and Database Systems

Chapter 13 File and Database Systems Chapter 13 File and Database Systems Outline 13.1 Introduction 13.2 Data Hierarchy 13.3 Files 13.4 File Systems 13.4.1 Directories 13.4. Metadata 13.4. Mounting 13.5 File Organization 13.6 File Allocation

More information

Compiling CAO: from Cryptographic Specifications to C Implementations

Compiling CAO: from Cryptographic Specifications to C Implementations Compiling CAO: from Cryptographic Specifications to C Implementations Manuel Barbosa David Castro Paulo Silva HASLab/INESC TEC Universidade do Minho Braga, Portugal April 8, 2014 Grenoble Motivation Developing

More information

Classifying Large Data Sets Using SVMs with Hierarchical Clusters. Presented by :Limou Wang

Classifying Large Data Sets Using SVMs with Hierarchical Clusters. Presented by :Limou Wang Classifying Large Data Sets Using SVMs with Hierarchical Clusters Presented by :Limou Wang Overview SVM Overview Motivation Hierarchical micro-clustering algorithm Clustering-Based SVM (CB-SVM) Experimental

More information

IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE OPERATORS

IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE OPERATORS Volume 2, No. 3, March 2011 Journal of Global Research in Computer Science RESEARCH PAPER Available Online at www.jgrcs.info IMPROVING PERFORMANCE OF RANDOMIZED SIGNATURE SORT USING HASHING AND BITWISE

More information

A Catalogue of the Steiner Triple Systems of Order 19

A Catalogue of the Steiner Triple Systems of Order 19 A Catalogue of the Steiner Triple Systems of Order 19 Petteri Kaski 1, Patric R. J. Östergård 2, Olli Pottonen 2, and Lasse Kiviluoto 3 1 Helsinki Institute for Information Technology HIIT University of

More information

Versatile weighting strategies for a citation-based research evaluation model

Versatile weighting strategies for a citation-based research evaluation model Versatile weighting strategies for a citation-based research evaluation model Gianna M. Del Corso, Francesco Romani joint work with Dario A. Binii Dipartimento di Informatica, Università di Pisa, Italy

More information

Apache Spark and Distributed Programming

Apache Spark and Distributed Programming Apache Spark and Distributed Programming Concurrent Programming Keijo Heljanko Department of Computer Science University School of Science November 25th, 2015 Slides by Keijo Heljanko Apache Spark Apache

More information

A Fast Pattern Matching Algorithm with Two Sliding Windows (TSW)

A Fast Pattern Matching Algorithm with Two Sliding Windows (TSW) Journal of Computer Science 4 (5): 393-401, 2008 ISSN 1549-3636 2008 Science Publications A Fast Pattern Matching Algorithm with Two Sliding Windows (TSW) Amjad Hudaib, Rola Al-Khalid, Dima Suleiman, Mariam

More information

Arithmetic Coding: Introduction

Arithmetic Coding: Introduction Data Compression Arithmetic coding Arithmetic Coding: Introduction Allows using fractional parts of bits!! Used in PPM, JPEG/MPEG (as option), Bzip More time costly than Huffman, but integer implementation

More information

Physical Data Organization

Physical Data Organization Physical Data Organization Database design using logical model of the database - appropriate level for users to focus on - user independence from implementation details Performance - other major factor

More information

Distributed Storage Networks and Computer Forensics

Distributed Storage Networks and Computer Forensics Distributed Storage Networks 5 Raid-6 Encoding Technical Faculty Winter Semester 2011/12 RAID Redundant Array of Independent Disks Patterson, Gibson, Katz, A Case for Redundant Array of Inexpensive Disks,

More information

Algorithms and Methods for Distributed Storage Networks 5 Raid-6 Encoding Christian Schindelhauer

Algorithms and Methods for Distributed Storage Networks 5 Raid-6 Encoding Christian Schindelhauer Algorithms and Methods for Distributed Storage Networks 5 Raid-6 Encoding Institut für Informatik Wintersemester 2007/08 RAID Redundant Array of Independent Disks Patterson, Gibson, Katz, A Case for Redundant

More information

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

More information

Algorithmic Techniques for Big Data Analysis. Barna Saha AT&T Lab-Research

Algorithmic Techniques for Big Data Analysis. Barna Saha AT&T Lab-Research Algorithmic Techniques for Big Data Analysis Barna Saha AT&T Lab-Research Challenges of Big Data VOLUME Large amount of data VELOCITY Needs to be analyzed quickly VARIETY Different types of structured

More information

Why you shouldn't use set (and what you should use instead) Matt Austern

Why you shouldn't use set (and what you should use instead) Matt Austern Why you shouldn't use set (and what you should use instead) Matt Austern Everything in the standard C++ library is there for a reason, but it isn't always obvious what that reason is. The standard isn't

More information

Load Distribution on a Linux Cluster using Load Balancing

Load Distribution on a Linux Cluster using Load Balancing Load Distribution on a Linux Cluster using Load Balancing Aravind Elango M. Mohammed Safiq Undergraduate Students of Engg. Dept. of Computer Science and Engg. PSG College of Technology India Abstract:

More information

Theoretical Aspects of Storage Systems Autumn 2009

Theoretical Aspects of Storage Systems Autumn 2009 Theoretical Aspects of Storage Systems Autumn 2009 Chapter 2: Double Disk Failures André Brinkmann Data Corruption in the Storage Stack What are Latent Sector Errors What is Silent Data Corruption Checksum

More information

Chapter Objectives. Chapter 9. Sequential Search. Search Algorithms. Search Algorithms. Binary Search

Chapter Objectives. Chapter 9. Sequential Search. Search Algorithms. Search Algorithms. Binary Search Chapter Objectives Chapter 9 Search Algorithms Data Structures Using C++ 1 Learn the various search algorithms Explore how to implement the sequential and binary search algorithms Discover how the sequential

More information

Secure Way of Storing Data in Cloud Using Third Party Auditor

Secure Way of Storing Data in Cloud Using Third Party Auditor IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661, p- ISSN: 2278-8727Volume 12, Issue 4 (Jul. - Aug. 2013), PP 69-74 Secure Way of Storing Data in Cloud Using Third Party Auditor 1 Miss.

More information

GraySort on Apache Spark by Databricks

GraySort on Apache Spark by Databricks GraySort on Apache Spark by Databricks Reynold Xin, Parviz Deyhim, Ali Ghodsi, Xiangrui Meng, Matei Zaharia Databricks Inc. Apache Spark Sorting in Spark Overview Sorting Within a Partition Range Partitioner

More information

Factoring Algorithms

Factoring Algorithms Institutionen för Informationsteknologi Lunds Tekniska Högskola Department of Information Technology Lund University Cryptology - Project 1 Factoring Algorithms The purpose of this project is to understand

More information

CSE-E5430 Scalable Cloud Computing Lecture 11

CSE-E5430 Scalable Cloud Computing Lecture 11 CSE-E5430 Scalable Cloud Computing Lecture 11 Keijo Heljanko Department of Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 30.11-2015 1/24 Distributed Coordination Systems Consensus

More information

Lecture 9 - Message Authentication Codes

Lecture 9 - Message Authentication Codes Lecture 9 - Message Authentication Codes Boaz Barak March 1, 2010 Reading: Boneh-Shoup chapter 6, Sections 9.1 9.3. Data integrity Until now we ve only been interested in protecting secrecy of data. However,

More information

Tolerating Multiple Faults in Multistage Interconnection Networks with Minimal Extra Stages

Tolerating Multiple Faults in Multistage Interconnection Networks with Minimal Extra Stages 998 IEEE TRANSACTIONS ON COMPUTERS, VOL. 49, NO. 9, SEPTEMBER 2000 Tolerating Multiple Faults in Multistage Interconnection Networks with Minimal Extra Stages Chenggong Charles Fan, Student Member, IEEE,

More information

Categorical Data Visualization and Clustering Using Subjective Factors

Categorical Data Visualization and Clustering Using Subjective Factors Categorical Data Visualization and Clustering Using Subjective Factors Chia-Hui Chang and Zhi-Kai Ding Department of Computer Science and Information Engineering, National Central University, Chung-Li,

More information

Distributed Dynamic Load Balancing for Iterative-Stencil Applications

Distributed Dynamic Load Balancing for Iterative-Stencil Applications Distributed Dynamic Load Balancing for Iterative-Stencil Applications G. Dethier 1, P. Marchot 2 and P.A. de Marneffe 1 1 EECS Department, University of Liege, Belgium 2 Chemical Engineering Department,

More information

Motivation for peer-to-peer

Motivation for peer-to-peer Peer-to-peer systems INF 5040 autumn 2007 lecturer: Roman Vitenberg INF5040, Frank Eliassen & Roman Vitenberg 1 Motivation for peer-to-peer Inherent restrictions of the standard client/server model Centralised

More information

The Complexity of Online Memory Checking

The Complexity of Online Memory Checking The Complexity of Online Memory Checking Moni Naor Guy N. Rothblum Abstract We consider the problem of storing a large file on a remote and unreliable server. To verify that the file has not been corrupted,

More information

BUSINESS ANALYTICS. Data Pre-processing. Lecture 3. Information Systems and Machine Learning Lab. University of Hildesheim.

BUSINESS ANALYTICS. Data Pre-processing. Lecture 3. Information Systems and Machine Learning Lab. University of Hildesheim. Tomáš Horváth BUSINESS ANALYTICS Lecture 3 Data Pre-processing Information Systems and Machine Learning Lab University of Hildesheim Germany Overview The aim of this lecture is to describe some data pre-processing

More information

12.0 Statistical Graphics and RNG

12.0 Statistical Graphics and RNG 12.0 Statistical Graphics and RNG 1 Answer Questions Statistical Graphics Random Number Generators 12.1 Statistical Graphics 2 John Snow helped to end the 1854 cholera outbreak through use of a statistical

More information

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1

Merge Sort. 2004 Goodrich, Tamassia. Merge Sort 1 Merge Sort 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4 Merge Sort 1 Divide-and-Conquer Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two disjoint subsets

More information

Fact Sheet In-Memory Analysis

Fact Sheet In-Memory Analysis Fact Sheet In-Memory Analysis 1 Copyright Yellowfin International 2010 Contents In Memory Overview...3 Benefits...3 Agile development & rapid delivery...3 Data types supported by the In-Memory Database...4

More information

Data Streams A Tutorial

Data Streams A Tutorial Data Streams A Tutorial Nicole Schweikardt Goethe-Universität Frankfurt am Main DEIS 10: GI-Dagstuhl Seminar on Data Exchange, Integration, and Streams Schloss Dagstuhl, November 8, 2010 Data Streams Situation:

More information

Big Data and Scripting map/reduce in Hadoop

Big Data and Scripting map/reduce in Hadoop Big Data and Scripting map/reduce in Hadoop 1, 2, parts of a Hadoop map/reduce implementation core framework provides customization via indivudual map and reduce functions e.g. implementation in mongodb

More information

Optimization Problems in Infrastructure Security

Optimization Problems in Infrastructure Security Evangelos Kranakis, School of Computer Science, Carleton University, Ottawa 1 Optimization Problems in Infrastructure Security Evangelos Kranakis Carleton University School of Computer Science Ottawa,

More information

A FAST STRING MATCHING ALGORITHM

A FAST STRING MATCHING ALGORITHM Ravendra Singh et al, Int. J. Comp. Tech. Appl., Vol 2 (6),877-883 A FAST STRING MATCHING ALGORITHM H N Verma, 2 Ravendra Singh Department of CSE, Sachdeva Institute of Technology, Mathura, India, hnverma@rediffmail.com

More information

Data Distribution Algorithms for Reliable. Reliable Parallel Storage on Flash Memories

Data Distribution Algorithms for Reliable. Reliable Parallel Storage on Flash Memories Data Distribution Algorithms for Reliable Parallel Storage on Flash Memories Zuse Institute Berlin November 2008, MEMICS Workshop Motivation Nonvolatile storage Flash memory - Invented by Dr. Fujio Masuoka

More information

CUDA Programming. Week 4. Shared memory and register

CUDA Programming. Week 4. Shared memory and register CUDA Programming Week 4. Shared memory and register Outline Shared memory and bank confliction Memory padding Register allocation Example of matrix-matrix multiplication Homework SHARED MEMORY AND BANK

More information

Hardware-Aware AlgorithmsandDataStructures. Gabriel Moruz BRICS University of Aarhus

Hardware-Aware AlgorithmsandDataStructures. Gabriel Moruz BRICS University of Aarhus Hardware-Aware AlgorithmsandDataStructures Gabriel Moruz BRICS University of Aarhus 1 Hardware/nm./: the part of the computer that you can kick. Geeky folklore. Gabriel Moruz: Hardware aware algorithms

More information

MLR Institute of Technology

MLR Institute of Technology MLR Institute of Technology DUNDIGAL 500 043, HYDERABAD COMPUTER SCIENCE AND ENGINEERING Computer Programming Lab List of Experiments S.No. Program Category List of Programs 1 Operators a) Write a C program

More information

Effective Data Mining Using Neural Networks

Effective Data Mining Using Neural Networks IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 8, NO. 6, DECEMBER 1996 957 Effective Data Mining Using Neural Networks Hongjun Lu, Member, IEEE Computer Society, Rudy Setiono, and Huan Liu,

More information

1 Scope and Motivation. Revised 2/4/2015 9:31 AM

1 Scope and Motivation. Revised 2/4/2015 9:31 AM N4355 : Shared Multidimensional Array with Polymorphic Layout Authors: Carter Edwards hcedwar@sandia.gov Christian Trott crtrott@sandia.gov Related papers: N4177 Multidimensional bounds, index and array_view,

More information

What s New in MATLAB and Simulink

What s New in MATLAB and Simulink What s New in MATLAB and Simulink Kevin Cohan Product Marketing, MATLAB Michael Carone Product Marketing, Simulink 2015 The MathWorks, Inc. 1 What was new for Simulink in R2012b? 2 What Was New for MATLAB

More information

RAM & ROM Based Digital Design. ECE 152A Winter 2012

RAM & ROM Based Digital Design. ECE 152A Winter 2012 RAM & ROM Based Digital Design ECE 152A Winter 212 Reading Assignment Brown and Vranesic 1 Digital System Design 1.1 Building Block Circuits 1.1.3 Static Random Access Memory (SRAM) 1.1.4 SRAM Blocks in

More information

New Hash Function Construction for Textual and Geometric Data Retrieval

New Hash Function Construction for Textual and Geometric Data Retrieval Latest Trends on Computers, Vol., pp.483-489, ISBN 978-96-474-3-4, ISSN 79-45, CSCC conference, Corfu, Greece, New Hash Function Construction for Textual and Geometric Data Retrieval Václav Skala, Jan

More information

Lossless Data Compression Standard Applications and the MapReduce Web Computing Framework

Lossless Data Compression Standard Applications and the MapReduce Web Computing Framework Lossless Data Compression Standard Applications and the MapReduce Web Computing Framework Sergio De Agostino Computer Science Department Sapienza University of Rome Internet as a Distributed System Modern

More information

Distributed Data Stores

Distributed Data Stores Distributed Data Stores 1 Distributed Persistent State MapReduce addresses distributed processing of aggregation-based queries Persistent state across a large number of machines? Distributed DBMS High

More information

Confinement Problem. The confinement problem Isolating entities. Example Problem. Server balances bank accounts for clients Server security issues:

Confinement Problem. The confinement problem Isolating entities. Example Problem. Server balances bank accounts for clients Server security issues: Confinement Problem The confinement problem Isolating entities Virtual machines Sandboxes Covert channels Mitigation 1 Example Problem Server balances bank accounts for clients Server security issues:

More information

HBase Schema Design. NoSQL Ma4ers, Cologne, April 2013. Lars George Director EMEA Services

HBase Schema Design. NoSQL Ma4ers, Cologne, April 2013. Lars George Director EMEA Services HBase Schema Design NoSQL Ma4ers, Cologne, April 2013 Lars George Director EMEA Services About Me Director EMEA Services @ Cloudera ConsulFng on Hadoop projects (everywhere) Apache Commi4er HBase and Whirr

More information

A Mathematical Programming Solution to the Mars Express Memory Dumping Problem

A Mathematical Programming Solution to the Mars Express Memory Dumping Problem A Mathematical Programming Solution to the Mars Express Memory Dumping Problem Giovanni Righini and Emanuele Tresoldi Dipartimento di Tecnologie dell Informazione Università degli Studi di Milano Via Bramante

More information

Big Data Storage Options for Hadoop Sam Fineberg, HP Storage

Big Data Storage Options for Hadoop Sam Fineberg, HP Storage Sam Fineberg, HP Storage SNIA Legal Notice The material contained in this tutorial is copyrighted by the SNIA unless otherwise noted. Member companies and individual members may use this material in presentations

More information

Scalable Data Analysis in R. Lee E. Edlefsen Chief Scientist UserR! 2011

Scalable Data Analysis in R. Lee E. Edlefsen Chief Scientist UserR! 2011 Scalable Data Analysis in R Lee E. Edlefsen Chief Scientist UserR! 2011 1 Introduction Our ability to collect and store data has rapidly been outpacing our ability to analyze it We need scalable data analysis

More information

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are

More information

Scalable Cloud Computing Solutions for Next Generation Sequencing Data

Scalable Cloud Computing Solutions for Next Generation Sequencing Data Scalable Cloud Computing Solutions for Next Generation Sequencing Data Matti Niemenmaa 1, Aleksi Kallio 2, André Schumacher 1, Petri Klemelä 2, Eija Korpelainen 2, and Keijo Heljanko 1 1 Department of

More information

BigData. An Overview of Several Approaches. David Mera 16/12/2013. Masaryk University Brno, Czech Republic

BigData. An Overview of Several Approaches. David Mera 16/12/2013. Masaryk University Brno, Czech Republic BigData An Overview of Several Approaches David Mera Masaryk University Brno, Czech Republic 16/12/2013 Table of Contents 1 Introduction 2 Terminology 3 Approaches focused on batch data processing MapReduce-Hadoop

More information

Parallel Computing for Data Science

Parallel Computing for Data Science Parallel Computing for Data Science With Examples in R, C++ and CUDA Norman Matloff University of California, Davis USA (g) CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an imprint

More information

MODELING RANDOMNESS IN NETWORK TRAFFIC

MODELING RANDOMNESS IN NETWORK TRAFFIC MODELING RANDOMNESS IN NETWORK TRAFFIC - LAVANYA JOSE, INDEPENDENT WORK FALL 11 ADVISED BY PROF. MOSES CHARIKAR ABSTRACT. Sketches are randomized data structures that allow one to record properties of

More information

Quantum Computing Lecture 7. Quantum Factoring. Anuj Dawar

Quantum Computing Lecture 7. Quantum Factoring. Anuj Dawar Quantum Computing Lecture 7 Quantum Factoring Anuj Dawar Quantum Factoring A polynomial time quantum algorithm for factoring numbers was published by Peter Shor in 1994. polynomial time here means that

More information

A Comparison of Five Methods for Signal Intensity Standardization in MRI

A Comparison of Five Methods for Signal Intensity Standardization in MRI A Comparison of Five Methods for Signal Intensity Standardization in MRI Jan-Philip Bergeest, Florian Jäger Lehrstuhl für Mustererkennung, Friedrich-Alexander-Universität Erlangen-Nürnberg jan.p.bergeest@informatik.stud.uni-erlangen.de

More information

PART-A Questions. 2. How does an enumerated statement differ from a typedef statement?

PART-A Questions. 2. How does an enumerated statement differ from a typedef statement? 1. Distinguish & and && operators. PART-A Questions 2. How does an enumerated statement differ from a typedef statement? 3. What are the various members of a class? 4. Who can access the protected members

More information

Architectures for massive data management

Architectures for massive data management Architectures for massive data management Apache Kafka, Samza, Storm Albert Bifet albert.bifet@telecom-paristech.fr October 20, 2015 Stream Engine Motivation Digital Universe EMC Digital Universe with

More information

Large-Scale Test Mining

Large-Scale Test Mining Large-Scale Test Mining SIAM Conference on Data Mining Text Mining 2010 Alan Ratner Northrop Grumman Information Systems NORTHROP GRUMMAN PRIVATE / PROPRIETARY LEVEL I Aim Identify topic and language/script/coding

More information

Dušan Bernát (bernat@fiit.stuba.sk)

Dušan Bernát (bernat@fiit.stuba.sk) Domain Name System as a Memory and Communication Medium Dušan Bernát (bernat@fiit.stuba.sk) Institute of Computer Systems and Networks, Faculty of Informatics and Information Technology, STU Bratislava,

More information

Spark ΕΡΓΑΣΤΗΡΙΟ 10. Prepared by George Nikolaides 4/19/2015 1

Spark ΕΡΓΑΣΤΗΡΙΟ 10. Prepared by George Nikolaides 4/19/2015 1 Spark ΕΡΓΑΣΤΗΡΙΟ 10 Prepared by George Nikolaides 4/19/2015 1 Introduction to Apache Spark Another cluster computing framework Developed in the AMPLab at UC Berkeley Started in 2009 Open-sourced in 2010

More information

Lecture 2: Universality

Lecture 2: Universality CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal

More information

Big Data with Component Based Software

Big Data with Component Based Software Big Data with Component Based Software Who am I Erik who? Erik Forsberg Linköping University, 1998-2003. Computer Science programme + lot's of time at Lysator ACS At Opera Software

More information

Cours de C++ Utilisations des conteneurs

Cours de C++ Utilisations des conteneurs Cours de C++ Utilisations des conteneurs Cécile Braunstein cecile.braunstein@lip6.fr 1 / 18 Introduction Containers - Why? Help to solve messy problems Provide useful function and data structure Consistency

More information

Efficient Fault-Tolerant Infrastructure for Cloud Computing

Efficient Fault-Tolerant Infrastructure for Cloud Computing Efficient Fault-Tolerant Infrastructure for Cloud Computing Xueyuan Su Candidate for Ph.D. in Computer Science, Yale University December 2013 Committee Michael J. Fischer (advisor) Dana Angluin James Aspnes

More information

Facebook: Cassandra. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation

Facebook: Cassandra. Smruti R. Sarangi. Department of Computer Science Indian Institute of Technology New Delhi, India. Overview Design Evaluation Facebook: Cassandra Smruti R. Sarangi Department of Computer Science Indian Institute of Technology New Delhi, India Smruti R. Sarangi Leader Election 1/24 Outline 1 2 3 Smruti R. Sarangi Leader Election

More information

Project DALKIT (informal working title)

Project DALKIT (informal working title) Project DALKIT (informal working title) Michael Axtmann, Timo Bingmann, Peter Sanders, Sebastian Schlag, and Students 2015-03-27 INSTITUTE OF THEORETICAL INFORMATICS ALGORITHMICS KIT University of the

More information

Hadoop Fair Scheduler Design Document

Hadoop Fair Scheduler Design Document Hadoop Fair Scheduler Design Document October 18, 2010 Contents 1 Introduction 2 2 Fair Scheduler Goals 2 3 Scheduler Features 2 3.1 Pools........................................ 2 3.2 Minimum Shares.................................

More information

Review of Hashing: Integer Keys

Review of Hashing: Integer Keys CSE 326 Lecture 13: Much ado about Hashing Today s munchies to munch on: Review of Hashing Collision Resolution by: Separate Chaining Open Addressing $ Linear/Quadratic Probing $ Double Hashing Rehashing

More information

Variable Base Interface

Variable Base Interface Chapter 6 Variable Base Interface 6.1 Introduction Finite element codes has been changed a lot during the evolution of the Finite Element Method, In its early times, finite element applications were developed

More information

Boolean Network Models

Boolean Network Models Boolean Network Models 2/5/03 History Kaufmann, 1970s Studied organization and dynamics properties of (N,k) Boolean Networks Found out that highly connected networks behave differently than lowly connected

More information

Outline. Database Management and Tuning. Overview. Hardware Tuning. Johann Gamper. Unit 12

Outline. Database Management and Tuning. Overview. Hardware Tuning. Johann Gamper. Unit 12 Outline Database Management and Tuning Hardware Tuning Johann Gamper 1 Free University of Bozen-Bolzano Faculty of Computer Science IDSE Unit 12 2 3 Conclusion Acknowledgements: The slides are provided

More information

C++ Programming Language

C++ Programming Language C++ Programming Language Lecturer: Yuri Nefedov 7th and 8th semesters Lectures: 34 hours (7th semester); 32 hours (8th semester). Seminars: 34 hours (7th semester); 32 hours (8th semester). Course abstract

More information

RAID Technology Overview

RAID Technology Overview RAID Technology Overview HP Smart Array RAID Controllers HP Part Number: J6369-90050 Published: September 2007 Edition: 1 Copyright 2007 Hewlett-Packard Development Company L.P. Legal Notices Copyright

More information

Support Vector Machines with Clustering for Training with Very Large Datasets

Support Vector Machines with Clustering for Training with Very Large Datasets Support Vector Machines with Clustering for Training with Very Large Datasets Theodoros Evgeniou Technology Management INSEAD Bd de Constance, Fontainebleau 77300, France theodoros.evgeniou@insead.fr Massimiliano

More information

Leonardo Aniello aniello@dis.uniroma1.it

Leonardo Aniello aniello@dis.uniroma1.it SDCI 2012 HOT TOPICS IN SECURE AND DEPENDABLE COMPUTING FOR CRITICAL INFRASTRUCTURES JANUARY 15 TH 19 TH, CORTINA D AMPEZZO, ITALY COLLABORATIVE EVENT PROCESSING FOR THE PROTECTION OF CRITICAL INFRASTRUCTURES

More information

A Partition-Based Efficient Algorithm for Large Scale. Multiple-Strings Matching

A Partition-Based Efficient Algorithm for Large Scale. Multiple-Strings Matching A Partition-Based Efficient Algorithm for Large Scale Multiple-Strings Matching Ping Liu Jianlong Tan, Yanbing Liu Software Division, Institute of Computing Technology, Chinese Academy of Sciences, Beijing,

More information

Longest Common Extensions via Fingerprinting

Longest Common Extensions via Fingerprinting Longest Common Extensions via Fingerprinting Philip Bille Inge Li Gørtz Jesper Kristensen Technical University of Denmark DTU Informatics LATA, March 9, 2012 1 / 17 Contents Introduction The LCE Problem

More information

StreamStorage: High-throughput and Scalable Storage Technology for Streaming Data

StreamStorage: High-throughput and Scalable Storage Technology for Streaming Data : High-throughput and Scalable Storage Technology for Streaming Data Munenori Maeda Toshihiro Ozawa Real-time analytical processing (RTAP) of vast amounts of time-series data from sensors, server logs,

More information

C programming: exercise sheet L2-STUE (2011-2012)

C programming: exercise sheet L2-STUE (2011-2012) C programming: exercise sheet L2-STUE (2011-2012) Algorithms and Flowcharts Exercise 1: comparison Write the flowchart and associated algorithm that compare two numbers a and b. Exercise 2: 2 nd order

More information

Lecture 03 Bits, Bytes and Data Types

Lecture 03 Bits, Bytes and Data Types Lecture 03 Bits, Bytes and Data Types In this lecture Computer Languages Assembly Language The compiler Operating system Data and program instructions Bits, Bytes and Data Types ASCII table Data Types

More information

Efficient LDPC Code Based Secret Sharing Schemes and Private Data Storage in Cloud without Encryption

Efficient LDPC Code Based Secret Sharing Schemes and Private Data Storage in Cloud without Encryption Efficient LDPC Code Based Secret Sharing Schemes and Private Data Storage in Cloud without Encryption Yongge Wang Department of SIS, UNC Charlotte, USA yonwang@uncc.edu Abstract. LDPC codes, LT codes,

More information

The Online Set Cover Problem

The Online Set Cover Problem The Online Set Cover Problem Noga Alon Baruch Awerbuch Yossi Azar Niv Buchbinder Joseph Seffi Naor ABSTRACT Let X = {, 2,..., n} be a ground set of n elements, and let S be a family of subsets of X, S

More information

Data Storage - II: Efficient Usage & Errors

Data Storage - II: Efficient Usage & Errors Data Storage - II: Efficient Usage & Errors Week 10, Spring 2005 Updated by M. Naci Akkøk, 27.02.2004, 03.03.2005 based upon slides by Pål Halvorsen, 12.3.2002. Contains slides from: Hector Garcia-Molina

More information

22S:295 Seminar in Applied Statistics High Performance Computing in Statistics

22S:295 Seminar in Applied Statistics High Performance Computing in Statistics 22S:295 Seminar in Applied Statistics High Performance Computing in Statistics Luke Tierney Department of Statistics & Actuarial Science University of Iowa August 30, 2007 Luke Tierney (U. of Iowa) HPC

More information

Business. Control Administration. Alessandro Colantonio. Bay31 GmbH, Switzerland. Roberto Di Pietro. Universita di Roma Tre, Italy.

Business. Control Administration. Alessandro Colantonio. Bay31 GmbH, Switzerland. Roberto Di Pietro. Universita di Roma Tre, Italy. BEIJING SHANGHAI HONG TAIPEI Role Mining in Business Taming Role-Based Access Control Administration Alessandro Colantonio Bay31 GmbH, Switzerland Roberto Di Pietro Universita di Roma Tre, Italy Alberto

More information

Find-The-Number. 1 Find-The-Number With Comps

Find-The-Number. 1 Find-The-Number With Comps Find-The-Number 1 Find-The-Number With Comps Consider the following two-person game, which we call Find-The-Number with Comps. Player A (for answerer) has a number x between 1 and 1000. Player Q (for questioner)

More information

A Tool for Generating Partition Schedules of Multiprocessor Systems

A Tool for Generating Partition Schedules of Multiprocessor Systems A Tool for Generating Partition Schedules of Multiprocessor Systems Hans-Joachim Goltz and Norbert Pieth Fraunhofer FIRST, Berlin, Germany {hans-joachim.goltz,nobert.pieth}@first.fraunhofer.de Abstract.

More information

Contents. SnapComms Data Protection Recommendations

Contents. SnapComms Data Protection Recommendations Contents Abstract... 2 SnapComms Solution Environment... 2 Concepts... 3 What to Protect... 3 Database Failure Scenarios... 3 Physical Infrastructure Failures... 3 Logical Data Failures... 3 Service Recovery

More information

Discrete Optimization

Discrete Optimization Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

More information

Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case.

Efficiency of algorithms. Algorithms. Efficiency of algorithms. Binary search and linear search. Best, worst and average case. Algorithms Efficiency of algorithms Computational resources: time and space Best, worst and average case performance How to compare algorithms: machine-independent measure of efficiency Growth rate Complexity

More information

Deploy App Orchestration 2.6 for High Availability and Disaster Recovery

Deploy App Orchestration 2.6 for High Availability and Disaster Recovery Deploy App Orchestration 2.6 for High Availability and Disaster Recovery Qiang Xu, Cloud Services Nanjing Team Last Updated: Mar 24, 2015 Contents Introduction... 2 Process Overview... 3 Before you begin...

More information

A Performance Comparison of Five Algorithms for Graph Isomorphism

A Performance Comparison of Five Algorithms for Graph Isomorphism A Performance Comparison of Five Algorithms for Graph Isomorphism P. Foggia, C.Sansone, M. Vento Dipartimento di Informatica e Sistemistica Via Claudio, 21 - I 80125 - Napoli, Italy {foggiapa, carlosan,

More information

1 Abstract Data Types Information Hiding

1 Abstract Data Types Information Hiding 1 1 Abstract Data Types Information Hiding 1.1 Data Types Data types are an integral part of every programming language. ANSI-C has int, double and char to name just a few. Programmers are rarely content

More information

Hadoop Architecture. Part 1

Hadoop Architecture. Part 1 Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,

More information

- Behind The Cloud -

- Behind The Cloud - - Behind The Cloud - Infrastructure and Technologies used for Cloud Computing Alexander Huemer, 0025380 Johann Taferl, 0320039 Florian Landolt, 0420673 Seminar aus Informatik, University of Salzburg Overview

More information