Improving Analytical Tools The Future of Interrogating Data London, October

Transcription

1 Improving Analytical Tools The Future of Interrogating Data London, October

2 OrbisIP Technology Services Consultancy Technology scouting and evaluation Product Distribution Technology Readiness Level enhancements Secure Software Development Horizon scanning & Technology watching Open innovation services

3 Security Technology Clusters Digital Forensics Biometrics Video Analytics Data Mining & Visualisation Cryptography Network Security Internet Security Secure Software Security Management & Architecture Data & Database Security Hardware, Embedded & Device Security Homeland Security Over 150 items of novel security technology exist in the OrbisIP Technology Tracker, available online

4 The OrbisIP Model Sources of IP Consumers of IP University Research Labs National Research Labs IP Requirements Tech. Transfer Advisory Board Technology Advisers IP Requirements IP Licensing Security Companies Primes Governments SMEs Revenue Share Operations Payment / Royalties

5 SITC conference challenge "... During the investigations into the London bombings in 2005, 90,000 hard drives and video tapes from CCTV systems were seized, together with 100 computers, 500,000 pages of photocopying, 4000 exhibits, 70 telephones and 10,000 statements a responsibility to capture, consolidate and interrogate massive volumes of structured and unstructured data. How do you make sense out of all the data coming in and then use it to make a difference? "... technologies which can assist users in making informed, strategic decisions."

6 The challenges of large scale data analysis Data tracking, capture, management and isolation Retrieval and analysis of large data sets Analytical impartiality Data interrogation Decision support Actionable intelligence Reportage and distribution

7 Helping to address the challenges Intercept Modernisation Programme IP data capture and interrogation Network Traffic Surveillance System High speed data retrieval and management Clusterpoint XML data base Decision Support and tracking technology SheBa Structured and Unstructured Data Analysis Leximancer

8 Intercept Modernisation Programme Every , phone call and website visit is to be recorded and stored after the Coalition Government revived controversial Big Brother snooping plans. The Telegraph 20 Oct 2010 "We will introduce a programme to preserve the ability of the security, intelligence and law enforcement agencies to obtain communication data and to intercept communications within the appropriate legal framework Communications data provides evidence in court to secure convictions of those engaged in activities that cause serious harm. It has played a role in every major Security Service counter terrorism operation and in 95 per cent of all serious organised crime investigations. UK Government Strategic Defence and Security Review October 2010

9 Intercept Modernisation Programme Technical challenges: Mass capture and storage of data by ISPs Gathering intelligence from data sets Interrogate to level of all packets generated at IP address Produce auditable interactions with the data set that conform to existing legislation and can be submitted in court to support prosecution

10 Intercept Modernisation Programme Technical challenges: Mass capture and storage of data by ISPs Gathering intelligence from data sets Interrogate to level of all packets generated at IP address Produce auditable interactions with the data set that conform to existing legislation and can be submitted in court to support prosecution Network Traffic Surveillance System (NTSS) - TRL: 9

11 NTSS collects data on ALL user network activities 1. All TCP & UDP traffic IP packets between customer and Internet get forwarded to NTSS Internet Traffic IP packets NTSS 2. IP packets are reengineered back to application level information units (web pages viewed, s sent, documents transferred). Decoder 3. All reengineered and analysed information is fully indexed and stored in Clusterpoint Server database. 4. Easy to use WEB interface provides necessary tools, to: get a quick situation overviews, search through the collected data, receive alerts on user defined criteria, follow up on network user activities, preview the reconstructed information. Entirely searchable & scalable NTSS database

12 High speed data retrieval and management The issues: Size of data base and scalability Retrieval or interrogation time The solution Clusterpoint DBMS TRL 9 create a fully scalable and fast response time XML based tagged database easily and lineary scalable no additional development required to scale the storage and necessary processing power can improve data retrieval times in such unstructured data storages 100 fold and more with response times of sub 5 seconds in multi terabyte databases

13 Clusterpoint DBMS Architecture AUTHORIZED USERS SECURITY APPLICATION SERVICES Clusterpoint API XML Entirely searchable & scalable NTSS database Existing databases SECURITY AUDITING AND MONITORING DATA CLUSTERPOINT NTS ( MULTI-SERVER CLUSTER )

14 Objective & Subjective Decision support The Challenges: Capture, manage and analyse information on various topics of relevance, including multiple items of information supplied at different times, and from different sources; Consider factors that degrade the certainty of the information, such as the length of time that has passed between its collection and its use; Represent the reliability or credibility of information and its source, including its provenance, and the source s objectivity, access, and specificity. Combine multiple sources of information with varying levels of reliability, and whose reliability may change over time;

15 Objective & Subjective Decision support The Solution: Sheba Estimative Intelligence Tool TRL 5 An application for performing predictive analysis under conditions of uncertainty a framework for users to structure and analyse estimative intelligence problems uses advanced probability theory to manage both likelihoods and certainty for problems in estimative intelligence transparent analytical structure necessary to allow the analyst to defend, and the consumer to understand, the judgments reached

16 Structured & Unstructured Data Analysis The challenge: How do you extract fully automated meaningful intelligence from vast scalable data sets of structured and unstructured data? Generating concept maps and an Automatic Sentiment Lens Avoiding onerous set up overhead, user manipulation or prejudices or priori rule sets? Providing multilingual analysis and integration into other applications through an API Offering full flexibility for analysis of outcomes and exportability into reporting platforms

17 Structured & Unstructured Data Analysis The solution: Leximancer TRL 9 Some applications: Intelligence Profiling: rapid information gathering, correlation, validation and analysis Analysis & Security: validating & predicting security classification of i Library Indexing & Search: indexing, search and retrieval within large databases/libraries Tendering: matching tender responses to questions coverage analysis Web searching and analysis: via Hypermancer

18 Leximancer Conversation Analysis As long as speaker labels are formatted correctly, Leximancer will automatically extract the speaker identifiers as variables, and associate these labels with their utterances. This allows content from selected speakers to be filtered in or out, and allows comparative analysis between speakers, normally using the discovered concepts as independent variables. Conversation analysis can be extended to incorporate: Blogging/forums etc. on the internet brand/product analysis etc. dialogues litigation ediscovery Voice to text translations call centre dialogues, meetings, scenario training/simulations

19 OrbisIP Contact Details Peter Jaco OrbisIP Limited 9 10 St. Andrew Square Edinburgh, EH2 2AF United Kingdom Tel: +44 (0) Fax: +44 (0) Cell: +44 (0)