Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities Tiffani Crawford, PhD
Big Analytics Characteristics Large quantities of many data types Structured Unstructured Human Machine 2 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Valuable Insights Getting to Value Quantitative Qualitative Correlation Longitudinal Social Search Operational Inferential 3 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Valuable Insights x/2 * 5 (x + y)/y + <10 sentiment blogs Getting to Value Quantitative Qualitative Correlation Longitudinal Social Search Operational Inferential Points * 3 if over 18 years old 3 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Proven in Industry Satellites and other geospatial systems Image processing and analysis Research & Development High technology and Internet Social media Telecommunications Knowledge bases and content systems Life sciences, health care and genetics Infrastructure systems and monitoring Operations and transportation Financial systems and compliance controls 4 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Key Benefits High speed data processing 1 big job => many small jobs (reduce complexity) Redundancies prevent data loss Temporary storage is faster Long term storage is easier to access High speed query and results processing (<10 sec) High speed predictive analytics & responsive models Low cost (2 10x cheaper) up to massive scale Easy to start small and practice before large builds Technology agnostic Open source, cloud or large supplier install base Proof of Concept, Proof of Value, Proof of Technology Early analytics modeling, Proof of Model 5 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Technology-Agnostic Industry-specific Big Solutions Execution Components Business Context Information Delivery Analytics & Insights Processing Service Offerings Big Dimensions Integration & Management Consulting Solution Engineering Governance Research & Development Volume Variety Complexity Velocity Space Science Space Exploration Space Entrepreneurship Space Policy 6 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics 4 Pillars Hadoop NoSQL Hybrid Systems Science 7 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Hadoop Map/Reduce Master Interconnect s Name Job Secondary (Passive) Name Job 8 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Hadoop Ecosystem HDFS Primary storage system used by Hadoop in a distributed environment Map/Reduce Cassandra Software framework for Clustered, Distributed data processing Highly scalable database which brings fully distributed design &Bigtable sdata model ZooKeeper Avro Chukwa HBase Hive HBASE Workflow Management Service Serialization Collection System to monitor Distributed Systems storage for distributed large tables warehouse For random, real time read/write access to Big Pig Scribe Sqoop High Level Query Language Platform for analyzing huge data sets which involves complex rules Log Collection Open source integration tool to integrate data into HDFS from sources outside of Hadoop platform 9 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Who uses Hadoop? 10 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics NoSQL NOT relational algebra (RDBMS tables/relationships) SQL is not the data manipulation language Google Bigtable 11 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Hybrid Systems Business Intelligence/ Visualization Analytics Appliance Integration Sources 12 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Science Making sense of data is both Art and Science Generate better insights Gain confidence in decisions Visualize the data Understand the data and communicate it Learn how to learn and adapt with agility Methods mining Machine learning Artificial intelligence Information retrieval Statistical analysis Gap analysis 13 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Visualization Context Start Date End Date Competitors Competitor1 Competitor2 Competitor3 Competitor4 Competitor5 14 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Visualization 15 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Visualization 16 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Big Analytics Visualization 17 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Space Entrepreneurship Opportunities Space exploration and environment data management Space systems operational data management Space investment data validation (mining studies, technology) Economic development analytics and decision making query results systems with fast processing power Social media, people interest matching, innovation discussions Genetics management for selection and health care 18 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Space Policy Opportunities & Implications Space science and business value delivery Research process automation, analysis and discovery Intellectual property Global, always on data availability Reduced cost and rework Collaborative leadership Learning opportunities Rewards and fairness (vertical and horizontal equity) Policy modeling before human cases (Kant Rawls) ubiquity, security and privacy 19 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Trends and Future for Big Analytics Innovate and adapt Streamline, automate and consolidate Discover, extend and renew 20 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
How to Adapt, Streamline and Grow? New data types and workflows Cluster management and node forecasting Automated deployment, network and application process monitoring Replay, backup, recovery and failover Cloud Social media and mobile integration SDN (software discovers needs and provisions the system) 21 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
How to Frame Our Thinking? What are we trying to accomplish? How is this valuable? What data questions do we have now? What questions may follow? Can the opportunity or problem benefit from Big Analytics? Does the solution have growing room? 22 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Use Cases Near-Term Business Value Mission to Mars system automation Robotics Life support Space Debris Image analysis Trajectory change alerts Satellite down alerts Material analysis Investment validation Space Manufacturing Process optimization Intelligent automation 23 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Use Cases Exploration and Settlement Locations and Objects Monitoring orbits Monitoring Lagrange Points Image analysis Kepler analysis 200 years for asteroids? No! Transportation Trajectory and path Fuel, power and life support Acquisitions en route High velocity options Communications Health impacts of space 24 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Use Cases Operationalizing Space Space data centers Unlimited storage Processing power Adaptive software Automation Space systems Intelligent controls Monitoring Maintenance Resource allocation Planning Research on demand Predictive analysis 25 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Use Cases Life Sciences & Health Care Genetics Genome analysis Health prediction Selection of pairs Tailoring for environments Biological Adaptation for Space Investigation of immunities Uses of bacteria Pre terraforming studies Find reality in futurist ideas Health Improvement Capabilities of mind Extended life 26 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Use Case Social Media and Sentiment Get a Big Space Program Up! Funding Investment Support Entrepreneurship Prove the Value of Space! Validate social models Design alternative living Explore compatibility for space Collaborate around the Earth Adapt policies before needing them Advance human learning 27 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Hands-On Exercise Step 1 Create a repeatable model for Big Analytics for a spaceship, colony or astrosettlement in a focus area that inspires you and your team. What information systems are involved? What data is collected? Is more needed? What analytics must be real time? Where can data be processed and analyzed? How can data and analyses be shared? What is the business value of your model over time? Step 2 Publicize your model online and ask for review. Step 3 Continue to discuss and develop your model and help review other models online. 28 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Hands-On Exercise Focus Areas Life support systems Air and water factories Farms and livestock Space operations and transportation Employment and Recruitment Education (preschool through adult) Human workforce, family, elderly and genetics planning Economic development and small business Community membership and law enforcement Community arts, entertainment and amenities People and social life Travel destination amenities and marketing Research projects Telescopes and probes Mining operations Terraforming operations Extraterrestrial investments Trade and treaty compliance New species discovery Non human adaptive language and distance communications 29 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities
Contact Information Tiffani Crawford, PhD 408 829 7096 zapquick@aol.com Tiffani Crawford, PhD, builds global Big Analytics systems. She has 20 years of high technology experience with Fortune 500 companies, including Cisco Systems, Cognizant Technology, Bank of America, VISA/Inovant, BAE Systems, Applied Competitive Technologies, Nuance/Ditech Networks, Big 4 financial firms, defense contractors and startups. She has worked in seminal technology development in Big, analytics, cloud, networking, telecommunications, software development, distributed multi tier applications, multimedia/digital, geographic information systems, intelligent transport systems, finance, security, policy systems and structural equation modeling. She is a credited software developer and published author. She earned her PhD from the University of Southern California in 2005. She has also earned her Master's, Bachelor's and various technology certifications. She is a member of NSS with various philanthropic contributions. 30 Big Analytics for Space Exploration, Entrepreneurship and Policy Opportunities