Digital Earth: Big Data, Heritage and Social Science

Transcription

1 Digital Earth: Big Data, Heritage and Social Science The impact on geographic information and GIS

2 Geographic Information Systems Analysis for Decision Support Impact of Big Data Digital Earth Citizen Engagement Analysis for Decision Support

3 Traditional GIS (self contained) imagery maps town plans cadastral data networks retrieval and processing software all data stored locally decision support product at agency level It is a decision support tool, in that it manipulates and presents data in a way that allows the user to form inferences or make decisions that otherwise would not be possible or as easy. Humans(resource managers) are still involved.

4 There has been a move towards web-serving data supplier 1 data supplier 2 data supplier 3 original GIS -all data stored locally problems with data duplication and maintenance data user 1 data user 2 value added products data maintenance handled by principal suppliers data access almost unlimited

5 Cloud served GIS has now been proposed (ESRI) in which both the data and processing software is on remote servers data supplier 1 data supplier 2 data supplier 3 GIS software data user 2 data user 1 value added products

6 The Geographic Information System The GIS model is variable different people have different needs and concepts What has been constant is the underlying spatial data reference such as a map grid. Will the planar grid in the GIS be replaced by the virtual globe? Explore the digital earth concept.

7 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science Digital Earth The concept is well established; it originated in 1998 and 1999 attributes addressed as on a globe Mercator globe from space Google Earth like A. Gore, The Digital Earth: Understanding our Planet in the 21 st Century, PERS, vol. 65, no 5., 1999, p528.

8 Digital Earth Because it has a virtual globe basis it is immediately accessible by anyone with a reasonable home computer. It is GIS-like because of the data layers added by other users and researchers. heritage, cultural, and social layers physical layers complex analysis for decision support The citizen scientist, although possibly untrained in the formal sense, may have much to add even simple facts, in aggregate over many people, can add important information to the data base.

9 What can we learn from the Internet about the role of the citizen, often an untrained amateur? My first suspicion of something born so instantaneously perfect is an alien origin. There was probably a meeting on a distant planet where they debated whether or not it was time for the earth to have the web. They ll only botch it up, the opponents probably argued. They ll put all kinds of garbage on it and ruin our good idea. R.W. Lucky, Bellcore, 1995

10 But the alien proponents and supporters of the Web carried the day. The earth has all sorts of talented people eager to get their material out to the public they undoubtedly said. Wonderful things will happen, just wait and see. R.W. Lucky, Bellcore, 1995

11 The Web is proving the efficacy of the long believed and hoped for Field of Dreams approach if we build it they will come. The people of the earth have been empowered by the Web and there has been an incredible outpouring of creativity When you empower a few, its not much competition. But when you empower tens of millions, surprising things will happen, some of them breathtakingly good R.W. Lucky, Bellcore, 1995

12 Citizen user interaction The internet and social networks give the citizen user ready and immediate access to data and results, give the citizen user a simple means for communicating, and allow the citizen user to communicatetheir own views to other citizens and to professionals alike.

13 We have seen an evolution from remote sensing, through GIS, to digital earth, and citizen science. remote sensing remote sensing satellites remote sensing aircraft UAVs navigation satellites GIS existing maps other spatial data sensor networks ground based sensors digital earth citizens as sensors social, cultural data

14 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science So, what sort of data future are we facing? Physical data (optical spectral, radar, thermal, lidar, ) from large satellites aircraft and UAVs satellite clusters ground based sensors Cultural, heritage and other forms of social data Citizen data over social networks that may be of questionable accuracy and precision ephemeral convergent over many citizens (crowd sourcing)

15 Thus, the emerging data environment is now like social media input heritage, cultural, and social layers physical layers analysis earth observation satellite and ground sensor inputs (the sensor web) clever servers, broadband internet social media data the challenge is to analyse this mixed data to provide meaningful decision support to the resource manager

16 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science Decision support in the digital earth framework TWO OBSERVATIONS 1. Physical sensors generate datathat is analysedto provide information. That information allows the human analyst to acquire knowledge about the problem domain. 2. Human (citizen) input to the digital earth model is generally in the form of knowledge(or facts) already.

17 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science Thus, knowledge propagation in the digital earth framework might look like: the GIS analysis problem sensor data analysis information human assimilation usable knowledge citizen input integrity checking, etc. usable knowledge knowledge fusion social network analysis

18 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science How are facts processed into knowledge? data type 1 (e.g. spectral) analytical process 1 labels data type 2 (e.g. radar) analytical process 2 labels knowledge processor (several candidates are available) product This is what we can do in GIS

19 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science How are facts processed into knowledge? data type 1 (e.g. spectral) analytical process 1 facts data type 2 (e.g. radar) cultural information analytical process 2 facts facts knowledge processor product citizen information facts can add to the digital earth archive

20 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science When does social network knowledge become useful? citizens C 1 C 2 C n social convergence and error correction useful knowledge (set of facts) citizen observations process could be deterministic: e.g. through issuing a call for help opportunistic: e.g. by using tourist photographs spontaneous: e.g. unsolicited input there is a whole body of research on social media, when messages stick and go viral, and what the major influences are this is still an open research topic, at the intersection of computer and social science

21 Now explore the spatial data domain a bit further, because the changing nature of data supply may impact on the way that GIS evolves. How much data are we now talking about? or will be soon!

22 kilobyte kb megabyte MB gigabyte GB terabyte TB petabyte PB exabyte EB zettabyte ZB estimated storage capacity of the human brain ~ 2.5PB October 2012 Internet Archive ~ 10PB (190 TB per month)

23 (courtesy of GuoHuadong, Director, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences)

24 (courtesy of GuoHuadong, Director, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences)

25 The digital universe is a measure of all the data created, replicated and consumed in a single year 1,000,000,000GB Source: IDC's Digital Universe Study Executive Summary, December 2012 (courtesy of GuoHuadong, Director, Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences)

26 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science In summary, what is the overall impact on GIS? apartfrom the virtual globe as a map base, the impact is largely through the data available, its increasing volume, and the fact that some data is ephemeral, and sometimes of questionable accuracy, particularly that delivered over social networks imagery maps town plans cadastral data networks citizen generated data retrieval and processing software decision support product how can we process this to provide more meaningful decision support, particularly in the face of a massive increase in data volume?

27 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science Look at technological considerations Big data implies, among other likely considerations, Large storage High processing power Effective knowledge extraction

28 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science STORAGE OF BIG DATA 4.00 Digital Universe absolute scales are relative; it is the trends that are important log scale Hard drive capacity growth in memory capacity is greater than the growth of the digital universe storage capacity will not be a problem but access and smart archiving may be Based on: en.wikipedia.org/wiki/file:hard_drive_capacity_over_time.svgand J. Gantzand D. Reinsel, The Digital Universe in 2020: Big Data, Bigger Digital Shadows, and Biggest Growth in the Far East, IDC IVIEW, December 2012

29 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science price USD/MB COST OF STORAGE the biggest technology impact on society will be the falling cost of computer memory price cents/mb $10/TB L. Free, Sydney, late 1970s Based on: hblok.net/blog/posts/2013/02/13/historical-cost-of-computer-memory-and-storage/

30 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science EVOLUTION OF COMPUTING POWER after Hans Moravec, Carnegie Mellon See

31 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science kilobyte megabyte kb Big Earth Data Challenges MB Generally, storage and processing power should not be GB limitations. 3 gigabyte petabyte PB terabyte TB Knowledge extraction may be; what approaches are available to us? exabyte EB zettabyte ZB See

32 Consider human eye information processing deg ~500Mpixel over 90deg x 90deg field of view ~1 million colours ~3 bytes per pixel, one for each colourprimary Can easily cope with a frame rate of about 20 per second, although not all pixels are viewed on each frame, but just a subset in the region of the field of view currently being looked at. If all pixels in a scene were assimilated then the processing rate, at a conservative maximum, could be 30GB per second. Let s use a much lower rate of 0.3GB per second. Then every hour about 1TB is processed, giving 10TB in a 10 hour day per person. This is big data! How is it processed? Not all is stored!

33 Take a citizen science example observes an event, such as a broken levee bank May take about a second to assimilate and cross check ~ 0.3GB Message passed to authorities, say 200 words (1200 characters at 8B per character) ~ 10kB This represents a compression of about 30,000! This is the result of the human reasoning system s ability to undertake complex analysis and to reduce large data volumes to important facts

34 Human Reasoning Can make complex, high level decisions; spatial reasoning is easy. Whiledata rates might be high, decision making is slow. Isprone to inconsistency and errors, especially when tired. Machine Processing High level reasoning ability is poor by comparison at this time; spatial reasoning is challenging. Very high processing speeds. Error free processing,in principle, with no fatigue. Best of both worlds would be achieved if we can emulate human reasoning by machine (artificial intelligence). General AI might be challenging, but applications-specific AI is not. This might be a pointer to the challenge of handling big data.

35 A simple, long-standing AIapproach, that works well in remote sensing and GIS, is the expert system based on production rules It can be used and can also be used to combine data types in a GIS

36 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science a classical data reduction exercise big data stores numerical process OR facts cultural information AI process facts facts knowledge processor product citizen information facts expert systems is this such a challenge?

37 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science So what might the GIS of the big data future look like? big data stores data data suppliers data AI interface data users data facts facts software suppliers software knowledge broker client citizen input facts web (cloud) knowledge infrastructure with a virtual globe basis facts data data user value added products an agent, skilled in turning client requirements into a set of specifications for data analysts, and skilled in combining knowledge

38 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science Decision support in the digital earth framework The advantage of the digital earth model is the diversity of data types, and the flexibility of data representation. Data and knowledge come from physical sensors, archives and citizens. They can be physical, social, cultural, heritage, visual, non-visual, with differing degrees of reliability, and available at different times and places. It is even envisaged that sound (bird calls?), smell and other attributes, with time, might be added.can anything be more complex? Further complexity is added by the rapidly increasing and massive amounts of data, particularly earth imaging data. Emulating human reasoning may be the way to extract relevant knowledge.

39 Will the Digital Earth Paradigm Change Digital GIS? Earth: Big Data, Heritage and Social Science