1 Free University Brussels FACULTY OF LAW Research group on Law, Science, Technology & Society (LSTS) GOA PROPOSAL Law and Autonomic Computing: Mutual Transformations Promotors : Prof. Dr. S. Gutwirth (spokesperson) Prof. Dr. P. De Hert Post-Doc Researchers Dr. M. Hildebrandt Drs. L. Desutter May 2 th 2006
2 I Research Proposal General Framework:...3 I.A Autonomic Computing: Three Scenarios for an Unknown Future...3 I.B What law got to do with it...5 I.C Mutual transformations: Legal Subjectivity and Ambient Law...7 I.D. Pertinence of the research...8 II Research Design...9 II.1. Mutual transformations: Autonomic computing and legal subjectivity...9 II.1.A. The central position of the human subject in law, in a constitutional democracy...9 II.1.B Task 1: Exploration of the impact of autonomic computing on the central position of the human subject in law, in a constitutional democracy...10 II.1.C. Tasks 2 and 3: Privacy, Data Protection and Intellectual Rights...12 II.1.D. Task 4: Legal subjectivity for transhumans and nonhuman consciousness?...15 II.2. Mutual transformations: Autonomic computing and ambient law...16 II.2.A. Task 5: Regulation of human behaviour in an AmI environment...17 II.2.B Task 6: Ambient law and the 'ecology of practices'...18 II.4 Methodology of the research...19 II.4.A Classical methodologies of the relevant disciplines...19 II.4.B Generic vocabulary and speculative thinking...19 III Work Distribution and Output...20 IV Personnel...23 V References...24 VI. Short description of LSTS and list of relevant running projects...30 VI.A. Short description of LSTS...30 VI.B. List of relevant running projects in LSTS...30 VII Curriculum Vitae applicants & postdoctoral researchers...32 VII.1 Serge Gutwirth...32 VII.2 Paul de Hert...60 VII.3 Mireille Hildebrandt...86 VII.4 Laurent de Sutter
3 3 I Research Proposal General Framework: I.A Autonomic Computing: Three Scenarios for an Unknown Future Ambient Intelligence and Autonomic Computing In the 2005 Report The Internet of Things (ITU 2005), 1 we are confronted with a detailed description of a world in which all things are wirelessly connected. Any movement or change is registered, allowing a permanent real-time monitoring that should enable the environment to adapt itself automatically to our inferred preferences. We are invited to imagine a world in which ubiquitous computing is no longer restricted to a proliferation of desktops or visible microprocessors, but embedded in a way that takes the technology out of our sight while providing us with unubtrusive and user friendly human-machine interfaces (HMI s). In fact, these HMI s may even be implanted into our bodies, 2 like the RFID-tags now used for livestock supply chain management, or the human implants used in healthcare institutions in the US. If ubiquitous computing meant connectivity anytime, any place for anyone, it will now come to anything, thus turning our offline world online. The Internet of Things seems to describe in more detail the Vision of Ambient Intelligence, initiated by Philips (Aarts and Marzano 2003) and the European Commission (ISTAG 2001) in recent years. Ambient Intelligence means an environment in which human beings are surrounded by pervasive, ubiquitous and interconnected computing devices that anticipate their preferences in order to adapt the environment to their inferred wishes. Ambient Intelligence is directed to user friendly human machine interfaces to such an extent that the environment itself becomes the interface (cf. SWAMI Deliverable 1, 2005) The most promising and challenging aspect of Ambient Intelligence and the Internet of Things is the fact that it will run on the basis of autonomic computing. In 2001 Paul Horn, IBM s senior vice president, introduced the idea of autonomic computing, a term that refers to our autonomic nervous system. He chose this term, because the autonomic nervous system governs our heart rate and body temperature, thus freeing our conscious brain from the burden of dealing with these and many other low-level, yet vital, functions (Kephart and Chess 2003). The Ambient Intelligent environment which should emerge if the Internet of Things catches on will do the same for us: providing a technological infrastructure that permanently adjusts the environment to our needs without bothering us with the enormous amount of decisions that have to be taken to allow such continuous adaption. Autonomic computing implies the collection and storage of our personal and many other data, to be mined for expected and unexpected patterns. This will allow the environment to anticipate our needs, desires and preferences. Most importantly, it will enable real time profiling and autonomic decision making by the machines that will run our networked environments for us. 1 ITU stands for International Telecommunications Union 2 In FIDIS we cooperate with Kevin Warwick, who has initiated experiments with human implants and evaluated some of the ethical implications, see Warwick 2003.
4 4 The technologies that will enable such autonomic adaptive environments are described as RFID-systems, biometric applications, sensors, miniaturization and nanotechnologies (ITU 2005). The wireless connections between them, online data bases and available processing power will see to it that these networked environments will start learning about us, anticipating our worries and possibly solving our problems before they have even occurred (to us). This is often referred to as machine intelligence, emerging not in separate machines, but out of the interconnections between things, sensors, online databases and processing power. Such networked, distributed intelligence may be based on algorithms, which means that in theory the autonomic decision making process is still predictable. However, in practice, the complexity of the software and the information to be processed, makes it virtually impossible to check what the system will come up with. On top of that, further development of neural networks and nanotechnologies will produce a type of intelligence that is no longer predictable, and therefore able to find solutions that human experts may not have discovered, yet (Kephart & Chess 2003). Ambient Intelligence, GRIN Technologies and Autonomic Computing: Three scenarios Ambient Intelligence is a vision on the role of emerging technologies, like the socalled GRIN's (Genetics, Robotics, Information and Nanotechs), 3 which have seen a drastic acceleration in the last two decades. Autonomic computing, which is a precondition for Ambient Intelligence, nourishes on these technologies. Without the exponential advances made in the field of GRIN technologies autonomic computing would not stand a chance and Ambient Intelligence would remain science fiction. All these interrelated developments have given birth to the parrallel development of a host of questionings around the meaning of the spectacular technological transformation of what Latour would call our common world (Latour 1999). These questionings have been articulated in three different scenarios (Garreau 2005). The first has been called the Heaven Scenario. Following this scenario, the development of GRIN technologies will give to human beings the possibility to fullfill their dreams way beyond their wildest expectations: immortality, collective intelligence, transcendance through machines, etc., will all become reality a reality that will enable humanity to take a step further into its political and social unification (Kurzweill, 1990, 1999). The second scenario has been called the Hell Scenario. Following this second scenario, the development of GRIN technologies will, on the contrary, drive humanity straight into a nightmarish world. In this world, there will not be one humanity anymore but a strict divide between those who can afford the benefits of GRIN technologies ( Enhanced ), and those who cannot or will not employ them ( Naturals ). This divide will multiply the tensions both inside and outside societies leading to new forms of discrimination and war (Joy, 2000). The third scenario has been called the Prevail Scenario. Following this scenario, it is not the development of GRIN technologies that will drive humanity to new places (Heaven or Hell), but the interaction between humans and GRIN technologies. Other than the other two scenarios, the Prevail Scenario does not consider the development of technology a stronger power than the power of human beings to discriminate 3 Other acronyms have been used, such as NBIC (nanotechnology, biological engineering, information technology, and cognitive engineering), GRAIN (genetics, robotics, artificial intelligence, and nanotechnology), GNR (genetics, nanotechnology, and robotics), etc.
5 5 between - and act upon - what they consider desirable or not (Lanier, 2000, 2004). While the first two scenarios seem to imply some form of technological determinism, the third seeks to articulate the possibility for human freedom (and responsibility). These three scenarios, describing the consequences of the current development of GRIN technologies share certain features: - Firstly, they all assume that the key for the development of GRIN technologies lies in the I of GRIN: in Information. It is the development of information technologies that allows the parallel development of genetics, robotics and nanotechs. - Secondly, they all assume that the development of technologies is never only a scientific question: it is a question that implies political and societal consequences. What differentiates one scenario from another is precisely the fact that these consequences and not the technologies by themselves - will lead humanity into one direction or another. - Thirdly, they all assume that, as scenarios, what they describe cannot be considered as what is commonly called science-fiction. Scenarios differ from science-fiction because of the fact that they use imagination within a framework of which the constraints are different: for science-fiction, imagination is a way to reach the possible, without being contrained by what is probable; for scenarios, it is a way to detect both the probable and the possible. This means that the researchers that have given birth to the Heaven, Hell and Prevail scenarios, are researchers who claim that the consequences they describe in part depend on very good present reasons (Garreau 2005). The fact that they are about probable and possible futures, imaginative stories about what the future might be like ( ) not predictions (Garreau 2005: 78), does not mean we should not take them serious. On the contrary, they allow us the kind of speculation we need in a world that is predictable only to a limited extent. This necessitates an imaginative anticipation of the foreseeable futures. 4 - Fourthly, they all nourish autonomic computing and, at the same time, depend on its widespread implementation. Ambient, transhuman and nonhuman intelligence will depend on the further exponential development of GRIN technologies that enable autonomic computing on an unprecedented scale. At the same time ambient, transhuman and nonhuman intelligence will nourish autonomic computing with vast amounts of data and the ability to process them in real time. I.B What law got to do with it The Heaven, Hell and Prevail scenarios all assume that law will be transformed by the development of GRIN technologies. The described developments will not only lead to the creation of new legal fields, but they may also lead to an alteration of the foundations of law. If Roman law was based on family, Medieval law on religion and Modern law on property what will a legal system that has to take into account the distinction between Enhanced and Natural be about? As the three scenarios are not science-fiction, this question is not juridic-fiction: from the point of view of both the Heaven and Hell scenarios, it will very practically be impossible to continue to maintain law as it is, to the extent that the legal objects as well as the legal subjects may be completely transformed. How to continue to elaborate general and abstract 4 In the Safeguards in a World of Ambient Intelligence (SWAMI)-project we have experienced that working with scenario s, in that case dark scenario s, is a very fruitful way of anticipating legal problems and solutions. See (SWAMI D1. & D.2., 2005).
6 6 norms when humanity is divided between Enhanced and Natural? How to continue to use categories such as property or originality when there will be collective intelligence (Lévy 1994 & 1997a)? How to continue to consider law as concerning humanity-only when the intelligence of the machines will make them more than human? But not only do the Heaven and Hell scenarios raise such questions: so does the Prevail scenario. Following the Prevail scenario, the type of control over technology that humans will begin to develop in order to choose between the probable and possible futures that are offered, may exceed substantively the type of control that is now offered for instance by intellectual property rights or by the fundamental rights attributed to the citizens of democratic constitutional states. In order to stop the investigating agents of a state department to exceed their competences, new legal means may for instance have to be invented. The Heaven and Hell scenarios both assume that the role of law regarding the consequences of the development of GRIN technologies will be passive. From the point of view of the Prevail scenario, however, law should play an active role in assessing these consequences. But whether active or passive, the role of law in the assessment of the consequences of GRIN technologies will necessarily lead to its transformation. Following the three scenarios, the current state of law and of legal concepts seems unable to meet the challenge represented by GRIN technologies for issues like legal subjectivity, agency, causality and liability. It is necessary already today, to imagine what kind of legal means and what kind of legal concepts must be invented, amended or put aside, for a scenario to develop its plot. Does this mean that law has nothing to say in the choice of scenario? The first bet of the present project is, on the contrary, that the voice of law will be crucial in the realisation of a scenario. To define the relationship between law and GRIN technologies in terms of passivity (following the lead of technology) or activity (co-shaping our common world) is to commit ourselves, as lawyers, into a debate on the role of law. This debate is not free of constraints because a choice between activity and passivity has take into account the consequences of that choice. The second bet of the present project is that to choose for activity is the only choice that could be faithful to what law is made for, whatever its foundation is (Gutwirth, 1993 & 1999). This bet is a bet in favour of the Prevail scenario, because it is the only one that integrates this faithfulness into its own plot. This means that law is not at the passive end of the transformations that may occur regarding, for instance, our subjectivity. Lawyers need to anticipate these transformations and reconstruct the legal framework. Instead of passively adapting to the demands of technological progress, the law needs to responsively fit the demands of a changing environment, while remaining true to the obligations lawyers consider crucial, even if these obligations may have to be re-configured to face the challenges detected in the set of scenarios described above. At the same time, lawyers cannot assume that technology will be at the passive end of the transformations they invent. This project aims to prepare active and mutual reconfigurations between the practice of law and the practices of technologists in the broad sense (computer scientists, engineers, mathematicians, data base experts, etc.). As the LSTS has been involved in a variety of relevant research projects (see section VI.B), which have confronted their researchers with the necessity of such mutual transformations, it can provide its lawyers, legal theorists and legal philosphers with the active context in which the ambition of this project can be succesfully pursued.
7 7 I.C Mutual transformations: Legal Subjectivity and Ambient Law Autonomic computing is what all three scenario feed on and produce. However, the way autonomic computing is confronted differs. The first two scenarios embrace optimistic or pessimistic technological determinism, which means they will take autonomic computing as a given (for the better or the worse). The third scenario will have to face the challenges of ambient, transhuman and nonhuman intelligence, without taking them for granted. Obviously, the possible legal implications are manyfold. In this project we will focus on two sets of implications that demand close attention. These will structure the two parts of his research project. Firstly, we will assess the impact of autonomic computing on the kind of human subjectivity that is both presumed and protected by law, within the framework of constitutional democracy. To what extent does autonomic decision making on the part of machines or human-machine hybrids impact the freedom of a human person to act autonomously (Bohn, Coroama et al. 2004)? How will this impact our privacy (Solove 2004)? How will autonomic decisions taken by interconnected intelligent machines impact the attribution of causality in cases of tort, breach of contract or criminal responsibility, how will it impact accountability and liability in private law, in administrative law and in criminal law? What happens if not only the critical infrastructure of the adaptive environment runs on autonomic decision-making, but if human persons use intelligent devices like personal digital assistents to delegate a host of decisions to them (Clarke 1994)? Secondly, after having assessed the possible impact, we turn to the issue of regulation. We will investigate to what extent the technological infrastructure of networked intelligent environments regulates the life of people, and how this compares to the way the legal infrastructure regulates life. The discussion on code as law, which has been initiated by Lessig (1999a) and Reidenberg (1998) and critically assessed by Tien (2004) Leenes & Koops (2005) and Brownsword (2005) will be mapped and its relevance for Ambient Intelligent environments will be assessed. After having identified shortcomings in current ways of understanding regulation by law on the one hand and technology on the other, an attempt will be made to develop a generic vocabulary that can adequately describe both technological and legal regulation of human interaction with the environment, after which the relevant differences between the two can be detected. Special attention will be given to the problems of data protection legislation to achieve adequate implementation. At the present moment the technological infrastructure does not empower end-users to effectively assess information on what happens to their data, let alone provide them with the instruments to regain some control over the knowledge built on the information that is recorded in their environments. Building on the generic vocabulary for the regulation of human interaction a concept of ambient law will be developed, which explores the way autonomic computing can be put to use for the implementation of mandatory data protection legislation. As such, ambient law is a technologically embodied law that empowers citizens to regain control over the information and knowledge that will impact decisions taken about them.
8 8 The idea of a technologically embodied law may be countered by those that fear a rule by technology. To prevent such undesired integration of law and technology the contraints of constitutional democracy need to be translated into such an ambient law. This raises the question how the practices of law and technology could be involved in such a project. This question will be researched from the perspective of the 'ecology of practices' (Stengers 1977 & 2006), in which the possibility and desirabiliy of mutual transformations of both legal and technological practices will be investigated. I.D. Pertinence of the research Why should lawyers get involved in the way their legal instruments affect human behaviour in comparison to the way technological instruments affect human interaction? The reason why this involvement is pertinent is the fact that technologies today affect human interaction to an extent that was previously unthinkable. At the same time this may interfere with the capacity of legal norms to have any effective impact on human behaviour. Turning a blind eye to this, with the excuse that as lawyers we would outstep our expertise by getting involved with technological regulation, may seem a return to a safe haven, but may in fact erode the safety this haven can deliver. If an Internet of Things appears, if we start living in a networked environment that forms the interface between human subjects and the technologies that monitor and profile their behaviours, we must come to terms with the way these technologies actually regulate our life (inducing certain actions and inhibiting others). Instead, however, of viewing technological regulation as a threat to legal regulation, we should identify the point at which effective legal regulation needs technological embodiement. For instance, it may be the case that Data Protection legislation cannot achieve its ends without a technological infrastructure that enforces the mandatory legal rules while also providing access to the information that will allow us informed negotiations with service providers. Such informed negotiations will have to be performed between autonomic machines on our side and those on the side of the AmI environment. The amount of trivial but still crucial - decisions that will have to be taken in the framework of an Internet of Things, is simply too massive to allow continuous human intervention (like consent). The proposed research is unique. The EC has been funding several large projects on ambient intelligence. In FP5 (ehealth) and FP6 (IST Audiovisual, IST Embedded Systems) approximately 36 projects have been funded in the area of ambient intelligence involving more than 242 organisations, for a total budget 95.5 M. Only a few had a specific workpackage devoted to ethical, legal, social and psychological implications. SWAMI (Safeguards in a World of Ambient Intelligence, is a project which focuses on a possible future in an Information Society where all ICT capacities are interconnected. The objective of SWAMI has mainly been to identify and explore the future social, economic, legal, technological and ethical issues and the policy options related to safeguards for privacy, security, trust, identity and digital divide. In this project LSTS took care of the legal aspects : LSTS made an overview of the law relevant for Ambient Intelligence and exploringly applied this body of law to the dark scenarios in order to identify strengths and weaknesses of the existing law. In the (current) final phase LSTS consequently tries to formulate relevant legal policy options.
9 9 FIDIS (Future of Identity in Information Society, is a project which focuses on advanced identification technologies and its impact on the identity of the European citizen. LSTS is workpackage leader of Profiling, which is the enabling technique for autonomic computing. This project explicitly takes a multidisciplinary perspective and would provide an excellent context for the researchers of the project on autonomic computing to exchange their findings with excellent researchers from technological and social sciences throughout the EU. The proposed research Law and Autonomic Computing: a Process of Mutual Reconfiguration will complement and further develop the research efforts initiated by SWAMI (completed in late 2006) and FIDIS (running until spring 2009). While SWAMI addresses the general issue of AmI and FIDIS adresses profiling from a multidisciplinary perspective, the proposed research specifically focuses on an assessment of the impact of autonomic computing on fundamental notions of law, specifically the notion of the legal person and the legal norm. It will thus face the regulatory challenges seeking the development of benchmarks beyond administrative law, in order to prepare the making of effective and legitimate regulation in the framework of the democratic constitutional state. The research we propose now is thus mainly conceptual and fundamental. II Research Design We propose a research design along the lines of the two identified interrelated subjects: Autonomic Computing and Legal Subjectivity and Autonomic Computing and Ambient Law. While describing these parts of the research we systematically identify the different research tasks we propose to undertake. II.1. Mutual transformations: Autonomic computing and legal subjectivity II.1.A. The central position of the human subject in law, in a constitutional democracy Constitutional democracy builds on the central position of the human subject. Firstly, democracy in the modern sense means self-rule for the citizens of a polity, whether this self-rule is understood in a deliberative, participatory or representative way. Only human beings qualify for citizenship and other than in Greek times citizenship is not restricted on the basis of class, race or gender. The legitimacy of democratic government depends on the fact that human beings are free to choose alternative courses of action, to deliberate on the best public policies, to participate in political decision making processes and to seek adequate representation for their public interests. It has been argued that this freedom, which is presumed by political theories of democracy, is not a given, but needs to be created by the framework that supports a viable democracy. In line with that, secondly, constitutional democracy depends on the rule of law that attributes a set of fundamental rights to its citizens. These fundamental rights confer legal subjectivity to each human being within the jurisdiction of the polity, providing the individual with a position from which she can
10 10 exercise her freedom to participate in public and private life and her freedom from illegitimate intrusions by private and public actors. To actually provide such freedom the legal persona must not be conflated with the person of flesh and blood. The legal persona is an artifical legal construction that aims to protect and empower the human person, whose nature is fundamentally underdetemined. This indeterminacy is one of the essential tenets of constitutional democracy. It means that a person can never be entirely defined by others (Foqué & t Hart 1990, Gutwirth 1993 & 2002, Hildebrandt 2002, De Hert & Gutwirth, 2006). The challenges of autonomic computing are directly connected with possible transformations of the human subject, raising the question what this should mean for the concept of the legal subject. Research into these issues will be the first task of the researchers. The findings of this research implicate a second research task, namely an investigation of the legal regulation of privacy and intellectual property, to assess to what extent the present legal framework offers adequate protection against possible risks, identified in the first task. The third task relating to the central position of the human subject in constitutional democracy, refers to the question to what extent legal subjectivity should be attributed to transhuman and nonhuman entities. II.1.B Task 1: Exploration of the impact of autonomic computing on the central position of the human subject in law, in a constitutional democracy To demonstrate the pertinence of this research we will describe some of the possible consequences of autonomic computing, discussed in the literature. All the tasks will feed on the issues raised in the research of this first task. Normalisation, customisation and filtering The potential effects of GRIN technologies, networked AmI environments and the Internet of Things may impact the human subject as it is presumed in and constituted by a constitutional democracy. One of the aims of this project is to investigate what this impact could be. Authors like Lessig (1999a), Solove (2004) and Sunstein (2001) have described how the profiling technologies that are at the core autonomated computing may engender processes of normalisation and customisation as well as filtering (see also Deleuze 1990 & Gutwirth 2002). Customisation and normalisation seem to change the balance of power between large organisations and individual citizens. These large organisations whether public or private will control an enormous amount of data and have access to detailed and dynamic profiles of categories of citizens. They can use the knowledge thus available to make decisions regarding individual persons, even without identifying them. In fact, as Lessig writes 5, autonomic computing implies that these decisions are taken by machines, which are continuously profiling us, in order to provide adequate services. The point may be that we are not necessarily facing a Big Brother, taking over our lives, but rather that we find ourselves in a labyrinth of digital dossiers, as Solove 5 'When the system seems to know what you want better and earlier than you do, how can you know where these desires really come from? ( ) profiles will begin to normalize the population from which the norm is drawn. The observing will affect the observed. The system watches what you do; it fits you into a pattern; the pattern is then fed back to you in the form of options set by the pattern; the options reinforce the patterns; the cycle begins again' (Lessig 1999: 154).
11 11 calls them. These dossiers or files are updated in real time by machines (sensors, RFID-readers, nanotechnologies, profiling software), which can provide access to a host of customised services, while recording, storing and processing our data. Under this view, the problem with databases and the practices currently associated with them is that they disempower people. They make people vulnerable by stripping them of control over their personal information (Solove 2004: 41). Solove rightly points out that the sheer existence of these files may invite people to change their behaviour out of fear that whatever they do, may be used against them at some point in time. Instead of the methaphor of the Big Brother, he turns to Kafka s The Trial to evoke the image of a person that lacks feedback on what may be held against him. Both Solove and Lessig indicate an intrusion on a person s identy or sense of self, mostly without the person even being aware of this intrusion and without necessarily bad intentions on the part of the relevant service providers or government authorities. We may need to rethink our concept of privacy to adequately assess the possible consequences of such a monitored existence (Solove 2002, De Hert & Gutwirth 2003, Hildebrandt 2006). Apart from the customisation and normalisation described, a third effect has been detected by (Sunstein 2001). This concerns the fact that people are increasingly empowered to filter incoming information by means of profiling technologies that select the information we prefer to take in, while blocking undesired information from our attention. Sunstein warns that this may impact the basic presumptions on which democracy especially a strong civil society thrives: First, people should be exposed to materials that they would not have chosen in advance. Unanticipated encounters, involving topics and points of view that people have not sought out and perhaps find irritating, are central to democracy and even to freedom itself. Second, many or most citizens should have a range of common experiences. Without shared experiences, a heterogeneous society will have a more difficult time addressing social problems and understanding one another.(sunstein 2001) The autonomic profiling prevalent in AmI environments will impact our identity, our sense of self, our privacy and the power balances between those that control the machines that monitor our behaviour and those that are monitored. The first task will consist of: 1. An exploration of the literature on such potential impact, to analyse how the further introduction of autonomic computing may interfere with the central tenets of democracy and rule of law. The focus will be on literature: a. in the field of philosophy of identity (e.g. Plessner 1975, Ricoeur 1992, De Mul 2005, 2003, ), b. philosophy of technology (e.g. Ihde 1990, 1991, 2002, Latour 1987, 1999, Lévy 1997b & 2000, Verbeek 2005) and c. legal philosophy with regard to privacy and ICT (e.g. Lessig 1999a, Solove 2002, 2004).
12 12 Within LSTS Hildebrandt has already worked extensively on this subject, integrating philosophy of identity, technology and law at his point (Hildebrandt 2005, 2006a, 2006b, 2006c). The concept of the legal person will be central to this analysis, because it may be that the rights and obligations that constitute the legal person need to be reconstructed in order to protect and empower the human person of flesh and blood in a world enhanced by machine intelligence. The research in this task will be performed mainly by Hildebrandt, who will share her findings with the promotors, the other postdoc and both predocs during discussions and presentations in the internal seminars, via the website and via discussions of the relevant publications she will prepare on the issue. II.1.C. Tasks 2 and 3: Privacy, Data Protection and Intellectual Rights Task 2: Privacy & Data Protection The literature study of the potential impacts of GRIN technologies on human and transhuman identity with regard to the subjectivity implied in constitutional democracy, will be complemented with an analysis of the present legal framework concerning privacy protection, including informational privacy. Research already performed withtin the LSTS indicates several lacunas that need further attention. Data protection legislation provides legal transparency tools to empower citizens to have some control (like the requirement of consent for the further processing of data), and access to their personal data (De Hert & Gutwirth 2006). However a number of salient problems do exist: data protection law only protects personal data. However, the protection that is needed may concern knowledge (profiles) rather than data, this renders data protection legislation inadequate to cope with the new types of knowledge generated by e.g. profiling technologies (Hildebrandt 2005) data protection law only protects personal data of identifiable persons, while most profiling is done on the basis of anonymised data to which the legislation does not apply, (Schreurs and Hildebrandt 2005). In the same way the application of a group profile to an anonymous person does not fall within the scope of data protection legislation, while it may have substantial consequences for this person (Custers 2004) data protection legislation provides more transparency to citizens whose data are processed. However the limited extent to which transparency of citizens is made possible may not allow adequate protection of privacy and fair treatment, demanding more than just transparency. In some cases the processing of data should not be made transparent, but simply prohibited. legal transparency tools may not provide any protection if the technological infrastructure is not in place to trace one s data, the profiles built with them and the profiles applied that are built on data of other people This second task will consist of an exploration of the literature, legislation and case law regarding data protection. At this moment the 1995 European Data Protection Directive is under review. It would be interesting to check whether any shortcomings relevant to autonomic computing were identified. Two sets of questions need to be answered:
13 13 1. Is Data Protection the adequate format to face the challenges of autonomic computing? If we ask the question which are the fundamental legal concepts that form the backbone of Data Protection legislation, we may point to the fair information principles formulated in the CoE Convention 108. However, the question remains which values or goods these principles aim to protect, like e.g. informational privacy and a free flow of information. The subsequent question is what we mean with informational privacy, whether we consider this a public or a private good, whether we think privacy should be at the disposal of citizens to trade with or to be protected even against their will (privacy as a commodity). Does Data Protection aim to protect human dignity, or does it aim to protect citizen s liberty, and whatever the answer may be: must values like dignity or liberty be understood as private goods or public goods? If data protection is focused too much on data instead of knowledge, what should this mean for the principles incorporated so far? As Gutwirth and De Hert have demonstrated, 'data protection' and 'privacy' should be seen as two different concepts. In that case the question arises to what extent privacy - in the sense of the protection of private life, art. 8 ECHR is the good or value that fully explains, justifies and validates Data Protection Law. It seems obvious that a set of other issues are at stake, which cannot be reduced to privacy. As described above, autonomic decision making will impact power balances between data controllers and data subjects (between those who know, and those who are known), possibly creating risks of undetected discrimination, unfair treatment and obstructions to due process. Is Data Protection legislation the adequate tool to counter such risks? 2. Are the fundamental legal notions of the Data Protection regulation still adequate? The concept of identity or identifiability is to be reconsidered, because, as indicated above, anonymity leads to a loss of protection (especially a loss of transparency regarding the knowledge inferred from your's and/or other's data). The protection of personal (identifiable) data seems inadequate in the case of profiling and autonomic computing, because the profiles that impact our life may be inferred from the personal data of others or from anonymised data, thus falling outside the scope of the directive. Autonomic computing demands a less static approach than the one taken in the directive (e.g. providing a set of rights on the moment of collecting data), because data processing becomes a real time dynamic process in a networked environment. Concepts like 'data controller' may need clarification as to the role played by new actors like communication service providers and terminal equipment producers. What about e.g. weblogs and personal websites, should they be be covered by data protection rules? What harm is there for 'privacy' in such cases, and who is the data controller of such applications? Another important legal notion to be evaluated is the consent of the data subject. Is this still a valid and acceptable basis for legitimising extensive processing of data, considering the fact that it is simply not possible to have an adequate prediction of the knowledge that may be inferred from these data? This also raises questions about the legal notion of 'purpose specifications', especially in the context of the interoperable applications that will prevail in Ambient Intelligent environments. The research within task 2 will be undertaken by PhDI, under supervision of Hildebrandt and Gutwirth & De Hert, who have published extensively on the issues involved.
14 14 Task 3: Intellectual Property Law (or Intellectual rights ) In the last decades the regime of intellectual property rights has known a radical transformation. The original balance between freedom of information (a public good) and the property right (a private good) that should provide an incentive for the creative mind has been transformed by a series of changes in the field of science and technology (Gutwirth, 1993). Firstly, the advance of information theory within a diversity of sciences, especially the life sciences, has given rise to a radical tranformation of scientific experiment, which becomes ever more dependent on complex technologies to produce new facts (Ihde 1991). Computer simulation, based on algorithms, have taken over part of the empirical test of traditional experimentation. Outside scientific inquiry, data mining has become a major tool for e.g. marketing and criminal investigation. This has stimulated a tendency towards the commodification of information (e.g. software, DRM), that crosses the already thin line between an idea and an invention. Secondly, the distinction between an invention and a discovery turns out to be more or less useless. Scientific facts are constructed in the interaction between human beings and the things they investigate (Latour 1987, Gutwirth 1993). The complexity of the technology employed to discover the facts already implicates human in(ter)vention in the construction of scientific reality. Thirdly, the capital investment in the technologies needed to do scientific experiments is way beyond what public finance can afford. This means pharmaceutical and other industries are financing the research, demanding a return on investment that forces the commercialisation of scientific facts (e.g. genetic testing, GMO s etc.) and the commodification of the information and knowledge used to produce these facts. The commodification of knowledge and information impacts the access of citizens to both, a fact that will influence the power relationships between citizens and the owners of such knowledge and information (De Beer 2004a & 2004b, Gutwirth 1987). The development of GRIN technologies could further disturb the balance between freedom of information and commodification of information. However, t, some authors claim that the problems of informational privacy could be solved by granting citizens property rights in their data, allowing them to trade with them to their own (dis)advantage (Lessig 1999a & 1999b). The European way of looking at privacy in terms of a personality right (a public good), of which the main characteristic is that it cannot be traded with, is contested by an American way of looking at privacy as a commodity (a private good) that should empower citizens to shape their priorities, see e.g. (Prins 2004). In task 3 this project aims to study these different perspectives in the light of the Internet of Things, which integrates a host of sensing and wireless technologies with online data bases, data mining techniques and the continuous storage and processing of data. Relevant questions will be: 1. What will happen to the balance between the users and the 'owners' of knowledge, if all these technologies, techniques, forms of knowledge and data can be endowed with intellectual property rights? 2. What will happen if humans have intellectual property rights in the intelligent and conscious robot or network they have created (which may lay claim to legal subjectivity).
15 15 3. What will happen if nonhuman intelligent entities, endowed with legal subjectivity, lay claim to intellectual rights in their inventions? The research in task 3 will be undertaken by PhDII, under supervision of Gutwirth & De Hert. Cooperation will be sought with Prof. De Sadeleer, Prof. F. Brison and with D. De Beer, all researchers at the LSTS, working in the field of intellectual property law (see : ) II.1.D. Task 4: Legal subjectivity for transhumans and nonhuman consciousness? Even though many jurisdictions attribute legal personality to nonhumans (limited liability corporations, the state etc.) this has always implied that the ensuiing legal subjects need to be represented at some point by a human being. The emerging GRIN technologies may confront us with a radically new situation. In a discussion of what is called transhumanism De Mul (2002) describes human minds that are upgrated by means of implanted computing devices that enlarge both the human memory and the processing powers, and may enable immortality and transmigration of the human spirit into other bodies or devices. Without falling prey to hypermodern fantasies of doom or boom, he sketches a course of evolution in which machines develop their own solutions to problems in their computer programs, thus creating unpredictable courses of action that may give rise to a new phase in the evolution of mankind: a phase in which human-machine hybrids (cyborgs) will transcend familiar limitations of human memory, physical strenght, age etc. New lifeforms may emerge that are entirely nonhuman, while still developing a consciousness (robots, networks, see the famous article by Joy, 2000).The social impact of such radical evolution will be enormous raising the question what is social in this age (who are we?), and there is no good reason to wait for all this to come about before anticipating the impact it may have on the subjectivity implied in and produced by constitutional democracy. As many researchers in this field point out, we are dealing with exponential increases in technological development: the coming about of a transhuman and even nonhuman consciousness could be closer to our present time than we can imagine (Garreau 2005, Warwick 2003). This brings us to the issue of how the concept of legal subjectivity applies to these transhuman and nonhuman entities; how can they be held accountable; what happens if a machine or a network claims a right to privacy? These are just two of the questions raised by the possible appearance of such new species. This taks will entail two types of research. 1. On the one hand there will be a straightforward legal analysis of civil and public law. Relevant questions will be: a. How will the legal consequences of birth, death and aging be transformed in the case of transhuman and nonhuman subjectivity? b. Are there starting points for legal regulation of new developments in current civil law, for instance in civil liability law? This area of law knows liability for goods and animals, next to liability for human behavior. Is liability of and for transhuman and nonhuman subjects to be conceived along the same lines?
16 16 c. In public law we may face the question whether a nonhuman subject could claim a right to privacy or a right to intellectual property. This part of the research in task 4 will be undertaken by both PhD's, under supervision of Hildebrandt, Gutwirth & De Hert. The work will be divided between the PhD's in accordance with their other research. 2. On the other hand there is more theoretical work to be done. Regarding this part of the research no other method is available but speculative thought. A method of speculative thought by lawyers trained in philosophy must not be confused with thinking free from obligations. In a way it will demand both a more rigorous awareness of the constraints of what counts as legal thinking, and the willingness to risk having to give up comfortable habits of thought about how the world fits with our legal system (and vice versa). This type of speculation is an experiment that will only provide interesting effects if the scenarios are taken serious as developments that will affect our near future as citizens and as lawyers. Relevant research will entail: a. A careful exploration of what these changes might come to mean to our sense of self and to the balance of power between enhanced (trans)humans and natural humans. b. An assessment of the the consequences for the framework of constitutional democracy. This part of the research of task 4 will be undertaken by mainly by Hildebrandt, who will build on the relational theory of law coined by Foqué and t Hart (1990) in the Netherlands as elaborated by Gutwirth (1993) and Hildebrandt (2002), in the fields of criminal law, privacy and data protection, intellectual property rights and law regarding ICT. The relational conception of law will be further extended with regard to these new technologies, to assess and rethink legal subjectivity, the meaning of the open texture of legal concepts and the importance of instituting checks and balances by means of a legal architecture that can provide robustness to the protection of values that uphold the core of constitutional democracy. II.2. Mutual transformations: Autonomic computing and ambient law The second part of the research concerns autonomic computing and ambient law. The claim is, that in intelligent environments infested with autonomic profiling and customised services the legal empowerment of citizens needs to be embodied in technogical devices that provide access to knowledge inferred from our data. Without the technological infrastructure such access is science fiction. However, such technologically embodied legal regulation raises two types of questions. First, to be investigated in task 5, the question must be answered how legal and technological regulations 'rule' the lives of human beings. Which are the differences and which are the similarities between the way these two practices influence out lives? Could it be that implementation of a technologically embodied law constitutes a rule of technology, which could actually overrule the law? Second, to be investigated in task 6, the question must be answered how the practices of law and technology can commit themselves to articulate such an ambient law. How to prevent law from colonizing
17 17 technology and vice versa, while also preventing a type of autism that would leave us paralysed in the face of the consequences of autonomic computing? II.2.A. Task 5: Regulation of human behaviour in an AmI environment Rule of technology In his Code and other laws of cyberspace Lawrence Lessig seems to suggest that computer code is a form a law, in the sense that it regulates human behaviour, just like the market, the law and social norms do. One can object that he conflates all types of regulators of human behaviour under the header of law, thus confusing issues and dismissing the specific safeguards of the law in a constitutional democracy (Brownsword 2005; Tien 2004). In fact one can warn that such conflation of the way technologies affect our habits and the way legal norms regulate our behaviour, opens the door to rule by technology, or, even worse referring to the opposition between rule by law and rule of law rule of technology. Development of a generic vocabulary for the regulation of human interaction Instead of endorsing the Pavlovian reaction of lawyers who advocate a strict separation between legal an other types of regulation, this project will seek to build a generic vocabulary for the regulation of human behaviour. This may obligate lawyers to give up formal positivism and invite them to rethink their concept of legal norms, especially demanding them to pay attention to the way legal norms should obligate citizens. At the same time it should not mean that lawyers must now embrace sociological positivism by equating regularity in behaviour with regulation of interaction. Rather, as lawyers, we should learn to appreciate the many ways in which human beings acquire and transform the habits that constitute their lifeform and investigate what could and what should be the role of both technology and law in this process of habit-formation. The term habit should not be understood in a pejorative sense here, but taken to refer to the fact that our actions are rule-based in a tacit way, as described by authors like Winch (1958), and Taylor (1995). It also means that to enact new legal norms, demands of people to change their habits, which is no easy endeavour. The construction of a generic vocabulary should enable both lawyers and technologists (in a broad sense: computer scientists, data mining specialist, computing engineers, mathematicians, biometric specialists) to tune the norm they enact (legislator), detect (judiciary), embed or enable (technologists, again in the broad sense), to the requirements of constitutional democracy. For technological ways of regulating behaviour, this could imply that the design of a technological infrastructure (the technological constitution of a society) and the design of technological devices should pass through the institutional devices of democracy (deliberation, participation and representation) and the rule of law (judicial control). It would also mean that decisions on the mix of legal and technological means to sustain constitutional democracy cannot be left to commercial enterprise or expert committee, but should be part of the constraints of both democratic and judicial practice. Task 5 will consist of three types of research: 1. A legal analysis of the way law deals with other types of regulation, like standards (e.g. ISO), computercode (e.g. open source vs patented) and technological design (like
18 18 e.g. speed humps). Relevant research questions within the domain of positive law will be, for example, whether these standards, codes and technological regulators are considered as facts to be proven on the basis of expert testimony; whether they provide reasons to excuse actions in the case of tort, breach of contract or criminal offence, etc. 2. An overview and analysis of the discussion between followers of Lessig, Reidenberg (propagating code as law) and Brownsword and Tien (propagating law should not be contaminated with technological implementation), and other salient interventions in the discussion, like Leenes and Koops (2005) will be assessed and described. 3. The development of a generic vocabulary for the regulation of human interaction, as described above. The first two parts of the research planned in task 5 will be divided between both PhD students, under supervision of Hildebrandt, Gutwirth & De Hert. Hildebrandt will undertake the research of the third part, in collaboration with De Sutter and both PhD's. II.2.B Task 6: Ambient law and the 'ecology of practices' The aim of task 6 is to formulate the conditions through which law, ambient intelligence and autonomic computing would peacefully coexist in an 'ecology of practices'. In order to succeed in such a formulation, it will first be necessary to develop a new concept of law as practice (Stengers, 2006). This concept will be developed starting from a perspective, which has been initiated in a dialogue between Bruno Latour and LSTS (Latour 2004, 2005, De Sutter and Gutwirth 2004), which has been initiated during the last five years. It will also be necessary to succeed in reformulating what a democratic constitutional state (and the principles which underlie him) may mean within such a perspective (Latour 1999, Audren and De Sutter 2005). This tasks focuses on ambient law or technologically embedded law from the perspective of the own constraints that respectively define law and technologies. As Latour and Stengers have shown both are respectively bound by a different set of obligations or by a different régime d énonciation or mode of existence (Latour 2004 & Stengers 1997 & 2006). When conceptualising ambient law as the conflation of law and technology, or when approaching technology, law, social norms and market mechanisms as forms of regulations to be mixed to the most accurate blend (Lessig 199b), the perspective taken in this task raises the issue of the differences and respective modes of existence of the practices implied: how can law and technology be articulated in such way that neither the lawyers nor the technologists betray their practice as lawyers and as technologists? This task confronts the following questions: 1. What does it mean to understand law as a practice, understood in terms of the concept developed by Isabelle Stengers?
19 19 2. Which are the consequences of such a perspective for the framework of constitutional democracy, for the concept of legal subjectivity, representation, consent, legal principles? 3. What difference could such a perspective make for the development of the type of ambient law, propagated in task 5? The research of task 6 will be undertaken by De Sutter, who will share his findings with the other researchers to nourish their research. Concerning the first of these questions De Sutter has already initiated a seminar on 'Is legal thinking a practice' within the framework of the IAP and the IVR, in which a variety of international scholars of legal theory have presented their findings (see II.4 Methodology of the research II.4.A Classical methodologies of the relevant disciplines This project demands a thorough understanding of how the methodologies of law, legal theory and philosophy can be employed to research the transversal theme of law and autonomic computing. This means that within the project each of these disciplines will function within the set of constraints that is specific to its domain, while taking good notice of the implications of findings in the other domains. Most research in all three domains is a matter of desk-top research, consisting of adequate overviews of existing literature, relevant legislation, case law and doctrine, analysis of the findings thereof, detection of the major challenges, reflexion and discussion of probable and possible solutions. In the tradition of the LSTS these disciplinary efforts will be complemented with a continuation of the intensive contacts with researchers from other disciplines, within the projects and networks in which the LSTS participates, e.g. the IAP, FIDIS, SWAMI (see section VI.B). 1.The seminars that have been initiated within the framework of the IAP will be continued (see further sub section III) 2. The contacts with the research groups of Jean-Paul Van Bendegem (CLWF, VUB), Bruno Latour (CSI, Paris), Isabelle Stengers (ULB), Marc Mormont & François Mélard (ULg), FIDIS, SWAMI, SPICE, TILT will be continued. 3. Newly initiated contacts with e.g. Yves Poullet (CRID, Namur), Jos de Mul (FW, EUR Rotterdam), Peter-Paul Verbeek (Department of Philosophy, University of Twente), Roger Brownsword (Faculty of Law, King s College, London), Emilio Mordini (Prof. Bioethics at La Sapienza, Rome and director of the Centre for Science, Society and Citizenship) will be extended. II.4.B Generic vocabulary and speculative thinking
20 20 At two points the project endeavours to work with necessarily risky methodologies, demanded by the cross-disciplinary problems raised by autonomic computing. 1. The development of a generic language for the legal and technological regulation of human behaviour in task 5 of this project, indicates that integration will be sought at a generic not general - level, providing a pragmatic conceptual framework that fits the different contexts in which it is to be applied. A generic vocabulary develops concepts that refer to different practices that share a family resemblance without necessarily sharing a common denominator (this differentiates such a vocabulary from a general vocabulary) (Glastra van Loon 1987/56, Wittgenstein 1992/45, Stengers 1997). In her doctoral dissertation Hildebrandt has provided such a generic vocabulary for the concept of the legal norm (Hildebrandt 2002, Chapter 1), which can serve as a basis for this part of the research. 2. The experimental speculation on the implications of transhuman and nonhuman consciousness in task 4 of this project demands a methodology that must not be confused with hypothetical thought-experiments free from any constraints. As mentioned above such speculative thinking will 'demand both a more rigorous awareness of the constraints of what counts as legal thinking, and the willingness to risk having to give up comfortable habits of thought about how the world fits with our legal system (and vice versa)'. See e.g. Stengers 2006 for an account of speculative thought in this sense. III Work Distribution and Output The promotors Gutwirth & De Hert will be responsible for the supervision of all tasks. Postdoc Hildebrandt will co-direct the work in tasks 1, 2, 4 and 5 and undertake the research of tasks 1, 2, 4 and 5 as described above. Postdoc De Sutter will focus on task 6. PhDI will undertake the research as described in 2, 4 and 5. PhDII will undertake the research as described in 3, 4 and 5. The two doctoral students will commit themselves to make a PhD in law, related to one of the different tasks of the project. We expect that one PhD-student will work on an issue related to autonomic computing and privacy (Task 2), while the other will concentrate on issues relating to Intellectual Property Law (Task 3). But we we do not consider this outcome as compulsory. Other PhD-projects (more theoretical ones for example) can be envisioned, depending on the interests of the candidates. Project dissemination plans will be primarily built on team senior members experience. All senior members are involved in quite a number of pan-european and global research networks. These networks will be used to disseminate early information on the project. In a further stage project dissemination plans will be principally addressed to EC funded research projects on ambient intelligence and scientific societies. A crucial step in dissemination plans will be represented by the public conference in which the main findings will be presented. We intend to involve in this conference representatives of EU senior citizens, EU policy makers, EGE
european cultural foundation www.eurocult.org Why we need European cultural policies by Nina Obuljen the impact of EU enlargement on cultural policies in transition countries Why we need European cultural
EUROPEAN COMMISSION Brussels, 6.5.2015 COM(2015) 192 final COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE
Good Research Practice What Is It? Explores some of the ethical issues that arise in research, and is intended to provide a basis for reflection and discussion. It is aimed at researchers in every field
Reducing risks, protecting people HSEBOOKS Reducing risks, protecting people Crown copyright 2001 Applications for reproduction should be made in writing to: Copyright Unit, Her Majesty s Stationery Office,
Comments very welcome Qualities of Democracy: How to Analyze Them. by Leonardo Morlino Istituto Italiano di Scienze Umane, Florence (Italy) September 2009 Old wine in new bottles? Since Plato s Republic
Report of the Task Force on General Education harvard university Faculty of Arts and Sciences Report of the Task Force on General Education copyright 2007 by the president and fellows of harvard college
PROJECT FINAL REPORT Grant Agreement number: 212117 Project acronym: FUTUREFARM Project title: FUTUREFARM-Integration of Farm Management Information Systems to support real-time management decisions and
POLITICAL WILL, CONSTITUENCY BUILDING, AND PUBLIC SUPPORT IN RULE OF LAW PROGRAMS August 1998 by Linn Hammergren Democracy Fellow Center for Democracy and Governance Bureau for Global Programs, Field Support,
Educational Design Research Edited by: Jan van den Akker University of Twente, the Netherlands Koeno Gravemeijer University of Utrecht, the Netherlands Susan McKenney University of Twente, the Netherlands
Climate Surveys: Useful Tools to Help Colleges and Universities in Their Efforts to Reduce and Prevent Sexual Assault Why are we releasing information about climate surveys? Sexual assault is a significant
System failure Why governments must learn to think differently Second edition Jake Chapman Open access. Some rights reserved. As the publisher of this work, Demos has an open access policy which enables
of the Spanish Cooperation Master Plan 2013 / 2016 of the Spanish Cooperation Master Plan 2013 / 2016 Prologue Prologue Spain is a supportive country that has always demonstrated its generosity with the
DL.org: Coordination Action on Digital Library Interoperability, Best Practices and Modelling Foundations Funded under the Seventh Framework Programme, ICT Programme Cultural Heritage and Technology Enhanced
People & Participation How to put citizens at the heart of decision-making Contents 1 2 3 4 Introduction by Geoff Mulgan, Involve Chair Introduction by Rt Hon Hazel Blears MP INTRODUCTION 1.1 Purpose of
FORUM ON TAX ADMINISTRATION: SME COMPLIANCE SUB-GROUP Information note Right from the Start: Influencing the Compliance Environment for Small and Medium Enterprises January 2012 TABLE OF CONTENTS FOREWORD...
British Journal of Social Work (2001) 31, 57 79 Some Considerations on the Validity of Evidence-based Practice in Social Work Stephen A. Webb Stephen Webb teaches social work at the University of Bradford.
Published in Studies in Continuing Education, 12, 2,1990, 61-80. Making the most of experience David Boud, University of New South Wales* and David Walker, The Educational Centre, Randwick Increasing interest
Close to home An inquiry into older people and human rights in home care Inquiry into older people and human rights in home care Acknowledgements This has been a collaborative inquiry, significantly enhanced
EUROPEAN COMMISSION Better Regulation "Toolbox" This Toolbox complements the Better Regulation Guideline presented in in SWD(2015) 111 It is presented here in the form of a single document and structured
WORKING WITH ETHICAL SYMMETRY IN SOCIAL RESEARCH WITH CHILDREN PIA CHRISTENSEN National Institute of Public Health ALAN PROUT University of Stirling Key words: childhood, children, ethics, methodology,
Science, Technology, and Sustainability: Building a Research Agenda National Science Foundation Supported Workshop Sept. 8-9, 2008 Report Prepared by: Clark Miller Arizona State University Daniel Sarewitz
What works in community involvement in area-based initiatives? A systematic review of the literature Paul Burton Jacqui Croft Annette Hastings Tom Slater Robina Goodlad Jo Abbott Geraldine Macdonald Home
OFFICE OF THE PRIME MINISTER S SCIENCE ADVISORY COMMITTEE Looking Ahead: Science Education for the Twenty-First Century A report from the Prime Minister s Chief Science Advisor April 2011 Science Education
About learning Report of the Learning Working Group Open access. Some rights reserved. As the publisher of this work, Demos has an open access policy which enables anyone to access our content electronically