CSCI E 98: Managed Environments for the Execution of Programs

CSCI E 98: Managed Environments for the Execution of Programs Draft Syllabus Instructor Phil McGachey, PhD Class Time: Mondays beginning Sept. 8, 5:30-7:30 pm Location: 1 Story Street, Room 304. Office Hours: On campus on Monday from 4:30-5:30pm and from 7:30-8:30pm by appointment. Online office hours to be arranged. Course Website: http://isites.harvard.edu/icb/icb.do?keyword=k105703 (to be updated). Managed Runtime Environments Traditionally, in order to run the computer programs that we write we would first edit the code in a language such as C, then pass it through a compiler to create an executable, and then run that executable directly on the local machine. The compiler would optimize the code based on a potentially sophisticated set of static analyses, tie in any architectureand OS-specific libraries and build a binary that is targeted to a particular platform. Today things have changed. When we program in the vast majority of modern languages (Java, Python, Ruby, Javascript, C#, ActionScript and more), we don t build a platform-specific executable. Instead we either skip the compilation stage entirely or we run a lightweight compilation step, producing a simplified format such as Java Bytecodes. We then pass either the bytecode or the source to a managed runtime environment (or Virtual Machine) that takes care of the actual execution of the program. The environment gives us features that weren t available before, the most obvious of which is garbage collection (or, technically, automatic dynamic memory management). But crucially it lets us focus on the code that we re writing without thinking too much about the minutiae of architecture and OS-specific details. For most application developers, that s where the story ends. Productivity goes up, bugs go down and the system Just Works. This course is aimed at those who want to pull back the curtain and find out what s really going on. Do we no longer need optimizing compilers (hint: we do)? Why do some languages use bytecode? How does the garbage collector figure out what to collect? What happens when we throw concurrency into the mix? And why is the Java VM now orders of magnitude faster than it was fifteen years ago? Just as a great C++ developer needs to understand computer architecture, a great Java developer should understand the advantages and limitations of the virtual machine executing their code to better understand performance bottlenecks, profile traces and some of the more counter-intuitive corners of their chosen language s semantics. 1

Students taking this class will build small but significant pieces of a Java virtual machine, learn VM concepts that progress from fundamental principles to the current state of the art, and discuss selected articles, blog posts and library documentation that illustrate how those concepts are being applied by VM developers today. Prerequisites Background. A good understanding of data structure and algorithmic fundamentals. The Java VM uses a stackbased architecture, so students may wish to re-familiarize themselves with stack operations. There will be a review of important concepts in class. Some familiarity with Operating Systems, Compilers or Programming Language design would be useful but not essential. Programming Experience. This course involves a substantial programming project in Java (see Assessment below), and much of the material and examples in the course will use the Java language. It is assumed that students have a good background in developing in Java. Experience in unit testing with JUnit and Mockito would be useful, but not essential since the basics will be covered in class. Computing Environment. It is assumed that students will be able to install, configure and run certain software in their computing environment, including Git, Maven, the Java Development Kit (JDK), and common third-party Java libraries such as JUnit and Mockito. The source code required for assignments will be made available as a Git repository, uses Version 7 of the JDK and will contain a Maven pom.xml file to resolve dependencies. Assignment 0 (see below) will ensure that students have a compatible environment available, and basic guidance on the use of these tools as relevant to the course will be provided in class and during office hours. Lectures There will be thirteen lectures in the course, meeting on Mondays beginning September 8th from 5:30-7:30. Lectures will be streamed live and posted online shortly after class. Students are expected to attend or view all lectures within the period between class meetings. See the Extension School Calendar for important dates and holidays. Readings There is no required text for this course, however readings of relevant online material will be assigned and discussed on forums and in class. Assessment Assessment for the course will consist of five programming assignments and a final exam. There will also be readings relevant to current topics assigned between classes, with credit given for participation in ensuing discussion either online or in class (or both). The final exam will cover all material presented in class, the assigned readings, and experience gained by working through the assignments. 2

Assignment Due Date Points Assignment 0: Environment Setup. Week 2 100 Assignment 1: Java Bytecode Interpretation. Week 4 200 Assignment 2: Just In Time Compilation. Week 6 200 Assignment 3: Garbage Collection. Week 8 200 Assignment 4: Exceptions and Error Handling Week 10 200 Final Project: Advanced Garbage Collection Week 13 400 Final Exam Week 14 500 Participation Week 14 200 Total Points Available 2000 Each student may use up to four late days over the course of the semester. Each late day extends the deadline for an assignment by 24 hours, and multiple late days may be used on a single assignment. The SimpleJava Virtual Machine The coding projects for this course will use the SimpleJava Virtual Machine. SimpleJava is a subset of the Java language designed to emphasize basic concepts in Virtual Machine design. It favors simplicity over completeness, and so is missing certain features of the full Java language such as long data types, custom class loaders, the Java Native Interface and others. The SimpleJava VM is an implementation of the SimpleJava runtime environment, itself implemented in Java. As with the language itself, the VM emphasizes simplicity of implementation over performance. Its component implementations are chosen for their approachability by students rather than to reflect the current state of the art in JVM implementation. A complete description of the SimpleJava language and VM, including architecture descriptions and API documentation, will appear on the course website and will be discussed in class. The course assignments build upon one another; students will have the option of continuing to use their own implementations or starting each assignment with a full implementation of the previous projects. Grading All of the development projects will require the student to fill in skeleton implementations of the required functionality, adhering to interface specifications in the code. Sample code will be provided that students can run on their completed virtual machine implementation to verify that the output matches that produced by the standard java command in the Oracle JDK. Initial grading of the projects will be performed both using these and other similar code samples, and with a comprehensive suite of unit tests that verify the expected behavior. Testing will focus on the end-to-end behavior of the system, and on the state of the virtual machine s internal data structures before and after the student code runs. The intention is to ensure that the implementation meets the specification, not to enforce aspects of programming style. In cases where the test suite does not pass completely, partial credit will be awarded based on the percentage of the suite that does pass, and on inspection of the code. Note that while JUnit tests are not required as part of the submission, it is strongly recommended that you unit test your code both to help in your own development and to make your progress and approach clear when evaluating code for extra credit. 3

Assignment 0: Environment Setup The initial project is aimed simply at ensuring that all students are in a position to build and run the SimpleJava virtual machine, and can submit assignments using the Canvas course website. A Git repository will be posted on the course website containing the code and building instructions, and the submission will be the output of a basic program executed by the SimpleJava VM. It is not anticipated that Assignment 0 should present major difficulties for somebody who is familiar with building and running Java code, but we do recommend working on the assignment early in the assignment period so that any issues that do arise can be fixed with the minimum of stress. Assignment 1: Interpretation Assignment 1 will focus on executing a number of Java bytecode instructions in the SimpleJava interpreter, following the pattern demonstrated in the rest of the interpreter. The instructions will include some stack-based arithmetic operations, some field accesses and method invocations. The semantics of the operations to be implemented, and the design of the interpreter will be discussed in class. Assignment 2: Just In Time Compilation Assignment 2 will cover the profiling decisions made by a JIT compiler in determining what code is worth the overhead of optimization. Note that we will not perform the optimizations themselves (students interested in the details of an optimizing compiler should consider taking CSCI E-95 in the Spring). This project will focus instead on the profiling data and algorithms used to select code for optimization to best takes advantage of a VM s runtime information. Assignment 3: Basic Garbage Collection Assignment 3 is the first of two garbage collection assignments. Students will implement a semi-space copying garbage collector algorithm. The SimpleJava VM s memory management interface is designed to allow students to focus on the algorithm itself in relative isolation from the rest of the VM s components. The design of the algorithm and its dependencies on the VM as a whole will be discussed in class. Assignment 4: Exception Handling Assignment 4 will focus on the implementation of Java s exception semantics. Students will implement the ATHROW instruction (the bytecode equivalent of Java s throw new Exception() idiom), including unwinding the program stack, building the stack trace and checking for declared handlers. The details of exception handling in Java and its impact on virtual machine design will be covered in class. Final Project: Generational Garbage Collection In the final project students will work alone or in pairs to implement a generational garbage collector, based on Appel s Simple Generational Collection algorithm. The heap will be split into two generations (the new space in which objects are allocated, and the old to which objects that have survived one collection will move). Each generation will be managed similarly to the semi-space heap implemented in Assignment 3. Generational collectors in general and the 4

algorithm to be exhibits in particular will be discussed in class, and sample code that demonstrates the generational hypotheses will be provided. If the final project is implemented in a pair, students are asked to turn in a one-paragraph statement that describes the division of work. In cases where students choose to work alone on the final project, consideration will be given if a partial credit evaluation is required. Participation At various points during the course, short online readings will be assigned. The intent of the readings is to demonstrate the real-world implications of the concepts discussed in class. When a reading is assigned, there will be a discussion topic created on the course discussion board, followed by an in-class discussion during the lecture following the assignment. Participation credit will be awarded for contributions to the discussion in either case. Student Responsibilities Students are expected to abide by the policies laid out in the Harvard Extension School Student Responsibilities. All development work is expected to be performed individually by students, with the exception of the final project which may be completed in pairs. Topics This section gives a provisional summary of topics to be covered in class. Note that topics will not map one-to-one with lectures. Introduction and Overview Course overview and syllabus. Overview of relevant background, including operating system, computer architecture and data structure topics. VM rationale and history. High-level VM component design. SimpleJava introduction. Language and Bytecode Design Bytecode vs. source code. Designing a language to target a virtual machine. Designing a bytecode instruction set to maximize performance and portability. 5

Classloading and Runtime Libraries Bringing code into the runtime environment. Advantages and disadvantages of dynamic classloading. Code verification. Optimizing the connection between the runtime library and VM. Object Representation Objects in traditionally-compiled and managed runtime environments. Data layout strategies. Object header optimization. Interpretation Stack machine architecture. Interpretation for Java bytecode. Dynamic dispatch for method invocation. Design of the SimpleJava interpreter. State of the art in interpreter design. Compilation Just In Time vs. Ahead Of Time. Profiling and selecting code for optimization. Allowing Interpreted code to run alongside JIT code. Introduction to Memory Management Manual memory management using malloc and free Reference counting and conservative GC for traditional languages. Bump pointer vs. free list allocation. Mark and sweep algorithms. Copying algorithms. 6

Advanced Garbage Collection The generational hypotheses and generational collection. Remembered sets, card tables and related data structures. Multi-region garbage collectors. Barriers, stack maps and other JIT contributions to GC. Exception Handling Java s language and bytecode-level exception model. Implementing both general and resource constrained exceptions. Optimizing null pointer and array bounds checks. Threading and Concurrency Memory consistency models and their impact on VM design. Thread models and mapping to the Operating System. Monitors, locks and other synchronization mechanisms. Concurrency impact on other VM components. Concurrent Garbage Collection Unique challenges for concurrent garbage collection. Parallel vs. concurrent collection. Minimizing GC pause times. Concurrent mark/sweep and copy phases. Concurrent GC barriers. Native Interfaces Designing a safe native interface. Native code impact on garbage collection and JIT compilation. Debugging and profiling interfaces. 7

Special Topics Weak, Soft and Phantom reference semantics and implementation. Finalization. Non-Uniform Memory Access (NUMA) architectures. Optimizing VM startup time. VM impact on cache behavior. Schedule The table below gives an approximate schedule of topics for each class period. Note that the schedule may change based on lecture pace and student interest. Students will be responsible only for material that is presented in class. Class Date September 8th September 15th September 22nd September 29th October 6th October 13th October 20th October 27th November 3rd November 10th November 17th November 24th December 1st December 8th December 15th Topic (provisional) Introduction and Overview Bytecode, Classloading, Runtime Libraries Object Representation, Interpretation JIT Compilation Memory Management and Simple Garbage Collection Columbus Day Generational Garbage Collection Advanced Garbage Collection Exception Handling Threading and Concurrency Memory Models and Distribution Concurrent Garbage Collection Native Interfaces Special Topics and Review Finals Week 8