White Paper: Localhost is Killing Software Delivery

Transcription

1 SECTION 1.0 White Paper: Localhost is Killing Software Delivery Localhost isn t secure, manageable, scalable, synchronized or collaborative ouch White Paper: Localhost is Killing Software Delivery 1

2 Contents 3 Introduction 4 Chapter 1: The Application Trinity 5 Chapter 2: The Developer Workspace 8 Chapter 3: The Codenvy Workspace 16 Conclusion White Paper: Localhost is Killing Software Delivery 2

3 INTRODUCTION Introduction The era of DevOps is upon us. DevOps is teaching us that when we do something repetitively, we make fewer mistakes. When applied to software, we find that continuous integration, deployment and feedback provided to developers is essential so that problems are rectified as quickly as they were introduced. This is the continuous software delivery goal. {v.3.0} {v.2.1} {v.2.0} {v.1.8} Continuous practices require the decomposition of software development to manageable building blocks. Agile, open source, containers, and microservices introduce modularity in the application building process which compliments DevOps. Less monolithic apps, packaged in smaller pieces not only let organizations get innovative functionality into the market faster, they are easier to use with continuous practices and even make for happier engineers (which helps retain the best). The landscape of vendors has shifted to reflect this new reality with over 200 vendors vying to win your business in the app development, software delivery and hosting categories. At Codenvy, we are obsessed with helping development teams achieve faster software delivery. In this article, we will explore the nature of this continuous world and the impact workspace configuration is having. White Paper: Localhost is Killing Software Delivery 3

4 CHAPTER 1: THE APPLICATION TRINITY Chapter 1: The Application Trinity Organizations are investing in Docker, Puppet, Chef, and Jenkins because these solutions are solving thorny problems around how to take code and then get it into the right environment in a consistent, repeatable way. These technologies are using recipes, automation, and management techniques to make this process not only continuous, but computerized, often performing their tasks repeatedly without a human launching the process. Develop Apps Deliver Apps Host Apps Issue Management Source Code Rep Check Out & Build Code Unit Test Quality Control Test Agile IDE Package & Archiving Integration Testing Deploy to Test Environment Pre-production Deploy to Pre-production Acceptance Testing Deploy to Production Production Workspace Continuous Integration Packaging & Building Tools Code Quality Testing Frameworks Artifact Repository Release Management 100 firms including Atlassian, JetBrains, Puppet, Chef, Ansible, ZeroTurnaround, IBM, Electric Cloud, Coverity, Sonatype, JFrog, and so on... This works well on centralized areas like testing, staging and production where these services are hosted and in control of the environments and resources. They have an awareness of the resource limits and coordinate these multitude of actions in such a way to optimize the delivery process to minimize time or failure rates. But what about the developer s workspace? Each developer needs a workspace to do their work, and these workspaces often require many of the environmental components that production systems require. White Paper: Localhost is Killing Software Delivery 4

5 CHAPTER 2: THE DEVELOPER WORKSPACE Chapter 2: The Developer Workspace The workspace is the central location to perform a unit of work against a set of like-minded projects. Developers often need many workspaces, each configured differently to support the unique constraints of their project. Developers demand choice and flexibility, so they will continue to use their favorite IDE integrated into those workspaces directly. At the same time, each workspace is the central location that a developer uses to coordinate with their corporate toolchain or targeted systems. Your Editors Responsible For Toolchain Security Isola on Resources Dependencies Repor ng Configura on Create Import Developer Workspace Build Edit Dev Cycle Transi on Commit Deploy Sync Your Targets APIs Middleware Databases PaaS Run Git SVN FTP Your Toolchain Code Repository Asset Repository CI / CD Issue Management System LDAP / Ac ve Directory The developer s workspace is a miniaturized staging environment, connected to the same resources that an organization s central systems are connected to, but with additional developer-centric services: debugging, code authoring, content assist, and file management. The system that hosts the workspace takes responsibility for a variety of infrastructure needs including toolchain integration, resource allocation, security, isolation, and reporting. White Paper: Localhost is Killing Software Delivery 5

6 CHAPTER 2: THE DEVELOPER WORKSPACE Today this workspace is almost always on the developer s desktop. Their workspace is localhost. It s commonly assumed that the workspace is bundled within the IDE, but the IDE is just acting as a smart client against the (assumed and embedded) localhost workspace. One question worth asking is: if continuous delivery demands automation and repeatability, then why is it that workspace configuration is manual and complex? Given this reality, it is probably not surprising that developers most commonly configure their workspace using manual wiki pages, which dictate a sequence of manual configuration steps that they often perform on localhost. Ugly Wiki Reality Sometimes they are able to automate these elements using Vagrant or Chef on localhost. But even with that automation, the developer quickly runs into a number of limits that cannot be overcome: Workspaces Defined by Images Must Be Constantly Synchronized This is difficult to manage at scale when changes are frequent. And, it s frankly not the Git-way. Workspaces Are Opaque It is not possible to remotely inspect the workspace on localhost or to query the image what its contents are. This leads to Workspaces Are Not Secure Organizations take great care to manage hosted source and issue management repositories, defining which resources have privileged access to intellectual property. But why bother when the first thing each developer executes is a repo clone onto their localhost workspace that isn t monitoring or controlled by the organization? The code once protected can be copied to other storage devices. The security of your IP is only as strong as the security strength of the collective workspaces that integrate with your hosted systems. With developers demanding and receiving control over their environments, questions are raised about the appropriate way to affect governance while not sacrificing the productivity of individuals. White Paper: Localhost is Killing Software Delivery 6

7 CHAPTER 2: THE DEVELOPER WORKSPACE Workspaces Are Not Shareable Localhost workspaces and images running within a localhost system are bound to the sharing rules dictated by the administrator of the computer running localhost the developer. Workspaces Are Not Programmable There is no programmatic way to create, update, or destroy the workspace. Or even more powerful, once the workspace has been created, a programmatic way to orchestrate and automate services that are performed against the projects hosted within. Localhost is Not Isolated For developers that configure localhost directly, they have to jump through many hoops to have one set of configurations isolated from others. With developers working on many versions and projects simultaneously, they need isolated workspaces to stay productive. While images can create this isolation natively, it comes at a cost. But more importantly localhost and image-based workspaces are not scalable. Have you tried running 20 workspaces simultaneously each consuming up to 100 GB of RAM so that you can fluidly switch between projects without worrying about configuration isolation, waiting for an admin to provision new resources, or the endless whirl of your CPU fan from the constant thrashing due to resource swapping? It s just not possible. White Paper: Localhost is Killing Software Delivery 7

8 CHAPTER 3: THE CODENVY SOLUTION Chapter 3: The Codenvy Solution While many think of Codenvy as a browser-based IDE, which is something we do provide, we are primarily a developer workspace cloud. Codenvy workspaces are designed to be redundant, scalable, secure, and collaborative. They are designed to eliminate the restrictions that have been holding developers back without limiting their preferences on experience. We want these workspaces to be free of compromises. All of the following are true with Codenvy workspaces: Use Any IDE Whether it is IntelliJ, Sublime, Eclipse, emacs, vi or our browser-based IDE, we are indifferent. Developers demand to choose their own experience so that they can make maximum use of their muscle memory or the plug-ins that their organization depends upon. We make this possible with a CLI that enables desktop cloning of workspaces for use by your local IDE and because White Paper: Localhost is Killing Software Delivery 8

9 CHAPTER 3: THE CODENVY SOLUTION Workspaces Are Programmable Each workspace is accessible through an opinionated REST API. The API includes meta elements around the management of the workspace s lifecycle, along with control over what is installed within the workspace and the actions that the workspace performs. There are over 100 possible methods to invoke with examples such as create project in workspace by importing zip, get list of all runner processes for workspace, create new temporary workspace, and add member to workspace. White Paper: Localhost is Killing Software Delivery 9

10 CHAPTER 3: THE CODENVY SOLUTION Workspaces Have Recipes Similar to the way a Chef recipe defines a server for execution or a Vagrantfile defines an image to be created, within Codenvy a workspace has a recipe that defines its structure. Recipes can be packaged into Factory URLs to enable repeatable re-creation of identical workspaces or they can be saved within your source code repository for versioning and inclusion as part of your configuration-as-code initiatives. Workspaces Never Run Out of Resources We take the various functions performed by users within their workspace and place them into microservices, which are executed within machines. Machines are lightweight services that can perform functions on your project in an isolated manner. We are able to place different machines onto different physical nodes, giving a single workspace access to a set of resources that far exceed what is available on localhost. We use Docker as our default machine implementation, but you can provide your own. You can even provide custom Dockerfiles that will allow us to use custom machines that are built on-demand to perform the functions in the workspace. When you run Codenvy On-Prem, you decide how many physical nodes to deploy into your cluster. With Codenvy SaaS, we provide these physical nodes on-demand. Codenvy IDE Codenvy Workspace Your Targets REST Services Containers Hardware or Your Editor syntax refactor compile debug build clone more... APIs Middleware Databases PaaS + many more Your Toolchain Code Repository: GitHub, BitBucket, Gitlab, Subversion +15 more White Paper: Localhost is Killing Software Delivery 10

11 CHAPTER 3: THE CODENVY SOLUTION Workspaces Are Extensible Every developer, project, team, and company (or ISV!) is different. Depending upon the perspective of the organization, they may need to integrate special tools, override atomic microservices, or provide custom environments. Codenvy is based upon Eclipse Che, which provides an extension mechanism that lets developers create custom machines and extensions that modify the behavior of workspaces. Che extensions, whether they are client or server-side can be packaged into custom assemblies for Codenvy On-Prem. These extensions can not only modify a workspace, but can change the entire nature of the system itself, including the branding. Eclipse Che Eclipse Che combines a kernel, developer REST services, machines and a JavaScript cloud IDE. Custom Clients (you author) reuses Browser loads Rich Clients Desktop IDE CLI Developers customize Eclipse Che by writing extensions at any tier, overriding any behavior. GWT Extension (you author) compiles into Cross-Browser JavaScript App Codenvy IDE Wizard Panels Editors uses invokes, registers, listens Codenvy is a multi-tenant system that runs Eclipse Che extensions. Java Extension (you author) deploys as Che REST Services Workspace Project Builder + many more launches runs in Custom Machines (you author) deploys as Machines Docker Process Rich Clients Servlet Runtime SDK White Paper: Localhost is Killing Software Delivery 11

12 CHAPTER 3: THE CODENVY SOLUTION Workspaces Understand the Projects Within Codenvy are project-centric. Projects that are contained within a workspace are structured and opinionated based upon the type that they inherit. The system alters its behaviors by the type of project including altering the plugins that are loaded, altering the behavioral policies, and adjusting the logic for key microservices. For example, building a maven multi-module project requires a different procedure than that of a node project and the system adjusts for this. With customization, extension authors can create their own project types. Additionally, projects may have complex structures including sibling and parent-child relationships, each that bring an associated set of behaviors. Take, for example, the common scenario where a developer is working on a 3 tier application, for which the source code is in 3 different GitHub repositories and the execution is within two machines (app server and database server). In this scenario, a developer needs a single workspace, three projects each mapped to a GitHub repository, and an environment that maps the projects to a set of 2 networked machines. White Paper: Localhost is Killing Software Delivery 12

13 CHAPTER 3: THE CODENVY SOLUTION Workspaces Are Collaborative It is always better to build software together rather than apart. Workspaces, their projects, their files, and their executing machines are all represented with URLs which can be shared with others. Increase the membership of a workspace by extending access privileges to others. Multiple developers are permitted to work on the same projects (each in their own isolated copy), sharing the projects, files and resources allocated within. Make projects public or private to further restrict access. White Paper: Localhost is Killing Software Delivery 13

14 CHAPTER 3: THE CODENVY SOLUTION Workspaces Are Continuous Codenvy embeds a structure that inherently understands the iterative developer cycle of code authoring. At the core of development, engineers write code, build it, and then debug it repeating this cycle with changes until the desired state of behavior is achieved. There are phases before this dealing with configuring the project and after this about moving the assets to another phase of the continuous integration cycle. With localhost workspaces, there was limited intelligence that was applied to this repetitive cycle and it became the domain of IDEs. IDEs would optimize this workflow for the technology each IDE specialized to, but limited by the resources and authorizations that it was offered as a client application that ran periodically. With hosted workspaces, they can and do run continuously, performing assist functions automatically based upon the state or events affecting the workspace. Codenvy workspaces have entities and structure that inherently understand this model of development with reusable APIs and microservices that can be applied to projects of any type, structure, language, or framework. This structure is baked into the system such that extension developers inherit these properties to their benefit allowing them to focus on implementing specialized logic unique to their project domain. ur ors Responsible For Toolchain Security Isola on Resources Dependencies Repor ng Configura on Create Import Developer Workspace Build Edit Dev Cycle Transi on Commit Deploy Sync You Targ AP Middle Datab Paa Run Git SVN FTP Your Toolchain White Paper: Localhost is Killing Software Delivery 14 Code Repository Asset Repository CI / CD Issue Management System LDAP / Ac ve Directory

15 CHAPTER 3: THE CODENVY SOLUTION Workspaces Are Manageable Codenvy lets account/owners, system/admins, and workspace/admins define access, behavioral and resource policies that the system will honor. These policies can be mapped against users or groups. This provides a variety of fine-grained controls that give those with super rights the ability to restrict the consumption of microservices, limit their access to certain behaviors, or to prevent access all together. For example, within your account, you can set RAM and Gigabyte hour caps on each workspace that will prevent new services from launching or even terminate existing ones if upper limits are reached. Beyond this configuration, we also track 100s of events across your workspace and those of all users, giving users and admins analytics to understand what people are doing within their workspace. Anyone can see their own analytics by accessing codenvy.com/analytics with their account credentials. White Paper: Localhost is Killing Software Delivery 15

16 CONCLUSION Conclusion So is this model of workspaces right for you? Beyond the flexibility and power that you will get with hosted workspaces, you ll also save money. Developers spent 100 gigabyte hours in 2014 on idle compute due to localhost. This problem is only going to get worse even though the cost of compute is declining. If you are tired of localhost being insecure, non-scalable, and unmanageable, it s time for you to think about hosted workspaces with Codenvy. You can get started for free as you get 20 free gbh every month in our SaaS self-service workspaces. And Codenvy On-Prem is free for use within your own systems as well. It s the power, flexibility, security, and manageability of Codenvy workspaces that enables leading development organizations like Intuit, WSO2, SAP, CCCIS and Youth Digital scale their workspaces and achieve their continuous software delivery goals. Configuration Wastes Time and Compute Billion gb hours Resources Wasted From Developer Config CAGR Developers 18.5M 22.8M 3.5% Config Hours 15.7B 19.3B 3.5% Resources Wasted 99B 213B 13.6% Input: IDC 2014 Report w/ # of hobbyist / professional devs world wide through 2020 Input: BLS statistics on growth of developers Average RAM per dev desktop: Assumption: Hobbyist RAM is 50% of professional Assumption: Pro works 2600 hours / year; hobbyist is 1040 Assumption: 100% of non-configuration work time is compiling + debugging Assumption: Compiling + debugging time consumes 100% of available RAM Value of Wasted Idle Compute $1.98B $3.45B 13.6% Configuration is constant and pervasive, costing organizations time and money. White Paper: Localhost is Killing Software Delivery 16

17 SECTION 1.0 White Paper: Localhost is Killing Software Delivery 17