Embracing Cloud for Efficient Development

Transcription

1 Embracing Cloud for Efficient Development Heikki Nousiainen Protecting the irreplaceable f-secure.com

2 Introduction Heikki Nousiainen Lead Architect, Cloud CSO-Technology Office Keen interest on technology Keen interest in improving software development lifecycle 2

3 Embracing Cloud for Efficient Development Context Continuous Integration at F-Secure Sandbox for Innovation Managing Complex Deployments and Configuration Continuous Integration, Take II Few Words on the Future 3

4 Context 4

5 Context Always Available Self-service Resource Consolidation 5

6 Context Always Available Infinite resource Services up in matter of minutes Time-to-market Minimal delays from decision to budgeting / procurement Grow as you go Prove business value and expand Elasticity, on-demand usage Fail-fast, cut resources at first opportunity 6

7 Context - Self-service Say no to handovers! No pre-clearance, low barrier of entry Bill per actual resource usage Time-to-market Get started in minutes Avoid dependencies and co-ordination outside sphere of control API-happy Enable easy automation 7

8 Context Resource Consolidation From servers to resource pools All users are equal Cost savings Manage utilization rates, supply and demand Flatten usage peaks Larger number of devices bring redundancy against total failure Commoditization of hardware 8

9 Continuous Integration at F-Secure 9

10 Continuous Integration Background: Large number of teams, number of components Continuous Integration as the only means to ensure overall system health Challenge is to balance between capabilities and cost: More targets == better coverage, faster feedback More targets == larger inventory, costlier maintenance Compromises: Reduced coverage, longer cycles, multi-stage results 10

11 Continuous Integration Clone from template Reclaim resources Execute tests 11

12 Continuous Integration Private cloud: built privately on top of KVM All testing done on instantiated templates Resources in single pool 12

13 Continuous Integration Benefits: Maintenance effort is limited Infinite Supply allows us to maintain full coverage Infinite Supply allows us to parallelize testing for faster feedback Self-service enables developers to fully manage their test loads Resource utilization is high, and supply vs. demand easily manageable 13

14 Continuous Integration 14

15 Sandbox for Innovation 15

16 Sandbox for Innovation Our CI usage peaks during working hours, a lot of free capacity available at night Following the principle of Self-service, for everyone to use for any purpose 16

17 Examples Continuous Integration, Take II MapReduce cluster of 100 nodes Tooling proof of concept Deployment, setup, processing of 28GB / 630 million rows of data in 30 minutes Software builders, tooling evaluation Evaluation of clustered filesystems, CEPH and GlusterFS Evaluation of distributed hash tables and their performance with high number of failures 17

18 Murphy Additional tester in team Model applications for Test Automation Capable of returning to any state with few clicks Capable of crawling applications and building models unaided 18

19 Managing Complex Deployments and Configuration 19

20 Continuous Deployment - DevOps Development wants continuous and incremental delivery Operations appreciate stability Development is more willing to accept failure, and fix it on the spot The deployed systems are growing in size It s hard to judge on the overall quality of the system With complex systems, it becomes near impossible to manage quick manual deployments deviations make or break the system 20

21 Continuous Deployments - DevOps LB FE LB FE FE AP AP AP AP DB DB DB DB 21

22 Continuous Deployment - DevOps Practice until perfect! Treat deployment as code Automate everything ban manual modifications Use same tooling during development and operations Build on similar templates in development and production no golden images 22

23 Continuous Deployment - DevOps Benefits Incremental, controlled changes Reduce time-to-market, gain early feedback Gain confidence from repeated deployments At all times, know the status and configuration of the deployed systems in version controlled repository! In case of problems, ability to re-create exact environments back in R&D for investigations Deploy on variety of cloud providers Our developers and operations work together on the tooling massively improving communications, trust and knowledge sharing 23

24 Continuous Deployment - DevOps Tooling: Poni, Puppet, Chef GIT Deployment Automation is a requirement and a part of every delivery 24

25 Continuous Integration

26 Continuous Integration 2.0 Traditional Continuous Integration cycle still has some drawbacks: Implementation Feedback cycle time Granularity of results Cost of fault Time to fix System Level Tests Commit to repository Unit-tests 26

27 Continuous Integration 2.0 Mandate positive Test Automation result before any merge to master Fault Detection Requirements: Locally package / build all local modifications Trigger Test Automation from local terminal Collect and communicate results back to the project Fault Prevention 27

28 Continuous Integration 2.0 Implementation Test Automation Merge Verify 28

29 Continuous Integration 2.0 Benefits: Faster feedback cycles Per-modification results available Less faults introduced in master Fewer disruptions to other developers More coherent modifications in source code history Test Automation as a first class citizen alongside implementation Breakages become actionable 29

30 Glimpse on the Future 30

31 Glimpse on the Future Platform as a Service Further cloudification of infrastructure Further operational freedom for developers: Beta programs A/B testing Technologies: Fluidic Systems 31

32 Conclusion 32

33 Conclusion Always Available Self-service Resource Consolidation 33

34 Thank you! 34

35