From the Monolith to Microservices: Evolving Your Architecture to Scale Randy Shoup @randyshoup linkedin.com/in/randyshoup
Background Consulting CTO at Randy Shoup Consulting o o Helping companies from large enterprises to small startups scale their organizations and technology CTO as a service CTO at KIXEYE o Real-time strategy games for web and mobile Director of Engineering for Google App Engine o o World s largest Platform-as-a-Service Part of Google Cloud Platform Chief Engineer at ebay o Multiple generations of ebay s real-time search infrastructure
Architecture Evolution ebay 5 th generation today Monolithic Perl à Monolithic C++ à Java à microservices Twitter 3 rd generation today Monolithic Rails à JS / Rails / Scala à microservices Amazon Nth generation today Monolithic C++ à Java / Scala à microservices
Evolution to Microservices The Monolith The Microservices Ecosystem Migrating to Microservices Life as a Service Owner
Evolution to Microservices The Monolith The Microservices Ecosystem Migrating to Microservices Life as a Service Owner
The Monolithic Architecture 2-3 monolithic tiers {JS, ios, Android} Presentation {PHP, Ruby, Python} Application {MySQL, Mongo} Database
The Monolithic Application Pros Cons Simple at first In- process latencies Coordination overhead as team grows Poor enforcement of modularity Single codebase, deploy unit Resource- efficient at small scale Poor scaling (vertical only) All- or- nothing deploy (downtime, failures) Long build times
The Monolithic Database Pros Cons Simple at first Join queries are easy Single schema, deployment Resource- efficient at small scale Coupling over time Poor scaling and redundancy (all- or- nothing, vertical only) Difficult to tune properly All- or- nothing schema management
If you don t end up regrering your early technology decisions, you probably over- engineered -- me
Evolution to Microservices The Monolith The Microservices Ecosystem Migrating to Microservices Life as a Service Owner
Microservices Single-purpose Simple, well-defined interface Modular and independent More graph of relationships than tiers Isolated persistence (!) *Not* a new idea o Nothing more than encapsulation and modularity A B C D E
Microservices Pros Cons Each unit is simple Many cooperating units Independent scaling and performance Independent testing and deployment Can optimally tune performance (caching, replication, etc.) Many small repos Requires more sophisticated tooling and dependency management Network latencies
Ecosystem of Services Hundreds to thousands of independent services Many layers of dependencies, no strict tiers Graph of relationships, not a hierarchy A C B D E F G
Evolution, not Intelligent Design No centralized, top-down design of the system o Most technology decisions made locally instead of globally o Better decisions in the field Variation and Natural selection o Create / extract new services when needed to solve a problem o Deprecate services when no longer used o Services justify their existence through usage Appearance of clean layering is an emergent property
Google Service Layering Cloud Datastore: NoSQL service o o o Highly scalable and resilient Strong transactional consistency SQL-like rich query capabilities Megastore: geo-scale structured database o o Multi-row transactions Synchronous cross-datacenter replication Bigtable: cluster-level structured storage o (row, column, timestamp) -> cell contents Colossus: next-generation clustered file system o Block distribution and replication Borg: cluster management infrastructure o Task scheduling, machine assignment Cloud Datastore Megastore Bigtable Colossus Borg
Standardization Standardized communication o Network protocols o Data formats o Interface schema / specification Standardized infrastructure o Source control o Configuration management o Cluster management o Monitoring, alerting, diagnosing, etc.
Standards become standards by being better than the alternatives!
Enforcing Standardization Encouraged via o Libraries o Support in underlying services o Code reviews o Searchable code
KIXEYE Service Chassis Goal: Make it easy to build and deploy Microservices Chassis core Configuration integration Registration and Discovery Monitoring and Metrics Load-balancing for downstream services Failure management for downstream services Development / Deployment Pipeline Transport layer over REST / JSON or WebSockets Service template in Maven Build pipeline through Puppet -> Packer -> AMI Red-black deployment via Asgard
KIXEYE Service Chassis Heavy use of NetflixOSS Asgard Hystrix Ribbon + WebSockets è Janus Eureka è Results 15 minutes from no code to running service in AWS (!) Open-sourced at https://github.com/kixeye
The easiest way to encourage best practices is with *code*!
Make it really easy to do the right thing, and harder to do the wrong thing!
Evolution to Microservices The Monolith The Microservices Ecosystem Migrating to Microservices Life as a Service Owner
Migrating Incrementally Find your worst scaling bottleneck Wall it off behind an interface Replace the implementation è Rinse and Repeat
Building Microservices Common Chassis / Framework o Make it trivially easy to build and maintain a service Define Service Interface (Formally!) o Propose o Discuss with client(s) o Agree Prototype Implementation o Simplest thing that could possibly work o Client can integrate with prototype o Implementor can learn what works and what does not
Building Microservices Real Implementation o Throw away the prototype (!) è Rinse and Repeat
Service Anti- PaRerns The Mega-Service o Overbroad area of responsibility is difficult to reason about, change o Leads to more upstream / downstream dependencies Shared persistence o Breaks encapsulation, encourages backdoor interface violations o Unhealthy and near-invisible coupling of services o (-) Initial ebay SOA efforts
Evolution to Microservices The Monolith The Microservices Ecosystem Migrating to Microservices Life as a Service Owner
Goals of a Service Owner Meet the needs of my clients Functionality Quality Performance Stability and reliability Constant improvement over time at minimum cost and effort Leverage common tools and infrastructure Leverage other services Automate building, deploying, and operating my service Optimize for efficient use of resources
Responsibilities of a Service Owner End-to-end Ownership o Team owns service from design to deployment to retirement o No separate maintenance or sustaining engineering team o DevOps philosophy of You build it, you run it Autonomy and Accountability o Freedom to choose technology, methodology, working environment o Responsibility for the results of those choices
Service as Bounded Context Primary focus on my service o Clients which depend on my service o Services which my service depends on o Cognitive load is very bounded Client A Client B Client C Very little worry about o The complete ecosystem Service o The underlying infrastructure è Small, nimble service teams
Microservice Relationships Vendor Customer Relationship o Friendly and cooperative, but structured o Clear ownership and division of responsibility o Customer can choose to use service or not (!) Service-Level Agreement (SLA) o Promise of service levels by the provider o Customer needs to be able to rely on the service, like a utility Charging and Cost Allocation o Charge customers for *usage* of the service o Aligns economic incentives of customer and provider o Motivates both sides to optimize
Maintaining Interface Stability Backward / forward compatibility of interfaces o Can *never* break your clients code o Often multiple interface versions o Sometimes multiple deployments o Majority of changes don t impact the interface in any way Explicit deprecation policy o Strong incentive to wean customers off old versions (!)
Recap: Evolution to Microservices The Monolith The Microservices Ecosystem Migrating to Microservices Life as a Service Owner
Obrigado! @randyshoup linkedin.com/in/randyshoup Slides will be at slideshare.net/randyshoup