The Art of Picking Your Poison

Transcription

1 The Art of Picking Your Poison Eugene Ciurana, Director, Systems Infrastructure Leap Frog Enterprises May 23, 2007

2 Who is this Eugene character? 15+ years of experience building mission-critical, highavailability systems infrastructure Not an end-user applications kinda guy 10+ years of Java work Engaged by the largest companies in the world Retail Finance Oil industry Background ranges from industrial robotics to on-line retail systems

3 And about open-source...? Eugene s open-source professional experience goes back to 1997 Geek! Contributed code to various projects Operating systems: Linux Java: Jetty servlet container Cross-platform tools: awke Advocacy OpenOffice.org - Computerworld anti-fud from Microsoft article 2004 Mule ESB Linux, Apache, CakePHP, RoR, others Enterprise Suit! Troll! Introduced Linux to Wal-Mart Stores in first official opensource product for production! Seeded first open-source education program there too - SourceLabs

4 In the beginning... A surgeon, a civil engineer, and an enterprise architect argued about whose profession came up first: Mine, said the surgeon. God created Eve from one of Adam s ribs. Wrong! The first profession was civil engineering, the next guy interrupted. Remember that God first created the universe from the chaos. You re both wrong, the enterprise architect said. Who do you think created the chaos?

5 What is the Architect s Job? Assuming that you are a system architect......what s the job description? Architecting is not only about programming Excellent coders are not, by default, excellent architects This is something that you grow into Architects must have a multi-disciplinary background Being a Java architect is a bit of an oxymoron Architects must articulate their works......in technological terms...in business terms Many tech folks want to become architects... why?

6 What is the Architect s Job? Commercial software is not the right answer Mixed environments are the right answer! Open-source software is not the right answer The architect must have a mix of skills In the past, tech kung f00 + problem domain knowledge was enough Today, the architect needs that, plus understanding of legal implications of open-source The best enterprise environments are a combination of commercial and open-source software. The right balance is up to you.

7 What is the Architect s Job? Commercial software is not the right answer Mixed environments are the right answer! Open-source software is not the right answer In an open-source world, the architect must not In the past, tech kung f00 + problem domain The only architect help in must shaping have the a enterprise mix of skills infrastructure and applications. The architect must lead in knowledge was enough Today, shaping the architect the company s needs that, policies. plus understanding of legal implications of open-source The best enterprise environments are a combination of commercial and open-source software. The right balance is up to you.

8 A Warning... Avoid Marketectures Don t let the vendors, commercial or open-source, bamboozle you.

9 Architectures......are often confused with frameworks In some cases, framework and architecture map 1:1 In other cases, vendors like to fan the confusion Is your architecture tied to a specific vendor or technology? Is this a good idea? Architectures are vendor- and technologyagnostic Lay out the architecture, then evaluate technology, and then figure out which products will become part of it J2EE?.Net? Open-source products? Why?

10 Open-source/Cutting Edge Software Larger enterprises sometimes don t benefit from using these tools Corporate policy gets in the way Fear of litigation Aversion to being first Inertia Vendor lock-in The smart companies, though, figure out ways of implementing these Pilot programs Partnerships

11 Open-source/Cutting Edge Software How do you go about changing corporate policy? Through education! Most large IT organizations are too busy with day-to-day business to pay attention Many lack, or forego, ongoing formal training for their engineering and engineering management groups Remember: vendors have their own best interest at heart! Are you Seek education from outside sources/third-parties spending at least 30 who are in that business, not minutes the business researching of selling new software Open Source Initiative Trade magazines technologies and trends every day? Why? WWW: TheServerSide, Slashdot, Reddit, java.net, Digg, InfoWeek, etc.

12 Open-source/Cutting Edge Software Multiple tiers Each tier is implement on different technology Coexistence of legacy and new systems Different skill sets for each tier Multiple physical locations Ad hoc interfaces between locations Multiple protocols Data and process consolidation Mainframes are at the core Cross-coupling between systems makes it difficult to add new functionality New integration projects == lengthy development cycle

13 Typical Application Architecture Architecture evolved toward maximum flexibility and quick development turn-around XML EDI CSV Provider poll, query Database Mid- or front-tier Server push, update Perl, Java, PHP, etc. many discrete modules Data dispatch in-line Systems are built as needed Point-only integration New systems require new interface definition Interface duplication or incompatibility Queuing (MQ Series, JMS, etc.) seen as silver bullets It works as long as all the participants support queuing

14 Typical Application Architecture Architecture evolved toward maximum flexibility and quick development turn-around XML EDI CSV Provider poll, query Point-only integration Systems are built as needed New systems require new interface definition Interface duplication or incompatibility Mid- or front-tier Database What s Server the problem Perl, Java, PHP, etc. Data dispatch in-line push, update many discrete modules with Point-to-Point? Queuing (MQ Series, JMS, etc.) seen as silver bullets It works as long as all the participants support queuing

15 Point-to-Point Problem: Case Study Top-3 in the US financial operation/integrion On-line banking system has a three-tier architecture Web browser front-end AIX/app server middle-tier Mainframe middle-tier Mainframe back-end MQ Series was chosen as the common conduit... Brittle data format COBOL-like data structures, positional Not scaleable Lots of processing overhead because of native-to-uniform data formats It worked, but it became very expensive to operate and enhance

16 Point-to-Point Problem: Case Study Web Browser Javascript AIX app server Javascript Java Other MQ Internet MQ Service Provider Mainframe zos COBOL PL/I MQ MQ everywhere! MQ MQ Series Common Conduit MQ MQ Bank Mainframe zos COBOL PL/I YUCK! Other Bank Applications Windows Server SQLServer etc.

17 Point-to-Point Problem Every problem becomes a new project Different systems may support different interoperability protocols JMS SOAP REST BPEL etc. Point-to-point integration is more expensive Consumes more resources Requires more regression testing Creates lock-in/artificial dependencies between systems

18 Point-to-Point Problem Application A Application E Applications run on different platforms JMS Application C SOAP Mainframes Windows *NIX JMS Adding a new application JMS FTP SOAP and protocol (i.e. BPEL) requires extensive changes Some applications become Application B Application D REST Application F hubs... and potential bottlenecks.

19 Solving the Integration Problem You picked your applications You leveraged open-source options Next comes the fun part: Integrating all application and subsystems to play together Assumption: each of the systems in the mix supports at least one way of communicating with the outside world Database SOAP BPEL REST FTP JMS/queueing JavaSpaces Et cetera!

20 Solving the Integration Problem It s buzzword time! SOA = service-oriented architectures Web services are *part* of this It means that functionality is deployed in dedicated systems that interact with one another asynchronously over some transport ROC = resource-oriented computing Think of it as SOA 2.0 Services provide not only data but computational capability

21 Solving the Integration Problem Shift effort from software development and data replication and consolidation toward integration Don t throw away your enterprise applications They are domain-specific Adopt new technology quickly The fastest pace of innovation now is in the open-source ecosystem Open-source software enables SOA and ROC Good open-source products come about to solve a problem, not to sell frameworks, products, or services

22 Solving the Integration Problem Shift effort from software development and data replication and consolidation toward integration Don t throw away your enterprise applications They are domain-specific Adopt new technology quickly The fastest pace of innovation now is in the open-source ecosystem Open-source software enables SOA and ROC Hibernate vs. EJBs Mule ESB vs. IBM Websphere ESB Good open-source products come about to solve a problem, not to sell frameworks, products, or services

23 Solving the Integration Problem Would you use open-source software for everything? Why? In-house applications and legacy - leave them where they are and continue their development/maintenance cycle Commercial software addresses domain-specific problems when is cheaper to acquire than to build in-house Open-source software works best for satisfying infrastructure requirements Why? Operating systems Network transports Programming languages Development frameworks Caching systems etc.

24 Why is Open-source Better for Infrastructure? Successful open-source projects attract many developers from all walks of life Most ARE NOT domain-specific Specialization is expensive The ecosystem evolves to become commercially viable as it thrives, giving birth to support subscription models Examples: Mule ESB Spring framework Apache projects Hibernate Linux

25 Open-source Architecture Application-specific components and services Vendor application (CRM, CMS, PIM) Web site (Wicket, tagging, templates,etc.) In-house customization to vendor tools (Ruby on Rails, PHP) Product CMS Database interfaces Services, ibatis, Hibernate, etc. 3rd party interfaces as services (ESB, SOA) Off-the-shelf application infrastructure (Oracle apps, SAP, other third-parties) (App server, caching systems, filtering, search, single sign-on technology) (Spring, Glassfish, JBoss, other) Java Virtual Machine Hardware and operating system (Solaris)

26 Open-source Architecture Web client (Firefox) or dedicated application Web services HTTP / HTML Internet Web Services Third-party.com HTTP HTML Database REST SOAP Mule ESB SOAP, REST, JMS, in-memory messaging Spring and server application mycompany.com Workflow, BPEL applications Content Management System Business Intelligence auth Servlet container ORM binary data Packages HTML etc. MySQL Database ERP systems, other databases OpenSSO, OpenID, other authentication systems Open LDAP Custom MySQL Other

27 Open-source Architecture Web client (Firefox) or dedicated application Web services HTTP / HTML Internet Web Services Third-party.com HTTP HTML Database REST SOAP Mule ESB SOAP, REST, JMS, in-memory messaging Spring and server application mycompany.com Workflow, BPEL applications Content Management System Business Intelligence auth Servlet container ORM binary data Packages HTML etc. MySQL Database ERP systems, other databases OpenSSO, OpenID, other authentication systems Open LDAP Custom MySQL Other

28 Conclusions Architecture is about figuring out what and why first Avoid marketectures Open-source is one of the answers to how Commercial In-house Open-source is as much as policy and governance issue as a technological one Shift effort toward integration rather than acquisition or development Open-srouce works best for infrastructure Domain-specific, specialized apps? In-house or commercial work better

29 Q&A Thanks for coming! Eugene Ciurana Director, Systems Infrastructure LeapFrog Enterprises, Inc. Download this presentation from: irc://irc.freenode.net ##java, #esb, #awk, #security