Best Overall Use of Technology Jaspersoft
Kerstin Klein Manager, Engineering Processes/ Infrastructure, Jaspersoft From requirements to release QA centric development
From Requirement to Release QA-Centric Development Kerstin Klein Manager, Engineering Processes/Infrastructure Jaspersoft May, 2014
Topic!! Jaspersoft introduces a QA centric, feature driven development approach based on Jaspersoft release cycles, outlining a highly integrated and automated build, test and release environment. During this talk, Kerstin will introduce simple, transparent and easy to adopt processes; outline best practices based on common tools; and will look at the integration points between those.!! The main focus will be on QA and test specific topics covering a full release cycle from customer requirement to release maintenance. Jaspersoft s challenge lies in frequent release cycles and a large number of new features that are shipped for an embeddable product suite consisting of three core products, multiple code bases, over 20 deliverables and supporting 200,000 platform combinations. Participants will leave this session with best practices and insight into a real-life scenario for product development.
Key Messages!! Message 1: QA is not just software testing.!! QA is quality in product, in deliveries, in process.!! QA in Jaspersoft does to quality control, quality assurance and testing.!! Message 2: Automate.!! Everything between code committed to SVN and Release is owned by QA.!! Automation on product (CI), deliveries (QAA), process (IaaS)!! The earlier in process automation applies the better.!! Good Quality with Better QA.
JASPERSOFT A TIBCO Product Group
What was announced? April 28 th, 2014 TIBCO Acquires Jaspersoft The Acquisition Accelerates TIBCO s Expansion into Embedded Business Intelligence and Reporting, Continuing to Build the Most Comprehensive Analytics Portfolio in the Market!!Includes all Jaspersoft products & employees!!jaspersoft will operate as a separate product group within TIBCO
Who are we?!! Acquired by TIBCO Software in April 2014!! Leading provider of embedded visualization, reporting and analytics!! Strong customer base across the globe, across every industry!! Founded in 2004!! Headquartered in San Francisco with offices across the globe!! A community of over 400,000 registered members!! Jaspersoft s commercial open source software has been downloaded millions of times
QA IS NOT JUST SOFTWARE TESTING A very brief take on Jaspersoft development methodology and QA philosophy.
Jaspersoft Engineering!! Implements applicable pattern and methods of agile, scrum, kanban, waterfall and other methodologies!! Example: stake holders, mostly agile development processes, iterations, continuous integration, long-term planning, features.!! Common sense driven approach!! Product company: We have deadlines.!! Environment: multiple source repositories, issue trackers etc. due to commercial and community offerings.!! 2 Releases per year!! Coordinated with Sales/Marketing/Services!! Moving release date : bad (and expensive)!! ~8-12 features per release + Bug fixes + customer enhancements!! +3-4 platforms per major release (total support ~200K permutations)!! ~30 release artifacts for JasperReports Server, JasperReports, Jaspersoft Studio, Mobile Apps, Rest Clients, doc, samples,
Jaspersoft Engineering!! New Features!! Separate projects!! Specific QA lead!! Target release!! Bug fixes and minor enhancements!! Team responsibility!! Hotfixes!! Maintenance fixes for previous releases apply to next release too!! Coordinated between development-, QA- and support teams New Features Bugfixes and minor enh Hotfixes New Release
Jaspersoft Engineering! QA acts as stake holder of feature! Review & Verification of Spec for completeness and consistency Specification Implementation! QA support depending on methodology (supporting sprints, sign offs)! Constant review and regression testing! Test plan development for feature sign off! Sign off when defined level of quality, stability and completeness are achieved Merge to core product Core product! Automation Runs! Regression testing! Release artifact testing! Platform testing
Jaspersoft Engineering - QA Development Cycle!! Collaboration with development team on new features and development!! Merge verification and sign off!! Performance testing Pre-Release Cycle!! Full Regression Testing!! Platform testing!! Update + Overlay testing!! i18n!! Distribution packages testing
The Dilemma About Software Releases Good quality choose 2 On Time Lots of new features
Jaspersoft QA Good Quality On Time Lots of new features
Why is Good Quality key?!! Without Good Quality no release!! Early identification of risks allows to scale/constantly review prioritizations towards release date!! Early involvement of QA in development process ensures feature quality!! The earlier a bug is found or a diversion from spec is detected, the cheaper/ easier/faster it is to fix it.!! Having QA acting as stake holder in agile process ensure active communication and constant review of implementation, good documentation, clarification and so helps to make development processes more efficient.
What is QA? Software testing!! Verifying installation of software on plethora of systems!! Verifying software is functional (no regressions)!! Verifying that new features in software are working as expected.
What is Better QA?!! Quality of product: Product is constantly and completely verified; Ensure that with that results are predictable and comparable!! Quality of deliveries: Ensure customer requirements are understood - Product is accompanied from requirement to delivery by QA!! Quality of process: Ensure traceable, standardized processes, efficient communication, simple workflows Product Quality!Completeness!Stability!Performance!Regression!Compliance!Vulnerabilities Customer focus!hotfix/escalation management!customer bugs verification!stakeholder for new features Release Delivery!On time!in Quality!Complete Continuous improvement!constantly review processes!collaboration/ Communication!Contribution
Challenges!! Limited team size!! Distributed team!! Limited time!! Large set of features!! Limited resources!! Large number of platforms
Constant verification. AUTOMATION. Regression Testing. Product deployment. Artifacts generation & testing. We aim to not waste time on repetitive tasks such as installing again and again or repeating same process again and again.
Jaspersoft Approach Development CI QAA QA Testing Production Cost!! The earlier a bug is found the cheaper is it to fix it as causing change can be easily determined.!! The less effort it takes to find a bug the better.
Continuous Integration Focus!! Fully automated process!! To retrieve source code!! To build the sources into binary artifacts!! To test code, functionality!! To provide releasable artifacts (JRS deliverables!! Transparency to all stake holders!! Constant reporting about code stability, quality and potential issues Builds!! Types!! Feature builds!! Core Product builds!! Special builds!! Basic build includes smoke test of deployed application and product packaging!! Additional builds ensure release artifacts stability, scan for vulnerabilities and code compliance; record code quality; generate documentation.
Continuous Integration!! CI environment: based on Jenkins CI + plugins!! Trace Sources!! Custom bug tracker plugin to resolve bug ID from change set to link to!! Bugzilla holds Phabricator link with change sets!! Sources of successful builds are tagged!! Trace deliverables!! Build id part of final build any installation can be traced down to source revisions
Continuous Integration Benefits!! Severe Regression on any release artifact can be tied to small change set.!! Parallel execution of builds/deployments makes processes more efficient.!! No human interaction required.!! Instantly deployed instances of features and product are available for QA team to test and verify latest changes.!! Change set can be easily determined by nondevelopers. Key Take Away!! Treat every build as release candidate!!! Generate full set of deliverables/release artifacts at every build.!! Develop a scalable CI environment.!! Bugs found late in the process drive improvements.!! Apply CI/CD patterns as they fit.!! Design simple processes, make it easy to use tools. CI Bug tracker Change Set
QA Automation!! Automated regression testing that performs common tasks within JasperReports Server and verifies output!! Results are constantly analyzed to detect regressions and deviations!! Java project!! constantly extended and updated to match latest JRS code!! Supports all DB / App Server JRS supports!! Coverage for all major workflows and services with varying complexity depending of feature stability, maturity and significance!! Integrated into CI to run daily on 7 defined, monitored platforms + on demand!! Combination of OS + JDK + App Server + DB Server!! Automation runs integrated / essential part of QA processes
QA Automation Test Runs!! Reset test environment!! Prepare: Download, configure and install latest version of product!! Verify instance!! Sanity check 6 tests to determine JRS runs in acceptable mode!! Run Tests!! Parallel executions of ~5,000 tests!! Screenshots of failures!! Re-Run Failed Tests!! Collect & Archive results!! Notify team!! Takes 6-8h depending on few factors
QA Automation Challenges!! Time!! Running full set of tests!! Analyzing results is time consuming!! Collaboration with QA/Dev while following up on findings!! Ever-changing Product!! Handling of known Bugs!! Dilute results if executed!! May only occur in subset of environments!! Allows compatibility/configuration to support multiple features, branches, versions and platforms!! Increase coverage!! Prioritization: Stability/Maturity of feature, # of bugs, planned features, relevance for users Scale automation Branches Tests Platforms Manual Run Grid Default Run v5.2 v5.5 v5.6 future
QA Automation Benefits!! Major Regression on any release artifact can be tied to relatively small change set.!! Platform compliance is assured within 12-36h of commit.!! Defects are found before manual QA starts testing due to daily automation full regression testing. With that expectations for manual QA are set with known limitations. Manual QA can focus on stable areas/more complex test scenarios.!! Human interaction required for results analysis. Key Take Away!! QA Automation saves time!!! Make your test environments scalable, monitor them!!! Each new build is a release candidate apply CD/DevOps pattern.!! Involvement of development team and architects leads to success. Test suite architecture has to be solid, maintainable and meet same quality standards as product itself.
Infrastructure as a Service Purpose!! Cross-functional for all engineering departments to better support processes!! Support QA/Dev processes by providing required platforms and environments!! Large amount of short living VM s!! Define and maintain reference environments!! Test environments (Big Data, Authentication Servers, )!! Provide tooling and integrations to support processes within engineering teams!! QA/Dev needs to operate efficiently to meet high expectations!! Improved workflows with ops Tooling!! Iterative process: QA Lab -> VMWare VCenter -> Scripted -> Provisioned: Provide self-service functionality to deploy any platform with any version of JRS pre-installed!! VCenter API, python scripts/saltstack
Infrastructure as a Service Benefits!! Faster deploy times!! Easier to reproduce/debug environment specific issues.!! Standardized, monitored, stable work environments to minimize outages (risk for deadline)!! Harvest fruits:!! Support pre-qualification and pre-certifications of platforms (Jenkins + QAA + IaaS)!! Automation indirectly drives improvements on product installers!! Process support by tooling QA loosely coupled to Ops. Key Take Away!! Infrastructure automation is an effort all groups across engineering can benefit from.!! Expertise on specific tooling/platforms can be bundled to small group and can be provided to all parties without delays/bottlenecks.!! Implementing IaaS/PaaS will be iterative process: manual -> templates -> scripts -> automated.!! Consistency makes infrastructure simple.!! Constantly improve and simplify environments.
BE BETTER.
Jaspersoft QA
Jaspersoft QA!! Process is key. Tooling is secondary. Tools support processes.!! Fail as early and often as you must. Learn from failures. Let failures drive innovation.!! Continuously improve.!! Automate. BE BETTER.
Tooling!! Jenkins CI and plugins for tailored CI/CD!! SaltStack, Ansible, Vagrant, VMWare for Infrastructure Services!! Xymon and Logstash for monitoring/logging!! Atlassian Jira, Bugzilla for issue management!! SVN, Phabricator for version control!! SonarCube for Code Quality!! Selenium Grid, Selenium WebDriver for QA Automation!! TestRail for Testcase management and test progress tracking!! JMeter for performance testing!! Blackduck for compliance and vulnerabilities testing!! JasperReports Server for advanced reporting
Try Out Jaspersoft Today Download!! Download 30-day Evaluation of Jaspersoft BI Enterprise at Jaspersoft.com!! Go to Jaspersoft.com/getting-started for setup tips On-Demand Trial!! Try our Live Online version NO INSTALLATION!!! Go to Jaspersoft.com/jaspersoft-live-trial!! Use for 7-days
Q & A @Jaspersoft Sales-emea@jaspersoft.com UK + 44 207 193 9321 openbookonbi.blogspot.com Brian Gentile CEO