Bricks Architecture Technical Whitepaper

Transcription

1 1 20steps Version: December 2014 Bricks Architecture Technical Whitepaper by Helmut Hoffer von Ankershoffen (Owner & CTO) Marc Ewert (Chief Software Architect & Co-CTO) Zimmerstraße 26 Aufgang B 8th Floor Berlin Tel. +49 (0) Fax. +49 (0) info@20steps.de

2 2 Overview Bricks by 20steps 1 offers a cloud based development platform for rapidly producing diverse kinds of digital products. This whitepaper explains a selection of the most relevant architectonical principles and features of the Bricks platform from a technical perspective. Bricks is tailored for use by digital agency networks and large digital agencies, facilitating their production of rapid prototypes, minimal viable products and full blown versions of digital products. From an agency perspective, Bricks can minimize technical risks and effort for the basic project setup, increase efficiency and speed and maximize strategic reuse. The digital products that can be produced using Bricks scale from simple landing pages and mobile single page apps, to full blown portals including e-commerce and search functionality, up to multi screen applications with custom workflows and a complex business logic. Every digital product based on Bricks provides integrated controlling features starting with simple web analytics and charting; personalized dashboards showing key performance indicators via connected marketing, accounting, CRM and VOIP services; up to custom reports including KPIs specific to the digital product itself. Bricks Architecture Diagram showing the selection of Bricks for a specific application 1 Hereafter referred to as Bricks

3 3 The inherent building block principle 2 of the Bricks architecture allows to construct complex digital prod- ucts 3 in an easy, fast and reusable way. The diagram shows that an application is being built just by injecting the required Bricks. There are many prebuilt Bricks that cover the commodities of any modern application such as content management, search, e-commerce functionality, controlling, workflow management, monitoring and many more. All technical wiring needed to setup and execute those functionalities is already done and ready to use. The application simply has to write it s own Custom Bricks for specific functionalities and business logic. The types of Bricks 4 will be explained in later sections of the document. The primary programming language for Bricks is PHP, and for good reasons: It is widely used in the field of agencies. Lots of open source libraries exist for common use cases. Wages for PHP developers are generally lower than those for J2EE developers 5 Modern versions of PHP provide all relevant features of modern object oriented languages. By using the HHVM 6 runtime, open sourced by Facebook, the performance is more than adequate. The SOA+MVC framework used by Bricks is Symfony2 7. It offers the same architectural design patterns as e.g. Java/Spring. The PHP language extension Hack 8, also developed by Facebook, enhances PHP with static type check- ing, thus allowing for very robust applications. In combination with Hack, HHVM and Symfony2, PHP is now fully enterprise ready. Bricks UI The Bricks platform offers a unified UI for all the Bricks contained in the system. The Bricks UI is used while producing or maintaining a digital product by the agency staff as well as by customers e.g. to look up current controlling data or for hands-on editorial work. 2 This is why we call them Bricks 3 Hereafter referred to as applications 4 Infrastructure, Basic, Partner and Custom 5 Which is why Facebook chose PHP as the primary programming language for facebook.com 6 Cp. 7 Cp. 8 Statically typed PHP-variant developed by Facebook Any Brick can provide a custom UI to be used by end users.

4 4 As shown in the following screenshot the UI supports whitelabelling. Agencies can use their own name and put some visual candy into the UX. The whitelabel configuration for an agency is called a Color in Bricks terminology. At start the user is being presented with a login dialog. The Bricks UI is only usable by authenticated users (agency staff or customer staff). Only the custom end user UI and public REST APIs may be accessed anonymously on demand. Screenshot 1: Login into the Bricks UI Depending on a user s roles/rights/access the navigation adapts accordingly. The fundamental structure of the navigation accommodates the different scopes a user may act in. The system scope allows for administrative work (such as adding new users, clients and projects). The client scope allows the setup for all the projects of a specific client. The project scope addresses issues belonging to a given project.

5 5 There is also personal scope where users can e.g. specify their preferences, change their profile picture or look up personal information that is being provided by any Brick. Screenshot 2: Dashboarding, notifications, personal scope The next screenshot shows the integration of the Found Brick in the UI for a universal search. Activated by clicking on the magnifying glass in the main navigation bar on the left, Found searches the entire system. The left main navigation contains a magnifier icon for executing a search over the whole system. Results are being retrieved from all Bricks and aggregated into a universal results view. The technical back- ground for extension points like the universal search will be described in later sections.

6 6 Screenshot 3: Universal Search via integration of the Found Brick Internal Architecture of a Brick As described above the Bricks is constructed of many individual elements - the Bricks. There are some fundamental Bricks generally used by all applications - the so called basic Bricks. But there are also specific Bricks, which are are relevant for some dedicated clients or projects only - the so called custom Bricks. All types of Bricks share the same internal architecture as shown below.

7 7 Brick Shell Diagram showing a Bricks architecture and it s connections to it s environment using the Pages Brick as an example Basically a Brick consist of a public interface, the so-called shell, and the kernel layer, consisting of one or multiple kernels. The shell of a Brick implements higher level functionality based on and complementing the low level functionality provided by the kernel layer of a Brick. Additionally the shell provides the custom User Interface for end users, as well as the User Interface for agency staff and customers - which is integrated into the Bricks UI as described above. A shell typically provides an API that other Bricks can call upon, allowing to easily implement complex behaviors and innovative features. The Pages Brick, for example, calls upon the SEO Brick to show indicators of the OnSite SEO quality of an article that was edited using the Pages UI. The shell API is meant to remain stable between software updates so as to ensure that Bricks may work together even after code refactorings of a Bricks internals.

8 8 Brick Kernel The kernel layer is encapsulated by the shell. Its interface is private to the shell 9 and may change during software updates. The kernel layer can provide one or multiple kernels. All kernels implement the same kernel interface. A kernel either implements the functionality required by the kernel interface itself or mediates to an existing basic technology - be it open source software such as Wordpress or Drupal, a SaaS solution such as the upcoming Google CMS or commercial products such as CoreMedia 10. Dynamic Kernel Injection Kernels can be activated and configured in the Bricks UI without having to change a single line of code. The Bricks platform automatically injects the kernel for the project as required. Via dynamic kernel injection the Bricks platform can easily adapt to the existing IT landscape of the customer. What is more, dynamic kernel injection allows for switching basic technologies without losing previous efforts when building the shell of a Brick. New kernels can be added if new basic technologies appear on the market or are required by a customer, but not yet implemented/wrapped in a Bricks kernel. Scoped Execution and Extension Points Beyond this fundamental principle of separating shell and kernel(s) there are further characteristics of a Brick. E.g. the code of both the shell and each kernel is provided as a Symfony2 Bundle 11. This way the dependencies and the offered functionality are defined in a standardized manner. A Brick may offer controllers, commands and web services. All those entry points are are executed in a scoped way, the scope being either system, client, project or user. The Bricks platform automatically rewires the application when entering a scope depending the configuration of the scope. Brick execution for a request resp. call 9 That means the interface of the kernel layer of a Brick cannot be executed by other Bricks. 10 Again the Pages Brick is used as an example - the same goes for other basic, partner or custom Bricks. 11 In Symfony2 the plugins or modules known from other frameworks are named bundles

9 9 Once Bricks detects and activates a scope all Bricks are wired up and ready to perform the given task. This can be e.g. the rendering of a Twig template or simply the JSON result of a REST API action. The web services that can be provided by a Brick are REST interfaces returning JSON (or XML) format. In the case of authenticated access to the REST API the needed API keys are managed and checked by the core Brick (and manageable in the Bricks UI). Additionally there are several standard extension points, which are called by the core or basic Bricks such as Pages, Found, Shop, Control and Workflow. By implementing these extension points it s easy for a Brick to e.g. provide key performance indicators for the Control Brick or make custom entities or other data searchable using the Found Brick. For the latter two examples there are so called Controlets resp. Foundlets which may be reused by the custom Brick. Another example would be a Workflowlet for registering custom workflows in the Workflow Brick. An in depth view of the internal architecture of a Brick is given in the developer manual and online documentation. The Brick types Bricks are structured in four different types. These are: Infrastructure: Fundamental Bricks offering the primary functionality of the Bricks platform. Bricks of this category are always activated and are used by all Bricks of the other types. Basic: This type offers standard Bricks, e.g. the Pages or the Workflow Brick. It s not mandatory that these Bricks are activated for a given project even though they are used by most applications. Partner: Partner Bricks are encapsulating external OpenSource applications or web services provided by partnering agencies or specialized IT service providers and specialists. E.g. the SEO Brick is provided by a partner company of 20steps specializing in SEO and SEA. Partner Bricks offer additional functionality for basic and custom Bricks. Custom: Customer specific Bricks are located in this category. These Bricks are generally only relevant for a given customer and project. If another customer of the same industry requires similar functionality a customer Brick is typically migrated to a basic Brick by stripping the customer specific functionality. The internal behavior of a Brick is independent from the Brick type. Brick types are simply a tool for structuring the Bricks code base and classifying them according the value and target audience of the Brick. Infrastructure layer The Infrastructure layer contains the corresponding Bricks (esp. the Core Brick) 12 but also the needed infrastructure for executing the Bricks platform. 12 Currently only the Core Brick

10 10 First of all there is the runtime environment. Currently 20steps and initial partner agencies use the regular PHP runtime on development stages and HHVM on Test, Pre-live and Live stages. HHVM is a virtual run- time environment for languages like PHP and Hack developed by Facebook, very similar to the JVM developed by Oracle. Above the runtime environment lays the SOA/MVC framework offering the infrastructure for building the Bricks platform. The framework used is Symfony2, the leading PHP framework for developing enterprise grade platforms or applications. It is highly scalable, well documented and backed by a large community. The Infrastructure layer contains the persistency of the Bricks platform used for storing all the information gathered by the Bricks. According to the nature of the information, a relational database is used - be it MySQL, MariaDB, Postgres or Oracle - or, when appropriate, a simple key/value store or document based NOSQL database system such as MongoDB. Brick integrates the Doctrine Object Relational Mapper (ORM) 13 to allow for the implementation of the model part of a Brick in a high level way without having to hassle with specifics of individual DBMS. Additionally the infrastructure layer provides basic libraries to further speed up the development of the required functionality. E.g. the AutoTables library enables the developer to render a dynamic HTML table for arbitrary entities. The table is automatically editable and will automatically persist the referenced enti- ties. Additional libraries wrap commonly used web services in an object oriented way. All basic libraries that are incorporated into the Bricks platform and that are not dependent on the Bricks architecture will be open sourced by 20steps. Core Brick The Core Brick offers the functionality needed by the rest of the Bricks platform. It is the glue for combining the multitude of Bricks to a running application and delivers the base classes for implementing various extension points by other Bricks. The following list shows the different modules that are covered in the Core Brick: Configuration: API for accessing and modifying properties. The properties correspond to one of the scopes user, project, client or system. The more specific scope will overwrite the more general one. So if a property cannot be found in the project scope it will automatically be searched in the client and system scope. Notification: Notifications may be triggered for any reason and scope and will be shown to the User (and sent via ) in the Bricks UI. 13 Cp.

11 11 Tasks: Tasks are assigned to a user and will appear in their personal task list in the UI.

12 12 Injection: The Core Brick is responsible for managing the Brick activation and kernel injection during scoped execution. Depending on the specific settings it rewires and prepares the Brick setup for each scoped request. Navigation: Each Brick may inject scoped routes to their controllers. The navigation module gathers the scoped routes and automatically injects them in different navigation sections of the Bricks UI. Personality: The personality module enables a Brick to enrich the profile of the current user with an additional personality provided by the Brick shell or a Brick kernel. E.g. the Pages Brick adds its personality to the current user including Access Control Lists of articles and webpages edited in the Pages UI. Monitoring: Bricks have to deliver health information on demand by implementing the Check interface. The monitoring module will gather the health of all Bricks and offers an endpoint for external monitoring services such as WeMonit, Nagios etc. The abstract superclasses for implementing the controllers and commands in the custom Bricks are located in the Core Brick. There are several different implementations handling each possible context and scope: public, authenticated, authenticated for a project, authenticated for a client, authenticated for the system and so on. By using the according superclass the custom Bricks doesn t have to care about the complex details going on in the background while wiring the Bricks during scoped execution. The controllers and views for rendering the general layout and base functionalities of the Bricks UI are also contained in the core Brick. E.g. project management functionalities including task management, bug tracking etc. are provided by the Core Brick. For authentication and authorization the Core Brick contains a client-/project- and user-management combined with a role and permission framework including ACLs for arbitrarily entities. Commands Additionally the Core Brick provides basic commands that can be executed on the command line by system administrators to setup a Bricks instance or check its configuration without having to use the Bricks UI. The following table gives an extract of all available commands provided by the Core Brick. A description of the command is shown by using the help option. Bricks:core:bootstrap Should be called for setting up a Bricks instance. Initializes the list of ACLs, creates an initial admin user, a first client and project and flags the system as being bootstrapped. Bricks:core:info Shows some general platform information about the Bricks instance such as the color (whitelabel variant) and stage (dev, test, pre-live, live).

13 13 Bricks:core:Brick:list Bricks:core:client:list Displays a list of all available Bricks with detailed information about them. Shows the list of clients known to the system. Bricks:core:client:info Shows the list of projects of a client with the specified id and a list of all its settings. Bricks:core:project:list Shows the list of projects known to the instance. Bricks:core:project:info Bricks:core:user:list Shows the list of Bricks enabled for the project with the specified ID and a list of all its settings. Shows the list of users known to the system. Bricks:core:user:info Shows the settings of a specified user. Bricks:generate:Brick Bricks:generate:kernel Generates a Brick stub with some example code to be used when developing a new Brick of any type. Generates an additional Brick kernel stub with some example code to be used when adding a new kernel to an existing Brick. Basic Bricks The following basic Bricks currently exist: Pages: Providing a user-friendly web content management system that can be integrated easily into custom applications, from building landing pages up to complex content driven portals. Shop: E-commerce functionality for digital products, from simple online shops to vertical online marketplaces. Found: A first-class search component for digital products like linguistic, semantic, instant, universal and guided search applications. From mobile applications up to vertical search engines for target audiences. Workflow: IT based implementation of flows of work, from production processes integrated into the platform to inter-departmental business specific workflows. Control: Holistic investigation, dashboarding, analysis and reporting of relevant key performance indicators: from web-tracking to performance monitoring of the customer care department. Cloud: Supplying a cloud based and secure file management for all Bricks. This may be used for example for the weekly transfer of a business report.

14 14 Deployment and Operations From the perspective of an individual agency Bricks is provided as a cloud service. The Bricks platform can run on any Mac OS X, Windows or Linux system. The Test stage is typically hosted on premise of the agency while the Pre-live and Live stages of the platform lie in an external datacenter. The Live stage is typically set up as a High Availability Cluster (HA-Cluster). For this 20steps designed a HA- Cluster dedicated for the needs of Bricks called Bricks Cluster 1. The Bricks Cluster consists of a redundant DBMS setup, redundant application servers and redundant loadbalancers, cache and SSL accelerators. 20steps currently manages two colors (whitelabel configurations) of the Bricks platform - one for itself, a second for the agency part of DocInsider GmbH (cp. pages.docinsider.de). Each color is hosted in a separate Bricks Cluster. Additional colors and corresponding Bricks Cluster can be setup easily via automated deployment procedures provided by 20steps. If required, HA clusters can be set up on the customer s premises. Brick Development The development of a new Brick starts with its creation - for which the Bricks platform provides a code generator, the Bricks:generate:Brick command. In a wizard-like dialogue the developer provides basic information (such as type and name of the Brick), afterwards the basic code of a Brick is generated (prefilled with some exemplary code). There are also generators for Brick kernels, commands and Twig extensions, which will be injected into the existing code base of a Brick. By using this generators the developer doesn t have to care about the concrete steps to wire the Brick code into the Bricks platform and is presented some exemplary code to ease the development of basic functionality of the given Brick. Experts at 20steps provide direct support for developers of cooperating agencies via trainings, co-development, documentation and request-based via live-chat, and phone. 1 More information about the Bricks Cluster architecture can be found in the Bricks Cluster - Technical Whitepaper

15 15 FAQ Has everything to be implemented in PHP? No. The shell and kernel of a Brick have to be implemented in PHP or Hack. Typically the kernel is thin mediating layer above an existing base technology, web service, commercial product or custom code which might be written in any programming language as needed. How do I integrate a self written custom Brick into the platform? The developer clones the Bricks platform from the GIT repository provided by 20steps into his local sandbox. Afterwards he will be able to generate a new Brick as described above. When he is done with the implementation and testing of a Brick the code is pushed into the GIT repository of the Test, Pre-live or Live stage that corresponds to the color of his agency (cp. Whitelabelling above). Automated deployment procedures at those stages automatically reinstall and reconfigure the new version of the platform on each push via a GIT Hook and activate the new version for execution.