Björn Kraus. Session Aware Full Page Caching For Magento With Varnish ESI

Size: px

Start display at page:

Download "Björn Kraus. Session Aware Full Page Caching For Magento With Varnish ESI"

Gerard Horton
8 years ago
Views:

1 Björn Kraus PHOENIX MEDIA Session Aware Full Page Caching For Magento With Varnish ESI

2 Was ist Varnish? Reverse Proxy Speichert beliebige Inhalte (HTML, CSS, JS, JPG/GIF) Gültigkeit über TTL/maxage im HTTP Header Steuerbar Volle Kontrolle durch Varnish Configuration Language (VCL) 5-10k Requests pro Sekunde möglich

3 Request-Ablauf (Cache Miss) GET / HTTP 1.1 Host: meinshop.de GET / HTTP 1.1 Host: meinshop.de Client Varnish Magento HTTP OK HTTP Cache-Control: max-age=20 HTTP Cache-Control: max-age=20

4 Request-Ablauf (Cache Hit) GET / HTTP 1.1 Host: meinshop.de Client Varnish Magento HTTP OK HTTP Cache-Control: max-age=20

7 Was ist ESI? Edge Side Includes oder ESI genannt, ist eine Auszeichnungssprache um Webseiten-Komponenten dynamisch zusammenzubauen und diese als fertige Seite auszuliefern. <html> <body> <p>the time is: <esi:include src="/cgi-bin/date.cgi"/> at this very moment.</p> </body> </html>

Webseiten-Komponenten dynamisch zusammenzubauen und diese als fertige Seite

8 Request-Ablauf mit ESI GET / HTTP 1.1 Host: meinshop.de Varnish Client ESI Processing GET /pagecache/esi/... HTTP 1.1 Magento HTTP OK HTTP Cache-Control: max-age=20 HTTP Cache-Control: max-age=5

.. HTTP 1.1 Magento HTTP 1.1 200 OK HTTP 1.

9 ESI Request-Ablauf (Cache Hit) GET / HTTP 1.1 Host: meinshop.de Varnish Client ESI Processing Magento HTTP OK HTTP Cache-Control: max-age=20

10 Bringing it all together Varnish Modul für Magento 2. Dynamische Blöcke ersetzen mit ESI-Tags <esi:include src= /pagecache/esi/mini-cart/ /> 3. ESI in VCL aktivieren set beresp.do_esi = true 4. Magento Controller für ESI-Block- Generierung

11 Demo

12 PageCache powered by Varnish Kostenfreie Community Version to-connect/pagecache-powered-by- varnish.html Enterprise Version mit ESI-Support

13 MySQL-Replication

14 Torsten Walluhn Magento Development Lead David Voge Infrastructure Lead

15 Need help with Magento performance?

16 Largest Magento Gold Partner in Europe with certified developers More than 40 Magento projects in 5 years with Magento Project implementation, Magento hosting, support & maintanence Focus on performance, migrations, integrations, time-to-market and TCO Working with medium and large retailers, manufacturers and brands First expert to launch Varnish ESI, Varnish Ajax, Redis, AWS and MongoDB

performance, migrations, integrations, time-to-market and TCO Working with medium and large

17 When to use a MySQL replication?

18 Use case #1: Large product catalog

19 Use case #2: High availability Reduce the risk Reduce single point of failure Disaster recovery Multi-location replication Mirroring

20 Use case #3: Load balancing Distribute workload across multiple computers Increase reliability Increase reliability through redundancy

21 Use case #4: Backing up from a slave File system snapshots Backup in a very short time No high computing load No outages during backup

22 We used MySQL replication for > 400k products Peak > 1MIO PI/day > 300/orders/hour Automatically scaled Web MySQL Cache

23 We used MySQL replication for

24 How did we achieve that?

25 Solutions Keep MySQL queries per page impression low Magento customization with good quality PHP 5.3.x Mysqlnd Optimize daemons and systems Web Server PHP MySQL Kernel Network File system Failover and disaster recovery plan Measuring and improving infrastructure performance

26 What can be done?

27 What can be done? Data Sharding Failover Sessions Backups Multiple slaves Location redundancy

28 Data Sharding write Magento read MySQL Master read MySQL Slave - Catalog - MySQL Slave - Other - sync

29 Data Sharding <catalog_read> <connection> <host><![cdata[srv]]></host> <username><![cdata[username]]></username> <password><![cdata[password]]></password> <dbname><![cdata[data_db]]></dbname> <model>mysql4</model> <initstatements>set NAMES utf8</initstatements> <type>pdo_mysql</type> <active>1</active> <use></use> </connection> </catalog_read>

30 What can be done? Data Sharding Failover Sessions Backups Multiple slaves Location redundancy

31 Basic failover implementation write Magento read MySQL Master MySQL Slave sync

32 Basic failover implementation Magento read / write MySQL Master MySQL Master

33 What can be done? Data Sharding Failover Sessions Backups Multiple slaves Location redundancy

34 Separation of the sessions write Magento read read / write MySQL Master MySQL Slave MySQL - Session - sync

35 Separation of the sessions write Magento read write read MySQL Master MySQL Slave MySQL Master - Session - MySQL Slave - Session - sync sync

36 What can be done? Data Sharding Failover Sessions Backups Multiple slaves Location redundancy

37 Create backups periodically in a short time Magento read / write MySQL Master 15 minutes sync Backup Host MySQL Slave snapshot Storage

38 What can be done? Data Sharding Failover Sessions Backups Multiple slaves Location redundancy

39 Each web server make use of it's own MySQL server running Magento running Webserver A write write Webserver B read read MySQL Slave sync MySQL Master sync MySQL Slave

40 What can be done? Data Sharding Failover Sessions Backups Multiple slaves Location redundancy

41 Location redundancy Magento Location A Magento Location B - standby- Webserver Webserver MySQL Master MySQL Slave MySQL Slave MySQL Slave sync

42 Where did Magento need help?

43 Core Problem: Backend Problem It is not possible to store the configurable product. Magento - Fronend - Magento - Backend - Workaround Backend server with it's own local.xml. read MySQL Slave write sync read / write MySQL Master

44 Core Problem: Failover Problem MySQL failover not possible directly by Zend Magento read / write Workaround Hardware load balancer Software load balancer Magento reconfiguration after outages. Programming a Zend extension. MySQL Master MySQL Master

45 Core Problem: Separation of the sessions Problem Currently not supported by Zend No separate session connector write MySQL Master Magento read MySQL Slave read / write Workaround Zend adjustment CGI has submitted a patch to Magento Inc. sync MySQL - Session -

46 Core Problem: Separation of the sessions --- a/build/app/code/core/mage/core/model/mysql4/session.php +++ b/build/app/code/core/mage/core/model/mysql4/session.php -73,8 +73,8 class Mage_Core_Model_Mysql4_Session implements Zend_Session_SaveHandler_Interfa public function construct() { $this->_sessiontable = Mage::getSingleton('core/resource')->getTableName('core/session'); - $this->_read = Mage::getSingleton('core/resource')->getConnection('core_read'); - $this->_write = Mage::getSingleton('core/resource')->getConnection('core_write'); + $this->_read = Mage::getSingleton('core/resource')->getConnection('session_read'); + $this->_write = Mage::getSingleton('core/resource')->getConnection('session_write'); }

47 What to measure, watch and optimize?

48 What to measure, watch and optimize? Replication status Failover process Synchronization of the database R/W splitting and more...

49 Pro / Contra

50 There are more advantages Pro Contra

51 Do s and Don ts Software Infrastructure Do understand how Magento uses MySQL resources. understand MySQL replication. check your shop in a test environment. monitor the performance and environment. implement a disaster and recovery plan. replication. implement a disaster and Don t trust 3rd Magento extensions - they can make use of wrong MySQL resources. use a small database to figure out an optimal architecture. enable replication on live system without testing. mix too much scenarios like data sharding and each node has it's own slave.

Varnish the Drupal way

Varnish the Drupal way About me Boyan Borisov Team Leader @ Propeople boyanb@propeople.dk @boyan_borisov Skype: boian.borisov hap://linkedin.com/in/ boyanborisov What is Varnish? Reverse proxy cache server...