Infinispan in 50 minutes. Sanne Grinovero

Transcription

1

2 Infinispan in 50 minutes Sanne Grinovero

3 Who s this guy? Sanne Grinovero Senior Software Engineer at Red Hat Hibernate team lead of Hibernate Search Hibernate OGM Infinispan Search, Query and Lucene integrations Apache Lucene JGroups

4 Agenda What is Infinispan API Key features Three use cases The path ahead

5 Infinispan l Open source highly scalable data grid platform l Distribution or Replication l Sync or Async l Transactional l Persists contents using a CacheLoader Write-through or write-behind Shared or per cluster node l Hibernate second-level cache l State of the art eviction strategies

6 Distributed Data Structure RemoteClient TCP TCP Node (JVM process) Node (JVM process) Memcached Hotrod Rest Memcached Hotrod Rest Transaction Transaction Storage Engine (RAM +Overflow) Query Map/Reduce Storage Engine (RAM +Overflow) Query Map/Reduce Monitoring Monitoring Transport (JGroups) TCP/UDP Transport (JGroups)

7 High Availability Memory is volatile Make redundant copies Total replication (Replication Mode) Partial replication (Distribution Mode) numowners Topology changes Nodes will crash! Re-arrange state

8 Elasticity Expect Node additions Node removals Topology changes are totally consistent do not "stop the world"

9

10 API Key-value store java.util.concurrent.concurrenthashmap JSR-107 compliant CDI support Spring support

11 Integration Hibernate integration 2nd level cache Lucene integration Query index storage ModeShape CapeDwarf CacheStores Cassandra, MongoDB,...

12 Key features Transactions Performance Persistence Map/Reduce Querying

13 Big Data / Fast Data?

14 Transactions JTA transactions support XA or Synchronization based enlistment recovery! Local transactions (batching) Transactional modes optimistic pessimistic coming: ergonomics!

15 Persistence/Cache Store Used for durability increased storage warm caches Various implementations HBase, Cassandra, BDBJE, JDBC, MongoDB, File, Cloud, Remote / Infinispan,... Extensible easy!

16 Network Performance & Reliability

17 Map/Reduce example

18 Map/Reduce

19 Map/Reduce

20 Infinispan on clouds l Cloud-friendly autodiscovery strategies l S3_PING, JDBC_PING, TCP_PING l Fully dynamic clustering l Add or kill nodes on the fly l Using a CacheLoader, scale back to zero nodes without data loss

21 Querying Based on Hibernate Search/Lucene Objects in the grid are indexed index management

22 Annotate your public class Book implements Serializable { String String String editor; public Book(String title, String author, String editor) { this.title = title; this.author = author; this.editor = editor; }

23 Run a Query SearchManager qf = Search.getSearchManager(cache); Query query = qf.buildquerybuilderforclass(book.class).get().phrase().onfield("title").sentence("in action").createquery(); List<Object> list = qf.getquery(query).list();

24 Use cases Local cache Cluster of caches Autonomous data store access protocols

25 Local cache

26 Features of a local cache Eviction Expiry Write through, write behind Preloading Notifications Statistics

27 Local cache not good enough...

28 Cluster of caches

29 Limitations of embedded mode Client is affected by cache topology changes Shared resources Tier management incompatible JVM tuning security garbage collection Non-JVM clients

30 Cache Servers

31 Client/Server Protocols REST Memcached Hotrod proprietary java, python, ruby

32 Lucene Index Stored in Infinispan

33 Single Node Performance Idea

34 Multi-Node Setup

35 Performance Warnings Set Lucene's maximum segment size to fit in LuceneDirectory chunk_size to avoid readlocks Verify blob sizes fit in JGroups network packets Check for CacheStores sweet spot size

36 The Path Ahead 5.1.x.Final February 2013 non-blocking state transfer cross-site replication rolling upgrades for hotrod-clients map/reduce marked stable Spring 2013 state transfer between sites 6.0 Beyond the current amazing performance Scalable index writing Advanced CacheStores

37 JBoss Data Grid (JDG) JBoss Data Grid Build on top of Infinispan

38 Thank