cache
Product: xTier™/CACHE 2.3	support@fitechlabs.com  Download  Buy
Depends: cluster  objpool marshal log  Related: grid  db
xTier™ Navigator: cache cluster config email i18n ioc info jmx jndi security log marshal objpool os fs tx uidgen workflow jobs db startup jms grid

Description
xTier™ comes with non-replicable distributed cache implementation. The main difference between replicable and non-replicable caches is that with replicable caches whenever an object is created or updated its state is replicated to every other node in the cluster (or configurable subset of nodes), while with non-replicable approach only a small invalidation message is sent out to nullify the references to given object on required nodes.

Replicable caches make possible to cache objects at application level whenever they are not backed by any secondary storage such as database. This would not be possible with non-replicable caches, since without secondary storage there is no way to recreate object's state. However, replicable caches suffer from certain disadvantages. Although an object may actually be accessed only on a single node, its state is always replicated across all other nodes, therefore unnecessary enlarging heap footprint for every node that may never be used to access this object. Full state replication, when it becomes moderately frequent, can also burden CPU's with expensive serialization/de-serialization operations on all nodes as well as possibly saturate the network.

Non-replicable caches take the path opposite from state replication to achieve cache coherency. Upon every data modification, a small invalidation message containing object key is sent to required nodes in order to effectively evict dirty references to the modified object from the cluster. The first following read request after invalidation will reload object from the underlying storage. Non-replicable caches can only be used for data backed by some secondary storage since the implied design requires reloading the state.

By minimizing size of invalidation messages non-replicable approach offers several benefits over replicable one since short invalidation messages allow for negligible serialization/de-serialization and impose minimal load on the network. Loading objects on demand allows for every node to cache data that has only been locally accessed and therefore minimizing heap utilization. In addition, since invalidation messages are small and fixed in size, they can be comfortably transmitted over lightweight UDP or IP-multicast protocols.

 Top

JCache
While not a standard yet, it is publicly known that JCache is based on java.util.Map interface. Using map-like interface for cache is natural and minimizes learning curve for end users since everyone is familiar with java.util.Map API. xTier cache is also based on java.util.Map and comes with 2 interfaces that closely resemble it: Cache and JCache. Cache interface does not extend java.util.Map, while JCache does. The reason for it is that given distributed nature of cache implementation, many operations on cache may fail and therefore throw an exception, however java.util.Map interface does not allow throwing any checked exceptions. That's why Cache interface has methods almost identical to java.util.Map methods but with added CacheException. Users who don't mind unchecked runtime exceptions may use JCache interface which converts checked CacheException into unchecked JCacheException. Note that both interfaces provide many methods in addition to the ones specified in java.util.Map interface. Also note tnay any cache defined in XML configuration can be obtained as Cache or JCache.

Main Features
Main features of 'cache' service include:

• Support for read-through and write-through behavior. Read-through behavior means that whenever objects are not in cache they are automatically loaded from secondary storage. Write-through behavior means that whenever an object is created, updated, or removed all modifications are automatically propagated to the secondary storage. See CacheLoader and CacheStore for details on how read-through and write-through semantics are supported.
• Based on java.util.Map interface just like JCache JSR 107.
• Support for ACID Two-Phase-Commit optimistic transactions with READ_COMMITTED and SERIALIZABLE isolation levels. Note that all transaction concurrency issues are resolved at application level. This allows, for example, to use database in light-weight READ_COMMITTED mode while application can still enjoy all flavors of SERIALIZABLE isolation level by utilizing cache transactions. For more information about cache transaction support see CacheTx documentation.
• Support for per-transaction caches. Objects modified within the same cache transaction can be retrieved from cache instead of going to a database.
• Support for pluggable expiration policies. xTier cache comes with four expiration policies out-of-the-box:
  • CacheAgeExpirationPolicy
  • CacheIdleExpirationPolicy
  • CacheLruExpirationPolicy
  • CacheLfuExpirationPolicy
  User is free to implemenent any custom expiration policy if none of the ones listed above fits. See CacheExpirationPolicy for more information about cache expiration policies.
• Support for pluggable cache topology. Cache topology is used for resolving to which nodes to send transaction invaidation messages. Providing ability to specify on which of the cluster nodes any cache object may reside singificantly reduces network load and eliminates single-point-of-bottleneck danger. See CacheTopology for more information about cache topology resolution.
• Out of the box support for caching query results. By properly assigning depended attribute to cache keys, cached query results will be invalidated automatically every time any of the constituent objects is modified or removed. See CacheKeyAttrs for more information about cache key attributes.
• There is no need to implement any interfaces for cache keys or cache values. Cache API works with Object just like java.util.Map interface.
• Lots of other goodies, such as statistical information on cache as a whole or any of the cached objects, subscribing for notifications to cache events and cache transaction events, support for bulk loading, bulk updates, and much more.

 Top

Cache Key Attributes
All keys in xTier cache must have key attributes assigned to them. Key attributes are specified with CacheKeyAttrs interface and are assigned to every cache key via CacheKeyAttrsResolver implementation provided by user in XML configuration. Cache implementation will ask CacheKeyAttrsResolver to provide key attributes for every cache key.

Cache key attributes are composed of 3 parameters:

• Type ID - type IDs are provided only for the purpose of minimizing hash-map collisions within internal cache storage. If user knows that 2 objects stored in cache belong to different types/Java-classes, then to improve performance their corresponding cache key attributes should have 2 different type IDs.
• Group ID - the way xTier cache works is that it allows to break all cache keys into 'groups' that are independent from one another. The main purpose of groups is to control the scope of updates. An update of an object belonging to some group can only affect one or more objects belonging to the same group – objects belonging to other groups will not be affected. Any cache key can only belong to one group.
• Depended-Flag - cache groups can have 'depended' and/or 'non-depended' objects within them. The contract is that whenever any object belonging to some cache group is updated, all 'depended' objects within that group will be invalidated (nullified). In a most common use case, 'non-depended' objects are used to cache single non-aggregate objects, and 'depended' objects are used for caching aggregate objects, such as query results.

Whenever an update happens to object A, all instances of object A on all nodes will be invalidated (nullified). Additionally, all instances of object C will be invalidated as well, because its key is specified as 'depended'. Object B will not be affected by updates to object A.

 Top

Configuration
'cache' service is configured via pre-defined xtier_grid.xml configuration file. This file follows standard xTier™ service configuration pattern that can be demonstrated by the following complete example of cache configuration (from examples):

1		<xtier-cache>
2		<region name="examples">
3		<!--
4		Cache start-up configuration.
5		*****************************
6		On start up  cache needs to coordinate with
7		other nodes in the cluster to avoid stale data.
8		port- TCP/IP port used for start up coordination.
9		-->
10		<startup port="64000"/>
11
12		<!--
13		Cache settings.
14		***************
15		name - Optional attribute to assign a name to
16		the cache. If omitted then this cahe is
17		assumed to be the default cache.
18		init-size - Initial size of the cache.
19		-->
20		<app-cache init-size="1000">
21		<!--
22		Invalidation related definitions that
23		define network settings used within
24		two-phase commit synchronous transaction
25		protocol.
26
27		Attributes:
28		***********
29		default-tx-timeout - Optional parameter,
30		default is 10000 ms.
31		-->
32		<app-tx>
33		<!--
34		Name of the thread pool dedicated for
35		cache message processing.
36		Note that this thread pool must be defined
37		in 'objpool' XML configuration. It is
38		recomended the 'grow' (default) policy will
39		be used for thread pooling to allow thread
40		pool to automatically adjust to required size.
41		-->
42		<thread-pool-name>
43		cache.thread.pool
44		</thread-pool-name>
45
46		<!--
47		Phase-one settings.
48		******************
49		port - Port to receive UDP replies on.
50		-->
51		<phase-one reply-port="20001">
52		<!--
53		UDP channels.
54		*************
55		port - UDP port for channel.
56		min - Minimum number of objects invalidated
57		on this channel.
58		max - Maximum number of objects invalidated
59		on this channel.
60		timeout - Time in ms. that sender will wait
61		for response when sending cluser-wide
62		notifications.
63		attempts - Number of attempts to send
64		cluster-wide notifications
65		in case if sender times out waiting for
66		reply.
67		-->
68		<channel port="20003" min="1" max="10"
69		timeout="1000" attempts="2"/>
70		<channel port="20004" min="11" max="20"
71		timeout="1000" attempts="2"/>
72		<channel port="20005" min="21" max="30"
73		timeout="1000" attempts="2"/>
74		</phase-one>
75
76		<!--
77		Phase-two settings.
78		*******************
79		port - UDP port for phase-two requests.
80		-->
81		<phase-two port="20002"/>
82
83		<!--
84		Distributed Garbage Collector (DGC) settings
85		to collect remote transactions.
86		********************************************
87		dgc-port - UDP port to which to send DGC
88		requests.
89		reply-port - UDP port on which to expect
90		replies to DGC requests.
91		timeout - Time in ms that DGC will wait for
92		response from other cluster nodes when
93		inquiring about a transaction before
94		collecting it.
95		tx-age - Optional parameter. Transaction age
96		at which it will be checked by DGC,
97		default is 5000.
98		-->
99		<dgc dgc-port="20006" reply-port="20007"
100		timeout="1500"/>
101		</app-tx>
102
103		<!--
104		Cache key attributes resolver used for cache example.
105		-->
106		<cache-key-attrs-resolver>
107		<ioc policy="singleton">
108		<java class="com.fitechlabs.xtier.examples.
109		services.cache.CacheExampleKeyAttrsResolver"/>
110		</ioc>
111		</cache-key-attrs-resolver>
112
113		<!--
114		Basic cache-topology which always returns all
115		non-local nodes in the cluster.
116		-->
117		<cache-topology>
118		<ioc policy="singleton">
119		<java class="com.fitechlabs.xtier.services.
120		cache.topology.CacheBasicTopology"/>
121		</ioc>
122		</cache-topology>
123
124		<!-- Cache expiration policy used for cache example. -->
125		<cache-expiration-policy>
126		<!-- LRU expiration policy. -->
127		<ioc uid="lru.expiration.policy" policy="singleton">
128		<java class="com.fitechlabs.xtier.services.cache.
129		expiration.CacheLruExpirationPolicy">
130		<ctor>
131		<!-- Initial size. -->
132		<arg type="int32">200000</arg>
133		<!-- Maximum size. -->
134		<arg type="int32">1000000</arg>
135		<!-- Access order. -->
136		<arg type="boolean">true</arg>
137		</ctor>
138		</java>
139		</ioc>
140		</cache-expiration-policy>
141
142		<!--
143		Cache store implementation used for cache example.
144		-->
145		<cache-store>
146		<ioc policy="singleton">
147		<java class="com.fitechlabs.xtier.examples.
148		services.cache.CacheExampleStore"/>
149		</ioc>
150		</cache-store>
151
152		<!--
153		Cache spooler implementation used for cache example.
154		-->
155		<cache-spooler>
156		<ioc policy="singleton">
157		<java class="com.fitechlabs.xtier.examples.
158		services.cache.CacheExampleSpooler"/>
159		</ioc>
160		</cache-spooler>
161		</app-cache>
162		</region>
163		</xtier-cache>

Formal sepcification for this service configuration can be found in xtier_cache.dtd file in ${XTIER_ROOT}/config/dtd folder. Generally, 'cache' service configuration consists of one <startup> XML tag element that defines TCP port for coordination of cache service startup on remote nodes and <app-cache> element that describes common application cache parameters and all pluggable ioc objects.

startup	This element specifies IP port that is used by cache service implementation in order to coordinate startup of cache service on remote nodes and avoid data integrity issues.
app-cache	This element specifies thread pool used for invalidation processing, transactional settings, and all pluggable IoC elements. As a whole, if defines all configurational setting for every cache in the system. Note that it is possible to have more than on cache configured in the system.

<app-cache> has the following attributes and subelements:

init-size	This attribute defines the initial size of the cache. Note that maximum cache size is controled through cache expiration policy. For more information about cache expiration policy refer to CacheExpirationPolicy documentation.
app-tx	This element defines configuration settings for cache 2-phase-commit transactions.
cache-key-attrs-resolver	This required element defines IoC object used to resolve cache key attributes. See IocService for more details on IoC usage. For more information about cache key attributes and how they get resolved refer to CacheKeyAttrs and CacheKeyAttrsResolver documentation.
cache-topology	This required element defines IoC object that represents topology resolver for cache service. See IocService for more details on IoC usage. See CacheTopology documentation for more information about what cache topology is.
cache-expiration-policy	This optional element defines IoC object that represents expiration policy utilized by cache. Note that although expiration policy is optional element, it is highly recomended that cache utilizes expiration policy of some sort. If no expiration policy is used user must take other precautions to make sure that cache size does not grow indefinitely. See IocService for more details on IoC usage. See CacheExpirationPolicy documentation for more information about what cache expiration policy is.
cache-spooler	This optional element defines IoC object that represents cache spool storage. Objects can be moved into spool storage after they expire in cache. Note that spool storage access (usually, local disk) is presumably much faster than secondary storage such as database. See IocService for more details on IoC usage. See CacheSpooler documentation for more information about cache spooler.
cache-loader	This element defines IoC object that represents cache loader. Whenever objects are not found in cache, they are loaded from secondary storage (e.g. database) using cache loader. Such behavior is usually called read-through behavior. Note that only one, cache loader or cache store, but not both, IoC objects may be specified since semantic of cache store include cache loader behavior. See IocService for more details on IoC usage. See CacheLoader documentation for more information about cache loader.
cache-store	This element defines IoC object that represents cache store. . Whenever objects are loaded, created, modified and updated the operation is propagated to a secondary storage (database) using cache store implementation. Note that cache store is used to ensure read-through and write-through behavior of cache. Also note that only one, cache loader or cache store, but not both, IoC objects may be specified since semantic of cache store include cache loader behavior. See IocService for more details on IoC usage. See CacheStore documentation for more information about cache store.

<app-tx> has the following subelements:

thread-pool-name	This element specifies the name of the thread pool used to process cache ivalidations. Note that thead pool is configured in object pool service XML configuration. See ObjectPoolService documentation for details about configuring thread pools. Note that it is recomended to use ThreadPoolStrictPolicy when configuring thread pool so cache load will be controlled. Priority of threads in the pool should also be set to the highest possible value. It is recomended to set thread priority to Thread#MAX_PRIORITY whenever possible.
phase-one	This element specifies configuration properties for phase one of cache 2-phase-commit (2PC) transactional protocol. During phase-one a 'prepare' request is sent out to the nodes participating in transction and reply must be received from all of them. Hence 'reply-port' attribute is used to specify on which port cache implementation should listen to phase-one replies. Since UDP messages must be of fixed size, but number of objects participating in transaction may vary, notion of 'channel' was introduced. Every channel is configured to receive different range of objects. Channel configuration parameters include: port - Port on which to receive messages designated to this channel. min - Minimum number of objects invalidated on this channel. max - Maximum number of objects invalidated on this channel. timeout - Timeout in milliseconds that sender will wait for reply after seding a message to this channel. attempts - Number of attempts to send a message to this channel in event if the sender times out waiting for response.
phase-two	This element specifies configuration properties for phase two of cache 2-phase-commit (2PC) transactional protocol. During phase-two a 'commit' or 'rollback' message is sent to all nodes participating in transaction. Note that in phase-two there is no need to wait for reply. If message does not reach any of its destination, then Distributed Garbage Collector will detect it and complete the transaction. Port configuration attribute specifies the listening port for phase-two messages.
dgc	This element specifies configuration properties for cache Distributed Garbage Collector (DGC). Since phase-two transaction messages are asynchronous and UDP is unreliable, it is possible that some nodes sometimes will not get the phase-two messages for some ongoing transaction. In this case, DGC will detect it and ask the originating node if the transaction is still active. If transaction is not active on originating node, then DGC will complete the transaction. DGC is configured using following configuration properties: dgc-port - Port to which DGC requests are sent. reply-port - Port on which replies to DGC requests are received. timeout - Time in milliseconds that DGC will wait for response from cluster node that originated the transaction in question. tx-age - Optional parameter. Transaction age at which it will be checked by DGC, default is 5000.

 Top

Examples
Usage of 'cache' service follows the standard pattern of using xTier™ service: you need to obtain an instance of xTier™ kernel that serves as a service registry. Once you have xTier™ kernel you can get an instance of any service, in our case the cache service. Once the service instance is obtained the service API can be used.

Note that usage of 'cache' service depends on 'object pool' and 'cluster' services. See ObjectPoolService and ClusterService services for details on their usage.

Following code snippet is taken out from cache service example:

1		private static void example() {
2		// 1. Get a handle on default cache instance.
3		Cache cache = XtierKernel.getInstance().cache().
4		getCache();
5
6		CacheTx tx = null;
7
8		try {
9		// 2. Start serializable cache transaction.
10		tx = cache.startTx(CacheTx.SERIALIZABLE);
11
12		// 3. Retrieve single orders from cache.
13		CacheOrder order1 = (CacheOrder)cache.get(
14		new CacheKey(1, 1));
15		CacheOrder order2 = (CacheOrder)cache.get(
16		new CacheKey(1, 2));
17		CacheOrder order3 = (CacheOrder)cache.get(
18		new CacheKey(2, 3));
19
20		logger.log("Retrieved order from cache: " +
21		order1);
22		logger.log("Retrieved order from cache: " +
23		order2);
24		logger.log("Retrieved order from cache: " +
25		order3);
26
27		// 4. Retrieve query result from cache.
28		List orders = (List)cache.get(new CacheKey(1,
29		"user_id=1 and qty > 5"));
30
31