Here's an interesting take on how an enterprise application can be made to achieve high performance, scale and high availability, using mid-tier caching.
Modern enterprise applications are being pushed to become faster, with a higher scale. Consequently, the 3-tier architecture model is changing to move data closer to the applications and to integrate data from several sources.
In addition, requirements of a higher scale are forcing applications to tolerate staleness and move towards weaker forms of consistency. These changes are being facilitated by:
- Hardware trends in 64-bit architecture, multi-core processors and inexpensive memory
- Ubiquitous data ‘in motion and everywhere'; and
- More and more enterprise applications require a faster and inexpensive scale, along with resilience to failure.
Welcome to the world of ‘mid-tier caching', also known as ‘cache tier' or simply ‘caching'.
The next generation applications are composite, aggregating data and business logic from sources that can be local, federated, or cloud-based. Data and applications can reside in different tiers with different semantics and access patterns. For example, data in back-end servers/clusters or in the cloud tends to be authoritative; data on the wire is message-oriented; data in the mid-tier is either cached data for performance or application session data; and data on the devices could be local data or data cached from back-end sources.
With the costs of memory going down, fairly large caches can be configured on the desktop and server machines. With the maturity of 64-bit hardware, 64-bit CPUs are becoming mainstream for client and server machines. True 64-bit architecture support dramatically increases memory limits (to bytes). For example, desktops can be configured with 16 GB (gigabyte) RAM, and servers can be configured with up to 2 TB (terabyte) of RAM. Large memory caches allow for data to be located close to the application, thereby providing significant performance benefits to applications. In addition, in a world where hundreds of gigabytes of storage is the norm, the ability to work with most data in memory (large caches) and easily shift from tables to trees and on to graphs of objects, is key to programmer productivity for next-generation applications. While the query-based access is a requirement, the data access, management requirements and semantics of caches are different from what commercial DBMSs support.
Caching in applications As described earlier, data can reside in different tiers (in different service boundaries) with different semantics. For example, data stored in the backend database is authoritative and requires a high degree of data consistency and integrity. Typically, there tends to be a single authoritative source for any data instance. Most data in the mid-tier, being operated by the business logic, can tend to be a copy of the authoritative data. Such copies are suitable for caching. Understanding the different types of data and their semantics in different tiers, helps define the different degrees of caching that is possible.
Reference data
Reference data is a version of the authoritative data. It is either a direct copy (version) of the original data or aggregated and transformed from multiple data sources. Reference data is practically immutable-changing the reference data (or the corresponding authoritative data) creates a new version of the reference data. This means that every reference data version is unique. Reference data is an ideal candidate for caching; as the reference data does not change, it can be shared across multiple applications (users), thereby increasing the scale and performance. Consider a product catalogue application aggregating product information across multiple backend applications and data sources. 
The most common operation on the catalogue data is read (or browse). A typical catalogue browse operation iterates over a large amount of product data, filters it, personalises it, and then presents the selected data to the users. Key-based and query-based access is a common form of operation. Caching is a critical requirement for catalogue access. If not cached, operations against such an aggregate catalogue require the operations to be broken up into operations on the underlying sources -- invoke the underlying operations, collect responses, and aggregate the results into cohesive responses. Accessing the large sets of backend data for every catalogue operation can be prohibitively expensive, and can significantly impact the response time and throughput of the application. Caching the backend product data closer to the catalogue application can significantly improve the performance and the scalability of the application. Similarly, aggregated flight schedules are another example of reference data.
Referenced data is refreshed periodically, usually at pre-configured intervals, from its sources, or refreshed when the authoritative data sources change. Access to reference data, though shared, is mostly read. Local updates are often performed for tagging (to better organise the data). To support a large scale, reference data can be replicated in multiple caches on different machines in a cluster. As shown below, in a social networking scenario, details of the friends list, usernames associated with a particular ID/user login, etc would be such reference data that is easy to cache and provides high scalability.
Activity-oriented data
Activity-oriented data is generated by the currently executing activity as part of a business transaction. The data originates as part of the business transaction and, eventually, at the close of the business transaction, it is retired to the backend data source as historical (or log) information. Consider the shopping cart data in an online buying application. There is one shopping cart, which is exclusive for each online buying session. During the buying session, the shopping cart is cached and updated with the products purchased. The shopping cart is visible and accessible only to the buying transaction. Upon checkout, once the payment is applied, the shopping cart is retired (from the cache) to a backend application for further processing. Once the business transaction is processed by the backend application, the shopping cart information is logged for auditing (and historical) purposes.
While the buying session is active, the shopping cart is accessed both for read and write operations; however, it is not shared. This exclusive access of the activity data makes it suitable for distributed caching. To support large scalability of the buying applications, the shopping carts can be distributed across the cluster of caches. Since these carts are not shared, the set of shopping carts can be partitioned across the distributed cache. By dynamically configuring the distributed cache, the degree of the scale can be controlled.
Resource-oriented data
Both reference (shared read) and activity (exclusive write) data are ideal for caching. However, not all application data falls into these two categories. There is data that is shared, concurrently read and written into, and accessed by a large number of transactions. For example, consider an inventory management application. The inventory of an item has the description of the item and the current quantity. The quantity information is authoritative, volatile, and concurrently accessed by a large number of users for read/write operations. Such data is known as the resource data; the business logic (e.g., the order application logic) runs close to the resource data (e.g., quantity data). The resource data is typically stored in the backend data stores. However, for performance reasons it is cached in the application tier. While caching the quantity data in memory on a single machine can provide performance improvements, a single cache cannot provide availability or scale when the order volume is high. For this purpose, the quantity data can be replicated in multiple caches across the distributed cache. However, since quantity is shared and concurrently updated, the consistency of the quantity values need to be maintained across the distributed cache. Maintaining consistency in distributed caches requires data replication, transactions, data change notifications, and invalidations. However, support for high performance and scale requires relaxed notions of consistency and transaction. For example, the seat inventory in a flight booking application is an example of resource-oriented data.
The following table illustrates some real-world application domains and scenarios that use distributed caching for application performance and scale. In this table, the application space is divided into Web and LOB (line of business) applications. Most Web applications use caches for reference and activity data, and occasional resource data (e.g., inventory). Enterprise Lines of Business are also increasingly leveraging distributed caches for reference and activity data scale up, to reduce the load on the backend authoritative database system.
Horizontal Scenarios
Web
|
User specific HTTP sessions. In-flight shopping carts for Web retail enabling online self service applications. Explicit storage of pre-computed or highly accessed data
|
| LOB |
Enterprise product catalogue for POS (point-of-sales), analytics caching, frequently used reference data for ERP (enterprise resource planning) applications |
This table gives examples of verticals where distributed caches are becoming essential.
Verticals
| Telco |
Cellular VoIP (voice over IP); compute utilisation, prepaid charges, call routing and session info. SMS; message content, notification, receipt, billing |
| Travel |
Aggregated flight pricing, availability of seats |
| Defence |
Sensor network data processing and threat deduction |
| Finance |
Per user portfolio data and delayed quote storage for trading. Aggregate and process ticket stream for algorithmic trading |
Competitive offeringsDistributed caching for application performance and scalability is not new. There are open source and commercial cache vendors in the market, particularly Java caches. JCache (JSR 107) is a specification targeting Java applications. By far, MemcacheD is one of the most widely used scalable cache for Web applications. It is an open source component and is typically used in the LAMP/J stack. MemcacheD is essentially an implementation of a distributed hash table across a cluster of machines, each with large amounts of main memory. It supports only object get and put operations; there is no support for transactions or query. Also, MemcacheD does not provide any support for availability.
Microsoft info bytes Microsoft's project, code named ‘Velocity', offers distributed caching functionality like competitive products, yet differs in the following ways:
1. Microsoft envisions Velocity to be a distributed application platform rather than just a cache. As applications become more data-centric, the company believes Velocity will evolve to providing additional data services like analytics, reporting, synchronisation, integration, and so on. 2. Velocity has been developed in C#, targeting the .NET applications. It is architected to take advantage and tightly integrate with a variety of .NET mechanisms like LINQ (Language Integrated Query), PLINQ (Parallel LINQ), ADO.NET Entity Framework, etc, to provide a broader distributed .NET application platform. 3. Unlike most of its competitors, Velocity is architected with a very large scale (1000s of cache instances) in mind. Microsoft believes distributed caches will be used from the enterprise, the Web and in cloud computing.
|
Most Web applications primarily use MemcacheD for caching large amounts of reference data. For high availability, these applications build custom solutions. Oracle (Tangosol) Coherence, Gemstone Gemfire are two of the leading cache providers in the enterprise application space. Coherence is a Java-based distributed cache that is highly scalable and available for enterprise applications. Like MemcacheD, Coherence supports a DHT (distributed hash table) for scalability. However, unlike MemcacheD, Coherence provides high availability by implementing replicated, quorum-based data consistency protocols. Coherence also supports distributed notifications and invalidations. It does not offer database capabilities like rich data modelling, rich data types and query. Gemfire, in addition to its enterprise cache capabilities, provides support for an SQL-like query over the cache.
References 1. MemcacheD - The open source distributed cache; http://www.danga.com/memcached/ 2. Java Caching API - https://jsr-107-interest.dev.java.net/ 3. Oracle Coherence - Java-based distributed cache; http://www.oracle.com/technology/products/coherence/index.html 4. Gemstone Gemfire enterprise data fabric; http://www.gemstone.com/products/gemfire/ 5. Microsoft Linq Project - http://msdn.microsoft.com/en-us/vbasic/aa904594.aspx 6. Oracle TimesTen - http://www.oracle.com/database/timesten.html 7. Azure Services Platform - http://www.microsoft.com/azure/default.mspx 8. Windows HPC Server - http://www.microsoft.com/hpc/en/us/default.aspx
|
In short, ‘distributed caching' can provide the following significant benefits: a) Performance, by storing application instances that may have been created from complex joins or integrating multiple data sources. Thus it can improve performance of applications since it can cache these data in memory b) It scales linearly as it uses a portioning mechanism to partition data. For example, adding three machines improves throughput nearly three times. In addition, adding new machines can increase the overall memory available, thus improving the cache hit ratio; and c) It enables high availability by keeping multiple copies of data around, thus ensuring that a process or machine failure does not lose data. This allows applications to be resilient to machine failures. The author is a program manager with 'Embedded Databases Group' at Microsoft India Development Center. He has over nine years of industry experience with over five years in Microsoft. He is a member of the team working on Microsoft project codenamed "Velocity"--which is a distributed data cache solution from Microsoft. He is very passionate about business solutions, database and systems domain, at the same time solving tough customer and business problems with the magic of software.
|