Terracotta Network-Attached Memory for Web App Session Clustering and Replication
By Matthew Sacks
Author Bio: http://matthewsacks.com/technical-articles/

Terracotta provides a service called Network-Attached Memory (NAM) for JEE applications. Similar to a distributed cache, NAM provides object-caching and session-clustering features. However, Terracotta sets itself apart from traditional Java distributed caches through its seamless integration with most Java application servers and frameworks and its ability to cache the entire framework and code, as well as provide session clustering and replication features on a level of high availability that is otherwise difficult to attain with such finesse and ease. This article demonstrates some of Terracotta’s session-clustering and high-availability features, using a demo application that the Terracotta engineers have created, called the Examinator.

The Examinator Demo Application

Terracotta recently released its Examinator application, which demonstrates high-performance application scaling. The application is based on real-world engagements, modeling a real-world JEE application that uses Spring and Hibernate. This application has been proven to scale out to 20,000 concurrent users, with an average response time of no more than 5 ms per request on only 16 application servers!

These are impressive metrics for any application farm. Because I’m interested in application performance, I decided to investigate Terracotta’s open source application-caching and clustering solution.

What is covered in this article, and what will the reader achieve by reading the article?

In this review, we will demonstrate how to set up and configure the Examinator application, which is a great way to test some of Terracotta’s capabilities. This is impressive enterprise infrastructure, representing a new breed of enterprise Java clustering and caching solutions. This article presents an introduction to Terracotta by providing a tutorial on using the Examinator application and its configuration. By following this tutorial, the user will become familiar with Terracotta and some of its features.

Article Scope

This article does not provide an analysis of Terracotta’s performance metrics. The focus of this article is Terracotta’s high-availability and session-clustering/replication features. Terracotta is capable of increasing application performance on a large scale due to its tight integration with the Java Virtual Machine itself and the sophisticated Terracotta Server Array technology. For more specific benchmarks on Terracotta’s performance increases, please refer to the Terracotta web site.

Although performance is not the focus of this particular review, a quick load test will reveal impressive performance results. (Stay tuned on The Bitsource for more information about Terracotta’s abilities in improving application performance.)

Terracotta in Three Quick Steps

1. Get Terracotta.
2. Get the Examinator application.
3. Set up a MySQL database and run the included schema-creation scripts.

Downloading and Configuring Terracotta

Because Terracotta is a Java-based application, there is little to do in terms of compilation and installation. All the user has to do is unzip the package and design an application to make use of Terracotta’s features. The installation of Terracotta itself is rather trivial.

NOTE: This tutorial assumes that the user already has a running MySQL 5+ server. For more information on configuring MySQL, see the “Further Reading” section of this article.

Step 1) Unzip the Terracotta package:

Step 2) Import the MySQL schema from conf/schema.sql
Step 3) Set the TC_INSTALL_DIR environment variable to the Terracotta server installation directory
Step 4) Set your JAVA_HOME environment variable to a 1.5+ JDK
Step 5) Execute bin/tim-get.sh -all

Installing and Configuring Examinator

The Examinator application has been designed to demonstrate Terracotta’s advanced high-availability and session-clustering features in a practical, real-world scenario: delivering online exams. Install and configure the Examinator to get it up and running by following these simple steps:

Step 1) Unzip the Examinator application binaries:

Step 2) Change the directory to the extracted Examinator directory:

Configuring Application Persistence

In order to save all of the session data to disk in the event of a failure, we can configure the application to utilize the Terracotta persistence feature. By default, the Examinator application does not enable this feature, so we need to make a simple change to the tc-config.xml that ships with Examinator to include the following lines:

<dso>
<persistence>
<mode>permanent-store</mode>
</persistence>
</dso>

Starting the Terracotta Server Array and Application Servers

Now that we have made the appropriate changes to the Examinator configuration, we can start the Server Array instance and application servers.

Step 1) Start the Server Array instance:

Step 2) Start the Examinator application servers:

Testing Session Clustering and Replication with Terracotta

On the command line, verify that both Examinator nodes are up and running. Make note of the process IDs for the Jetty application server running on port 8080 and the other application server running on port 8081. We will shut down one of these nodes to demonstrate the session-clustering and replication features:

Point a browser at http://localhost:8080/examinator/login.do and log into the Examinator with the username and password student1:passwd1.

The following screen will allow you to take another exam, or display previous exam results.

Copy the URL for the Student welcome screen, http://localhost:8080/examinator/welcome.do, into another browser tab, and change the port to the other Examinator application server. The user will not be prompted to log in. The session is automatically replicated over to the other Examinator application server because it is making use of the Terracotta session-clustering features.

Now change the URL to point to the same URL on the other application server:

http://localhost:8081/examinator/welcome.do.

The same screen appears, and the user will not notice a difference, as the same user session is replicated seamlessly.

To demonstrate the high-availability features of Terracotta, start taking an exam by clicking on the Terracotta exam, and then click the Take Exam link to start the exam.

Click an answer, and mark the question for review. Mark an answer for one or two exam questions.

Notice that the marked question for review displays in the left-hand column. The marked questions for review are stored in the user’s session context, and are replicated in the cluster to ensure high availability of the user’s session.

Copy the URL that is currently listed in the browser address bar. Then kill the process that is running the application server on port 8080.

Change the URL to point to the second application server, now that the application server on which the user was taking a test with the Examinator has “crashed.”

The same session remains, and there has been no loss of data.

All of the session data has been flushed to disk. Terracotta uses a hybrid memory model, similar to the Unix Virtual Memory system, which caches objects that are most frequently used, and flushes other data to disk. On a failure, session information and object caches are not lost, even in the event of an entire grid failure.

Shut down the other application server. If this had been a production site, the entire application would be offline.

Get the process ID for the last remaining Examinator application running on port 8081, and issue a kill command to take the service offline entirely – grid, application, database, and everything.

Kill the last application server:

Kill the database:

What happened to the user’s session? After all, nobody likes taking tests. The test taker has invested a great deal of time and effort into taking this exam, and cannot possibly remember all of his or her responses if he or she would have to retake the test. Fortunately, all of the session data is saved in the grid. In the event that the grid would fail at this point, there still would be no loss of data, because persistence is enabled.

Start the application:

Now try hitting the same URL that was in the browser when the service went offline. We can even change the port to the other Examinator node by changing the port from 8081 back to 8080:

http://localhost:8080/examinator/flow/takeexam.do?execution=e1s1

The session and data are still intact, even after a failure of the database.

You can even verify that the cache is active, by using the Terracotta dev console.

Click Connect to log into the dev-console.

Drill into the clients section of the Terracotta admin console, and the user can see the objects being cached, check runtime performance metrics in real time, and take thread dumps for debugging analysis.

Further Reading

Anyone can download and demo the ready-made Examinator application from Terracotta.org to evaluate the application architecture for his or her needs (or, as in my case, simply out of curiosity), and test drive it to see what it can do.

This article presents a brief introduction to Terracotta and only touches on some of its reliability and clustering features. Terracotta has many rich features for providing high availability, reliability, scalability, and performance increases for enterprise applications. To learn more about the Terracotta server array architecture, see the following document: http://www.terracotta.org/web/display/docs/Terracotta+Server+Arrays

To get the Terracotta Server and Examinator application:
Go to http://www.terracotta.org/web/display/orgsite/Download to download Terracotta.

Examinator Quick Start Guide: http://www.terracotta.org/web/display/orgsite/Exam+App+Quick+Start

MySQL documentation: http://dev.mysql.com/doc/

Installing MySQL: http://dev.mysql.com/doc/refman/5.1/en/installing.html

FAQs

What does tim-get.sh do?
tim-get.sh is an installation package manager, like RPM for Red Hat. Terracotta integrates with various programming frameworks and provides additional functionality through packages (plugins) that are called Terracotta Integration Modules, hence the name “tim”. tim-get manages installing and updating these TIMs on the end user’s local machine.

Without the Terracotta server, are the TC jars providing replication features and object caching?
Not exactly. Terracotta is designed to always work with a Terracotta Server Array. One or more instances in the Terracotta Server Array may be configured to provide (independent) reliability (disk persistence), high availability (replicated servers and automated failover), and scalability (additional throughput). The client jars provide automated connectivity to the Terracotta Server Array. The Terracotta clients (the client jars) work with the Terracotta Server Array to manage coordination and replication.

Does starting the Terracotta server only provide session replication?
As indicated above, it’s an integral part of the clustering solution.

Do you still need hardware load-balancers in light of TC clustering features?
It depends on your use case, so not always, but usually yes. Especially for a web application – you need some kind of load-balancing strategy to route incoming requests to different application servers.