Finally, we have reached the final blog in this series about the new features in SDL Tridion Sites 9. As you might have noticed, there is a gap between the previous post and this one, the reason being that we (EXLRT) have attended the annual Tridion Developer Summit in Amsterdam which was held in the meantime, and my focus was on preparing my presentation for the conference. Speaking of which, I intend to write a blog about TDS/my presentation so stay tuned.

Bye-bye Fredhopper

As the title says, one of the biggest changes in XO 9.0 is replacing the indexing/querying engine, Fredhopper with Elasticsearch. Although Fredhopper was capable of performing that role, there were numerus reasons for doing so. Fredhopper was once owned by SDL, but since then it was divested, and additionally, only a fraction of its capabilities were used for the XO purposes in the first place. It was somewhat complicated to install, configure and maintain (some configuration changes needed to be applied manually to all instances in the cluster, etc.). The other big reason is that Elasticsearch is already used in Docs to provide search functionality and the same is planned for future releases of Sites. Having both search and personalization functionalities on the same technology stack is definitely the right choice. Elasticsearch is a modern open source engine which means it’s a good foundation for all the future goodies planned for us, and maybe most important of them all, it is much more scalable than Fredhopper.

You can find more information about Elasticsearch here.

Architecture

For comparison reasons, both the old and new architecture of XO is shown below:

By comparing the two, we can observe the following:

• Fredhopper is completely replaced
• There is a single logical entity for both indexing and querying the items, the Elasticsearch cluster VS Fredhopper’s multiple, indexer instance and query instance(s)
• There is no need to have an additional entity to forward the index data (in Fredhopper this was the SmartTarget Deployment Webservice). The index data is directly sent by the Deployer extension
• There are more scaling options with Elasticsearch

Although the architecture is somewhat different, some things didn’t change:

• The XO Management and Query functionality is the same
• The XO UI Extension is the same
• The API’s are backward compatible

 Additional comparison

SmartTarget UI

The UI has not changed and behind the scenes it still uses the XO management service to store XO data, but this time in Elasticsearch. The same item types are still present and behave exactly the same way:

• Promotions and Experiments
• Triggers and Trigger Types
• Regions
• Content (dynamic component presentations- DCPs)

Deployment

The Deployer still deploys active DCPs, and their folders or other hierarchies into Elasticsearch. One difference is that the configuration of all configurable items is now centralized. No more adjusting the Regions and Trigger-Types on each and every server. Finally! :)

Fredhopper

Because Fredhopper is gone, so is the Business Manager (BM). The main usage of the BM was for putting attributes live so that they could be used in Triggers, but this is no longer needed. Elasticsearch for XO is configured in such a way that it can cope with large amounts of data and putting attributes live is no longer relevant. Fredhopper provided us with two additional functionalities, and their statuses are listed below:

• Search – Deprecated
• Navigation – Dropped

Querying

The QueryBuilder API, which hasn’t changed, is the only supported official API from now on, .NET Server Controls and Java Tags work as is. Using the QueryBuilder API and Query clients, it’s possible to query the Query Service (recommended) or Elasticsearch directly. Querying the Query Service is the recommended approach because the service acts as a load balancer for the queries and it holds additional processing which would otherwise needed to be embedded in the web application, causing performance hits for each request that deals with XO.

Upgrade

To upgrade, you need to perform the following steps:

• Install Elasticsearch
• Migrate data from Fredhopper to Elasticsearch. At this point you already have access to the CM XO item types (Promotions, Experiments, etc.).
• Recompile web apps which are using XO against the new dependencies that have been introduces in the Sites 9 release

Migrating Fredhopper data

As we’ve seen earlier, the second step in an upgrade scenario is migrating data from Fredhopper to Elasticsearch, and SDL has built a dedicated tool for this purpose which will migrate all the data, including:

• Promotions
• Experiments
• Triggers
• Trigger Types
• Attributes
• Regions
• Content

The tool is a Java jar file and can be ran multiple times, but in order to be able to export data from Fredhopper, an extension to FH is needed. The process is quite simple, basically it consists of copying a jar file (also supplied by SDL) in the FH indexer and restarting the instance. After the extension is installed, it’s possible to export the data by using the following command:

Importing the data is done using the following command:

The config parameter is the path to the config file which needs to be edited with configuration related to your environment prior to running the export/import commands.

The image below shows the migration process:

And with this we have reached the end of our journey for now. Looking back overall, there are a looooot of nice features and functionalities which I’m eager to start playing around as soon as possible. I can’t wait for Sites 9 to be out to try some of them for myself. As always, if you have any questions or would like to share your thoughts, don’t hesitate to get in contact.

Disclaimer: All images are copied from SDL with their permission.