Updates for the Composum AI: content generation, content analysis and transparent translation in AEM

A while ago I wrote about the implementation of the AI based translation for AEM with which I extended the Composum AI that I've had the pleasure to do. Quite some time has passed and there have been a number of changes and improvements since then, so I thought it would be a good idea to summarize the status and interesting points. It turned out to work nicely: it's now used in an international site that is translated into some 20 languages for about 50 countries and the customer is very happy with the results - to use the translation mechanism that is integrated into AEM live copies is considerably simpler than the standard AEM translation mechanism. There is now also a translation review tool that will show you all translations that appear in a page so that you can review those and approve or edit all them if necessary in one go. And that's in addition to the nice content creation features for editing texts or just asking the AI in a chat for ideas or references (it can search the current site now).

In the following I'll give a brief overview of the current features - many of the implementation details are in the mentioned previous blog.

In case you are wondering: all that is available open source with MIT licence, so you are invited to give it a try. But please tell me if you did with some feedback, even if you did not like it. :-)

Content creation

There is a content creation dialog that can be opened from richtext fields or text areas in the AEM editor:

has a configurable library of example prompts, or you can of course edit them / enter or dictate your prompts
offers you to select a source text that the prompt can transform: the text in the field, the text in the component, the text of the current page, referenced pages, images linked in components near the edited component, the text from external URLs.
has various settings for text length etc.
a history that allows you to go back to previous attempts and continue from there
is able to search the current site for relevant information using retrieval augmented generation (RAG) and use that for the content creation

Content analysis ("Sidebar AI")

It has a chat interface integrated into the AEM editor sidebar that allows you to ask the AI questions, discuss with it, have it analyze the content etc. Some of the features are:

The text of the current page or of the currently selected component can be used as context for the chat
It also has a configurable library of example prompts and dictation.
There is a history to go back to previous chats.
It can search the current site for relevant information using retrieval augmented generation (RAG), answer questions based on that or create content suggesitions

Transparent translation during live copy rollout

AEM has a nice speciality as a CMS: live copies are a very nice way to update various sites with similar content from one or more master sites ("blueprint"). If you change the master page you can roll out the changes to some or all live copies, where the parts of the page that are unchanged will be overwritten with the content from the master page. On all such pages you can cancel the inheritance of components and edit them - they won't be touched by this process until you reenable inheritance again. Now, the Composum AI translation mechanism takes that to the next level by hooking into that process with an automatic translation of the rolled out content. This makes it really easy to set up a multilingual site.

Translates a full page at a time using AI, thus providing the AI with context
There are several ways to give additional instructions to the AI - either general ones ("Do not translate word by word but for readability") or specific ones for certain pages or whenever certain words or phrases appear in the page ("Translate 'Leiter' as 'leader'", in German that could also be a ladder, though the AI would very likely guess that from context. :-)
For word / phrase translation suggestions it's possible to upload a spreadsheet with the source and target words / phrases, which is then used to create the instructions for the AI.
There is a review tool that allows you to easily view all translations in the page and edit them if necessary.

The review tool allows you to see even hidden text properties like alt texts of images or page title and description together in one page and edit them quickly, without having to go through the (relatively slow) AEM editor dialogs. It also does support a workflow: it presents you all new translations and you can either approve them (so that they aren't shown again until the source text changes) or cancel the inheritance and edit them. If the source text changes again, you'll be offered the new automatical translation for review and either approve that or merge in the changes, automatically or manually.

General features

As backend it can use LLM (large language models)
of various providers as long as they have an OpenAI chat completion API compatible interface. I tried OpenAI API, Azure OpenAI, Anthropic, Google Gemini API and local models.
It gives the AI an easy possibilty to call local actions aka tools / function calls simply by implementing the interface AITool
There is a tool GetPageMarkdownAITool for accessing the text of other pages and SearchPageAITool that does provide the AI (for the content completion dialog an sidebar AI chat) with the ability to search the current site.
Of course there are tooltips for all buttons and fields and also help pages.

More information

Composum AI - the main page with links to the documentation and the prototype and some videos
Composum AI source on GitHub.
Composum AI youtube playlist has several demonstrations of many of the features.
RAG and AI supported search with Apache Sling / AEM discusses how embeddings and Retrieval Augmented Generation (RAG) techniques can be implemented in AEM / Sling without much effort, which is used in the Composum AI dialogs.
AI translation implementation ideas how to deal with AI translation covers the details of the implementation of the AI-based translation in Composum AI, which goes beyond the usual machine translation approaches in that the AI translates a whole page at a time and can be given additional instructions
AI-triggering local actions describes ways to enable AI to trigger local system actions via APIs and scripts. Composum AI has an implementation that can easily be extended to have the AI perform local actions from the chat - currently search / RAG are implemented, and some initial experiments to have the AI modify page texts.

Conclusion

Even if it's free (MIT license) - the Composum AI is a very powerful tool to support content creation, content analysis and translation in AEM. It is easy to use, integrates nicely into the AEM editor and supports a wide range of AI backends. It was a real pleasure to implement it and to support it's implementation into a pretty large multinational site, to work out the needs of the customer and improve things and implement new tools to support the authors. Please try it out and make sure to tell me about your experiences!