Alfresco Transform Services 4.0.0
0 likes209 views
The document outlines the architecture and components of a transform services and repository system, detailing both community and enterprise versions. It covers the functionality of various transformation engines, shared file storage, configuration options, and secure communication settings. Additionally, it provides guidance on creating new transformation engines and best practices for security and configuration management.
![Transform Services & Repository
Configuration
Described in T-Engines as JSON files
• libreoffice_engine_config.json
• imagemagick_engine_config.json
• misc_engine_config.json
• pdfrenderer_engine_config.json
• tika_engine_config.json
{
"transformOptions": {
"metadataOptions": [
{"value": {"name": "extractMapping"}}
]
},
"transformers": [
{
"transformerName": "Tika",
"supportedSourceAndTargetList": [
{ "sourceMediaType": "application/pdf",
"maxSourceSizeBytes": 26214400,
"priority": 55,
"targetMediaType": "text/plain"},
],
"transformOptions": [
"metadataOptions"
]
}
]
}](https://image.slidesharecdn.com/alfresco-transform-services-4-230720151236-40c21498/85/Alfresco-Transform-Services-4-0-0-10-320.jpg)
![Transform Services & Repository
Configuration changes
External JSON file configured as environment variable
TRANSFORM_CONFIG_FILE_<NAME>
• Add
• Override
• Removal
• Default values modification
{
"addSupported": [
{
"transformerName": "Archive",
"sourceMediaType": "application/zip",
"targetMediaType": "text/csv",
"priority": 60,
"maxSourceSizeBytes": 18874368
}
]
}
https://github.com/aborroy/alfresco-transform-service-3.0.0#changing-default-configuration](https://image.slidesharecdn.com/alfresco-transform-services-4-230720151236-40c21498/85/Alfresco-Transform-Services-4-0-0-11-320.jpg)
![Transform Services & Repository
Creating a new T-Engine
• Define pipelines
{
"transformers":
[
{
"transformerName": "markdown",
"supportedSourceAndTargetList":
[
{"sourceMediaType": "text/markdown", "targetMediaType": "application/pdf" },
{"sourceMediaType": "text/x-markdown", "targetMediaType": "application/pdf" }
]
},
{
"transformerName": "latex",
"supportedSourceAndTargetList":
[
{"sourceMediaType": "application/x-tex", "targetMediaType": "application/pdf" }
]
}
]
}](https://image.slidesharecdn.com/alfresco-transform-services-4-230720151236-40c21498/85/Alfresco-Transform-Services-4-0-0-13-320.jpg)
Alfresco Transform Services 4.0.0
- 1. Transform Services & Repository
- 2. Transform Services & Repository Components • Transform Core (AIO) • Single-step transformations via HTTP • Individual T-Engines can be also deployed independently • Transform Router • Multi-step transformations with pipelines and failover mechanism via Messaging (ActiveMQ) • Shared File Store • Filesystem repository via HTTP
- 3. Transform Services & Repository Community Transform Core AIO LibreOffice ImageMagick PDF Renderer Tika Misc Alfresco Repository Transform Service SYNC Custom local.transform.service.enabled=true
- 4. Transform Services & Repository Transform Core AIO (Community) • Spring Boot App • HTTP services • Configuration GET transform/config • Transform POST / • Test Page GET / • Log GET /log • Live GET /live • Ready GET /ready • Synchronous • Unique option for Alfresco Community deployments Transform Core AIO
- 5. Transform Services & Repository Enterprise Shared File Store Transform Router Transform Core AIO ActiveMQ Alfresco Repository Transform Service SYNC ASYNC ASYNC LibreOffice ImageMagick PDF Renderer Tika Misc Custom DTE AIS SYNC (config) SYNC ASYNC SYNC transform.service.enabled=true
- 6. Transform Services & Repository Transform Core AIO (Enterprise) • Spring Boot App • HTTP services • Synchronous • Configuration, some Share actions and Text extraction (Solr) • Messaging services (ActiveMQ) • Asynchronous • Requests queue: org.alfresco.transform.t-request.acs • Replies queue: org.alfresco.transform.engine.t-reply.acs Transform Core AIO
- 7. Transform Services & Repository Transform Engines • LibreOffice Office Documents including Microsoft formats • ImageMagick Images and thumbnails • PDF Renderer Generate PNG of first page from a PDF document • Tika Compressed files and metadata extraction • Misc HTML, iWorks, EML, XML, Markdown, source code… • DTE (Document Transformation Engine): Word, Excel and PowerPoint • AIS (Alfresco Intelligence Services): Enrichment of content metadata • Custom Transformation and enrichment for additional formats Transform Core AIO Out of the Box Paid Modules Dev
- 8. Transform Router Transform Services & Repository Transform Router • Spring Boot App • Retrying policy • Max retries (3 by default) • Timeout (10 secs by default) • Types of Transformers • Single-step • Pipeline • Error handling • 400 BAD REQUEST • Invalid JSON • Invalid values • Unsupported transformation • 403 FORBIDDEN • mTLS fails due to wrong client certificate • 500 INTERNAL SERVER ERROR
- 9. Shared File Store Transform Services & Repository Shared File Store • Spring Boot App • HTTP service wrapping a shared filesystem • A place to store and retrieve files from different services (like ACS and ATS) • Configuration • scheduler.content.age.millis 86400000 Content retention period • scheduler.cleanup.interval 86400000 Cleanup Scheduler interval
- 10. Transform Services & Repository Configuration Described in T-Engines as JSON files • libreoffice_engine_config.json • imagemagick_engine_config.json • misc_engine_config.json • pdfrenderer_engine_config.json • tika_engine_config.json { "transformOptions": { "metadataOptions": [ {"value": {"name": "extractMapping"}} ] }, "transformers": [ { "transformerName": "Tika", "supportedSourceAndTargetList": [ { "sourceMediaType": "application/pdf", "maxSourceSizeBytes": 26214400, "priority": 55, "targetMediaType": "text/plain"}, ], "transformOptions": [ "metadataOptions" ] } ] }
- 11. Transform Services & Repository Configuration changes External JSON file configured as environment variable TRANSFORM_CONFIG_FILE_<NAME> • Add • Override • Removal • Default values modification { "addSupported": [ { "transformerName": "Archive", "sourceMediaType": "application/zip", "targetMediaType": "text/csv", "priority": 60, "maxSourceSizeBytes": 18874368 } ] } https://github.com/aborroy/alfresco-transform-service-3.0.0#changing-default-configuration
- 12. Transform Services & Repository Creating a new T-Engine • Maven <parent> <groupId>org.alfresco</groupId> <artifactId>alfresco-transform-core</artifactId> <version>3.0.0</version> </parent> <dependencies> <dependency> <groupId>org.alfresco</groupId> <artifactId>alfresco-base-t-engine</artifactId> <version>3.0.0</version> </dependency> </dependencies> • Java • org.alfresco.transform.base.TransformEngine to provide basic information about the Engine and the Transform pipelines • org.alfresco.transform.base.CustomTransformer to implement the transformation operation https://github.com/aborroy/alfresco-transform-service-3.0.0/tree/main/pandoc-t-engine
- 13. Transform Services & Repository Creating a new T-Engine • Define pipelines { "transformers": [ { "transformerName": "markdown", "supportedSourceAndTargetList": [ {"sourceMediaType": "text/markdown", "targetMediaType": "application/pdf" }, {"sourceMediaType": "text/x-markdown", "targetMediaType": "application/pdf" } ] }, { "transformerName": "latex", "supportedSourceAndTargetList": [ {"sourceMediaType": "application/x-tex", "targetMediaType": "application/pdf" } ] } ] }
- 14. Transform Services & Repository Creating a new T-Engine • Create the Dockerfile FROM pandoc/core:latest-ubuntu COPY target/pandoc-t-engine.jar /usr/bin EXPOSE 8090 USER ${USERNAME} ENTRYPOINT java $JAVA_OPTS -jar /usr/bin/pandoc-t-engine.jar
- 15. Transform Services & Repository Creating a new T-Engine • Deployment in Docker Compose services: alfresco: image: alfresco/alfresco-content-repository-community:7.4.0 environment: JAVA_OPTS: >- -DlocalTransform.pandoc.url=http://transform-pandoc:8090/ transform-pandoc: image: alfresco/pandoc-t-engine:latest ports: - 8096:8090
- 16. Transform Services & Repository Resources usage and logging level transform-core-aio: image: alfresco/alfresco-transform-core-aio:3.0.0 environment: JAVA_OPTS: " -Dserver.tomcat.threads.min=4 -Dserver.tomcat.threads.max=12 -Dlogging.level.org.alfresco.transform.common.TransformerDebug=ERROR "
- 17. Transform Services & Repository Scaling up https://github.com/aborroy/acs-transform-cluster Community Enterprise
- 18. Secure Communications & Settings
- 19. Secure Communications & Settings • From ACS 7.4, mTLS communication between Transform Service and Repository is supported • Alfresco SSL Generator tool has been updated to produce required keystore and truststore files • Generate a self-signed CA* • Generate certificates, keystores and truststores for Repository & Transform $ ./run_additional.sh -servicename transform-core-aio -rootcapass kT9X6oe68t -keysize 2048 -keystoretype PKCS12 -keystorepass kT9X6oe68t -truststoretype PKCS12 -truststorepass kT9X6oe68t -certdname "/C=GB/ST=UK/L=Maidenhead/O=Alfresco Software Ltd./OU=Unknown/CN=Transform Core AIO" -servername transform-core-aio -alfrescoformat current • Apply configuration settings for properties and keystores * You may use your own software or PKI infrastructure to create CA and certificates, but configuration related to keystore type and certificate alias should be followed as described by Alfresco SSL Generator
- 20. Secure Communications & Settings https://github.com/aborroy/alfresco-secure-connection Community
- 21. Community – Repository alfresco: environment: JAVA_TOOL_OPTIONS: >- -Dencryption.ssl.keystore.type=PKCS12 -Dencryption.ssl.keystore.location=/usr/local/tomcat/alfresco.keystore -Dssl-keystore.password=kT9X6oe68t -Dencryption.ssl.truststore.type=PKCS12 -Dencryption.ssl.truststore.location=/usr/local/tomcat/alfresco.truststore -Dssl-truststore.password=kT9X6oe68t JAVA_OPTS: >- -DlocalTransform.core-aio.url=https://transform-core-aio:8090/ -Dhttpclient.config.transform.mTLSEnabled=true -Dhttpclient.config.transform.hostnameVerificationDisabled=true -Djdk.tls.client.protocols=TLSv1.2 volumes: - ./keystore/alfresco/alfresco.keystore:/usr/local/tomcat/alfresco.keystore - ./keystore/alfresco/alfresco.truststore:/usr/local/tomcat/alfresco.truststore Secure Communications & Settings Alfresco SSL Generator
- 22. Secure Communications & Settings Community – Transform Service transform-core-aio: environment: SERVER_SSL_ENABLED: "true" SERVER_SSL_CLIENT_AUTH: "need" SERVER_SSL_ENABLED_PROTOCOLS: "TLSv1.2" SERVER_SSL_KEY_PASSWORD: "kT9X6oe68t" SERVER_SSL_KEY_STORE: "file:/transform-core-aio.keystore" SERVER_SSL_KEY_STORE_PASSWORD: "kT9X6oe68t" SERVER_SSL_KEY_STORE_TYPE: "PKCS12" SERVER_SSL_TRUST_STORE: "file:/transform-core-aio.truststore" SERVER_SSL_TRUST_STORE_PASSWORD: "kT9X6oe68t" SERVER_SSL_TRUST_STORE_TYPE: "PKCS12" volumes: - ./keystore/tengineAIO/transform-core-aio.keystore:/transform-core-aio.keystore - ./keystore/tengineAIO/transform-core-aio.truststore:/transform-core-aio.truststore Alfresco SSL Generator
- 23. Secure Communications & Settings Enterprise Shared File Store Transform Router Transform Core AIO Alfresco Repository Transform Service mTLS LibreOffice ImageMagick PDF Renderer Tika Misc Custom DTE AIS mTLS mTLS mTLS mTLS
- 24. Enterprise – Repository alfresco: environment: JAVA_TOOL_OPTIONS: >- -Dencryption.ssl.keystore.type=PKCS12 -Dencryption.ssl.keystore.location=/usr/local/tomcat/alfresco.keystore -Dssl-keystore.password=kT9X6oe68t -Dencryption.ssl.truststore.type=PKCS12 -Dencryption.ssl.truststore.location=/usr/local/tomcat/alfresco.truststore -Dssl-truststore.password=kT9X6oe68t JAVA_OPTS: >- -Dtransform.service.enabled=true -Dtransform.service.url=https://transform-router:8095 -DlocalTransform.core-aio.url=https://transform-core-aio:8090/ -Dsfs.url=https://shared-file-store:8099/ -Dhttpclient.config.transform.mTLSEnabled=true -Dhttpclient.config.transform.hostnameVerificationDisabled=true -Djdk.tls.client.protocols=TLSv1.2 volumes: - ./keystore/alfresco/alfresco.keystore:/usr/local/tomcat/alfresco.keystore - ./keystore/alfresco/alfresco.truststore:/usr/local/tomcat/alfresco.truststore Secure Communications & Settings Alfresco SSL Generator
- 25. Secure Communications & Settings Enterprise – Transform Router transform-router: environment: CORE_AIO_URL: https://transform-core-aio:8090 FILE_STORE_URL: https://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file SERVER_SSL_ENABLED: "true" SERVER_SSL_CLIENT_AUTH: "need" SERVER_SSL_ENABLED_PROTOCOLS: "TLSv1.2" SERVER_SSL_KEY_PASSWORD: "password" SERVER_SSL_KEY_STORE: "file:/transformRouter.keystore" SERVER_SSL_KEY_STORE_PASSWORD: "password" SERVER_SSL_KEY_STORE_TYPE: "PKCS12" SERVER_SSL_TRUST_STORE: "file:/transformRouter.truststore" SERVER_SSL_TRUST_STORE_PASSWORD: "password" SERVER_SSL_TRUST_STORE_TYPE: "PKCS12" CLIENT_SSL_KEY_STORE: "file:/transformRouter.keystore" CLIENT_SSL_KEY_STORE_PASSWORD: "password" CLIENT_SSL_KEY_STORE_TYPE: "PKCS12" CLIENT_SSL_TRUST_STORE: "file:/transformRouter.truststore" CLIENT_SSL_TRUST_STORE_PASSWORD: "password" CLIENT_SSL_TRUST_STORE_TYPE: "PKCS12" CLIENT_SSL_HOSTNAME_VERIFICATION_DISABLED: true volumes: - ./keystore/transformRouter/transformRouter.truststore:/transformRouter.truststore - ./keystore/transformRouter/transformRouter.keystore:/transformRouter.keystore Alfresco SSL Generator
- 26. Secure Communications & Settings Enterprise – Transform Core transform-core-aio: environment: FILE_STORE_URL: >- https://shared-file-store:8099/alfresco/api/-default-/private/sfs/versions/1/file SERVER_SSL_ENABLED: "true" SERVER_SSL_CLIENT_AUTH: "need" SERVER_SSL_ENABLED_PROTOCOLS: "TLSv1.2" SERVER_SSL_KEY_PASSWORD: "password" SERVER_SSL_KEY_STORE: "file:/tengineAIO.keystore" SERVER_SSL_KEY_STORE_PASSWORD: "password" SERVER_SSL_KEY_STORE_TYPE: "PKCS12" SERVER_SSL_TRUST_STORE: "file:/tengineAIO.truststore" SERVER_SSL_TRUST_STORE_PASSWORD: "password" SERVER_SSL_TRUST_STORE_TYPE: "PKCS12" CLIENT_SSL_KEY_STORE: "file:/tengineAIO.keystore" CLIENT_SSL_KEY_STORE_PASSWORD: "password" CLIENT_SSL_KEY_STORE_TYPE: "PKCS12" CLIENT_SSL_TRUST_STORE: "file:/tengineAIO.truststore" CLIENT_SSL_TRUST_STORE_PASSWORD: "password" CLIENT_SSL_TRUST_STORE_TYPE: "PKCS12" CLIENT_SSL_HOSTNAME_VERIFICATION_DISABLED: true volumes: - ./keystore/tengineAIO/tengineAIO.truststore:/tengineAIO.truststore - ./keystore/tengineAIO/tengineAIO.keystore:/tengineAIO.keystore Alfresco SSL Generator
- 27. Secure Communications & Settings Enterprise – Shared File Store shared-file-store: environment: SERVER_SSL_ENABLED: "true" SERVER_SSL_CLIENT_AUTH: "need" SERVER_SSL_ENABLED_PROTOCOLS: "TLSv1.2" SERVER_SSL_KEY_PASSWORD: "password" SERVER_SSL_KEY_STORE: "file:/sharedFileStore.keystore" SERVER_SSL_KEY_STORE_PASSWORD: "password" SERVER_SSL_KEY_STORE_TYPE: "PKCS12" SERVER_SSL_TRUST_STORE: "file:/sharedFileStore.truststore" SERVER_SSL_TRUST_STORE_PASSWORD: "password" SERVER_SSL_TRUST_STORE_TYPE: "PKCS12" CLIENT_SSL_KEY_STORE: "file:/sharedFileStore.keystore" CLIENT_SSL_KEY_STORE_PASSWORD: "password" CLIENT_SSL_KEY_STORE_TYPE: "PKCS12" CLIENT_SSL_TRUST_STORE: "file:/sharedFileStore.truststore" CLIENT_SSL_TRUST_STORE_PASSWORD: "password" CLIENT_SSL_TRUST_STORE_TYPE: "PKCS12" CLIENT_SSL_HOSTNAME_VERIFICATION_DISABLED: true volumes: - ./keystore/sharedFileStore/sharedFileStore.truststore:/sharedFileStore.truststore - ./keystore/sharedFileStore/sharedFileStore.keystore:/sharedFileStore.keystore Alfresco SSL Generator
- 28. Tips & Tricks
- 29. Tips & Tricks TLS • Use TLSv1.3 where available • When using TLSv1.2, restrict accepted algorithms to • AES-128 with SHA-256 • AES-256 with SHA-384 • The use of SSL, TLSv1.0 and TLSv1.1 is severely discouraged Keystores • Use PKCS#12 keystore and truststore types • P#12 keystores built with OpenSSL don’t work with Java, using keytool is required * Keys • Use at least 2048 bits for RSA keys * https://angelborroy.wordpress.com/2022/08/12/building-a-custom-pkcs12-truststore-for-java/