Running Language Weaver Edge on Docker

Language Weaver Edge could easily be deployed in Docker & the deployment architecture is similar to an on-prem Windows/Linux deployment. Docker image and sample “docker-compose.yml” files are provided by RWS for easy deployment. A docker host with a GPU is preferred for Training Engines.
 
Minimum requirement
  1. Docker host with CPU/RAM totalling up to the recommended minimum requirement.
    e.g. Azure, EC2, VMWare
  2. Docker & docker-compose installed on the host.
  3. Sudo access for user mtedge.
  4. Local storage as recommended for the image & language pair models. 

Architecture
  1. Edge Controller host is deployed as a container in docker and Edge UI & API are published as external ports in the docker overlay network.
  2. Job Engines, Translation Engines & Training Engines are deployed as containers and could be manually configured to scale up/down based on the predefined number of containers as manifested in the docker-compose.yml file.
  3. Language Pairs and the Edge configuration is saved in docker volumes.
  4. All containers use the same docker base image.
Deployment
  1. Prerequisites
    1. Create mtedge user & group with the id 10001.

      $ groupadd mtedge -g 10001
      $ useradd mtedge -u 10001 -g 10001 -m -s /sbin/nologin

    2. Grant sudo access to user mtedge.
      eg. on a Ubuntu EC2 host, edit /etc/sudoers.d/90-cloud-init-users file and add a new line as below.
      $ vi /etc/sudoers.d/90-cloud-init-users
      mtedge ALL=(ALL) NOPASSWD:ALL

    3. Install unzip, docker & docker-compose on the host.
      eg. on a Ubuntu EC2 host, run.

      $ sudo apt update
      $ sudo apt install -y docker.io docker-compose unzip
      $ sudo usermod -a -G docker mtedge

  2. Installation
    1. Login to docker repo and pull LW Edge docker image. 
      $ docker login docker.edge.languageweaver.com
      $ docker pull docker.edge.languageweaver.com/sdl/mtedge:8.6.4



    2. Copy docker-compose.yml, sdl-ets.lic and .env files provided by RWS to current user's home folder.
    3. Download & install required language pairs on docker host. (can be removed after a successful docker deployment).
      $ sftp user@files.edge.languageweaver.com
      $ get Language_Weaver_Edge_XxxXxx_Generic_SRV_TNM_X_X_x_1_installer.zip
      $ unzip Language_Weaver_Edge_XxxXxx_Generic_SRV_TNM_X_X_x_1_installer.zip
      $ sudo ./Language_Weaver_Edge_XxxXxx_Generic_SRV_TNM_X_X_x_1_installer/setup-linux-x64.run --mode unattended

    4. Start containers by running docker-compose up. This will also create docker volumes mtedge-controller-pvc & mtedge-lp-pvc

      $ docker-compose up -d

    5. Copy license file sdl-ets.lic to mtedge-controller-pvc.

      $       docker cp sdl-ets.lic mtedge-controller:/mnt/controller/conf/sdl-ets.lic

    6. Copy language pair models to mtedge-lp-pvc.

      $ sudo docker cp /opt/sdl/ets/lps/. mtedge-controller:/opt/sdl/ets/lps

    7. Wait until all language pair models are copied and then restart translation engines from UI or run a docker-compose down & up
       
  3. Cleanup
    1. Delete locally installed language pair models, installation zip files & folders from the host.

      $ sudo rm -rf /opt/sdl
      $ sudo rm -rf /etc/opt/sdl
      $ sudo rm -rf Language_Weaver_Edge_*
Reconfigure
Language Weaver Edge instance in a docker host could be reconfigured/resized by modifying the docker-compose.yml and Edge admin user details by modifying the .env file. All modifications require a restart of docker containers by running docker-compose up -d.

Typical values that could be changed are:
  1. Bootstrap Language Weaver Edge admin user.
  2. Install new Language Pairs.
  3. Increase/decrease number of Job Engines, Training Engines or add/modify Translation Engines or PUs for a given Language Pair.

 

Environment Variables
Custom environment variables could be set independently for:
  1. Controller pod.
  2. Job Engine pods - all pods share the same environment variables.
  3. Translation Engine pods - each Language Pair has its own environment variables.
  4. Training Engine pods - all pods share the same environment variables.
Edge API
The API key is automatically prefixed with “u_” + username + ”_” when created.
For an example, to make a REST API call for user admin@example.com with API key myapikey1234
$ curl -u u_admin@example.com_myapikey1234: https://myhost.example.com/api/v2/
Related
Recommended