You are currently reviewing an older revision of this page.

Running Language Weaver Edge on Docker

Language Weaver Edge could easily be deployed in Docker & the deployment architecture is similar to an on-prem Windows/Linux deployment. Docker image and sample “docker-compose.yml” files are provided by RWS for easy deployment. A docker host with a GPU is preferred for Training Engines.
 
Minimum requirement
  1. Docker host with CPU/RAM totalling up to the minimum requirement.
    e.g. Azure, EC2, VMWare
  2. Docker & Docker-Compose installed on the host.
  3. Sudo access granted to user mtedge.
  4. Recommended size of the local storage for the image & Language Pair models. 

Architecture
  1. Edge Controller host is deployed as a container in docker and Edge UI & API are published as a service in the docker overlay network.
  2. Job Engines, Translation Engines & Training Engines are deployed as containers and could be manually configured to scale up/down based on the predefine number of containers defined in the docker-compose.yml file.
  3. Language Pairs and the Edge configuration is saved in docker volumes.
  4. All containers use the same docker base image.
Deployment
  1. Prerequisites
    1. Create mtedge user & group with the id 10001.

      $ groupadd mtedge -g 10001
      $ useradd mtedge -u 10001 -g 10001 -m -s /sbin/nologin

    2. Grant sudo access to user mtedge.
      eg. on a Ubuntu EC2 host, edit /etc/sudoers.d/90-cloud-init-users file and add a new line as below.
      $ vi /etc/sudoers.d/90-cloud-init-users
      mtedge ALL=(ALL) NOPASSWD:ALL

    3. Install unzip, docker & docker-compose on the host.
      eg. on a Ubuntu EC2 host, run.

      $ sudo apt update
      $ sudo apt install -y docker.io docker-compose unzip
      $ sudo usermod -a -G docker mtedge

  2. Installation
    1. Login to docker repo and pull LW Edge docker image. 
      $ docker login docker.edge.languageweaver.com
      $ docker pull docker.edge.languageweaver.com/sdl/mtedge:8.6.4
    2. Copy docker-compose.yml, sdl-ets.lic & .env files provided by RWS to current users home folder.
    3. Download & install required language pairs. (can be uninstalled after a successful docker deployment).
      $ sftp user@files.edge.languageweaver.com
      $ get Language_Weaver_Edge_XxxXxx_Generic_SRV_TNM_X_X_x_1_installer.zip
      $ unzip Language_Weaver_Edge_XxxXxx_Generic_SRV_TNM_X_X_x_1_installer.zip
      $ sudo ./Language_Weaver_Edge_XxxXxx_Generic_SRV_TNM_X_X_x_1_installer/setup-linux-x64.run --mode unattended

    4. Start containers by running docker-compose up.

      $ docker-compose up -d

    5. Copy license file sdl-ets.lic to mtedge-controller-pvc.

      $       docker cp sdl-ets.lic mtedge-controller:/mnt/controller/conf/sdl-ets.lic
    6. Copy language pair models to mtedge-lp-pvc.

      $ sudo docker cp /opt/sdl/ets/lps/. mtedge-controller:/opt/sdl/ets/lps

    7. Wait until all language pair models are copied & restart translation engines from UI or run a docker-compose down & up
       
  3. Cleanup
    1. Delete language pair models, installation zip files & folders from the host.

      $ sudo rm -rf /opt/sdl
      $ sudo rm -rf /etc/opt/sdl
      $ sudo rm -rf Language_Weaver_Edge_*
Reconfigure
Language Weaver Edge instance in a docker host could be reconfigured/resized while the Language Weaver Edge service is running. Reconfiguration can be performed using a helm upgrade or directly modifying relevant Kubernetes objects. The simplest way is to modify the “values.yaml” and run a helm upgrade.

Typical values that could be changed while the system is running are:
  1. Bootstrap Language Weaver Edge admin user.
  2. Install new Language Pairs.
  3. Increase/Decrease the size of the pool of Job Engines, Training Engines or add/modify the pool of Translation Engines for a given Language Pair.
  4. Auto scale the number of Translation Engines, depending on the system load.

 

Environment Variables
Custom environment variables could be set independently for:
  1. Controller pod.
  2. Job Engine pods - all pods share the same environment variables.
  3. Translation Engine pods - each Language Pair has its own environment variables.
  4. Training Engine pods - all pods share the same environment variables.
Edge API
The API key is automatically prefixed with “u_” + username + ”_” when created.
For an example, to make a REST API call for user admin@example.com with API key myapikey1234
$ curl -u u_admin@example.com_myapikey1234: https://myhost.example.com/api/v2/