This post is copied from New versions for ISHServer v1.5 and ISHBootstrap v1.0. Support for Amazon Machine Image (AMI), Docker containers, Vagrant Boxes and many more.

Background

For those encountering the terms ISHServer and ISHBootstrap for the first time, they are automation modules for SDL’s Knowledge Center Content Manager also identified with the code name ISH.

ISHServer

ISHServer started initially as a PowerShell module that offered cmdlets that aligned with the steps required to prepare a Windows Server for the installation of Content Manager (ISH). This process is discussed in the official documentation and has proven to be error prone. Initially it was part of ISHBootstrap but it was separated, as ISHServer has a more generic purpose and can be used by anyone even without ISHBootstrap. Overtime, ISHServer extended beyond the simple task of automating the pre-requisites installation and server configuration.

ISHBootstrap

ISHBootstrap is a PowerShell script library that drives the flow. It has the ability to bootstrap a clean Windows Server instance into a fully operational server with Content Manager (ISH) deployments. The supported operating systems are Windows Server 2012R2 and 2016. The bootstrapping can be executed locally or against a remote server using Windows remoting. ISHBootstrap has embedded an examples directory to showcase how the complete process could be represented and executed by a json file, but keep in mind that this is just an example. By default the instance of Content Manager has zero configuration applied to it, a state that is also known as the Vanilla deployment. The examples folder in ISHBootstrap offer a collection of ISHDeploy scripts that showcase certain configuration examples using scripts that use ISHDeploy cmdlets. In general, ISHBootstrap scripts are a very good example of how to use ISHServer cmdlets. Although disconnected, I would advice always to use ISHBootstrap over ISHServer.

New features

The main mission of ISHServer has been accomplished since its first stable version. Usually all enhancements to the module are in favour of supporting enhancements for ISHBootstrap when the functionality is considered generic enough. With the latest version of 1.5 those additional features are summarized as:

• Support for FTP, Amazon S3, Azure Blob and File storage as source for the dependencies, ISH CD and AntennaHouse license.
• Create, update and initialize the user account that is used for Content Manager’s processes.
• Download and expand the Content Manager CD in a consistent manner.

All above features are developed to support execution under the following conditions:

• Local administrator.
• Windows Remoting. This is important because packer uses windows remoting to provision scripts while initializing an Amazon AMI or an Hyper-V instance.
• Local SYSTEM account. This is important because for Amazon EC2 instances, both the userdata and CodeDeploy execute under the SYSTEM account and this really complicates things a lot. I’ll try to make a post about the difficulties we faced when executing under the SYSTEM account.

ISHBootstrap had always the ability to prepare an operating system with a fully operational Content Manager. With the first stable version 1.0, ISHBootstrap is now able to do something slightly different but very important. It can prepare a Windows Server instance with an almost ready Vanilla deployment.

What is an almost ready Vanilla deployment?

The last step of installing Content Manager takes around 5-6 minutes alone. We also need to consider that the installer was not build for producing a deployment that just needs a couple of parameters to very quickly become operational but it always approached the problem as from zero to all. But when working with cloud services and containers that is not acceptable. Imagine a docker container that takes 5-6 minutes to run, because it needs to install Content Manager first. Same concept applies for cloud and vagrant boxes. You need something that is ready to launch as quick as possible. In my experience the containers technology is the best advocate of this concept. For example Microsoft’s SQL Server container image, is in kind of almost ready state because the services are not running, the sa password is not set and only the master database is available. It is the job of the initialization script to fill in this gaps and provide an operational SQL Server as part of the container’s run step. Imagine the container installing SQL Server when running. Imagine the delay. This is exactly what we want to avoid with Content Manager. They are both fairly complicated products to install with a time consuming installation process, that we don’t want to spend every time we want to run the instance.

To solve this problem, ISHBootstrap introduces the Builder concept. The builder concept works around the limitations of the installer of Content Manager by producing a reusable template of one Content Manager deployment, that previous was referred to as almost ready. It actually mocks temporarily certain dependencies to allow Content Manager’s installer to execute and then clears them out. At this point the deployment is inoperable just because the mocked dependencies used are already cleared out. To turn this non-operational deployment into a operational one, a script is provided that will replace the mocked values. This last step could be considered as the run step of this almost ready artifact. The process is very similar to the sql server image but is not restricted to docker containers only.

The produced artifact can one of the following:

• Amazon Machine Image (AMI). An AMI can be used to spin up an Amazon EC2, that is a Virtual Machine. As the AMI includes all the time consuming process, the EC2 instance is ready for use in the least possible amount of time.
• Vagrant Box on top of Hyper-V. A Vagrant Box can spin up virtual machines for any host. As the box includes all the time consuming process, the VM instance is ready for use in the least possible amount of time.
• Docker Container. This is for experimentation and learning process.

Each of this types of artifacts is ready to become operational by providing the following parameters

• A db connections string.
• A user to run Content Manager processes, known also as the osuser.
• A hostname.
• A valid certificate for the hostname.

The builder concept goes even one step further and can embed a local sql server express in the artifact. This is not meant for production purposes but if you want to quickly spin up an SDL Knowledge Center instance as a Hyper-V VM, or Amazon EC2 or a container, then you don’t have to worry about the external dependency to a database server. It’s already ready and executing. I’m actually looking for ideas to reduce dependency to the rest of the parameters for this use case. For example hard-code the osuser’s credentials.

Unfortunately this is not yet possible for the Vagrant boxes because we are facing problems installing the database as part of the packer provisioning process. Luckily, very good workarounds exist for AMIs and containers. Amazon offers windows AMIs with SQL Server Express already installed and for containers, I’ve created this asarafian/mssql-server-windows-express:2014SP2.

Before I mentioned that the container variant is not to be used in production because Content Manager needs to go under some core architecture changes to become more container friendly. It’s not that it won’t work as far as ISHBootstrap is concerned but ISHBootstrap cannot and will not amend any core product limitations. As with everything, try it out but if you productize it, then the risk is yours.

With that in mind, I find containers an amazing learning tool. The docker team, has actually solidified the concept of image and instance but more importantly they fully support inheritance between the containers. This means, anyone can use as starting point a container that someone else has created. As such, a Content Manager image would be an excellent starting point to create customized images ready to run. To be fair, Amazon had already achieved this with the Amazon Machine Image technology but it’s still virtual machine technology based and kind of dirty. As far as I know packer and vagrant don’t support this model and this is the reason there isn’t yet the ability to produce a vagrant box with SQL Server embedded.

The business case driving the features

SDL is moving to the Amazon for various reasons and there can’t be any cloud without full automation. During the last year, I’m driving the DEVOPS angle for Knowledge Center by making sure that we (the engineering team) produce the necessary tooling for automation such as ISHDeploy, ISHServer and ISHBootstrap. My colleague Dave De Meyer, is also pushing the ISHRemote PowerShell module. Although DEVOPS is focused on operations, I’m a believer that DEVOPS should be a mental approach embraced by all software engineers involved. It should be a discipline of “Automate as much as possible”. For this reason, the automation toolkits are not specific only to DEVOPS nor to a specific cloud provider and can be used by anyone to improve the interaction with a Content Manager deployment. The main goal is to help everyone improve their interaction with Content Manager by automating the uninteresting and error-prone steps. Therefore, I think that a ready to launch deployment is a great offering.

Concepts such as containers, AMIs and Vagrant Boxes can become the deliverable and replace the old fashioned CD that as an approach is too error prone. Instead a ready to execute artifact has much less chances to go wrong.

As SDL is moving to Amazon, we need to build AMIs that allow the creation of a cloud formation stack within a reasonable amount of time. ISHBootstrap’s codebase is the main engine for SDL’s AMI builder process. The only difference is that here at SDL we want to hard wire certain dependencies for ISHBootstrap and Content Manager where ISHBootstrap uses the latest and greatest.

I’ll provide some rough numbers to understand the significance of having a ready to launch template such as an AMI, Vagrant Box or docker container. The following table presents the amount of time in minutes required to get an operational Content Manager deployment with and without the builder concept.

(*) Both with Amazon and Docker, we usually get the operating system with all the latest updates. But with Hyper-V that is not true. Installing the latest windows updates on Windows Server 2016 takes an additional of 1 hour minimum. That is a lot of time to waste if you ask me.

Sadly at the moment of writing this post, the containers are not fully tested. They build just fine but something during the run step does not allow them be become operational. This is probably because the necessary parameters required are too many and a mix of filesystem and in memory variables. The reason this didn’t stop us from releasing is because containers have an experimental and learning purpose and therefore a lower and non-blocking priority.

Start using ISHBootstrap - Get in touch with your SDL liaison.

Some basic documentation is available on ISHBootstrap’s tutorials section. The main limitation is that the server dependencies for Content Manager and the CD itself are not freely accessible to everyone. If you want to get access to these files, then you need to address your SDL contact. Although the dependencies can become public, the current policy of SDL restricts the CD of becoming one. For this reason using ISHBootstrap requires some initial alignment with your SDL liaison.

For the same reason any artifact such as an AMI, Docker container or Vagrant box will not be published as it implicitly contains the above. Instead, you will have to build them using your provided access.

Extending your deliverable

As with containers, the generated artifact can be extended and modified in composition. For SDL’s cloud, we actually do this but the core implementation is common and resides in ISHBootstrap. In the same manner, you can extend the process to produce your own customized AMIs, docker containers or Vagrant boxes.

ISHBootstrap and ISHServer we’ll do the heavy lifting and you just need to add the extra magic afterwards.