Accepted, Not Yet Planned

Make publishing of containers default to publishing the contents individually

Currently, if you publish a publication or a structure group, you end up with a single publishing transaction. We've accepted this for years because it's "just the way Tridion works".

I would generally suggest that creating one big publish transaction is not what's wanted. In most reasonably large sites, if you published the root structure group, you'd probably break something. Common approaches usually involve deliberately breaking publish actions down to a smaller granularity. This gives you firstly some resilience against failures: you can republish only the things that failed. Secondly, assuming a finite amount of render threads, by publishing a lot of large SGs, you can easily tie up all your available threads for several minutes at a time, which means you won't be able to get your small-but-important job to publish straight away, even with high prio.

In general, I have my doubts about the usefulness of large transactions. There may once have been a theoretical possibility of transactional deployment of an entire site, but in practice this isn't a high priority for many people. My suggestion would be to make all publish actions on containers default to queuing the contents as individual items. Maybe you'd want to keep the current behaviour as a checkbox option. Maybe there'd need to be support for controlling the default setting for the checkbox.

I don't know how this would impact the content delivery side. At first sight, you'd imagine that smaller transactions would have less collisions/rollbacks/retries.

Dominic Cronin

  • "Perhaps historical data might be a separate idea to know what might fail for some reason."

    This reminds me of build history in Jenkins. Each task keeps track of the last X number of build jobs (five or so), with how long each step took, who or what triggered it, success/failure, etc. It can use the history of previous builds to estimate how long the next build will take.

    Tridion publishing tracks much of the same kind of metadata, but from the UI side, it doesn't show a history of anything before the last publish.

    On a loosely related note, one of the most important features for our users whenever anything is incorrectly published, is to know who, when, and what. Once an issue is discovered, corrected, and republished, it is then too late to do a post-morten to find out who caused the problem by publishing something incorrect and when it happened, because the publish queue has updated to show the subsequent publish.

  • I suspect there are fewer render failures with DXA, DD4T, or similar template approaches. I'm actually wondering if we could remove the "iginore failures" option from the UI.

    Transactionality

    Where can we vote for instant publishing where we wouldn't need queues or priority? ;-) I believe transactions for publishing would be okay with really fast publishing (and still desired with Workflow). In a different, but related queue, we don't necessarily want transactions for Translation Manager. If a single Component fails or needs rework, we want the rest of the work to continue. Like publishing, translation involves time but the people and money factor favor some items over failing everything.

    Hogging of publisher capacity / Transport Package size

    I like the publisher configuration ideas, but I'd worry about the expertise needed to understand and tweak these settings. And even if change-able, they wouldn't necessarily adapt to publishing needs, which might be based on time of day, day of week, or calendar date.

    My 2 cents is we'd first want more awareness on how publishing is working, before more control. In the long run, I see our own Cloud and the community picking up automatic scaling solutions (which might still include complex options, but managed by scripts).

    Publish Transaction granularity / Who decides?

    On the configuration, we can compare to requests for Translation Manager for more granularity on included items. The discussions ultimately end with the idea of making a Content Porter-like interface to granularly select options for each item in some resolved list. :-(

    I would expect Workflow setups would still be as transactional as ever with no changes to their existing workflow designs.

    For users, I can imagine a default option to break up Structure Group publishing into separate Pages. Perhaps they could choose to keep the transaction in advanced options, which either stays the same until the user changes it (or maybe it's a personal preference).

    For an even smaller idea, this might be system-wide similar to the Workflow as Collaboration and Translate Minor version settings. These make support/testing harder for us, but at least users don't have to choose each time.

    To point, ideally publishing could proactively warn users of problems. But since we don't what would fail without trying to render, resolve, publish, and trying to commit, the best we have are reasonable guesses. Perhaps historical data might be a separate idea to know what might fail for some reason.

  • Regarding workarounds, the Alchemy extension is not the only way - as Jonathan says, scripts are common. I've written several publishing scripts. My "standard" approach is to mimic what end users do, and publish second-level structure groups (or third or whatever depending on your content). The other old favourite is simply to search within a structure group and then select and publish all the pages in your search results.

  • Automatically splitting publish transactions would be good. That would separate the functional notion of a publish action from the technical/infrastructure constraints that the end user shouldn't be worried about. Infra people would be very happy to just be able to say "keep it smaller than X"

  • Thanks for submitting this idea.  It seems there are multiple reasons for wanting such functionality. We would like to get a better picture on the different aspects; it seems there may be alternative solutions for some of the aspects. Please provide feedback on the questions in the orange box.