XPP Rest return values and order of operations

In javascript running on a content server, I am using axios.get to run an XPP Rest "User command," which successfully executes a Perl script on the XPP server to copy a file from one folder to another on that server. I am separately signed into the XPP server, and can see that the file has been copied successfully. On the javascript side, I am receiving status code 200 (success), which I think means that XPP Rest found the Perl script and ran it. Unfortunately, code 200 does not mean that the file copied successfully on the XPP server, because the Perl script fails silently when I ask it to copy a nonexistent file. I am looking for a way please to provide a meaningful return value from the Perl script back to the javascript, to verify that the file copy took place and therefore that subsequent actions can proceed. Thank you!

emoji
  • Well, in your perl, you could just check for existence of source file (before copying) and destination file (after copying) and if doesn't exist, exit with error code that javascript will also interpret as fail.

    emoji
  • Thank you, Shawn, exiting the Perl with an error code (if the file copy return value indicates failure) seems to work!

    In the javascript, I believe that I have written the code correctly with promises to wait for a return value before moving on to the next function.  I make another User Command call after the first one, to perform a similar file copy function.  Observing the filetimes on the XPP side with "ls --full-time", it looks like the second file is copied a few milliseconds after the first, as expected.

    The next function call in the javascript is XPP RESTful function Put File, to copy a file from the content server to the XPP server.  Based on filetimes, the Put File appears to complete its file operation a few milliseconds before the second User Command file copy.  This seems to indicate that one or more file handling functions on the XPP side are asynchronous or returning positive values before the file operations are complete, or possibly, that the filetimes obtained with "ls --full-time" are not accurate.

    I could try to introduce additional checks or delays on the javascript side, though that would be less elegant.  I am open to suggestions for how to ensure the correct sequence of operations, thank you!

    emoji
  • Peter,

    As is typical for a REST interface things run asynchronously. That is if you do 2 separate calls, the second call will not wait for the first call to end. They both run as independent processes. So there is no way of telling which one will complete first.
    If you want to stay in control, you have to set the "nowait=true" option on each of the calls.
    As the doc says: 
    If "nowait=true", the command will return immediately with the id, code, and time the command started (default="false"). The user can check for command completion, by using the 'xpp/docs/processes/id' endpoint;
    Checking the command status will return either 'running' or the exit code of the process you started.

    emoji
  • Thank you, Bart!  After sending my previous message, with further testing I realized that the Linux command "ls --full-time" was showing me last-modified file times, not file creation times.  My implementation in javascript appears to be working so far.  Each RESTful call is wrapped in a function that returns a promise, the function calls have .then blocks that wait for the responses, and the next function call is embedded in the previous function's .then block.  I may still need to use nowait=true, we'll see!  Best regards, Peter

    emoji
  • Just to follow-up, using promises with .then blocks was successful!  Also, rather than while or for loops, I used recursive functions to enforce the order of operations.  Thank you for your assistance!  Best regards, Peter

    emoji
  • Over a year ago now I was sent some information on this XPP RESTful WS discussion of async/sync behavior by an "XPP expert" (you can probably guess who). At the time I didn't do anything with the information that was sent to me, because I knew nothing really about XPP RESTful WS and little of it made any sense to me. I didn't feel comfortable passing on something that I didn't understand at all.

    I have a better understanding now of XPP RESTful WS and so I will pass on the information that was sent to me; it might prove useful to others:

    From a low-level server-side socket implementation of the XPP RESTful API, each “get”, “post”, “del”, etc. URL API call is actually synchronous, i.e. the URL will not return information back to the socket calling the API via the HTTP command until it has been completed. The node.js implementation of XPPrest goes out of its way to do this. The “nowait” supported commands (like compose and print) are also "synchronous" in that they start the requested functionality in the background but return an ID that can be used to determine if the command has completed. It will not return until this has happened. The "asynchronous" aspect of these commands is an artifact of the API itself and not due to async behavior. As stated, they won't return until everything is established and hence the HTTP “get” is "synchronous" (even though background tasks have started).

    However, the XPP RESTful connections are via HTTP clients, and this is where is gets more complicated.

    JavaScript client-side modules (and possibly Java) speaking REST are most often ‘async’. It is up to the client program to make URL calls but manage when they are completed via “callback” functions and or newer sync/async functionality using “Promises”. You will have to read up on that as the scope of that topic is large. So, if you are using these types of modules without attending to these details, and just structure successive XPP RESTful API calls they might fail because the call has come back but the command has not finished because you have not captured the return of the asynchronous call you made.

    The XPP Restful JavaScript test suite uses a “super-agent” module for testing and assistance for doing this kind of thing. You will see calls to ‘async’ and ‘await’ to force sequences to complete before moving onto the next one.

    The standard PERL HTTP/Socket module implements HTTP requests as synchronous. So, it is very easy to make a call and when it is completed you know that the XPP command has finished. The Perl test suite puts this to good use and steps through each of the API calls and in fact depends on the order since it is doing things like creating divisions, testing existence, toxsf, etc., and these steps have to be done in a very specific application order.

    XPP QA’s cloud speed test suite uses another PERL module, and it is asynchronous. A call is made to a REST URL (compose without the ‘nowait’), and it returns immediately. This is client side, and a thread has been created to monitor the API call and wait for the REST call to finish but it immediately returns an object ID that can be used to monitor the task. Again, this is client side and not to be confused with the “nowait” that has been built into the XPP RESTful API. The actual ‘get’ is waiting for the response back from XPP REST, but it has returned back to the client program so it can make other ‘async’ calls as it sees fit.

    I wanted to share this information because of Bart’s comment that XPP RESTful is "asynchronous".  From a pure perspective this is patently not true, as at the lowest lever, the HTTP ‘socket’ will not return data until the command has finished. However, as I tried to explain above, you have to know (and correctly use) the client API in your development language to handle async/sync behavior. Use it incorrectly and it will bite you.

    Jonathan Dagresta
    RWS Group/
    XPP Development

    emoji
  • Thanks Jonathan (and the secret XPP expert Grinning) for going deeper into this and providing us with the correct explanation on how things work
    My use of the word ' asynchronous' was indeed very misleading.
    I was thinking more in the lines of that it is very well possible to send out another http request before the first one is finished. But I did not think things through correctly and was pointing in the wrong direction by calling the http request asynchronous which it clearly isn't.
    The possible problem lays indeed on the calling end where with something like javascript it is very easy to make a request and not wait for the response before you move on to the next thing.
    So sorry, my mistake.
    But happy to see this corrected.


    emoji