Malformed XML using s3.upload

Today I ran into one of those problems that makes you question your sanity. A well used API to upload content to AWS S3 was returning an indecipherable error:

MalformedXML: The XML you provided was not well-formed or did not validate against our published schema

This was kind of funny, because I was writing the code in Typescript, and uploading JSON. XML was the furthest thing from my mind.

The code was running in an AWS Lambda, on NodeJS 14. The upload function I mentioned takes a NodeJS Stream as a parameter. I did a quick bit of internet research and found that the easiest way to turn the string that I had into a Stream was to use Readable.from like this:

const stream = Readable.from(payloadJson)

However, while this code _runs_, it does not work correctly! Don’t ask me why — or please do tell me why in the comments! What’s the fix?

  const stream = new Readable();
  stream._read = () => {}
  stream.push(payloadJson);
  stream.push(null);

That is some serious black magic. What an awful API! But it works.