Today I ran into one of those problems that makes you question your sanity. A well used API to upload content to AWS S3 was returning an indecipherable error:
MalformedXML: The XML you provided was not well-formed or did not validate against our published schema
This was kind of funny, because I was writing the code in Typescript, and uploading JSON. XML was the furthest thing from my mind.
The code was running in an AWS Lambda, on NodeJS 14. The upload function I mentioned takes a NodeJS Stream as a parameter. I did a quick bit of internet research and found that the easiest way to turn the string that I had into a Stream was to use Readable.from like this:
const stream = Readable.from(payloadJson)
However, while this code _runs_, it does not work correctly! Don’t ask me why — or please do tell me why in the comments! What’s the fix?
The other day I went to install Xcode and I was out of disk space. Deleting files helped but after a while, I noticed that no matter how many files I deleted, the disk free did not change.
I suspected that APFS Snapshots could be to blame. Because a snapshot keeps a reference to files at a point back in time. Potentially those big files I was throwing away in the Finder were still accessible.
I also uninstalled ARQ hoping this would help. it did not — somehow Arq backup had left two APFS snapshots behind, and those snapshots were causing many 10s of GB of data to be kept.
Use this command to list the snapshots for your disk
? ~ diskutil apfs listSnapshots /
Snapshot for disk1s6s1 (1 found)
NOTE: This snapshot limits the minimum size of APFS Container disk1
? ~ diskutil apfs listSnapshots /System/Volumes/Data
Snapshots for disk1s1 (2 found)
| Name: com_haystacksoftware_arqagent_0XXXX000-0XX0-00X0-X000-0X0X0XX00X00_0
| XID: 00000000
| Purgeable: Yes
| NOTE: This snapshot limits the minimum size of APFS Container disk1
Why would I mention the Testy McTestface on this blog? It’s because of S (sea rch) E (en gine) O (optim ization). So I can test that people who search for Testy McTestface in a “sir chengine” will find my new server.
Xcode’s debugger is bad about letting you see what the contents of a Data / NSData object are. Here’s my workaround:
Use po to print out the base64 version of the object, e.g.
2. Copy that to the clipboard, then use the command line in Terminal to turn that into readable content
pbpaste | base64 -D | xxd
If you know it is plain text you can omit the xxd part.
For example, here’s the output for a recent AlamoFire debugging session:
(lldb) po defaultDownloadResponse.data?.base64EncodedString()
some : "eyJzdWNjZXNzIjp0cnVlLCJwcm9maWxlIjp7InByb2ZpbGVzIjpbeyJmaWx0ZXJhYmxlIjpmYWxzZSwiZmlsdGVyX3ByaW9yaXR5IjpudWxsLCJmaWx0ZXJfdGV4dCI6bnVsbCwiZmlsdGVyX3BhcmFtZXRlcnMiOnt9LCJxdWVzdGlvbl9pY29uIjoiQmFzaWNzXzAwMiIsInF1ZXN0aW9uX2lkIjoyLCJ0YWdzIjpbIm9waW5pb25zIl0sImRlY2xpbmVkX3F1ZXN0aW9uIjpmYWxzZSwiY29tbWVudCI6IiIsImNyZWF0ZWRfYXQiOiIyMDIwLTEyLTA4VDE3OjM1OjEyLjk3NVoiLCJ1cGRhdGVkX2F0IjoiMjAyMC0xMi0wOFQxNzozNToxMi45NzVaIiwiY2hvaWNlcyI6W3sib3JkZXIiOjAsImFjdGl2ZSI6dHJ1ZSwiaWQiOjExMTk4LCJjaG9pY2VfaWQiOjI5LCJjdXN0b21fZW50cmllcyI6W10sImNyZWF0ZWRfYXQiOiIyMDIwLTEyLTEwVDE1OjQ5OjI5LjQwOFoifV19XX19"
The code generator for AppSync gets confused in this case and creates incorrect Subscription definitions. They lack the `owner` parameter, as you can see if you sign into the AppSync dashboard for your environment, and tap on the Schema link.
This is the way it should look:
onCreateUser(owner: String!): User @aws_subscribe(mutations: ["createUser"])
But this is what gets created
onCreateUser: User @aws_subscribe(mutations: ["createUser"]
Just change your schema.graphql back to having only one cognito auth type and push. AppSync schema should be back to normal and the error on launch will go away!
Has Xcode failed to export an Ad Hoc distribution file or upload to AppstoreConnect? Read on for a possible fix.
Xcode uses internal processes to do a lot of its work. Some of those tools can be influenced by environment variables that may be set up by your terminal shell.
Normally when Xcode is launched from the Finder, properties you set in your terminal shell don’t matter. But it’s different if you’ve learned to use the command line program open to launch Xcode. When you do that, the Xcode process inherits the terminal environment. Including virtual environment settings such as virtualenv, pyenv, or a version manager such as rvm. It’s that last one that can cause this inexplicable error:
If you click Show Logs you might find:
2020-09-29 02:38:10 +0000 /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require': incompatible library version - /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/date-3.0.0/lib/date_core.bundle (LoadError) from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/date-3.0.0/lib/date.rb:4:in <top (required)>' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require' from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/CFPropertyList-3.0.2/lib/cfpropertylist/rbCFPropertyList.rb:4:in'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/CFPropertyList-3.0.2/lib/cfpropertylist.rb:3:in <top (required)>' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:130:inrequire'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:130:in `rescue in require'
The fix suggested here is clever, but didn’t actually work for me. But it got me wondering — why would Xcode invoke my .bashrc or .zshrc file anyhow? It shouldn’t be creating a shell process at all to run its commands. That got me thinking maybe the Xcode process itself was tainted with a bad environment. Then I remembered, I often use open to start Xcode, e.g. from a source code directory with an ios directory containing an Xcode workspace:
$ open ./ios/myproject.xcworkspace
I open to be easier than clicking around in the Finder. Especially when my current working directory is already there.
But be forewarned — if you see Xcode starting to generate bizarre errors, it’s because of the Ruby Version Manager selecting a version of Ruby that doesn’t have the necessary Gems.
The fix couldn’t be easier. Quit Xcode. Then relaunch it — using the Dock, the Finder, or Apple Menu recent applications. Just not the shell command, open.
Suppose you have two related projects with a similar file/folder structure. You want to compare the files by the same name in the two projects. You can use some magic of pipes to build a list of filenames that are the same (and in the same relative directories) in the two projects like this:
Have you ever used AWS Amplify with AppSync and done the command amplify push only to have it work for about 5 minutes and then fail, with an annoying error, “Attempting to edit the global secondary index”?
It happens sometimes, especially in development when iterating on the design of relationship in your schema. GraphQL is a powerful language for expressing relationships between objects in your data model. This limitation of DynamoDB makes this iteration painful. DynamoDB cannot update, modify, create, or delete more than on GSI (Global Secondary Index) at a time on a particular table.
How Did I Get Here
You probably made multiple changes to your schema. Every time you add a line like @key(name: "keyName", field: ["objectId"]) you’re ultimately causing Amplify to add a GSI to your DynamoDB Table. Add more than one between calls to amplify push and you’re going to encounter this error.
The Solution: Easy Part
The first thing to do is to log into the AWS Console, navigate to DynamoDB, and examine your table and it’s indexes. amplify push will complete BEFORE the indexes are finished being created in DynamoDB. If you try to push a second time before DynamoDB is finished, you’ll get an error.
Look for any line that says “Creating”. If it’s there, you just have to wait. Click the refresh button periodically to find out if it’s finished yet. Then try amplify push again.
The Harder, But Sure Fix
Some folks recommend amplify remove api and then amplify add api as a way to fix this problem. That is a BIG hammer. For one thing, it will DELETE ALL YOUR DATA. For another, it’s not needed. And also, if you have inter-dependencies, the remove call might fail, for example if you have an amplify function connected to Cognito as a post-user-sign-up hook.
Step 1: Find the CloudFormation Template for each Table
Use AWS Console to navigate to CloudFormation. Find the top level cloud formation stack for your environment. Click on that and look at its resources until you find api. Then click on that until you find the templates for each of the Object types in your schema. We’ll need to go through the following steps for *each* of those.
Step 2: Download the Current Template
I simply copied the current template onto my clipboard, and then saved it as a new file:
X=Segmentpbpaste > cur_$X.json
Use YOUR object name, not Segment. Forgive me, I am a Command Line geek. Use whatever tools you want, e.g. create a new file in a text editor, paste it, and then use your favorite file comparison tool for the next step.
Step 3: Compare AND EDIT the Template
I’m a vim user so this was all I had to do to compare them
vim -d cur_$X.json amplify/backend/api/appname/build/stacks/$X.json
Use YOUR appname, not appname.
Look at the section GlobalSecondaryIndexes. Are there more than 1 difference between the current template and the new one the Amplify want’s to create? Edit the downloaded JSON, bringing over only one of the changes, so that there’s only one difference. We’ll be adding those over one at a time, deploying the change after each edit. Yes, I know, this is tedious. But it’s better than destroying your entire environment and losing all the data.
Step 4: Update the Template in CloudFormation
Now go to the template for that Object. Click Update from the Template tab. It will say “It is recommended to update through the root stack“. Choose “Update nested stack”.
Select “Replace current template” and “Upload a template file”
and upload your edited JSON file
Then click Next a bunch
You will have to give it permission to do something with IAM. Click the box.
With luck, it will succeed
Step 5: Goto Step 3
If you had to remove more than one GSI change, add it back — again one-at-a-time and repeat starting at Step 3.
Step 6: Goto Step 2
Repeat this for all your Objects (which each has their own Table).
Step 7: Amplify Push Will Work!
Now that the GSIs have all been created, amplify push will be successful!
? Generated GraphQL operations successfully and saved at src/graphql
? Code generated successfully and saved in file src/API.ts
? All resources are updated in the cloud
GraphQL endpoint: https://<redacted>.appsync-api.us-east-1.amazonaws.com/graphql