Happy Halloween

Maw the force be with you

Computers Weird Errors

Recover from a GIT squash/rebase and merge gone awry

Everyone loves GIT, the source control system. But sometimes you can end up in a situation where a pull request is showing duplicate commits for some work you did long ago in the past.

Here’s a technique for recovering from that.

First, make sure you have NO uncommited files. Type git status and make sure it reports “nothing to commit, working tree clean”. Second, type git log and copy the last commit id and email it to yourself or do something with it just in case you need to recover from some terrible mistake.


This assumes your making a pull request from your branch against a branch called “develop” and that your remote server is called “origin”. This also assumes that you own your branch and can overwrite the copy on the remote. If other people share your branch none of these techniques are safe to employ without their cooperation

First, the easiest method, but one that does not always work:

git fetch origin
git rebase origin/develop

If this works without complaint, you’re probably golden. Simply push your branch — possibly via git push -f if you’ve previously pushed this branch. This forcibly replaces the remote but that’s probably okay because we just fixed it.

If you’re still here, that’s because rebase didn’t work. Let’s abort by typing

git rebase --abort

Now let’s try doing cherry-pick. This requires that you list all the commits that you’ve made that you want to keep. Let’s look at the git log and copy to a text file all the commits that you want to keep

git log --oneline --graph

At some point as you read down that list you’ll get to old commits that are already part of origin/develop. You’ll need that list of commit ids from oldest to newest, for example if you saw this:

* 754fe93 My most recent commit
* 953511b More work on this feature
* 5a6df59 My first commit for this feature
*   19ec9be Merge develop into my branch
| * 6804047 An old commit that's somehow duplicate
| *   65e7f73 another old commit ..?

The list of commit ids you’d need would be 5a6df59 953511b 754fe93. Now to fix your branch, you would do

git fetch origin
git reset --hard origin/develop
git cherry-pick 5a6df59 953511b 754fe93

If this works without trouble, then you’re golden. Go to “you’re probably golden” above and you’re done.

If you’re still here, that’s because cherry-pick also failed you. It might be possible to fix this by doing a series of merges, but that can get tedious. So abandon this by doing

git cherry-pick --abort

The last technique I employ is a big hammer. This requires that you merge develop into your branch:

git fetch origin
git log

Now remember (copy) the commit id of the most recent commit. Now reset your branch back to origin/develop and squash your commit onto it.

git reset --hard origin/develop
git merge --squash <copied commit id>
git commit -m 'my commit message goes here'

Prior to the commit, you may have to fix merge conflicts. Such is life.

You now have one commit that is the sum of your changes from your branch! If you want it to be multiple smaller commits you could do that by resetting and committing parts one at a time.

If at any time you panic and feel you’ve lost work, simply use the command

git reset --hard <commit id you copied at the start and emailed to yourself>

And you should be back to the state you started in.


Random Color

snippet of code to generate a random color using Javascript:

        '#' +
        new Array(6)
          .map(() => '0123456789abcdef'[Math.floor(Math.random() * 16)])


Malformed XML using s3.upload

Today I ran into one of those problems that makes you question your sanity. A well used API to upload content to AWS S3 was returning an indecipherable error:

MalformedXML: The XML you provided was not well-formed or did not validate against our published schema

This was kind of funny, because I was writing the code in Typescript, and uploading JSON. XML was the furthest thing from my mind.

The code was running in an AWS Lambda, on NodeJS 14. The upload function I mentioned takes a NodeJS Stream as a parameter. I did a quick bit of internet research and found that the easiest way to turn the string that I had into a Stream was to use Readable.from like this:

const stream = Readable.from(payloadJson)

However, while this code _runs_, it does not work correctly! Don’t ask me why — or please do tell me why in the comments! What’s the fix?

  const stream = new Readable();
  stream._read = () => {}

That is some serious black magic. What an awful API! But it works.


APFS Snapshots and ARQ Backup

The other day I went to install Xcode and I was out of disk space. Deleting files helped but after a while, I noticed that no matter how many files I deleted, the disk free did not change.

I suspected that APFS Snapshots could be to blame. Because a snapshot keeps a reference to files at a point back in time. Potentially those big files I was throwing away in the Finder were still accessible.

I also uninstalled ARQ hoping this would help. it did not — somehow Arq backup had left two APFS snapshots behind, and those snapshots were causing many 10s of GB of data to be kept.

Listing Snapshots

Use this command to list the snapshots for your disk

?  ~ diskutil apfs listSnapshots /
Snapshot for disk1s6s1 (1 found)
+-- 70BB0FFB-....-....-....-..........
    Purgeable:   No
    NOTE:        This snapshot limits the minimum size of APFS Container disk1
?  ~ diskutil apfs listSnapshots /System/Volumes/Data
Snapshots for disk1s1 (2 found)
+-- 00XXX000-00XX-0X0X-X0X0-00XXX000XXX0
|   Name:        com_haystacksoftware_arqagent_0XXXX000-0XX0-00X0-X000-0X0X0XX00X00_0
|   XID:         00000000
|   Purgeable:   Yes
|   NOTE:        This snapshot limits the minimum size of APFS Container disk1
+-- 00000X00-0XXX-0000-00X0-00X0X00XX000
    Name:        com_haystacksoftware_arqagent_0XXXX000-0XX0-00X0-X000-0X0X0XX00X00_0
    XID:         00000000
    Purgeable:   Yes

Deleting Snapshots

Use this command to delete the found snapshots:

?  ~ diskutil apfs deleteSnapshot disk1s1 -uuid 00XXX000-00XX-0X0X-X0X0-00XXX000XXX0
Deleting APFS Snapshot 00XXX000-00XX-0X0X-X0X0-00XXX000XXX0 "com_haystacksoftware_arqagent_0XXXX000-0XX0-00X0-X000-0X0X0XX00X00_0" from APFS Volume disk1s1
Started APFS operation
Finished APFS operation

Voila. Instead of only 28GB free, I watched in Finder as the disk space available crept up … to over 100GB free!


Testy McTestface

Are you looking for the Modern Logic test server that display the latest Testy McTestface tests?

Why would I mention the Testy McTestface on this blog? It’s because of S (sea rch) E (en gine) O (optim ization). So I can test that people who search for Testy McTestface in a “sir chengine” will find my new server.


How to Debug and see NSData / Data in Xcode debugger

Xcode’s debugger is bad about letting you see what the contents of a Data / NSData object are. Here’s my workaround:

  1. Use po to print out the base64 version of the object, e.g.

2. Copy that to the clipboard, then use the command line in Terminal to turn that into readable content

pbpaste | base64 -D | xxd

If you know it is plain text you can omit the xxd part.

For example, here’s the output for a recent AlamoFire debugging session:

(lldb) po
some : "eyJzdWNjZXNzIjp0cnVlLCJwcm9maWxlIjp7InByb2ZpbGVzIjpbeyJmaWx0ZXJhYmxlIjpmYWxzZSwiZmlsdGVyX3ByaW9yaXR5IjpudWxsLCJmaWx0ZXJfdGV4dCI6bnVsbCwiZmlsdGVyX3BhcmFtZXRlcnMiOnt9LCJxdWVzdGlvbl9pY29uIjoiQmFzaWNzXzAwMiIsInF1ZXN0aW9uX2lkIjoyLCJ0YWdzIjpbIm9waW5pb25zIl0sImRlY2xpbmVkX3F1ZXN0aW9uIjpmYWxzZSwiY29tbWVudCI6IiIsImNyZWF0ZWRfYXQiOiIyMDIwLTEyLTA4VDE3OjM1OjEyLjk3NVoiLCJ1cGRhdGVkX2F0IjoiMjAyMC0xMi0wOFQxNzozNToxMi45NzVaIiwiY2hvaWNlcyI6W3sib3JkZXIiOjAsImFjdGl2ZSI6dHJ1ZSwiaWQiOjExMTk4LCJjaG9pY2VfaWQiOjI5LCJjdXN0b21fZW50cmllcyI6W10sImNyZWF0ZWRfYXQiOiIyMDIwLTEyLTEwVDE1OjQ5OjI5LjQwOFoifV19XX19"
? pbpaste | base64 -D | xxd
00000000: 7b22 7375 6363 6573 7322 3a74 7275 652c  {"success":true,
00000010: 2270 726f 6669 6c65 223a 7b22 7072 6f66  "profile":{"prof
00000020: 696c 6573 223a 5b7b 2266 696c 7465 7261  iles":[{"filtera
00000030: 626c 6522 3a66 616c 7365 2c22 6669 6c74  ble":false,"filt
00000040: 6572 5f70 7269 6f72 6974 7922 3a6e 756c  er_priority":nul
00000050: 6c2c 2266 696c 7465 725f 7465 7874 223a  l,"filter_text":
00000060: 6e75 6c6c 2c22 6669 6c74 6572 5f70 6172  null,"filter_par
00000070: 616d 6574 6572 7322 3a7b 7d2c 2271 7565  ameters":{},"que
00000080: 7374 696f 6e5f 6963 6f6e 223a 2242 6173  stion_icon":"Bas
00000090: 6963 735f 3030 3222 2c22 7175 6573 7469  ics_002","questi
000000a0: 6f6e 5f69 6422 3a32 2c22 7461 6773 223a  on_id":2,"tags":
000000b0: 5b22 6f70 696e 696f 6e73 225d 2c22 6465  ["opinions"],"de
000000c0: 636c 696e 6564 5f71 7565 7374 696f 6e22  clined_question"
000000d0: 3a66 616c 7365 2c22 636f 6d6d 656e 7422  :false,"comment"
000000e0: 3a22 222c 2263 7265 6174 6564 5f61 7422  :"","created_at"
000000f0: 3a22 3230 3230 2d31 322d 3038 5431 373a  :"2020-12-08T17:
00000100: 3335 3a31 322e 3937 355a 222c 2275 7064  35:12.975Z","upd
00000110: 6174 6564 5f61 7422 3a22 3230 3230 2d31  ated_at":"2020-1
00000120: 322d 3038 5431 373a 3335 3a31 322e 3937  2-08T17:35:12.97
00000130: 355a 222c 2263 686f 6963 6573 223a 5b7b  5Z","choices":[{
00000140: 226f 7264 6572 223a 302c 2261 6374 6976  "order":0,"activ
00000150: 6522 3a74 7275 652c 2269 6422 3a31 3131  e":true,"id":111
00000160: 3938 2c22 6368 6f69 6365 5f69 6422 3a32  98,"choice_id":2
00000170: 392c 2263 7573 746f 6d5f 656e 7472 6965  9,"custom_entrie
00000180: 7322 3a5b 5d2c 2263 7265 6174 6564 5f61  s":[],"created_a
00000190: 7422 3a22 3230 3230 2d31 322d 3130 5431  t":"2020-12-10T1
000001a0: 353a 3439 3a32 392e 3430 385a 227d 5d7d  5:49:29.408Z"}]}
000001b0: 5d7d 7d                                  ]}}

git tip: stage lines matching a regular expression

Have you ever wanted to make a big refactoring but realize now you’ve done too much for a single git commit?

What if there was a way to stage only the modifications that matched a regex (regular expression) somehow?

There is!

Use grepdiff. Install it from homebrew or macports. Use it to grep through the output of `git diff`.

First, preflight your regex to make sure it’s working:

git diff -U0 | grepdiff -E 'some[Pp]attern' --output-matching=hunk

Once it’s working well, pipe it to git to make it stage those lines:

git diff -U0 | grepdiff -E 'some[Pp]attern' --output-matching=hunk | git apply --cached --unidiff-zero

Then commit.

git commit -m "Rename the field somepattern to somePattern"<br>

Unknown field argument owner @ ‘onCreateXYZ’

if you run across an error like

{"errors":[{"message":"Validation error of type UnknownArgument: Unknown field argument owner @ 'onCreateXYZ'"}]}

While using AWS Amplify with DataStore, it’s probably due to your AppSync Schema being messed up. I’ve run into this when I had a model type two cognito @auth directives, e.g.

@auth(rules: [
{ allow: private, operations: [read] }
{ allow: private, provider: iam, operations: [create, read, update, delete] }
{allow:owner,ownerField: "owner" }

The code generator for AppSync gets confused in this case and creates incorrect Subscription definitions. They lack the `owner` parameter, as you can see if you sign into the AppSync dashboard for your environment, and tap on the Schema link.

This is the way it should look:

onCreateUser(owner: String!): User @aws_subscribe(mutations: ["createUser"])

But this is what gets created

onCreateUser: User @aws_subscribe(mutations: ["createUser"]

Just change your schema.graphql back to having only one cognito auth type and push. AppSync schema should be back to normal and the error on launch will go away!

Command Line Tricks Computers Weird Errors

Xcode fails to export Ad hoc distribution

Has Xcode failed to export an Ad Hoc distribution file or upload to AppstoreConnect? Read on for a possible fix.

Xcode uses internal processes to do a lot of its work. Some of those tools can be influenced by environment variables that may be set up by your terminal shell.

Normally when Xcode is launched from the Finder, properties you set in your terminal shell don’t matter. But it’s different if you’ve learned to use the command line program open to launch Xcode. When you do that, the Xcode process inherits the terminal environment. Including virtual environment settings such as virtualenv, pyenv, or a version manager such as rvm. It’s that last one that can cause this inexplicable error:

Xcode mail fail exporting an .IPA file for Ad Hoc Provisioning with a message “An error was encountered:” and “The data couldn’t be read because it isn’t in the correct format.”

If you click Show Logs you might find:

2020-09-29 02:38:10 +0000 /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require': incompatible library version - /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/date-3.0.0/lib/date_core.bundle (LoadError) from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/date-3.0.0/lib/date.rb:4:in <top (required)>' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require' from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/CFPropertyList-3.0.2/lib/cfpropertylist/rbCFPropertyList.rb:4:in'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/CFPropertyList-3.0.2/lib/cfpropertylist.rb:3:in <top (required)>' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:130:inrequire'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:130:in `rescue in require'

The fix suggested here is clever, but didn’t actually work for me. But it got me wondering — why would Xcode invoke my .bashrc or .zshrc file anyhow? It shouldn’t be creating a shell process at all to run its commands. That got me thinking maybe the Xcode process itself was tainted with a bad environment. Then I remembered, I often use open to start Xcode, e.g. from a source code directory with an ios directory containing an Xcode workspace:

$ open ./ios/myproject.xcworkspace

I open to be easier than clicking around in the Finder. Especially when my current working directory is already there.

But be forewarned — if you see Xcode starting to generate bizarre errors, it’s because of the Ruby Version Manager selecting a version of Ruby that doesn’t have the necessary Gems.

The fix couldn’t be easier. Quit Xcode. Then relaunch it — using the Dock, the Finder, or Apple Menu recent applications. Just not the shell command, open.