Category: Weird Errors

Everyone loves GIT, the source control system. But sometimes you can end up in a situation where a pull request is showing duplicate commits for some work you did long ago in the past.

Here’s a technique for recovering from that.

First, make sure you have NO uncommited files. Type git status and make sure it reports “nothing to commit, working tree clean”. Second, type git log and copy the last commit id and email it to yourself or do something with it just in case you need to recover from some terrible mistake.

First, the easiest method, but one that does not always work:

git fetch origin
git rebase origin/develop

If this works without complaint, you’re probably golden. Simply push your branch — possibly via git push -f if you’ve previously pushed this branch. This forcibly replaces the remote but that’s probably okay because we just fixed it.

If you’re still here, that’s because rebase didn’t work. Let’s abort by typing

git rebase --abort

Now let’s try doing cherry-pick. This requires that you list all the commits that you’ve made that you want to keep. Let’s look at the git log and copy to a text file all the commits that you want to keep

git log --oneline --graph

At some point as you read down that list you’ll get to old commits that are already part of origin/develop. You’ll need that list of commit ids from oldest to newest, for example if you saw this:

* 754fe93 My most recent commit
* 953511b More work on this feature
* 5a6df59 My first commit for this feature
*   19ec9be Merge develop into my branch
|\  
| * 6804047 An old commit that's somehow duplicate
| *   65e7f73 another old commit ..?

The list of commit ids you’d need would be 5a6df59 953511b 754fe93. Now to fix your branch, you would do

git fetch origin
git reset --hard origin/develop
git cherry-pick 5a6df59 953511b 754fe93

If this works without trouble, then you’re golden. Go to “you’re probably golden” above and you’re done.

If you’re still here, that’s because cherry-pick also failed you. It might be possible to fix this by doing a series of merges, but that can get tedious. So abandon this by doing

git cherry-pick --abort

The last technique I employ is a big hammer. This requires that you merge develop into your branch:

git fetch origin
git log

Now remember (copy) the commit id of the most recent commit. Now reset your branch back to origin/develop and squash your commit onto it.

git reset --hard origin/develop
git merge --squash <copied commit id>
git commit -m 'my commit message goes here'

Prior to the commit, you may have to fix merge conflicts. Such is life.

You now have one commit that is the sum of your changes from your branch! If you want it to be multiple smaller commits you could do that by resetting and committing parts one at a time.

If at any time you panic and feel you’ve lost work, simply use the command

git reset --hard <commit id you copied at the start and emailed to yourself>

And you should be back to the state you started in.

Have you ever used AWS Amplify with AppSync and done the command amplify push only to have it work for about 5 minutes and then fail, with an annoying error, “Attempting to edit the global secondary index”?

It happens sometimes, especially in development when iterating on the design of relationship in your schema. GraphQL is a powerful language for expressing relationships between objects in your data model. This limitation of DynamoDB makes this iteration painful. DynamoDB cannot update, modify, create, or delete more than on GSI (Global Secondary Index) at a time on a particular table.

How Did I Get Here

You probably made multiple changes to your schema. Every time you add a line like @key(name: "keyName", field: ["objectId"]) you’re ultimately causing Amplify to add a GSI to your DynamoDB Table. Add more than one between calls to amplify push and you’re going to encounter this error.

The Solution: Easy Part

The first thing to do is to log into the AWS Console, navigate to DynamoDB, and examine your table and it’s indexes. amplify push will complete BEFORE the indexes are finished being created in DynamoDB. If you try to push a second time before DynamoDB is finished, you’ll get an error.

Look for any line that says “Creating”. If it’s there, you just have to wait. Click the refresh button periodically to find out if it’s finished yet. Then try amplify push again.

The Harder, But Sure Fix

Some folks recommend amplify remove api and then amplify add api as a way to fix this problem. That is a BIG hammer. For one thing, it will DELETE ALL YOUR DATA. For another, it’s not needed. And also, if you have inter-dependencies, the remove call might fail, for example if you have an amplify function connected to Cognito as a post-user-sign-up hook.

Step 1: Find the CloudFormation Template for each Table

Use AWS Console to navigate to CloudFormation. Find the top level cloud formation stack for your environment. Click on that and look at its resources until you find api. Then click on that until you find the templates for each of the Object types in your schema. We’ll need to go through the following steps for *each* of those.

Step 2: Download the Current Template

I simply copied the current template onto my clipboard, and then saved it as a new file:

X=Segment
pbpaste > cur_$X.json

Use YOUR object name, not Segment. Forgive me, I am a Command Line geek. Use whatever tools you want, e.g. create a new file in a text editor, paste it, and then use your favorite file comparison tool for the next step.

Step 3: Compare AND EDIT the Template

I’m a vim user so this was all I had to do to compare them

vim -d cur_$X.json amplify/backend/api/appname/build/stacks/$X.json

Use YOUR appname, not appname.

Look at the section GlobalSecondaryIndexes. Are there more than 1 difference between the current template and the new one the Amplify want’s to create? Edit the downloaded JSON, bringing over only one of the changes, so that there’s only one difference. We’ll be adding those over one at a time, deploying the change after each edit. Yes, I know, this is tedious. But it’s better than destroying your entire environment and losing all the data.

Step 4: Update the Template in CloudFormation

Now go to the template for that Object. Click Update from the Template tab. It will say “It is recommended to update through the root stack“. Choose “Update nested stack”.

Select “Replace current template” and “Upload a template file”

and upload your edited JSON file

Then click Next a bunch

You will have to give it permission to do something with IAM. Click the box.

With luck, it will succeed

Step 5: Goto Step 3

If you had to remove more than one GSI change, add it back — again one-at-a-time and repeat starting at Step 3.

Step 6: Goto Step 2

Repeat this for all your Objects (which each has their own Table).

Step 7: Amplify Push Will Work!

Now that the GSIs have all been created, amplify push will be successful!

? Generated GraphQL operations successfully and saved at src/graphql
? Code generated successfully and saved in file src/API.ts
? All resources are updated in the cloud
GraphQL endpoint: https://<redacted>.appsync-api.us-east-1.amazonaws.com/graphql