Categories
Computers Weird Errors

Recover from a GIT squash/rebase and merge gone awry

Everyone loves GIT, the source control system. But sometimes you can end up in a situation where a pull request is showing duplicate commits for some work you did long ago in the past.

Here’s a technique for recovering from that.

First, make sure you have NO uncommited files. Type git status and make sure it reports “nothing to commit, working tree clean”. Second, type git log and copy the last commit id and email it to yourself or do something with it just in case you need to recover from some terrible mistake.

NOTE

This assumes your making a pull request from your branch against a branch called “develop” and that your remote server is called “origin”. This also assumes that you own your branch and can overwrite the copy on the remote. If other people share your branch none of these techniques are safe to employ without their cooperation

First, the easiest method, but one that does not always work:

git fetch origin
git rebase origin/develop

If this works without complaint, you’re probably golden. Simply push your branch — possibly via git push -f if you’ve previously pushed this branch. This forcibly replaces the remote but that’s probably okay because we just fixed it.

If you’re still here, that’s because rebase didn’t work. Let’s abort by typing

git rebase --abort

Now let’s try doing cherry-pick. This requires that you list all the commits that you’ve made that you want to keep. Let’s look at the git log and copy to a text file all the commits that you want to keep

git log --oneline --graph

At some point as you read down that list you’ll get to old commits that are already part of origin/develop. You’ll need that list of commit ids from oldest to newest, for example if you saw this:

* 754fe93 My most recent commit
* 953511b More work on this feature
* 5a6df59 My first commit for this feature
*   19ec9be Merge develop into my branch
|\  
| * 6804047 An old commit that's somehow duplicate
| *   65e7f73 another old commit ..?

The list of commit ids you’d need would be 5a6df59 953511b 754fe93. Now to fix your branch, you would do

git fetch origin
git reset --hard origin/develop
git cherry-pick 5a6df59 953511b 754fe93

If this works without trouble, then you’re golden. Go to “you’re probably golden” above and you’re done.

If you’re still here, that’s because cherry-pick also failed you. It might be possible to fix this by doing a series of merges, but that can get tedious. So abandon this by doing

git cherry-pick --abort

The last technique I employ is a big hammer. This requires that you merge develop into your branch:

git fetch origin
git log

Now remember (copy) the commit id of the most recent commit. Now reset your branch back to origin/develop and squash your commit onto it.

git reset --hard origin/develop
git merge --squash <copied commit id>
git commit -m 'my commit message goes here'

Prior to the commit, you may have to fix merge conflicts. Such is life.

You now have one commit that is the sum of your changes from your branch! If you want it to be multiple smaller commits you could do that by resetting and committing parts one at a time.

If at any time you panic and feel you’ve lost work, simply use the command

git reset --hard <commit id you copied at the start and emailed to yourself>

And you should be back to the state you started in.

Categories
Command Line Tricks Computers Weird Errors

Xcode fails to export Ad hoc distribution

Has Xcode failed to export an Ad Hoc distribution file or upload to AppstoreConnect? Read on for a possible fix.

Xcode uses internal processes to do a lot of its work. Some of those tools can be influenced by environment variables that may be set up by your terminal shell.

Normally when Xcode is launched from the Finder, properties you set in your terminal shell don’t matter. But it’s different if you’ve learned to use the command line program open to launch Xcode. When you do that, the Xcode process inherits the terminal environment. Including virtual environment settings such as virtualenv, pyenv, or a version manager such as rvm. It’s that last one that can cause this inexplicable error:

Xcode mail fail exporting an .IPA file for Ad Hoc Provisioning with a message “An error was encountered:” and “The data couldn’t be read because it isn’t in the correct format.”

If you click Show Logs you might find:

2020-09-29 02:38:10 +0000 /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require': incompatible library version - /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/date-3.0.0/lib/date_core.bundle (LoadError) from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/date-3.0.0/lib/date.rb:4:in <top (required)>' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require' from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/CFPropertyList-3.0.2/lib/cfpropertylist/rbCFPropertyList.rb:4:in'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in require' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:inrequire'
from /Users/paddlefish/.rvm/gems/ruby-2.6.3/gems/CFPropertyList-3.0.2/lib/cfpropertylist.rb:3:in <top (required)>' from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:130:inrequire'
from /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:130:in `rescue in require'

The fix suggested here is clever, but didn’t actually work for me. But it got me wondering — why would Xcode invoke my .bashrc or .zshrc file anyhow? It shouldn’t be creating a shell process at all to run its commands. That got me thinking maybe the Xcode process itself was tainted with a bad environment. Then I remembered, I often use open to start Xcode, e.g. from a source code directory with an ios directory containing an Xcode workspace:

$ open ./ios/myproject.xcworkspace

I open to be easier than clicking around in the Finder. Especially when my current working directory is already there.

But be forewarned — if you see Xcode starting to generate bizarre errors, it’s because of the Ruby Version Manager selecting a version of Ruby that doesn’t have the necessary Gems.

The fix couldn’t be easier. Quit Xcode. Then relaunch it — using the Dock, the Finder, or Apple Menu recent applications. Just not the shell command, open.

Categories
Command Line Tricks Computers

comm to compare files with the same name in the same directory (another bash command-line interface [cli] trick)

Suppose you have two related projects with a similar file/folder structure. You want to compare the files by the same name in the two projects. You can use some magic of pipes to build a list of filenames that are the same (and in the same relative directories) in the two projects like this:

(find src -name '*.ts'| sort) | comm -12 - <(cd ../other_project && find src -name '*.ts' | sort)

It’s simple to then feed that into vimdiff or whatever is your favorite command-line diff tool using a for expression

for x in `(find src -name '*.ts'| sort) | comm -12 - <(cd ../other_project && find src -name '*.ts' | sort)`; do vim -d $x ../other_project/$x; done

Replace ../other_project with the path to the other directory to compare.

Categories
Computers Weird Errors

GSI (Global Secondary Index) errors with Amplify push

Have you ever used AWS Amplify with AppSync and done the command amplify push only to have it work for about 5 minutes and then fail, with an annoying error, “Attempting to edit the global secondary index”?

It happens sometimes, especially in development when iterating on the design of relationship in your schema. GraphQL is a powerful language for expressing relationships between objects in your data model. This limitation of DynamoDB makes this iteration painful. DynamoDB cannot update, modify, create, or delete more than on GSI (Global Secondary Index) at a time on a particular table.

How Did I Get Here

You probably made multiple changes to your schema. Every time you add a line like @key(name: "keyName", field: ["objectId"]) you’re ultimately causing Amplify to add a GSI to your DynamoDB Table. Add more than one between calls to amplify push and you’re going to encounter this error.

The Solution: Easy Part

The first thing to do is to log into the AWS Console, navigate to DynamoDB, and examine your table and it’s indexes. amplify push will complete BEFORE the indexes are finished being created in DynamoDB. If you try to push a second time before DynamoDB is finished, you’ll get an error.

Look for any line that says “Creating”. If it’s there, you just have to wait. Click the refresh button periodically to find out if it’s finished yet. Then try amplify push again.

The Harder, But Sure Fix

Some folks recommend amplify remove api and then amplify add api as a way to fix this problem. That is a BIG hammer. For one thing, it will DELETE ALL YOUR DATA. For another, it’s not needed. And also, if you have inter-dependencies, the remove call might fail, for example if you have an amplify function connected to Cognito as a post-user-sign-up hook.

Step 1: Find the CloudFormation Template for each Table

Use AWS Console to navigate to CloudFormation. Find the top level cloud formation stack for your environment. Click on that and look at its resources until you find api. Then click on that until you find the templates for each of the Object types in your schema. We’ll need to go through the following steps for *each* of those.

Step 2: Download the Current Template

I simply copied the current template onto my clipboard, and then saved it as a new file:

X=Segment
pbpaste > cur_$X.json

Use YOUR object name, not Segment. Forgive me, I am a Command Line geek. Use whatever tools you want, e.g. create a new file in a text editor, paste it, and then use your favorite file comparison tool for the next step.

Step 3: Compare AND EDIT the Template

I’m a vim user so this was all I had to do to compare them

vim -d cur_$X.json amplify/backend/api/appname/build/stacks/$X.json

Use YOUR appname, not appname.

Look at the section GlobalSecondaryIndexes. Are there more than 1 difference between the current template and the new one the Amplify want’s to create? Edit the downloaded JSON, bringing over only one of the changes, so that there’s only one difference. We’ll be adding those over one at a time, deploying the change after each edit. Yes, I know, this is tedious. But it’s better than destroying your entire environment and losing all the data.

Step 4: Update the Template in CloudFormation

Now go to the template for that Object. Click Update from the Template tab. It will say “It is recommended to update through the root stack“. Choose “Update nested stack”.

The screen that recommends you update the root stack.

Select “Replace current template” and “Upload a template file”

The Screen you see when going to update a CloudFormation Template

and upload your edited JSON file

Select your Edited CloudFormation JSON template

Then click Next a bunch

Updating a CloudFormation Template requires you answer a bunch of questions. Fortunately it has the right answers already selected.

You will have to give it permission to do something with IAM. Click the box.

A scary warning that CloudFormation is going to update IAM

With luck, it will succeed

When the CloudFormation template is properly updated, you’ll see the words “UPDATE_COMPLETE”. If it failed, you’ll see “UPDATE_ROLLBACK_COMPLETE” in red.

Step 5: Goto Step 3

If you had to remove more than one GSI change, add it back — again one-at-a-time and repeat starting at Step 3.

Step 6: Goto Step 2

Repeat this for all your Objects (which each has their own Table).

Step 7: Amplify Push Will Work!

Now that the GSIs have all been created, amplify push will be successful!

? Generated GraphQL operations successfully and saved at src/graphql
? Code generated successfully and saved in file src/API.ts
? All resources are updated in the cloud
GraphQL endpoint: https://<redacted>.appsync-api.us-east-1.amazonaws.com/graphql

Categories
Command Line Tricks Computers

TIL: git show <commit>:path/file.txt

Have you ever asked yourself, “How do I show a file at a particular commit or revision in its entirety?” “How do I print out a file not a diff in git?” “Show me a git file at a commit on stdout!” Here’s how you do it.

I knew about git show <commit> which shows a diff of all the files changed in a commit. You can even scope it with git show <commit> path/file.txt to show the diff for a particular file. Today I learned if you use a : (colon) instead of a (space) between the commit id and the file path git will instead show you the file in its entirety at that revision.

$ cd temp/
$ mkdir git_show_example
$ cd git_show_example/
$ git init
Initialized empty Git repository in .../temp/git_show_example/.git/
$ commit --allow-empty -m 'Initial commit'
[master (root-commit) 2f2f99d] Initial commit
$ echo 'Hello, World' > file.txt
$ git add file.txt
$ git commit -m 'Hello world'
[master 4bc4156] Hello world
 1 file changed, 1 insertion(+)
 create mode 100644 file.txt
$ echo 'Goodbye' > file.txt
$ git commit -a -m 'Goodbye'
[master 8af3b90] Goodbye
 1 file changed, 1 insertion(+), 1 deletion(-)
$ git show 4bc4156:file.txt
Hello, World
Categories
Computers

Kotlin, please allow return from closures

Edit: After writing this, I was helped by Andrey on Twitter and now I know about the return@

The simple answer is:


sequence.flatMap {
if (it.containsKey("connnected") {
return@flatMap Observable.just(it.getBoolean("connected"))
}
return@flatMap Observable.empty()
}

Here is the original post.


Today’s kotlin frustration… trying to map a Bundle into types. E.g. if the bundle has the key “connected” get it out as a boolean. otherwise, skip the event.

so I do

sequence.flatMap {
if (it.containsKey("connnected") {
return Observable.just(it.getBoolean("connected"))
}
return Observable.empty()
}

Except, oh Kotlin, you back-stabbing-friend… closures don’t allow `return`

So I edit it…

sequence.flatMap {
if (it.containsKey("connnected") {
Observable.just(it.getBoolean("connected"))
}
Observable.empty()
}

That’s probably obviously wrong to you now, but it took me a long time to see the problem.

And the fix.


sequence.flatMap {
if (it.containsKey("connnected") {
return Observable.just(it.getBoolean("connected"))
} else {
return Observable.empty()
}
}

I don’t see why Kotlin disallows the return — and it makes this kind of error much less likely. The fact that closures evaluate to the last expression is handy for doing tiny bits of work, but I find that reactive networks in RxJava are much easier to express with blocks — and if they get more than a few lines long, you sometimes really want the ability to do an early return without introducing the nested scope of an `if` statement.

Categories
Computers Geekery

How to create a git repo with the SQLite amalgamation source

Suppose you use GIT for your source control and would like to include SQLite as a third-party component via `git submodule` or `git subtree`. How does one go about getting access to the SQLite amalgamation as a GIT repo? When I tried this I found one on github.com, but it wasn’t being maintained any longer. It occurred to me that I didn’t need to depend on anyone else — I could do this myself. These steps use the bash command line shell. I’ve tried it on Mac OS, but should also work on Linux or using a bash shell on Windows. SQLite source is kept in a source control management tool called Fossil (http://fossil-scm.org). (It’s written by the authors of SQLite and uses SQLite internally, naturally.) The authors courteously tag each major of release in Fossil. So it’s simply a matter of cloning the SQLite source, checking out each tag, running the makefile to produce the amalgamation and checking in the amalgamated code into a new git repo.

Setup:

  1. Install Fossil from http://www.fossil-scm.org/download.html
  2. Install dev tools necessary for building fossil’s amalgamation (make, etc..)
  3. Install git
  4. Clone the SQLite source repo using fossil:

    mkdir sqlite-fossil
    cd sqlite-fossil
    fossil clone http://www.sqlite.org/cgi/src sqlite.fossil
    fossil open sqlite.fossil
    cd ..
  5. Create a new git repo to hold the amalgamation:

    mkdir sqlite-amalgamation
    cd sqlite-amalgamation
    git init
  6. Optional: Create a README.md file with a copy of these instructions or a link to this page and commit it.

In the future you won’t need to clone from fossil fresh, nor create a new repo — you’ll still have them. To add new SQLite releases to your repo you will use `fossil update` and `git pull` to update the two repos.

Adding one release:

  1. Perform these steps, replacing $1 with the release name — OR just copy this into a file called build-sqlite.sh next to your two repos, make it executable using `chmod +x build-sqlite.sh` and invoke it with the name of the release.
    cd sqlite-fossil
    fossil update $1
    rm -rf ../sqlite_build
    mkdir ../sqlite-build
    cd ../sqlite-build/
    ../sqlite-fossil/configure
    make sqlite3.c
    cd ../sqlite-amalgamation/
    mv ../sqlite-build/sqlite3.c .
    mv ../sqlite-fossil/src/shell.c .
    mv ../sqlite-build/sqlite3.h .
    mv ../sqlite-fossil/src/sqlite3ext.h .
    git commit -a -m $1
    cd ..
  2. For example:


    ./build-sqlite.sh version-3.8.11.1

    Not sure what all the tags are? Use `fossil tags` to display all the tags. Search for the ones that start with 3. as the major releases. Warning: The releases do not sort in alphabetical order (e.g. 3.8.11 sorts before 3.8.9 but is a later release) so use care when picking which tags to build, otherwise your git repo could end up with the commits not in the correct order.

Categories
Computers Geekery

Monitor Progress of dd Command

If you ever use the “dd” command do duplicate a drive, you’re probably familiar with the unnerving wait that accompanies its use. The command generally takes on the order of *minutes* to complete, and gives no feedback.

If you read the MAN page, you’ll see you can monitor the progress by sending a SIGINFO signal.

Open another terminal window, and do

kill -SIGINFO <dd pid>

Where <dd pid> is the process id of your dd command. For example, when I was formatting the SD card for use in my Raspberry Pi via dd on the command line, I was able to monitor progress this way:

sudo dd if=~/Downloads/2013-02-09-wheezy-raspbian.img of=/dev/disk2 bs=2048k
193+0 records in
192+0 records out
402653184 bytes transferred in 314.132226 secs (1281795 bytes/sec)

UPDATE: OMG! I just realized that the moment in time when I sampled my software installation on my Raspberry Pi was 100π to 4 digits of precision. What a wonderful omen!

Categories
Computers

CSS tricks

I’m trying to learn more about CSS, so here are a few tricks that I’ve picked up….

From http://phark.typepad.com/phark/2003/08/accessible_imag.html

A picture of Orion, my D&D figurine

The above image is actually an H2 text element that has been restyled to hide the text and show an image. This technique is useful for skinning a web page — you can simply put in placeholder text in <div> element with an appropriate ID. Then the CSS converts that to an image.

A block (from a div element) with position attribute bottom set to negative 10.

Note that in the previous example, there seems to be no way to push a contained box to the bottom of its container.

A block (from a div element) with float left
A block (from a div element) with float left
A block (from a div element) with float left
A block (from a div element) with float left

Next : how do you center horizontally in a box: use margin:auto;

A block (from a div element) with margin: auto

Next : How do you center vertically? The containing block must have display: table-cell and vertical-align: middle.

A block (from a div element) with margin: auto
Categories
Computers

Updated USB PicKit

A while back I put up a modified version of USB PicKit that works with Mac OS X.
Well, Jeff Boly has taken up the lead now so cruise on over to his site for the latest USB PicKit software for Mac OS X and Linux.