Homographies in Adobe Photoshop

The main goal of this project was to add new methods for manually defining homographies in Adobe Photoshop to better facilitate multi-viewpoint image integration. Although Adobe Photoshop offers a few homography related methods, these are geared more towards working with an image or set of images shot from a single point in space. When trying to combine images taken from different positions in relation to a space, parallax almost always means that there will be no single homography that will cause all corresponding points to map onto each other. The philosophy taken with this project is that an artist is better equipped than a computer algorithm to weigh the tradeoffs between various mappings and that the computers role should be to allow the artist to visualize possible mappings as quickly, intuitively and non-destructively as possible.

An overarching benefit of all these features is that they allow an artist to work in a more rapid, experimental and iterative fashion. The speed and accuracy with which one can "converge" upon a desired transform will allow for the creation of images of greater complexity.

A key building block to these homography related additions is the ability to embed inside Photoshop's built-in structures the steps needed to recreate a transformed layer. Two pieces of information needs to be stored in order to reload a layer from source with all applied perspective transformations:

1. The file path of the source image.
2. A set of 4 pairs of points defining the homography applied to the source image.

The first part of this recipe can be stored in a layer's name using the following form:

layer_description { loadLayer('DSC_1045.jpg'); }

Since most of the code for these additions interact with Photoshop's DOM using Javascript, the above form makes it easy to use regular expressions to pull out multiple commands from a layer name (between curly brackets, separated by semicolons). Once the commands were isolated it was simple to pass them to the Javascripts eval() function. Duplicating layers by normal means preserves this metadata. As a side note, in the future other functions will also be added to handle filter calls such as those that would correct for barrel or pincushion distortion.

The second part of the recipe is a real key to this project. As mentioned above, 4 pairs of points are required to define a homography. Transformations in Photoshop are done by moving the 4 bounding box corners of a layer or layer set to new positions. If we can always reload an image from source, the original bounding box corners for an image represent 4 of the points needed to define a homography; with a source image (e.g. *.jpg, *.dng) this just means knowing the dimensions of the image in pixels. To store the information needed to fully capture a transformed layer's homography, it is only necessary to keep track of where the original 4 bounding box corners have ended up.

This can be tracked by creating a vector path with 4 corner points placed at the 4 bounding box corners of the original source image. This is done when the image is loaded, at the same time as the layer name is altered to include the layer source metadata. This path is stored as a vector mask for this layer. A vector mask is a special type of vector path that is associated, and by default "linked", with a particular layer (in much the same way as a layer mask is a channel attached to a particular layer). A vector mask has many properties which make it ideal for the task of tracking a homography:

1. When left in its default linked state, normal perspective free transforms, perspective cropping, and Auto-Align Layers affect the points in the vector mask in the same way as the pixels in the layer. That is, the points over the corner bounds of the layer will be moved to wherever there corresponding bound corners are moved.
2. Vector path points, like all path points, are floating point and thus have sub-pixel accuracy.
3. Multiple transformations can be applied to a layer, but the vector mask will always store the information for the single homography that goes from source to cumulative transformed state. Since each transformation degrades image quality slightly, it is advantageous to replace a layer with versions which has only once been transformed.
4. Path points can lie outside of the document bounds (e.g. path points won't get clipped)

This feature takes advantage of two of the undocumented action events for cropping with perspective and interactive free transforming with initial transform corner positions. The crop feature in Photoshop has a perspective option, which, when checked, allows one to define 4 points that will become the corner points of the newly cropped and rectified image. By duplicating a selected layer into a new document, we can use this crop feature to force points of interest to be at the bounding box corners. Duplicating this newly cropped image back to the original composition document gives a layer that can be transformed with control points lying at the points of defined interest.

It should be noted that the closer a point of interest is to a control handle, the easier it is to place it. And just as important, being closer to a handle means that image points are less likely to be affected by subsequently moving the other three control points (better local control).

Normally this procedure wouldn't be of much help, since these steps involve deleting much of the layer we wanted to transform. Also, these two transformation processes can result in strong image degradation. However, our vector path and layer source metadata make it possible to reload the entire layer with a single computed homography, preserving as much image detail as possible.

This feature relies upon a user defined 'Work Path' to define point correspondences between a base image - or set of images - and the image to be mapped onto it. The Javascript code that handles this feature makes an outside call to another executable I wrote. This takes a list of input points, a list of corresponding output points, and the 4 bounding points of the image to be transformed:

854,706,872,630,831,490,700,660
1172,712,1198,639,1171,497,1012,665
-51,167,949,167,949,832,-51,832


The point correspondences (4 or more pairs) allow for a homography to be computed using singular value decomposition. Passing the bounding corner points through this homography yields the coordinates, in the Photoshop documents space, to which the corner points should be moved when calling the transform command. As with the other features added in this project, the free transform destination points are not committed until confirmed by the user; they may be tweaked further. In the end, multiple transformations often need to be done on different regions of the same image because of parallax. These tools allow for hierarchies of transformations to be made with no image degradation penalty from its depth.