Pixtra PanoStitcher™ - Advanced Topics

    Real world photos for real world panoramas

    PanoStitcher's solutions 

1. Camera exposure: fixed or auto? 

2. Wide-angle lenses 

3. Focal length: known, same unknown, or different unknown? 

4. How much overlap between neighbor photos 

5. Moving Objects: eliminate ghosts! 

6. People panorama 

7. Parallax problems 

8. Straighten the horizon  

9. Auto/Marker stitching 

10. Panorama resolution 

11. Visually guided printing utility

12. Auto/manual intensity adjustment 

13. Reset stitching  

14. Other photo editing software and PanoStitcher panoramas 
    

Real world photos for real world panoramas

For making an ideal panorama, the ideal set of photos would have the following:

• The photos would be taken smoothly around a fixed point, e.g., rotating horizontally
  but not tilting the camera up or down during shooting. A tripod would be used.

• The scene would be the same during photo-taking: e.g., no moving objects, no change
  in lighting condition, etc.

• Each neighboring photo pair would have a large enough overlap (>20%) with a sufficient amount of features in the overlap region.

• The camera would have a fixed exposure, hence each photo would have the same intensity.

• The camera's lens would be high-quality, with no distortion and no chromatic aberrations.
  The focal length would always be known.

But , carrying a tripod is inconvenient, and the real world conditions do not always match the ideal. Let's assume taking hand-held photos. The real set of photos would have the following:

• The photos were taken as you turned around an imaginary point. In reality the "point" moved around during shooting introducing the parallax phenomenon.

• During the photo-taking period: people walked around, cars were driven away, trees were shaken by the wind, the water was waving, and the sun was setting.

• Even though you tried hard to consistently achieve an overlap of 30%, the range was really 10-40%. And what's worse, in taking a full 360 degree panorama, when you looped around, the overlap was unpredictable and the camera up-down tilt level was quite different from the starting level. 

• You might have preferred not to fix the camera exposure, hence the intensities between the neighbor photos were dramatically different (imagine one photo included the sun and the next did not).

• The camera was not high-end and you wanted to use the simple wide-angle lens: straight lines in the scene became curved; edge features looked like rainbows.

• You did not know your camera system's focal length. You might have used zoom, now it is impossible to know the focal length.

Welcome to the real world!

PanoStitcher's solutions 
Due to the nature of the job, it is much harder to stitch a variety of photos from the real world than those of the ideal world. PanoStitcher is the first stitching software to be truly fully functional in the real world with robustness, flexibility and ease of use. Some of its distinguishing unique features are:

• Auto/Manual stitching: Since there can be all sorts of images from the real world, there can never be 100% automation. PanoStitcher guarantees success via full automation for most cases and manual tools for difficult occasions (especially for photos with motion and parallax). 

• Handheld capability: This is supported by PanoStitcher's fundamental design principle. A tripod becomes optional, not essential, equipment.

• Automatic detection of focal length: Focal length is a foreign concept for many consumers. Furthermore, with zooming there is no way to know the exact focal length used. PanoStitcher eliminates the guesswork.

• Correction of lens distortion and chromatic aberration: While present to some degree in many lenses, the problem is particularly noticeable for the wide-angle lens of digital cameras. PanoStitcher automatically corrects distortion and chromatic aberration.

1. Camera exposure: fixed or auto?
Fixed: The overlap area between neighbor photos will have the same exposure. Hence no intensity balancing is needed and the blending can be perfect. But if the surrounding scene has drastic differences in lighting, some photo areas will be over-exposed while some others will be under-exposed. You should try to avoid over-exposure since there is no way to correct a white-out, while overly dark regions typically can be corrected by photo editing software.

Auto: This is typically the default setting of most cameras --- for a good reason: the exposure problem is minimized within each photo. But it will introduce dramatic exposure differences between the neighbor photos, which can be inherently difficult to correct in stitching. The unevenness of the resulting panorama might be noticeable but the saturation problem is less than using fixed exposure.

Either fixed or auto can be used when the surrounding scene does not have extreme contrast. Otherwise it will depend on your camera, the scene and your emphasis.

A hybrid solution: Take a photo set of the full scene with one fixed exposure. Then take photos selectively of the scene areas with extreme brightness or darkness with auto exposure (or other manual exposures). Stitch the photo set of the fixed exposure with PanoStitcher first. If the result is not satisfactory due to the intensity extremes, experiment by substituting some photos of fixed exposure with their equivalents of auto exposure in PhotoBench and click Stitch/Blend again (make sure auto intensity
balancing is turned on in the Blend Option setting). But more likely you will need to take full control by customizing with PanoStitcher's Manual intensity adjustment tool (see Topic 12). 

2. Wide-angle lenses 
A wide-angle lens may be used if you want to have a wider vertical view of the panorama and/or you want to use fewer photos for a full panorama. Inexpensive, simple lenses may have severe distortions and chromatic aberrations, but these can be corrected by PanoStitcher with its Camera Info files.

3. Focal length: known, same unknown, or different unknown?
The default "same unknown" is good for most practical cases, which assumes you did not use zoom or otherwise change focal length during photo-taking. If you know the focal length of your lens, setting to "known" and setting the correct focal length value can help with motion and parallax problems. But if zooming is changed during shooting, you must use "different unknown", and you will likely need to use Marker Mode in the Stitch step. In Marker Mode, you need only two marker pairs with "Known" focal length, while you need three for "Same unknown" and four for "Different unknown".

If your camera produces photos with distortion or chromatic aberration you should use the corresponding Camera Info file, which also contains a "known" focal length value.

4. How much overlap between neighbor photos
When taking a series of photos for making a panorama, the more overlap between photos the more information there will be for stitching, but also the more photos are needed for the same scene coverage. If you use Auto Mode the rule of thumb is 30%. For Marker Mode, a minimum of 5% is needed if the focal length is known, or 20% otherwise. How much overlap between each neighbor photo pair also depends on the scene itself. A larger overlap is needed if there are very few distinctive features in the overlap area. Larger overlaps can help deal with moving objects, as discussed in Topic 5. To minimize parallax problems, adjust the overlaps so that close-by objects are in the center of one photo rather than in the overlap area, as discussed in Topic 7.

5. Moving Objects: eliminate ghosts!
More often than not there are people in the scene. The overlap area of a neighbor photo pair may not be the same, e.g., a person may appear in one of the neighbor pairs' overlap region, but not the other. A moving object in the stitched panorama will be "ghost"-like if it is on the blending line. The Select pair blend mode feature in PhotoBench usually solves this problem by shifting the blend line of the corresponding photo pair.

Sometimes moving objects' "ghosts" cannot be eliminated by shifting the blend line, depending on a number of factors such as whether a moving object is fully/partially included in a photo or in its overlapping area with a neighbor photo. To minimize such scenarios you can shoot with a larger overlap between the involved neighbor photo pair, and/or you can time your shooting to avoid moving objects around the overlap area.

6. People panorama
You may want your family, your friends or yourself to be in a panorama. You may even want the same people to move around so that they show up in each photo that contributes to a panorama. If people are to be in both of a neighbor photo pair,  1) take each photo with people in its center area, 2) make sure there are no people in the overlap area between the neighbor photos. If the two conditions cannot be satisfied at the same time, you should have people show up in every other photo.

7. Parallax problems 
Parallax can be the most ominous problem for stitching. You should try hard to reduce it. Its severity is proportional to the camera's spatial movement relative to the scene features' distance. Tripods can often eliminate many parallax problems. A tripod is suggested when taking indoor panoramas since the objects are usually closer to the camera than when taking outdoor photos. But the following suggestions can help for those instances you do not have or do not want to use a tripod.

There is one trick to reduce parallax when taking photos handheld. Remember that the parallax effect is more prominent with objects that are close. A lot of times the nearby object is a pole, a tree or a table with the background scene far away.  You should try to adjust your angle of photo-taking around the pole such that the pole is in the center of one photo while its two neighbor photos do not contain or
contain little of the pole. Then in the Blend step of PanoStitcher the whole pole in the panorama is contributed by the one photo with the pole in the middle (Use the Select pair blend mode feature to achieve if necessary). Very often such a maneuver totally avoids parallax.

But what can be done if parallax is obvious?   Foreground features may not be well-registered when auto stitching succeeds as it favors the majority of the scene, which is normally the background. But if Auto stitching fails or you do not like its result, Marker method or Overlay method is used.   With Marker method you should choose markers from features of the same scene depth (i.e., markers from close-up objects, or from background objects, not both). Otherwise the panorama will fail to stitch and you can be repeatedly asked to edit the markers due to the detected inconsistency.   If you want to have the images stitched despite the mismatch, set the focal length to "Known" with the correct value.   If you do not know the correct focal length, you can experiment by changing it in the Stitch Settings dialog box in the overlap window and using the value that gives the best overlapping among the photos. On the other hand, since the focal length is known now the Overlay method should be preferable.   With this method you can simply visually overlay each image pair to your liking.

8. Straighten the horizon  
When you take successive photos especially with hand-held cameras, it is difficult to keep your camera level. Further, you might want to tilt your camera to cover the scene above or below " level", e.g., tilting 20 degrees downward while standing on a ledge or cliff. If there are enough photos PanoStitcher normally makes a good adjustment to straighten the panorama. But if there are only a few photos for a panorama you may want to change PanoStitcher's automatic adjustment. PanoStitcher provides features to rotate/unbend the panorama. Remember that a panorama is a world-map type of image which has warping, e.g., straight lines in the scenes are generally not straight in the world-map. But with the above features, the horizon can be straightened (e.g., "unbent"), and if necessary tilted (e.g., "rotated") to be horizontal. Here is an example,


(a) Raw photos


(b) Initial panorama preview


(c) Panorama preview interactively rotated and unbent.


(d) Final panorama, blended and cropped.


9. Auto/Marker stitching

PanoStitcher has the unique duo of Auto and Manual stitching. Auto Mode lets you create panoramas with ease, while Manual Mode provides the backup. For real world photos, Auto Mode can never be 100% successful and correct. Manual Mode provides the means to handle even complicated photos created under less than ideal conditions, turning them into not just credible, but great panoramas.

While Auto Mode is generally successful, it prefers to err on the conservative side. If there is some doubt about the stitching between a neighbor image pair, the Auto Mode would rather decide to fail than to "risk" a false success (e.g., mismatched overlap features). In such cases you would be prompted to use the "secure" Manual Mode.

To manually stitch an image pair two methods are provided: Marker method and Overlay method. With the Overlay method the focal lengths of the involved image pair are assumed to be correctly provided.   By contrast, with the Marker method the focal lengths do not have to be known. But the Overlay method is a visually direct approach. What-you-see-is-what-you-get (wyziwyg) in term of overlapping. While you need to identify feature points as marker pairs with Marker method. These two methods are complement to each other. Which one to use for an image pair will depend on the focal length condition and the image features.

10. Panorama resolution 

At what resolution should you save panoramas? We strongly recommend saving the first panorama of each project with high resolution (e.g., at least a Resolution Ratio of 75%) and  in .TIF format (with no compression). You can always reduce the resolution later with PanoStitcher to generate panorama files of any size and at various .jpg compression levels. If you initially save at a low resolution you cannot then use that image to generate panoramas of higher resolution.

PanoStitcher does not have a limit on the sizes of photos for stitching and sizes of resultant panoramas. It is designed to handle very large photos. With a 3 Mega-pixel camera, each photo can be 2000x1600 pixels.  When taking photos in the portrait orientation, a 360° panorama with the resolution ratio of 75% can have a size of 10,000x1500. This will occupy 45 megabyte of memory space during stitching. Depending on your computer system, CPU, RAM, etc., PanoStitcher may run slowly when making panoramas of this size. If you will be stitching very large panoramas regularly, your computer should have enough RAM, roughly three times the memory one photo takes plus two times the memory a panorama takes. Assuming photos of 1200x1000, this gives 34MB RAM for a panorama of 5000x800, 58MB for 8000x1000 and 100MB for 10,000x1500.

If you use a scanner to get very large photos (e.g., 4000x6000) for stitching,   PanoStitcher's working memory can exceed 300MB. You should ensure that your computer's virtual memory size is at least this amount. Otherwise you may see an "Out of memory" error message.

11. Visually guided printing utility
Since a panorama image can be very long, you may need to print it onto multiple pages. In PanoStitcher you can visually set the number of pages to print to and the margin settings of each page. More often all you need is to select a setting in the print floating menu. PanoStitcher will automatically fit the panorama to the desired number of pages.

The multiple printed pages will need to be taped or glued, and will have visible seams. A better way is to tape or glue the blank papers together before printing, or to buy panorama photo paper (8"x22"). Then in Print setting, choose the proper paper size in Page Setup and "Fit One Page" in print floating menu. The panorama will be printed to a single long paper. Alternatively you can upload your panoramas to an online company for printing.

12. Auto/manual intensity adjustment

PanoStitcher's "Auto-balance intensity" makes smooth brightness transitions between photos when blending. But achieving such smooth transitions can result in a panorama with regions of extreme brightness/darkness. Note that a digital image has a limited intensity range (from 0 to 255). Sometimes compromises are necessary to obtain visibility of the entire scene and to smooth intensity transition in each pair's overlap area. Each camera model also has its own exposure characteristics. Even for the same camera, different settings can have different effects on different color channels. In these complicated cases you need to experiment to achieve your preferred effects.

Typically you can use "Auto-balance intensity" first when Blending. If the panorama is not to your liking you can manually adjust the PhotoBench images and blend again. You might need to go through several rounds of experimentation. Since blending a very high-resolution panorama can take a while you can reduce the Resolution Ratio to blend panoramas more quickly. You can increase the Ratio when you decide to make the final master panorama.

Please note that even if you have manually adjusted the intensity, you can choose to blend with or without "Auto-balance intensity".   You might need to experiment with both to see which one gives the preferred result.   Make sure the blend option "Auto-balance intensity" is checked or unchecked per your preference before clicking on Blend.  

PanoTutorial tutorial 3 provides a detailed example.

13. Reset stitching 
Sometimes stitched panoramas will appear grossly mismatched. This is usually due to wrong settings or marker pair placement. Main menu Edit | Reset Stitching will reset the project to the Sort step, leaving intact only marker pairs that were manually placed. By resetting stitching you are given the chance to try stitching again after the relevant conditions are corrected. These are usually 1) some wrong marker pairs
were picked, 2) the wrong focal length value was set when the focal length was set to Known, or 3) the wrong Camera Info file was selected. 

14. Other photo editing software and PanoStitcher panoramas
If needed, you can use other photo editing software to do special processing of your panoramas. You should save the edited image to a different file name from the original panorama. The original panorama made by PanoStitcher contains panorama-specific information which is used by PanoStitcher and PanoViewer. To use these with your edited panorama you will need to transfer the panorama-specific information from the original panorama to your edited one. 

When finished editing, load the saved edited image into PanoStitcher. Right click on the image and choose "Re-convert to panorama format". Click "Select original panorama file" to get your original panorama info. Click "OK" to transfer the panorama information from your original panorama to your edited image. Remember to save the re-converted panorama.