6.3 Photogrammetry in Post Production: An In-depth Analysis of Techniques and Applications
This portion of my thesis project is still being adapted into a web ready format. However, you can read the early version below
Challenges in Capturing a Room for Photogrammetry with Memento
Capturing a room for photogrammetry with Memento proved challenging, particularly for the sky bridge. The large windows and small hallway contributed to these issues. Tutorials often advise against capturing shiny or reflective surfaces. A possible solution could have been to cover the windows with black polyethylene sheeting, but the small hallway might have still caused stitching problems.
A regular room was used to experiment with the process (Studio B in Gannett on RIT campus) and the Memento was unable to stitch that together. At the time this software was better at recreating objects than it was rooms.
Due to these limitations I had to resort back to recreating the skybridge in Maya.
Learning from Tutorials and Software Limitations
Memento presented numerous challenges during this project, which may be lessened now that the software has been in use for a few years. [No longer exists] A significant portion of the project was dedicated to obtaining a successful stitch and a solid 3D model with minimal errors. Texturing could contain errors, as they can be fixed later in applications like Maya, ZBrush, or Katana.
Following tutorials didn't guarantee a good stitch, and making adjustments in Photoshop RAW sometimes led to better models, albeit with degraded textures. A high f-stop number is used in the camera to compensate for this. A lot of light is needed to compensate for a small aperture, however there is such a thing as too much light in this context. The Memento software involved a lot of trial and error, slowing down the project considerably. The majority of the project time was spent trying to achieve a good stitch. As a beta-tested software, Memento is likely to improve in future releases.
Ideal Conditions and Techniques for Photogrammetry
Overcast weather is ideal for stitching models, as it eliminates shadows that the software might interpret as edges. Harsh lighting from a bright sun or studio lights can also cast shadows. If these are the only available options, soften the light using a silk or, if outdoors, wait for cloud cover. Clouds act as a natural diffuser, similar to how a silk functions on set.
Incorporating high-frequency content and additional objects into the scene can significantly improve the stitching of the desired object. The added elements provide more reference points for the software, enhancing the overall quality of the resulting 3D model.
Experiments and Improvements in Stitching Process
There was a point when photographing the actor was paused, and a Hulk action figure was used in various experiments to determine how to achieve a better stitch. As an experiment, a fluffy stuffed poodle from the MPS room was added to the scene to see if it would improve the stitching, and it did! Although it didn't result in a complete 3D model, enough of the front of the two toys was captured to create a decent stitch for that section.
Adding more items to the scene can be helpful, as the software can better understand the spatial relationships between objects. Previously, the actor was photographed outdoors with the nearest object being 5 to 10 feet away. When photos are taken in close proximity, the high-frequency pattern of an object like the poodle can be easily matched by the software from one frame to another, enhancing the stitching process.
Other experiments conducted during the stitching process involved adjusting the photos in post-production. Reversing the order of the same set of photos in the software actually produced a completely different stitch. When the same set of photos was imported twice, the resulting models were nearly identical. Memento has a feature that allows users to compare differences between two models, which is useful for marine biologists studying coral reefs. They can photograph the coral over time and compare the differences to observe growth or decay. However, this feature wasn't used for the nearly identical models in this case.
Removing one or two images from the sequence also affected the resulting model. Experimenting with this feature led to many different versions produced from the same set of photos, all with varying results. Another approach involved adjusting the photos in Photoshop Raw. Each time adjustments were made, the outcome differed. The first step involved tweaking highlights and shadows, followed by increasing contrast and adjusting sharpness. While these changes sometimes improved the model, the texture often suffered. As an experiment, a set of grayscale images was added, resulting in a model with more surface bumps as the software interpreted freckles as bumps.
Using Memento to create a good human model proved to be more challenging than expected. Initially, it was believed that the issue stemmed from the quality of the photos taken. However, even after addressing this problem, the software continued to struggle with producing a satisfactory model. Following tutorials didn't guarantee a successful stitch. Adjusting photos in Photoshop Raw sometimes resulted in a better model, but often at the expense of the texture.
Trial and Error with Memento and Future Outlook
The trial-and-error nature of working with Memento significantly slowed down the project, with a considerable amount of time dedicated to obtaining a good stitch. As the software was still in beta testing during this period, it is likely that future releases will bring improvements and enhancements to the process.
Notes:
Memento no longer exists as it was replaced by Remake, and then later Recap
This process is not advisable for most indie film and 3D artists as there are better and cheaper alternatives now
I currently use Scaniverse and Polycam
The current workflow I use now is just using these tools as better references to import into Maya to model from for accuracy and for proxy models. I do not spend too much time worrying about my lighting or other objects in the scene
I have not used a NERF system yet, but some of the results look fantastic
