Choosing Camera Settings - an AI approach

by Ryan Stout

Choosing great settings for a photo is a complex process. First, you need to determine what attributes of a photo you should be optimizing. And second you need to know how your gear handled different situations to optimize your goals in step #1.

What to Optimize

Unlike phone cameras, which up until recently have mostly tried to maximize sharpness, DSLR/mirrorless cameras need to balance a lot of factors. "Good" settings depend on the scene. A human might ask questions like:

  • How important is sharpness in this photo?
  • Should I blur the background to focus on the subject?
  • Should I lengthen the shutter to show movement?
  • Should I shorten the shutter to freeze movement?
  • Where should I focus to capture multiple subjects?
  • What aperture captures my subject without stopping down so far as to lower sharpness? (due to diffraction)
  • etc...

Arsenal doesn't ask these questions directly. Instead, Arsenal's artificial intelligence was trained by looking at the photos of pro photographers. Using a deep convolutional neural network (similar to the ones used in self driving cars), Arsenal can find similar photos in it's database of settings. (Stored on Arsenal -- no need for internet access)

Currently, Arsenal finds the 30 closest photos in it's database and clusters them based on settings. It takes the largest cluster and uses those settings as the "baseline." By looking at what worked well in similar photos, Arsenal knows what goals it should be optimizing and how to prioritize conflicting constraints.

Conditional Constraints

Because conditions are never the same, we can't just suggest the baseline settings. Instead, Arsenal looks at the constraints for the current camera/lens, and the current conditions.

The following are the constraints Arsenal uses when choosing settings:

(Note: Not every constraint will end up being used in every photo)

Vibration

Arsenal has a very sensitive accelerometer inside of it. Using the data from the accelerometer and the focal length, Arsenal can determine the minimum shutter speed needed to prevent a blurry photo. (Even on a tripod, camera shake can be a problem)

Arsenal also factors in vibration compensation (lens or in body.)

Subject Motion

Arsenal uses what's known as an "optical flow" algorithm to determine how much an object is moving in the scene. Before a shot is taken, Arsenal reads data from the sensor (multiple times) and determines the minimum shutter speed to freeze the motion.

Dynamic Range

Arsenal can read data from the sensor to determine how much dynamic range is in a scene. The higher the ISO, the lower the camera's dynamic range. Arsenal can compute the max ISO before you start losing detail in the highlights or shadows.

In many scenes, some iso can be traded for faster shutter speed or higher aperture.

Color Range

Similar to dynamic range, color accuracy also drops as the ISO increases. Arsenal uses a similar process to dynamic range to show the max ISO before you start losing color detail (due partly to noise)

Depth of Field

Arsenal's Focus screen lets you choose what you want in focus in your shot. You can choose to focus on 1) a single point, 2) multiple points, 3) or to have Arsenal get the whole scene in focus.

For multiple points and whole scene, Arsenal computes a modified version of the hyperfocal distance. This tells Arsenal where to focus and what aperture is required to get everything in "perfect sharpness" (more on this later)

Noise Profiles

Different sensors produce noise at different levels. Arsenal has a sensor database that includes data on how noise changes as ISO and incoming light changes.

Lens Sharpness Profile

Arsenal contains a database of lens sharpness profiles. Different lenses are sharper at different apertures. Arsenal uses this data to try optimize the aperture at which photos are taken.

The lens sharpness profiles take into account optical factors like diffraction, aberrations, and coma.

Circle of Confusion

Arsenal's camera database contains information on sensor size and megapixels. From there we can compute how physically large each photosite on the sensor is. The circle of confusion determines how far an edge can blur before it spills into the next photosite. The circle of confusion gives us a baseline for "max sharpness", over which there is no gain in perceived final image resolution.

Reconciling Constraints

After Arsenal knows the constraints, it needs to find the best way to reconcile competing interests. This is where the the baseline settings come in. Arsenal uses the clustered settings to determine what should be high priority, low priority, or ignored all together.

For example, for a scene with a runner, you might want to freeze their motion (the subject motion), but for a wave at the beach, you might want to ignore the subject motion constraint.

Hopefully this has given you a little more background on how Arsenal recommends settings.