Taking photos at night on your phone
used to look terrible.
But if you purchased a new smartphone recently,
you may have noticed that your night photos have improved.
Ah, much better.
You can even take photos of stars.
I’m Julian Chokkattu, reviews editor at Wired,
and I’ve been reviewing smartphones for over five years.
How has smartphone photography gone from this,
to this beautiful photo?
Before we get into the technology
behind the new night modes,
let’s first have a little chat about bad photos.
Take a look at this photo here,
taken on an iPhone Five around 2014.
A couple elements stand out to me,
like that classic lens flare, or the blur.
No matter how nice or advanced the camera is,
it’s always going to need a good source of light.
That’s exposure, the amount of light
that reaches your camera sensor.
Right now, this lovely crew has lit me really well.
Let me show you.
If they cut the lights, now I’m back lit and underexposed.
This is the iPhone 3G in low light,
and this is the iPhone 13 Pro in low light.
Let’s get the lights back on.
Part of the reason the iPhone 3G looks so underexposed
is because it didn’t spend a lot of time taking the photo.
That’s shutter speed.
That’s the length of time the camera’s little door is open,
exposing light onto the camera sensor.
One of the main reasons night mode on your phone
asks you to stay still is because
the longer you have the shutter open,
the more light you can let in,
which will produce a brighter photo.
But here’s the thing.
In night photos, the seconds it’s asking you to wait,
it’s actually taking more and more photos
to make a composite with machine learning algorithms.
So night mode is a part of the field
of computational photography.
I’m going to call up Ramesh Raskar at the MIT Photo Lab
to get into the technical element of how it works.
[Ramesh] Hi Julian.
Would you be able to tell me
what exactly is happening when you take a night photo
in a modern day smartphone?
There are three elements in any photography.
There is capture, there is process,
and then there’s display.
And what we have seen over the last 10 years
is there is amazing improvement in all three areas.
So how is the software actually changing
what the photo will look like?
You will hear all these terms, HDR, HDR plus, night mode,
smart HDR, but all of them are roughly doing the same thing.
This key idea of so-called deep fusion,
where you’re fusing the photos by using machine learning
and computer vision, is really the breakthrough
into today’s low light photography.
Could you explain HDR?
So HDR, traditionally high dynamic range, simply means
whether it’s bright scene or a dark scene,
you can capture that in a single photo.
A smartphone, it has seen millions of photos of a sunset,
or a food, or a human face.
It has learned over time, what are the best ways
to enhance such a photo, and how to either
reduce the graininess, or how to make it look more vibrant
and choose the right saturation?
Choosing those parameters is basically machine learning
when it comes to photography.
Now let’s take a look at this machine learning in action
by comparing some photos.
The one on the left is the iPhone 3G,
so quite a long time ago.
And the one on the right is the iPhone 12.
What are your first thoughts
in what they’re doing differently?
So you can see that the previous phones
just gave you a photo from a single instant.
The photo on the right is actually not physically real,
in the sense that there were different things.
People were bobbling their heads,
and the lights were flashing.
And so the photo’s actually composed by multiple instances.
So when you try to fuse these multiple photos,
the light in one photo could be one direction,
light in the later photo could be in a different direction.
And it’s taking some clever decisions
to create an illusion, as if this photo was taken
at that single instant.
Here you can also see the HDR into effect,
where the audience is completely dark
in the iPhone 3G photo, whereas you can actually see
everyone’s heads in the other one.
If an AI is learning how to color correct a night scene
based on what it thinks it should be,
are we moving away from photo realism?
Julian, I think photo realism is dead.
We should just bury it, and it’s all about hallucination.
The photo you get today has almost nothing to do
with what the physics of the photo says.
It’s all based on what these companies are saying
the photo should look like.
So yeah, I took one of these with the Pixel Six
and one of these with the iPhone 13 Max Pro.
What happened there that would’ve caused those colors
to be very different between the two photos?
These two companies have decided to give you
a very different photo experience.
The Pixel might have taken 20 photos.
It’s also recognizing certain features
whether there’s a sky, is it outdoor?
What kind of wide balance it has?
There’s some automatic beautification also being applied.
So most of the photos we see are hallucinations,
but not the physical representation of the world out there.
These companies are providing us with ways to
control some of that, like turn off
that beautification feature or maybe make it even stronger.
Do you think that’s where the compromise will lie
with the people that do want to maybe
tailor some of their own shots to give them that control,
and those options to tweak their settings?
The innovations in all these three areas
have actually taken the control away from us.
But in reality, it’s not that difficult
for these companies to provide those controls back to us.
They’re just making an assumption
that most consumers would like to just take a photo,
click a button, and get something they really
would like to see, whether it matches the reality or not.
I think the thing that we really care about is
we go on a trip, and you reach Paris,
and the Eiffel Tower is in a haze.
And what you would like to see is take a photo
with your family with Eiffel Tower in the back
as if it’s a bright sunny day, right?
And that’s where as a consumer,
you yourself are willing to separate the physics,
the reality from hallucination,
because if somebody can paste just a bright, sunny photo
of Eiffel Tower behind your family,
you’ll be pretty happy about it.
So we focused on night photography.
Every time we look at the nighttime photos,
those actually do seem to be improving year over year.
But broadly, what would you say are some of those challenges
that are left for photography in general
when it comes to smartphones?
In terms of night mode,
there are lots of challenges right now.
If you want do something that’s high speed,
it’s very difficult to capture that at nighttime.
It’s also difficult to capture very good color
in nighttime, because nighttime photos have,
when they use burst mode, the challenge with burst mode
is that every frame has a so-called read noise.
So there’s a cost a camera pays
every time it reads the photos.
But the other technique many companies are using
is just using lots of tiny lenses.
Now some phone companies have five lenses,
and that’s one trick to capture just five times more light.
How does that affect the rest of the phone’s capabilities?
What can we expect in the future?
Photography or imaging should give us superhuman powers,
so we should be able to see through fog,
we should be able to see through rain.
we should be able to see a butterfly
and see all the spectrums, not just the three colors.
I think the notion that
we should just see what we are seemingly experiencing
is not in different displays,
but I would like to see a beautiful view finder.
If I’m in Paris and as I’m moving my view finder,
it should tell me, hey, if I take a picture
of the Eiffel Tower, it’s very jaded.
A lot of people are out taking a photo.
But if you keep rolling and there is this tiny statue,
actually not enough people have taken the photo of this.
So I think we’re going to see this very interesting progress
in capture, processing, and display.
And I’m very excited about
what photography of tomorrow will look like.
I’m going to show you some of my favorite features
with the iPhone 13 Pro and the Google Pixel Six.
We’re doing low light photography, so let’s cut the lights.
Let’s open up the camera
and see what happens with night mode.
You can see that I’m already in a pretty dark area,
so night mode has been triggered here.
Once you tap it,
you can actually control the length of the exposure.
So if you think that you might need a longer shot,
sometimes that might produce a brighter image.
If I tap on the background, it’ll expose for the background
and it will also change the focus there.
So you can actually slide it up and down
to change the brightness, or the shadows in the shot.
Those are just a couple of features
in the camera app themselves.
All right, let’s bring the lights back on.
So we have to talk about tripods.
Tripods are an easy way to up your photo game,
especially at night.
Of course, a large problem of taking photos at night
is the hand shake of when you’re taking a photo.
Once more, can we cut the lights?
Can I get a volunteer?
So now I’m going to first take a photo without a tripod,
and see how it reacts then.
So you can just basically switch over to the night site mode
and tap the photo.
But now if I switch over to a tripod,
it’s going to be much more stable.
And if I tap the button, it knows that it’s on a tripod,
and you can see it is taking a lot longer to take the photo.
It’s taking multiple, multiple images
of different exposures.
Shooting handheld is a problem, because the shutter speed
is trying to take in as much light as possible.
And that means your hands are shaking,
and that’s influencing the shot.
That’s what makes it impossible
taking photos of stars without a tripod.
Certain phones like the Pixel Six
let you take photos of the star
with a certain astrophotography mode.
And essentially it’s doing what night mode is doing,
but for a much longer period of time,
like two, three, sometimes even five minutes.
And what it really needs is the phone to be on a tripod.
If you’re curious about what some of our favorite phones are
for taking photos, or maybe just looking at
other camera gear that might help you take
some of these better photos,
well, we have guides on wired.com.
And as Ramesh said, it’s going to be really interesting
to see how our cameras improve in the future,
whether they’ll completely decide on their own
exactly what photo you should take,
or if you’ll have any control left.
Photo realism is dead.
No, that’s dark.
I hope this video helped you understand a little bit more
about night photography, and I hope
you continue going out there taking lots and lots of photos.