[silence]
Hello, and thank you for the opportunity to talk to you about the
work we’ve been doing on deep learning for earthquake monitoring.
My name is Greg Beroza. I’m at Stanford University.
The funding sources for this work are shown on the left,
and the participants in it are shown on the right.
Initially, all at Stanford, but they’re starting to scatter to the forewinds.
So this figure shows schematically what is involved in earthquake monitoring.
We have our input, which is ground motion from a seismic network.
We don’t know ahead of time when the earthquakes are going to happen,
so we have to work with that data to detect the events, to pick the phases,
to associate those phases with events, then to locate and characterize the
earthquake and – earthquakes. And then, the output from that
is a comprehensive catalog of discrete seismic events –
earthquakes, quarry blasts, whatever.
My group has been working systematically to develop machine
learning-based approaches to plug into this workflow.
I’ll talk a bit about PhaseNet, which is a machine learning-based
method for picking seismic phases. I’m not going to talk about
DeepDenoiser, but that’s a method for separating signal
from noise, such that we get better catalogs.
We’ve developed a deep Bayesian neural network for single-station
earthquake location. This is relevant when we have
sparse networks or when the earthquakes are small and difficult
to detect on multiple stations. Same goes for magnitude
determination. We have a single-station
magnitude determination neural network that will allow that.
And our sort of second-generation earthquake detection network –
I’ll talk a bit about that. It’s a multi-task network in that
it does both event detection and phase picking.
And, because it does those together, it does better at both.
So mention that deep learning is data-hungry.
And what I mean by that is that it requires a lot of data – many examples
for a neural network to learn to, say, pick seismic phases in a –
in a way that’s reliable and that can work generally.
Now, seismology is very fortunate in that we have – through the hard work
of analysts, we have very large data sets. These data sets are labeled already,
which is fantastic. But the labels are not perfect.
That is, there are lots of errors and omissions in the catalog.
And these have the potential to undermine our deep learning-based
approaches if we don’t eliminate to the extent that we can.
So, some issues with earthquake catalogs are sometimes there are
earthquakes that aren’t really there. And, hence, we would be teaching
a neural network the wrong thing. This is probably not too much
of a problem. Much more insidious is the fact that we have extra earthquakes.
So shown here is our waveforms – three-component waveforms,
P wave and S wave picks. So an earthquake detection there.
Here’s an earthquake – a smaller earthquake aftershock
that’s not in [audio cuts out].
And, if we were to not have a label for that, or to label it as noise, our network
would learn exactly the wrong thing. That is, our neural network,
not our seismic network. So we absolutely have to
get rid of these extra earthquakes or label them as earthquakes.
And then, as all seismologists know, that there are errors in picks.
And so, if we’re trying to pick arrival times,
we have to deal with those – with those errors.
So, in order to get a very high-quality data set, we curated a data set.
That is, we collected many millions of waveforms.
We did quality control on them, and called this – we added labels.
We called this the Stanford Earthquake Data Set, or STEAD –
1.2 million seismograms. So it’s not a gigantic data set.
But it comes from 500,000 earthquakes recorded at different depths and
around the world, as shown here, in a variety of tectonic settings.
All these seismographs are – or, seismograms are from local
distances only – less than a couple hundred kilometers.
And the reason for that is that our initial motivation is to locate small local
earthquakes because these are the ones most likely to be missing from catalogs.
So this STEAD waveform database is collected from a range of
different seismometers, different kinds of sensors.
We have examples of signal and of noise.
We’ve added many labels, like coda duration.
And I mentioned the quality control. And we’ve also put in an HDF5 format
for easy I/O and to recruit expertise from data scientists who don’t like –
you know, who shouldn’t be burdened with learning
our seismological exchange formats and so forth.
So first, I’ll talk briefly about PhaseNet, which was developed using data from
the northern California seismic network. PhaseNet – the input is
three-component seismic data. The output is pics of P waves
and picks of S waves. So that probability – the green –
the green here is a truncated Gaussian centered on the
P wave arrival time in the catalog. This one is centered on the
S wave arrival time. And the idea is that we engineer it such that it picks the
probability peaks on the P wave arrival time and on the S wave arrival time.
And it works. This just shows examples from different kinds of data.
Nice data, clipped data, noisy data. There’s a lot of each kind of these
situations in the – in the data set. So it learns, just like a seismologist
would, to make the pick despite challenges that it might encounter.
So it works quite well. So at the top is a histogram
of PhaseNet for P and S waves. Below for the sort of standard
non-machine learning-based approach – characteristic function-based approach.
We can see that the P wave is zero mean with a narrow – with a small variance –
not much dispersion about that mean. The S wave is not as good as the
P wave, but it’s dramatically better for the machine learning-based
catalog than for the – for the sort of standard approach.
And I should mention that our ground truth here is not exactly truth.
These are analyst-reviewed picks, which are taken as correct,
but, of course, themselves can include errors as well.
Mentioned Earthquake Transformer is our next-generation
earthquake detection and phase-picking algorithm.
This is a very deep – meaning many-layered neural network.
In addition to its depth, there are different kinds of layers.
So there are convolutional layers, which are meant to characterize
the characteristics of – characteristic features of waveforms.
Those are shown in pink.
There are – shown in yellow, there are recurrent layers.
These are meant to capture the sequencing [audio cuts out], say, the
P wave before the S wave to properly represent what we know to be the case.
And then there are these so-called attention mechanisms that use the –
that focus the attention of the neural network on the parts that matter for
detection, P wave arrival picking and S wave arrival picking.
So there’s seismogram output – or, input and three different outputs.
One is when the earthquake is, P wave arrival time, and the
S wave arrival time – the latitude just like PhaseNet.
It works better than PhaseNet. And the reason it works better
is because, like a seismologist, it uses the waveform context
to pick the P waves and the S waves.
So it doesn’t just focus on the narrow little P wave arrival.
It works better, not only than PhaseNet, but it works better than other machine
learning-based approaches for picking P waves and S waves.
In addition to PhaseNet, there are four other approaches shown here.
Same kind of histogram – P waves and S waves.
And Earthquake Transformer, for now, is the – is the best-performing
of the – of the lot. So let me show you some results that we’ve gotten
using PhaseNet – not Earthquake Transformer, but PhaseNet –
for phase picking as part of this earthquake monitoring workflow.
We applied it to the Guy-Greenbrier Arkansas earthquake, already
a well-studied earthquake sequence thought to be induced.
We developed a complete machine learning-based catalog
for this using PhaseNet. We did not re-train it.
That is, this is PhaseNet as trained on NCSN data, yet it generalizes well
to these smaller, closer, shallower earthquakes that occur in a different
geologic setting, perhaps by a different mechanism.
There are 90,000 events in the catalog. There would be hundreds of thousands
if we had more stations to work with. Our stations are very limited.
These events, as I said, were thought to be induced – or thought to be induced
through a combination of hydraulic stimulation and deep disposal wells.
Clara Yoon showed the importance of hydraulic stimulation
to the early parts of this sequence.
So that did we learn? Well, we learned that this is actually
the superposition of two sequences. So what this just shows – latitude
versus calendar time, the evolution of seismicity in our catalog.
You can see these characteristic parabolic shapes.
You can fit a diffusivity to that. It’s quite high.
Order of 10 to 100 meters squared per second.
Also notice that the sequence starts up here, propagates to the south,
and then sort of putters on, and then it re-nucleates – or, another
sequence nucleates down here and spreads back towards where the
sequence began as well as to the south. And it’s the second part of the sequence
that had the magnitude 4.6 that shut down the –
shut down the wells – 1 and 5. And yet, this – you know,
there’s a chance that this Well 2 was involved in triggering this sequence
because that sequence actually nucleated farther to the south.
So some interesting new insights into this already
well-studied earthquake sequence.
We’ve applied PhaseNet to the Amatrice – well, to the Apennines
sequence – the 2016-2017 part of that. It’s a complex normal-faulting
sequence. Killed hundreds of people. Left tens of thousands of people
homeless in central Italy. And here, again, is a catalog comparison.
The top four panels show the standard catalog – a very good catalog
developed by scientists at INGV. Has 82,000 events in it during
that one-year time period. That’s about one earthquake every six minutes.
The PhaseNet catalog has over 10 times as many events – over 900,000 events.
That’s one every 35 seconds. These are all really tiny earthquakes –
all the new ones. But they are illuminating new structures, and we’re
in the very early stages of figuring out what more we might learn from them.
Now I’d like to close by previewing some things that are coming soon –
within the next year or so. One is a comprehensive application
of PhaseNet to NCSN data led by Ian McBrearty and in collaboration
with the Berkeley Seismo Lab. Richard Allen, thank you for buying
a workstation so that we could run it over there and not move
all that data to Stanford. Preliminary sampling –
sort of random sampling suggests hundreds of thousands of phase picks
per day and about one earthquake per minute from this – from decades
of continuous seismic data. So we’re going to have a lot to sift through.
Every time I talk about this to regional seismic network operators,
they ask about Pn and Sn. We have not worked on that.
You know, because of the distances we use, we don’t have those in the catalog,
so we’re working on a augmented label data set that has these difficult-to-pick
phases that we will add to the STEAD database of local event phases.
The one thing we haven’t done with deep learning is association.
We have two approaches to that – one by Ian McBrearty using graph
convolutional neural networks. The other by Weiqiang Zhu using
a Gaussian mixture model approach. Both these approaches are designed
to work for heterogeneous catalogs that are large, that vary with time,
on which small events are detected by only a few stations.
Both methods use arrival time and amplitude information –
at least they can. Also coming soon, led by Weiqiang Zhu, is an end-to-end
method that combines these different modules into a single linked neural
network. The advantage of this is that information is not lost through
thresholding and, as suggested by the inset in the lower right,
we’ve applied it to Ridgecrest, and it works – it seems to work quite well.
Finally, working with former postdoc Lise Retailleau and engineers at IPGP,
we’ve applied PhaseNet to seismicity on the volcano – this is actually
a volcano near Mayotte. It detects more events,
despite not being retrained, than other methods, including SeisComP3.
It improves locations. It’s now working in real time.
It will be installed at the Martinique Volcano and Seismic Observatory
later this year. And we, or they, are working on a modification
of binder to exploit PhaseNet’s ability to pick P and S separately,
which will lead to further improvements in the catalog.
And you can see your PhaseNet versus standard catalog.
Again, we have much, much more information to work with.
So I’ll stop there and take any questions when the time comes. Thank you.
[silence]