Implementation of a reduced rank square-root smoother for high resolution ocean data assimilation
Introduction
The development of data assimilation in geophysics has been triggered by the need of an accurate representation of the current atmospheric state to initialize a numerical weather forecast. In the framework of estimation theory, this issue is a filtering problem of estimating a dynamical state given a model (numerical), past and present observations (the only available data for a forecast). The spearhead of dynamical estimation methods is the Kalman filter (Kalman, 1960). The Kalman filter has received much interest in geophysics. This is due to its solid roots in estimation theory on the one hand, and to its possible implementation – provided some simplifications are made – with large numerical models on the other hand [e.g. Parrish and Cohn, 1985, Todling and Cohn, 1994, Evensen, 1994, Fukumori and Malanotte-Rizzoli, 1995, Houtekamer and Mitchell, 1998]. However, many oceanographic applications now expect more from data assimilation than only initialize a prediction. The study of climate variability and evolution, the targeted case studies of ocean dynamics, or biogeochemistry, require reanalyses of the ocean circulation: complete, consistent, and accurate datasets of the variables that describe the ocean, over a continuous time period in the past. The estimation of past ocean states implies that posterior observations exist (up to present), and may clearly be used retrospectively to improve the data assimilation outputs. Such estimation problem is tackled by smoothers.
Optimal linear smoothers stemming from estimation theory can be considered as an extension of the Kalman filter that takes future observations into account. Actually, all optimal linear smoother algorithms involve the Kalman filter. For a detailed description of the various types of smoothers based on Kalman’s theory, and their algorithms, we refer the reader to textbooks such as Anderson and Moore (1979) or Simon (2006). In this paper, we are concerned with the sequential approach of smoothing, as exposed by Cohn et al. (1994) or Evensen and van Leeuwen (2000): the Kalman filter analysis is followed by retrospective analyses, i.e. corrections of the past state estimates using the Kalman filter innovation.
Smoother algorithms involve the Kalman filter, however, it is well known that the Kalman filter cannot be implemented in its canonical form with realistic models of the ocean. First, models are generally nonlinear. Linearity is an essential hypothesis for the Kalman filter to be optimal. Various strategies can be adopted to extend the Kalman filter to nonlinear systems. The Extended Kalman filter is the straightforward, first order generalization of the Kalman filter. Higher order approaches exist, such as the unscented Kalman filter (Julier and Uhlmann, 1997). Evensen (1994) proposed an ensemble approach to nonlinear Kalman filtering. The second hindrance to the application of the standard Kalman filter is due to computer storage and CPU requirements. The Kalman filter is computationally untractable, and approximations have to be made for implementation. The most common strategy in oceanography consists in reducing the dimension of the error space. The Ensemble Kalman Filter (EnKF) of Evensen (1994) does it implicitly, by handling an ensemble of states far smaller than the state vector dimension. The Reduced-Rank SQuare-RooT filter (Verlaan and Heemink, 1997), the Error Subspace Statistical Estimation (ESSE) algorithm (Lermusiaux and Robinson, 1999), and the Singular Evolutive Extended Kalman (SEEK) filter (Pham et al., 1998, Brasseur and Verron, 2006) are explicitly founded on the order reduction.
Some algorithms approximating the Kalman filter have been extended for smoothing and applied in problems connected to oceanography. It is the case of the EnKF (van Leeuwen and Evensen, 1996, Evensen and van Leeuwen, 2000), with applications with a 2-layer quasigeostrophic model (van Leeuwen, 1999, van Leeuwen, 2001). Lermusiaux and Robinson (1999) have derived a ESSE smoother. This algorithm has been run in real data assimilation experiments at high resolution by Lermusiaux, 1999a, Lermusiaux, 1999b, and Lermusiaux et al. (2002). Fukumori (2002) have tested a Kalman filter and an approximation of the RTS smoother (Rauch et al., 1965), based on a state-partitioning approach, with a one-dimensional shallow-water model. Todling and Cohn (1996) and Todling et al. (1998) have introduced various strategies to make the Kalman filter and optimal smoothers applicable with large systems and tested some of them with a linear shallow-water model. Gaspar and Wunsch (1989) have experimented the RTS smoother with an over-simplified equation of ocean dynamics. Recently, Ravela and McLaughlin (2007) have illustrated fast ensemble smoothing algorithms in identical twin experiments with the Lorenz-95 100-variable model.
The work presented in this paper basically aims at pioneering the implementation of a smoother algorithm, based on the SEEK filter algorithm, in a high resolution ocean data assimilation system imbedding a primitive equations ocean circulation model. The mid-term perspective is to make high quality, high resolution reanalyses of the ocean circulation. In this prospect, the chosen smoothing algorithm is sequential, in the spirit of Evensen and van Leeuwen (2000) or Cohn et al. (1994). The first part of the paper is dedicated to the theoretical derivation and the algorithmic aspects of the SEEK extension to smoothing, including a method to handle a common parameterization of the model error. It is shown that the implementation of this sequential smoother is straightforward, and results in negligible additional cost, when the SEEK filter is already in place. In the second part of the paper, the application of the SEEK smoother with a mesoscale ocean circulation model is demonstrated and evaluated. Three issues expected to be tricky or relevant in a real data assimilation context are examined.
Section 2 recapitulates the Kalman filter and the sequential smoothing ingredients and algorithm. Section 3 presents the SEEK filter and the SEEK smoother algorithm. The SEEK filter is well known (Pham et al., 1998, Rozier et al., 2007). In Section 3.1, we stress the aspects particularly relevant or problematical for the smoother extension. In Section 3.2, the equations of the new SEEK smoother algorithm are written down. For clarity’s sake, the perfect model and imperfect model cases are considered separately. Two practical aspects are examined in Section 4: the computational complexity and the numerical implementation. In Section 5, we present the set-up of a twin experiment carried out with the SEEK smoother and an ocean circulation model in a high resolution, idealized configuration. Results are examined in Section 6, and Section 7 reports three examinations and numerical experiments concerning key issues with the SEEK smoother. Section 8 concludes.
Section snippets
Optimal linear smoothers
The Kalman filter and the most common optimal linear smoothers are derived and described in many textbooks (Anderson and Moore, 1979, Simon, 2006, Evensen, 2007). Here we first recall the well-known Kalman filter algorithm to set down the notations that will be used next. Then we provide an intuitive view and the equations of the less-known optimal smoothers. Note that we are interested in the smoothers of the sequential type here.
Square-root transformation and order reduction
The square-root transformation and the order reduction are introduced in the smoother equations. This is done here with the Singular Evolutive Extended Kalman (SEEK) filter, but adaptation to any other square-root filter is similar [e.g. Ravela and McLaughlin, 2007].
Algorithm complexity
Considering that the expensive operations in the Kalman filter are the model integrations (Eqs. (1a), (1b)) and the inversion of the innovation error covariance (Eqs. (2a), (2c)), the Kalman filter algorithm complexity approaches:where n is the size of the state vector, N the number of operations in one model integration, and s the number of observations.
The full rank fixed-lag smoother is far more expensive than the Kalman filter, for it involves extra model integrations
Twin experiment with a nonlinear, ocean circulation model: description
The smoother algorithm is implemented and applied with an ocean circulation model. In this section, the experimental set-up is described. Results are presented in the next section.
Results
Concerning the smoother, we focus on the 8-days retrospective analysis in this section. The reason will be given in Section 7.1, where the choice of the lag is discussed.
On the smoother lag
In the framework of the Kalman filter hypotheses, the largest lag theoretically provides the best error reduction and smoothing. But it generally makes sense to consider a limited number of retrospective analyses. In presence of unstable or dissipative dynamics in particular, most of the smoother improvements are due to the first few retrospective analyses, the others having a minor impact on the results (Cohn et al., 1994). High resolution ocean circulation models develop nonlinear, unstable
Conclusion
In theory, the Kalman filter is well designed to provide an initial state for a prediction. To build a re-analysis of the ocean circulation, optimal smoothers are more appropriate. In particular, the impacts of gaps in the observation network are efficiently smoothed out by smoothers. In this paper, a smoother algorithm has been derived based on a reduced rank square-root filter, the SEEK filter, and implemented with a high resolution ocean circulation model.
The theoretical derivation of the
Acknowledgements
We thank Jean-Marc Molines for sharing his skills with the NEMO model. This work was supported by CNRS/INSU through the LEFE/ASSIM program and by the ANR program. Partial support of the European Commission under Grant Agreement FP7-SPACE-2007-1-CT-218812-MYOCEAN is gratefully acknowledged. Calculations were performed using HPC resources from GENCI-IDRIS (Grant 2009-011279). The original manuscript has been well improved thanks to Pr Pierre Lermusiaux and two other anonymous referees.
References (54)
- et al.
Assimilation of altimetric data in the mid-latitude oceans using the seek filter with an eddy-resolving primitive equation model
Journal of Marine Systems
(1999) Estimation and study of mesoscale variability in the strait of Sicily
Dynamics of Atmospheres and Oceans
(1999)Uncertainty estimation and prediction for interdisciplinary ocean dynamics
Journal of Comparative Physiology
(2006)- et al.
The representer method, the ensemble Kalman filter and the ensemble Kalman smoother: a comparison study using a nonlinear reduced gravity ocean model
Ocean Modelling
(2006) - et al.
Assimilation of sea-surface temperature and altimetric observations during 1992–1993 into an eddy permitting primitive equation model of the North Atlantic Ocean
Journal of Marine Systems
(2003) - et al.
Optimal Filtering
(1979) Inverse methods in physical oceanography
- Brankart, J., Testut, C., Parent, L., 2002. An integrated system of sequential assimilation modules: sesam reference...
- Brankart, J.-M., Testut, C.-E., Brasseur, P., Verron, J., 2003. Implementation of a multivariate data assimilation...
- et al.
Efficient parameterization of the observation error covariance matrix for square root or ensemble Kalman filters: application to ocean altimetry
Monthly Weather Review
(2009)
The seek filter method for data assimilation in oceanography: a synthesis
Ocean Dynamics
Data assimilation for marine monitoring and prediction: the MERCATOR operational assimilation systems and the MERSEA developments
Quarterly Journal of the Royal Meteorological Society
Mapping tropical Pacific sea level: data assimilation via a reduced state space Kalman filter
Journal of Geophysical Research
On the role of the GRACE mission in the joint assimilation of altimetry and TAO data in a tropical Pacific ocean model
Geophysical Research Letters
The influence of boundary conditions on midlatitude jet separation in ocean numerical models
Journal of Physical Oceanography
A fixed-lag Kalman smoother for retrospective data assimilation
Monthly Weather Review
Sequential data assimilation with a non linear quasigeostrophic model using monte carlo methods to forecast error statistics
Journal of Geophysical Research
The ensemble Kalman filter: theoretical formulation and practical implementation
Ocean Dynamics
Data Assimilation. The Ensemble Kalman Filter
An ensemble Kalman smoother for nonlinear dynamics
Monthly Weather Review
Fast data assimilation using a nonlinear Kalman filter and a model surrogate: an application to the columbia river estuary
Dynamics of Atmospheres and Oceans
A partitioned Kalman filter and smoother
Monthly Weather Review
An approximate Kalman filter for ocean data assimilation; an example with an idealized Gulf stream model
Journal of Geophysical Research
Estimates from altimeter data of barotropic Rossby waves in Northwestern Atlantic ocean
Journal of Physical Oceanography
Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter
Monthly Weather Review
The role of mesoscale eddies in the general circulation of the ocean—numerical experiments using a wind-driven quasi-geostrophic model
Journal of Physical Oceanography
Data assimilation using an Ensemble Kalman Filter technique
Monthly Weather Review
Cited by (38)
A direct insertion technique to assimilate sea surface height into a storm surge model
2024, Journal of HydrologyImproved thermal structure simulation and optimized sampling strategy for Lake Erie using a data assimilative model
2020, Journal of Great Lakes ResearchCitation Excerpt :Ultimately, developing a GLOFS-DA framework with a time-varying error covariance using rank-reduction (e.g. Fukumori and Malanotte‐Rizzoli, 1995; Verlaan and Heemink, 1997; Pham et al., 1998; Fukumori, 2002; Cao et al., 2007; Cosme and et al., 2010; Xue et al., 2011) and ensemble representation (Evensen 2009; Anderson 2001; Houtekamer and Mitchell, 2001; Whitaker et al., 2002; Houtekamer et al., 2005; Torn and et al., 2009; Xue et al., 2011, 2012; Houtekamer and Zhang, 2016) is the long-term goal.
Comparison of different incremental analysis update schemes in a realistic assimilation system with Ensemble Kalman Filter
2017, Ocean ModellingCitation Excerpt :In all these works, satisfactory results have been obtained and the capacity of the IAU techniques to act like a continuous assimilation and to reduce the high frequency analysis-induced oscillations has been proven. Yan et al. (2014) compared three IAU schemes (IAU 0, IAU 50 and IAU 100) to the conventional intermittent scheme with an idealised configuration (the so called square-box configuration (Cosme et al., 2010)) of the NEMO (Nucleus for European Modelling of the Ocean) primitive equation ocean model in a twin experiment using the Ensemble Kalman Filter (EnKF) (Evensen, 2004). They concluded that 1) in case with the same ensemble size, the three IAU schemes outperform the intermittent scheme.
Comparison of different assimilation schemes in a sequential Kalman filter assimilation system
2014, Ocean ModellingCitation Excerpt :Moreover, comparisons with other similar assimilation methods such as the smoother (Cosme et al., 2010), the Incremental Digital Filtering (IDF) (Polavarapu et al., 2004), different varieties of nudging (Sandery et al., 2011; Lei et al., 2012), etc., seem very interesting to provide a complete view of the performance of these assimilation schemes.
Simplified Kalman smoother and ensemble Kalman smoother for improving reanalyses
2023, Geoscientific Model Development