On methods for assessment of the value of observations in convection-permitting data assimilation and numerical weather forecasting

In numerical weather prediction (NWP), a large number of observations are used to create initial conditions for weather forecasting through a process known as data assimilation. An assessment of the value of these observations for NWP can guide us in the design of future observation networks, help us to identify problems with the assimilation system, and allow us to assess changes to the assimilation system. However, assessment can be challenging in convection-permitting NWP. This is because verification of convection-permitting forecasts is not easy, the forecast model is strongly nonlinear, a limited-area model is used, and the observations used often contain complex error statistics and are often associated with nonlinear observation operators. We compare methods that can be used to assess the value of observations in convection-permitting NWP and discuss operational considerations when using these methods. We focus on their applicability to ensemble forecasting systems, as these systems are becoming increasingly dominant for convection-permitting NWP. We also identify several future research directions, which include comparing results from different methods, comparing forecast validation using analyses versus using observations, applying flow-dependent covariance localization, investigating the effect of ensemble size on the assessment, and generating and validating the nature run in observing-system simulation experiments.