metobs_toolkit.Dataset.fill_gaps_automatic#

Dataset.fill_gaps_automatic(modeldata, obstype='temp', max_interpolate_duration_str=None, overwrite_fill=False)[source]#

Fill the gaps by using linear interpolation or debiased modeldata.

This method serves as a triage to select the gaps to be filled with linear interpolation and those to be filled using a diurnal debias gapfill. When the duration of a gap is smaller or equal than max_interpolation_duration, the linear interpolation method is applied else the debiased modeldata method.

For a detailed description of these methods, we refer to the corresponding metobs_toolkit.Dataset.fill_gaps_linear() and metobs_toolkit.Dataset.fill_gaps_era5().

Parameters:

modeldata (metobs_toolkit.Modeldata) – The modeldata to use for the gapfill. This model data should the required timeseries to fill all gaps present in the dataset.
obstype (String, optional) – Name of the observationtype you want to apply gap filling on. The modeldata must contain this observation type as well. The default is ‘temp’.
max_interpolate_duration_str (Timedelta or str, optional) – Maximum duration to apply interpolation for gapfill when using the automatic gapfill method. Gaps with longer durations will be filled using debiased modeldata. The default is None.
overwrite_fill (bool, optional) – If a gap has already filled values, the interpolation of this gap is skipped if overwrite_fill is False. If set to True, the gapfill values and info will be overwitten. The default is False.

Returns:

comb_df – A dataframe containing all the filled records.

Return type:

pandas.DataFrame

Examples

import metobs_toolkit

your_dataset = metobs_toolkit.Dataset()
your_dataset.update_settings(
    input_data_file=metobs_toolkit.demo_datafile, # path to the data file
    input_metadata_file=metobs_toolkit.demo_metadatafile,
    template_file=metobs_toolkit.demo_template,
)
# Specify the gap defenition
your_dataset.update_qc_settings(gapsize_in_records = 20)

#Update the gapsize BEFORE importing the data
your_dataset.import_data_from_file()

#Update the settings (definition of the period to calculate biases for)
your_dataset.update_gap_and_missing_fill_settings(
                                                  gap_debias_prefered_leading_period_hours=24,
                                                  gap_debias_prefered_trailing_period_hours=24,
                                                  gap_debias_minimum_leading_period_hours=6,
                                                  gap_debias_minimum_trailing_period_hours=6,
                                                  )
#(As a demonstration, we will fill the gaps of a single station. The following functions can also be
# directly applied to the dataset.)
your_station = your_dataset.get_station('vlinder05')

#Get ERA5 modeldata at the location of your stations and period.
ERA5_modeldata = your_station.get_modeldata(modelname='ERA5_hourly',
                                            obstype='temp')

#Use the debias method to fill the gaps
gapfill_df = your_station.fill_gaps_automatic(modeldata=ERA5_modeldata,
                                              max_interpolate_duration_str='6h', # <6 hours will be interpolated
                                              obstype='temp')