Skip to content

KeyError: 'model_name' #84

@ningmeng-Mo

Description

@ningmeng-Mo

Hello, whenever I run the programme, whether using your dataset or my own, I consistently encounter the following issue.
root@I2286be2f7200301457:/hy-tmp/TFB# python ./scripts/run_benchmark.py --config-path "rolling_forecast_config.json" --data-name-list "ILI.csv" --strategy-args '{"horizon": 24}' --model-name "duet.DUET" --model-hyper-params '{"CI": 1, "batch_size": 8, "d_ff": 512, "d_model": 256, "dropout": 0.1, "e_layers": 2, "factor": 3, "fc_dropout": 0, "horizon": 24, "k": 2, "loss": "MAE", "lr": 0.001, "lradj": "type1", "n_heads": 1, "norm": true, "num_epochs": 100, "num_experts": 2, "patch_len": 48, "patience": 5, "seq_len": 104}' --deterministic "full" --gpus 0 --num-workers 1 --timeout 60000 --save-path "ILI/DUET"
2025-09-19 16:34:08 [INFO] ts_benchmark.data.data_source(126): Start loading 1 series in parallel
2025-09-19 16:34:08 [INFO] ts_benchmark.data.data_source(135): Data loading finished.
2025-09-19 16:34:08 [INFO] ts_benchmark.data.suites.global_storage(40): Data server starting...
2025-09-19 16:34:08 [INFO] ts_benchmark.data.suites.global_storage(41): Start sending data to the global storage.
2025-09-19 16:34:08 [INFO] ts_benchmark.data.suites.global_storage(46): Notifying all workers to sync data from the global storage.
2025-09-19 16:34:08 [INFO] ts_benchmark.data.suites.global_storage(49): Data server started.
2025-09-19 16:34:08 [INFO] ts_benchmark.models.model_loader(102): Trying to load model ts_benchmark.baselines.duet.DUET
scheduling DUET: 0%| | 0/1 [00:00<?, ?it/s]2025-09-19 16:34:09 [WARNING] ts_benchmark.baselines.deep_forecasting_model_base(54): The model parameter horizon is deprecated. Please use pred_len.
---------------------------------------------------------- DUET
Total trainable parameters: 1225507
Validation loss decreased (inf --> 0.309561). Saving model ...
Updating learning rate to 0.001
EarlyStopping counter: 1 out of 5
Updating learning rate to 0.0005
Validation loss decreased (0.309561 --> 0.296071). Saving model ...
Updating learning rate to 0.00025
Validation loss decreased (0.296071 --> 0.283024). Saving model ...
Updating learning rate to 0.000125
EarlyStopping counter: 1 out of 5
Updating learning rate to 6.25e-05
Validation loss decreased (0.283024 --> 0.281689). Saving model ...
Updating learning rate to 3.125e-05
EarlyStopping counter: 1 out of 5
Updating learning rate to 1.5625e-05
Validation loss decreased (0.281689 --> 0.274102). Saving model ...
Updating learning rate to 7.8125e-06
EarlyStopping counter: 1 out of 5
Updating learning rate to 3.90625e-06
EarlyStopping counter: 2 out of 5
Updating learning rate to 1.953125e-06
EarlyStopping counter: 3 out of 5
Updating learning rate to 9.765625e-07
EarlyStopping counter: 4 out of 5
Updating learning rate to 4.8828125e-07
Validation loss decreased (0.274102 --> 0.271266). Saving model ...
Updating learning rate to 2.44140625e-07
EarlyStopping counter: 1 out of 5
Updating learning rate to 1.220703125e-07
EarlyStopping counter: 2 out of 5
Updating learning rate to 6.103515625e-08
EarlyStopping counter: 3 out of 5
Updating learning rate to 3.0517578125e-08
EarlyStopping counter: 4 out of 5
Updating learning rate to 1.52587890625e-08
EarlyStopping counter: 5 out of 5
scheduling DUET: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:42<00:00, 42.27s/it]
collecting DUET: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 4364.52it/s]
2025-09-19 16:34:52 [INFO] ts_benchmark.recording(103): loading log file /hy-tmp/TFB/result/ILI/DUET/DUET.1758270892.I2286be2f7200301457.1771.csv.tar.gz
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py", line 3621, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'model_name'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "./scripts/run_benchmark.py", line 357, in
report(report_config, report_method=args.report_method)
File "/hy-tmp/TFB/ts_benchmark/report/init.py", line 12, in report
report_csv.report(report_config)
File "/hy-tmp/TFB/ts_benchmark/report/report_csv.py", line 52, in report
leaderboard_df = get_leaderboard(
File "/hy-tmp/TFB/ts_benchmark/report/utils/leaderboard.py", line 154, in get_leaderboard
single_metric_result = _calculate_single_metric_result(
File "/hy-tmp/TFB/ts_benchmark/report/utils/leaderboard.py", line 53, in _calculate_single_metric_result
metric_df["model_name"] + ";" + metric_df["model_params"]
File "/usr/local/lib/python3.8/dist-packages/pandas/core/frame.py", line 3505, in getitem
indexer = self.columns.get_loc(key)
File "/usr/local/lib/python3.8/dist-packages/pandas/core/indexes/base.py", line 3623, in get_loc
raise KeyError(key) from err
KeyError: 'model_name'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions