LoginSignup
maruten
@maruten

Are you sure you want to delete the question?

If your question is resolved, you may close it.

Leaving a resolved question undeleted may help others!

We hope you find it useful!

Keras tunerでcheckpointファイルが生成されない

解決したいこと

keras tunerでcheckpointファイルが生成されない
例)
時系列予測でダイレクトアプローチを行うモデルのハイパーパラメータをkeras tunerで行おうとしています。インプットはある時点から30日前までの日データ、アウトプットは未来7日分の予測で、予測期間の各日について別々のモデルを作成します(モデルは7つ)。keras tunerを実行したところ、新しいファイル(hp_tuning)は生成されたのですが、checkpointが生成されずエラーが起きました。前半でkeras tunerの中身を作り、後半でtunerのサーチを7回分forで回しています。;forecast_horizon=7

実行環境:

  • VScode jupyter notebook
  • python 3.9.18 (tensorflowでGPUを使用するため古め)
  • keras tuner 1.4.6

解決方法を教えて下さい。

発生している問題・エラー

Trial 5 Complete [00h 00m 31s]
fit: 91.20452880859375

Best fit So Far: 53.17115020751953
Total elapsed time: 00h 02m 40s
---------------------------------------------------------------------------
NotFoundError                             Traceback (most recent call last)
c:\Users\admin\Desktop\Dissertation\code\Multi-step forecasting.ipynb Cell 42 line 1
    109 with tf.device('/device:GPU:0'):
    110     tuner.search(train_data=scaled_train_data, val_data=scaled_val_data)
--> 112 best_model = tuner.get_best_models(num_models=5)[0]
    113 best_hyperparameters = tuner.get_best_hyperparameters(num_trials=1)[0]
    115 # 辞書に保存

File c:\Users\admin\anaconda3\envs\tf_gpu\lib\site-packages\keras_tuner\src\engine\tuner.py:400, in Tuner.get_best_models(self, num_models)
    382 """Returns the best model(s), as determined by the tuner's objective.
    383 
    384 The models are loaded with the weights corresponding to
   (...)
    397     List of trained model instances sorted from the best to the worst.
    398 """
    399 # Method only exists in this class for the docstring override.
--> 400 return super().get_best_models(num_models)

File c:\Users\admin\anaconda3\envs\tf_gpu\lib\site-packages\keras_tuner\src\engine\base_tuner.py:365, in BaseTuner.get_best_models(self, num_models)
    350 """Returns the best model(s), as determined by the objective.
    351 
    352 This method is for querying the models trained during the search.
   (...)
    362     List of trained models sorted from the best to the worst.
...
     34     'not '
     35     'supported') in error_message:
     36   raise errors_impl.UnimplementedError(None, None, error_message)

NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for hp_direct\untitled_project\trial_2\checkpoint

該当するソースコード

def slice_direct(i):
    # use input as it is
    # y[:, i] ... : = all parts of a batch.  i = prediction-period (forecast horizon)
    
    return lambda x, y: (x, y[:, i])
i=0
class MyHyperModel(keras_tuner.HyperModel):

    def build(self, hp):
        tuned_window_size = hp.Choice('window_size', [10,20,30])
        activation = 'tanh'
        model = tf.keras.models.Sequential()
        # Tune the number of hidden layers.
        second_hidden_layer = hp.Choice('second_hidden_layer?', [0, 1])
        third_hidden_layer = hp.Choice('third_hidden_layer?', [0, 1])

        if second_hidden_layer == 0:
            model.add(GRU(
                    units=hp.Int("units_first_layer", min_value=10, max_value=200, step=10),  # tuning the num of units
                    activation=activation,
                    input_shape=[tuned_window_size, feature_nums],  # use self.window_size here
                    return_sequences=False
                ))
        else:
            model.add(GRU(
                    units=hp.Int("units_first_layer", min_value=10, max_value=200, step=10),  # tuning the num of units
                    activation=activation,
                    input_shape=[tuned_window_size, feature_nums],  # use self.window_size here
                    return_sequences=True
                ))
            
            if third_hidden_layer == 0:
                model.add(GRU(
                        units=hp.Int("units_last_hidden", min_value=10, max_value=200, step=10),  # tuning the num of units
                        activation=activation,
                        return_sequences=False
                        ))
            else:
                model.add(GRU(
                        units=hp.Int("units_last_hidden", min_value=10, max_value=200, step=10),  # tuning the num of units
                        activation=activation,
                        return_sequences=True
                        ))
                model.add(GRU(
                            units=hp.Int("units_last_hidden", min_value=10, max_value=200, step=10),  # tuning the num of units
                            activation=activation,
                            return_sequences=False
                ))
        
       
        model.add(Dense(1))
  
        optimizer_name = hp.Choice('optimizer', ['adam', 'sgd'])
        if optimizer_name == 'adam':
            optimizer = keras.optimizers.Adam(learning_rate=0.001)
        elif optimizer_name == 'sgd':
            learning_rate = hp.Float("learning_rate", min_value=1e-5, max_value=1e-2, sampling="log")
            optimizer = keras.optimizers.SGD(learning_rate=learning_rate, momentum=0.9)
  
        
        model.compile(
            optimizer=optimizer,
            loss=hp.Choice('compiled_loss', ["mse",'mae']),
            metrics=[tf.keras.metrics.MeanAbsoluteError(), tf.keras.metrics.RootMeanSquaredError()],
        )
        
        return model
    
    def fit(self, hp, model, train_data, val_data, i=i, callbacks=None):
        # convert train_data into train_dataset
        tuned_window_size = hp.get('window_size')
        fd = Form_data(forecast_horizon, tuned_window_size, batch_size)
        train_x, train_y = fd.split_data_into_xy_multi_direct(train_data)
        val_x, val_y = fd.split_data_into_xy_multi_direct(val_data)
        train_dataset = fd.transform_npdataset_tfdataset(train_x, train_y)
        val_dataset = fd.transform_npdataset_tfdataset(val_x, val_y)
        model.fit(train_dataset.map(slice_direct(i)), epochs=50, verbose=0, validation_data=val_dataset.map(slice_direct(i)), callbacks=[early_stopping])
        
        # evaluate the performance for one of the prediction days
        predictions = []
        actuals = []
        for x, y in val_dataset.map(slice_direct(i)):
            prediction = model.predict(x, verbose=0) # (409, 7) slide windows * 409 times
            predictions.append(prediction)
            actuals.append(y.numpy())
        
        # Convert lists to numpy arrays
        predictions = np.concatenate(predictions)
        actuals = np.concatenate(actuals)

        # unscaling
        unscaled_predictions = np.copy(predictions)
        unscaled_actuals = np.copy(actuals)
        for i in range(predictions.shape[0]):
            unscaled_predictions[i] = scaler.inverse_transform(predictions[i].reshape(-1, 1)).flatten()
            unscaled_actuals[i] = scaler.inverse_transform(actuals[i].reshape(-1, 1)).flatten()
        mse = MSE(unscaled_actuals, unscaled_predictions)
       
        return mse



models = {}
hyperparameters = {}
for i in range(forecast_horizon):
    tuner = keras_tuner.BayesianOptimization(
            MyHyperModel(),
            objective=keras_tuner.Objective("fit", direction="min"),
            max_trials=5,
            overwrite=True,
            directory="hp_tuning",
            project_name='direct'
        )
    tuner.search_space_summary()
    with tf.device('/device:GPU:0'):
        tuner.search(train_data=scaled_train_data, val_data=scaled_val_data)

    best_model = tuner.get_best_models(num_models=5)[0]
    best_hyperparameters = tuner.get_best_hyperparameters(num_trials=1)[0]

    # 辞書に保存
    models[i] = best_model
    hyperparameters[i] = best_hyperparameters

自分で試したこと

Google Colablatoryでも同様のエラーが起きました。

0

1Answer

エラーがその1行なわけないですよね...?
現状の提供されたエラー内容では解決のしようがありません

0

Comments

  1. @maruten

    Questioner

    失礼しました。全文掲載しました。

  2. jupyter notebookらしくやはりまだ中略されている箇所があって全文ではないですね.

Your answer might help someone💌