{"id":313,"date":"2020-04-19T17:00:33","date_gmt":"2020-04-19T08:00:33","guid":{"rendered":"https:\/\/www.crestboz.co.jp\/techblog\/?p=313"},"modified":"2024-01-03T12:53:19","modified_gmt":"2024-01-03T03:53:19","slug":"hyperopt%e3%81%a8optuna%e3%81%ab%e3%82%88%e3%82%8b%e3%83%8f%e3%82%a4%e3%83%91%e3%83%bc%e3%83%91%e3%83%a9%e3%83%a1%e3%83%bc%e3%82%bf%e3%83%81%e3%83%a5%e3%83%bc%e3%83%8b%e3%83%b3%e3%82%b0%e3%81%ae","status":"publish","type":"post","link":"https:\/\/www.crestboz.co.jp\/techblog\/archives\/313\/","title":{"rendered":"Hyperopt\u3068Optuna\u306b\u3088\u308b\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u306e\u4f8b"},"content":{"rendered":"

\u6a5f\u68b0\u5b66\u7fd2\u306b\u304a\u3044\u3066\u3001\u4f5c\u6210\u3057\u305f\u30e2\u30c7\u30eb\u306e\u7cbe\u5ea6\u3092\u9ad8\u3081\u308b\u305f\u3081\u306e\u30c6\u30af\u30cb\u30c3\u30af\u306e\u3072\u3068\u3064\u304c\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u306e\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\uff08\u8abf\u6574\uff09\u3067\u3059\u3002
\n\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u306e\u30c1\u30e5\u30fc\u30cb\u30f3\u30b0\u306e\u65b9\u6cd5\u3068\u3057\u3066\u306f\u3001\u624b\u52d5\u3001\u30b0\u30ea\u30c3\u30c9\u30b5\u30fc\u30c1\u3001\u30e9\u30f3\u30c0\u30e0\u30b5\u30fc\u30c1\u3001\u30d9\u30a4\u30ba\u6700\u9069\u5316\u306a\u3069\u306e\u65b9\u6cd5\u304c\u3042\u308a\u307e\u3059\u3002
\n\u4eca\u56de\u306f\u3001\u30d9\u30a4\u30ba\u6700\u9069\u5316\u306b\u3088\u308b\u65b9\u6cd5\u3067\u3088\u304f\u4f7f\u308f\u308c\u308b\u30e9\u30a4\u30d6\u30e9\u30ea\u3067\u3042\u308bHyperopt<\/a>\u3068Optuna<\/a>\u306e\u4f7f\u3044\u65b9\u3092\u6bd4\u8f03\u3059\u308b\u305f\u3081\u3001\u540c\u3058\u554f\u984c\uff08Boston house prices dataset\uff09\u306b\u5bfe\u3057\u3066\u3001\u540c\u3058\u30e2\u30c7\u30eb\uff08\u30ea\u30c3\u30b8\u56de\u5e30; sklearn.linear_model.Ridge<\/code>\uff09\u3092\u7528\u3044\u3066\u3001\u7c21\u5358\u306a\u30b3\u30fc\u30c9\u3092\u4f5c\u6210\u3057\u3066\u307f\u307e\u3057\u305f\u3002\uff08\u3042\u304f\u307e\u3067Hyperopt\u3068Optuna\u306e\u4f7f\u7528\u4f8b\u3092\u793a\u3059\u305f\u3081\u306e\u3082\u306e\u3067\u3001\u3053\u306e\u554f\u984c\u306e\u89e3\u6cd5\u3092\u793a\u3057\u3066\u3044\u308b\u308f\u3051\u3067\u306f\u3054\u3056\u3044\u307e\u305b\u3093\u306e\u3067\u3001\u3054\u4e86\u627f\u304f\u3060\u3055\u3044\u3002)<\/p>\n

from sklearn import datasets\r\nfrom sklearn.linear_model import Ridge\r\nfrom sklearn.model_selection import train_test_split\r\nfrom sklearn.metrics import mean_absolute_error\r\nfrom hyperopt import fmin, tpe, hp, STATUS_OK, Trials, space_eval\r\n\r\ndef score(params):\r\n    reg = Ridge(**params)\r\n    reg.fit(X_train, y_train)\r\n    y_pred = reg.predict(X_test)\r\n\r\n    score = mean_absolute_error(y_test, y_pred)\r\n    print(f'params: {params}, score: {score:.4f}')\r\n\r\n    return {'loss': score, 'status': STATUS_OK}\r\n\r\nspace = {\r\n    'alpha': hp.loguniform('alpha', 0.1, 5.0),\r\n    'fit_intercept': hp.choice('fit_intercept', [True, False]), \r\n    'normalize': hp.choice('normalize', [True, False]), \r\n}\r\n\r\nif __name__ == '__main__':\r\n    # \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u30ed\u30fc\u30c9\r\n    (X, y) = datasets.load_boston(return_X_y=True)\r\n    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\r\n\r\n    # hyperopt \u306b\u3088\u308b\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u6700\u9069\u5316\r\n    trials = Trials()\r\n    best = fmin(score, space, algo=tpe.suggest, trials=trials, max_evals=100)\r\n\r\n    # \u7d50\u679c\u3092\u8868\u793a\r\n    sorted_lst = sorted(trials.trials, key=lambda x: x['result']['loss'])\r\n    min_loss = sorted_lst[0]['result']['loss']\r\n    print(f'best score: {min_loss:.4f}, best params: {space_eval(space, best)}')<\/pre>\n
from sklearn import datasets\r\nfrom sklearn.linear_model import Ridge\r\nfrom sklearn.model_selection import train_test_split\r\nfrom sklearn.metrics import mean_absolute_error\r\nimport optuna\r\n\r\ndef objective(trial):\r\n    params = {\r\n        'alpha': trial.suggest_loguniform(\"alpha\", 0.1, 5), \r\n        'fit_intercept': trial.suggest_categorical('fit_intercept', [True, False]),\r\n        'normalize': trial.suggest_categorical('normalize', [True, False]),\r\n    }\r\n\r\n    reg = Ridge(**params)\r\n    reg.fit(X_train, y_train)\r\n    y_pred = reg.predict(X_test)\r\n\r\n    mae = mean_absolute_error(y_test, y_pred)\r\n    return mae\r\n\r\n\r\nif __name__ == '__main__':\r\n    # \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u30ed\u30fc\u30c9\r\n    (X, y) = datasets.load_boston(return_X_y=True)\r\n    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\r\n\r\n    # optuna \u306b\u3088\u308b\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u6700\u9069\u5316\r\n    study = optuna.create_study()\r\n    study.optimize(objective, n_trials=100)\r\n\r\n    # \u7d50\u679c\u3092\u8868\u793a\r\n    print(f'best score: {study.best_value:.4f}, best params: {study.best_params}')<\/pre>\n
\u4e0a\u304cHyperopt\u306e\u30b3\u30fc\u30c9\u4f8b\u3001\u4e0b\u304cOptuna\u306e\u30b3\u30fc\u30c9\u4f8b\u3067\u3059\u3002\u30b3\u30fc\u30c9\u3092\u6bd4\u8f03\u3059\u308b\u3068\u308f\u304b\u308b\u3088\u3046\u306b\u3001Hyperopt\u306e\u65b9\u306f\u6700\u9069\u5316\u3059\u308b\u30e2\u30c7\u30eb\uff08\u95a2\u6570\uff09\u3068\u306f\u5225\u306b\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u306e\u7bc4\u56f2\u3092\u5b9a\u7fa9\u3057\u3066fmin<\/code>\u95a2\u6570\u306b\u95a2\u6570\u3068\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u306e\u7bc4\u56f2\u3092\u5b9a\u7fa9\u3057\u305f\u3082\u306e\u3092\u6e21\u3057\u3066\u3044\u307e\u3059\u304c\u3001Optuna\u306e\u65b9\u306f\u6700\u9069\u5316\u3059\u308b\u30e2\u30c7\u30eb\uff08\u95a2\u6570\uff09\u306e\u4e2d\u3067\u30cf\u30a4\u30d1\u30fc\u30d1\u30e9\u30e1\u30fc\u30bf\u306e\u7bc4\u56f2\u3092\u5b9a\u7fa9\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n
\u4eca\u56de\u306e\u4f8b\u3067\u306f\u3001\u4ee5\u4e0b\u306e\u901a\u308a\u3001\u5b9f\u884c\u7d50\u679c\u306f\u307b\u307c\u540c\u3058\u306b\u306a\u308a\u307e\u3057\u305f\u3002<\/p>\n
best score: 3.1175, best params: {'alpha': 4.321534442189128, 'fit_intercept': True, 'normalize': False}<\/pre>\n
best score: 3.1174, best params: {'alpha': 4.373698954104559, 'fit_intercept': True, 'normalize': False}<\/pre>\n
Hyperopt\u3068Optuna\u306f\u3001TPE\uff08Tree-structured Parzen Estimator\uff09\u3068\u3044\u3046\u540c\u4e00\u306e\u6700\u9069\u5316\u30a8\u30f3\u30b8\u30f3\u3092\u4f7f\u3063\u3066\u3044\u307e\u3059\u304c\u3001\u679d\u5208\u308a\u306e\u6a5f\u80fd\u306e\u8ca2\u732e\u306b\u3088\u308a\u3001Optuna \u306e\u65b9\u304c\u52b9\u7387\u7684\u306b\u6700\u9069\u5316\u306e\u51e6\u7406\u3092\u3059\u308b\u3088\u3046\u3067\u3059\u3002<\/p>\n
Hyperopt\u3068Optuna\u306b\u3064\u3044\u3066\u306e\u8a73\u3057\u3044\u60c5\u5831\u306f\u3001\u4ee5\u4e0b\u306e\u30ea\u30f3\u30af\u3092\u3054\u53c2\u7167\u304f\u3060\u3055\u3044\u3002<\/p>\n
\u53c2\u7167<\/h2>\n