Note
Click here to download the full example code
Active learning
This example shows how to use the ActiveLearning
class to iteratively
build a Gaussian process emulator for an unnormalized posterior involving a
simulator. It should be noted that this example is only for illustration
purpose, rather than a real case. For simplicity, required arguments for
ActiveLearning
(including simulator, likelihood, data, etc.) are purely
made up. For a realistic case of active learning, one can refer to Zhao and Kowalski [2022].
First, we define the simulator, prior distribution of its variable parameters, observed data, and likelihood function. They basically define the Bayesian inference problem.
import numpy as np
ndim = 2 # dimension of variable parameters of the simulator
bounds = bounds = np.array([[-5,5],[-5,5]]) # bounds of variable parameters
data = np.array([1,0])
def simulator(x1, x2):
"""Simulator y=f(x)."""
y1, y2 = x1, x2
return np.array([y1, y2])
def prior(x):
"""Uniform prior."""
return 1/(10*10)
def likelihood(y, data):
"""Likelihood function L(y,data)."""
return np.exp(-(y[0]-data[0])**2/100 - (y[0]**2-y[1]-data[1])**2)
Imagine that the simulator is a complex solver. It is not computationally feasible to compute the posterior distribution of the variable parameters using grid estimation or Metropolis Hastings estimation. This is because they require evaluating the likelihood many times which essentially leads to many evaluations of the simulator. Therefore, we resort to use active learning to build a Gaussian process emulator for the unnormalized posterior (prior times likelihood) based on a small number of evaluations of the simulator. The the posterior can be estimated using the emulator.
To do so, we need to pass arguments to following parameters of the ActiveLearning
class:
run_sim_obj : instance of class
RunSimulator
. It carries information on how to run the simulator.lhs_sampler : instance of class
LHS
. It is used to draw initial samples to run simulations in order to train an inital Gaussian process emulator.scalar_gasp : instance of class
ScalarGaSP
. It sets up the emulator structure.
from psimpy.simulator import RunSimulator
from psimpy.sampler import LHS
from psimpy.emulator import ScalarGaSP
run_simulator = RunSimulator(simulator, var_inp_parameter=['x1','x2'])
lhs_sampler = LHS(ndim=ndim, bounds=bounds, seed=1)
scalar_gasp = ScalarGaSP(ndim=ndim)
Next, we create an object of the ActiveLearning
class by
from psimpy.inference import ActiveLearning
active_learner = ActiveLearning(ndim, bounds, data, run_simulator, prior, likelihood,
lhs_sampler, scalar_gasp)
Then we can call the ActiveLearning.initial_simulation()
method to run initial
simulations and call the ActiveLearning.iterative_emulation()
method to
iteratively run new simulation and build emulator. Here we allocate 40 simulations for
initial emulator training and 60 simulations for adaptive training.
n0 = 40
niter = 60
init_var_samples, init_sim_outputs = active_learner.initial_simulation(
n0, mode='parallel', max_workers=4)
var_samples, _, _ = active_learner.iterative_emulation(
n0, init_var_samples, init_sim_outputs, niter=niter)
The upper bounds of the range parameters are 401,3482 397,4694
The initial values of range parameters are 8,026963 7,949389
Start of the optimization 1 :
The number of iterations is 22
The value of the marginal posterior function is -146,5038
Optimized range parameters are 30,59853 130,6077
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,9828751 0,9733764
Start of the optimization 2 :
The number of iterations is 36
The value of the marginal posterior function is -146,5038
Optimized range parameters are 30,59827 130,6067
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 423,194 420,3365
The initial values of range parameters are 8,463881 8,40673
Start of the optimization 1 :
The number of iterations is 13
The value of the marginal posterior function is -148,6792
Optimized range parameters are 30,19534 133,3286
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,9589025 0,9524277
Start of the optimization 2 :
The number of iterations is 12
The value of the marginal posterior function is -148,6792
Optimized range parameters are 30,19539 133,329
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 422,8426 419,9874
The initial values of range parameters are 8,456851 8,399747
Start of the optimization 1 :
The number of iterations is 34
The value of the marginal posterior function is -149,9595
Optimized range parameters are 30,64771 135,0597
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,9360715 0,9297509
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -149,9595
Optimized range parameters are 30,64696 135,0536
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 423,2876 420,4294
The initial values of range parameters are 8,465751 8,408588
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -151,304
Optimized range parameters are 31,21605 136,8571
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,9143024 0,9081287
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -151,304
Optimized range parameters are 31,21617 136,8537
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 389,6167 386,9859
The initial values of range parameters are 7,792334 7,739717
Start of the optimization 1 :
The number of iterations is 13
The value of the marginal posterior function is -152,6009
Optimized range parameters are 32,285 141,3784
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,8935228 0,8874894
Start of the optimization 2 :
The number of iterations is 17
The value of the marginal posterior function is -152,6009
Optimized range parameters are 32,28096 141,3737
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 368,8855 366,3947
The initial values of range parameters are 7,377711 7,327894
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -153,5419
Optimized range parameters are 32,43481 141,7852
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,8736668 0,8677675
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -153,5419
Optimized range parameters are 32,43357 141,7844
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 339,4555 337,1634
The initial values of range parameters are 6,789111 6,743268
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -154,9929
Optimized range parameters are 33,69448 151,2103
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,854674 0,848903
Start of the optimization 2 :
The number of iterations is 22
The value of the marginal posterior function is -154,9929
Optimized range parameters are 33,69468 151,2111
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 369,9581 367,46
The initial values of range parameters are 7,399163 7,349201
Start of the optimization 1 :
The number of iterations is 26
The value of the marginal posterior function is -155,4764
Optimized range parameters are 33,90556 152,0139
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,8364895 0,8308412
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -155,4764
Optimized range parameters are 33,90541 152,0137
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 299,8308 297,8062
The initial values of range parameters are 5,996615 5,956124
Start of the optimization 1 :
The number of iterations is 27
The value of the marginal posterior function is -155,8802
Optimized range parameters are 34,04389 152,6133
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,8190626 0,813532
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -155,8802
Optimized range parameters are 34,04381 152,6131
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 272,589 270,7484
The initial values of range parameters are 5,45178 5,414968
Start of the optimization 1 :
The number of iterations is 22
The value of the marginal posterior function is -156,1383
Optimized range parameters are 34,46966 151,4246
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,802347 0,7969293
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -156,1383
Optimized range parameters are 34,46972 151,4261
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 217,1413 215,6751
The initial values of range parameters are 4,342826 4,313501
Start of the optimization 1 :
The number of iterations is 16
The value of the marginal posterior function is -156,184
Optimized range parameters are 34,52228 152,8428
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,7863001 0,7809907
Start of the optimization 2 :
The number of iterations is 29
The value of the marginal posterior function is -156,184
Optimized range parameters are 34,5049 152,8145
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 206,6122 205,2171
The initial values of range parameters are 4,132244 4,104342
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -156,2348
Optimized range parameters are 34,79805 155,1646
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,7708824 0,7656772
Start of the optimization 2 :
The number of iterations is 22
The value of the marginal posterior function is -156,2348
Optimized range parameters are 34,82503 155,2481
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 264,8501 263,0618
The initial values of range parameters are 5,297002 5,261235
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -156,4063
Optimized range parameters are 34,74024 156,9499
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,7560578 0,7509526
Start of the optimization 2 :
The number of iterations is 35
The value of the marginal posterior function is -156,4062
Optimized range parameters are 34,71806 156,8712
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 255,2317 253,5083
The initial values of range parameters are 5,104633 5,070165
Start of the optimization 1 :
The number of iterations is 20
The value of the marginal posterior function is -156,5552
Optimized range parameters are 35,37305 157,5615
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,7417925 0,7367837
Start of the optimization 2 :
The number of iterations is 37
The value of the marginal posterior function is -156,5552
Optimized range parameters are 35,37704 157,591
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 234,5139 232,9304
The initial values of range parameters are 4,690278 4,658607
Start of the optimization 1 :
The number of iterations is 25
The value of the marginal posterior function is -156,8767
Optimized range parameters are 35,91747 161,0899
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,7280556 0,7231396
Start of the optimization 2 :
The number of iterations is 32
The value of the marginal posterior function is -156,8767
Optimized range parameters are 35,92577 161,0559
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 237,6023 235,9979
The initial values of range parameters are 4,752045 4,719958
Start of the optimization 1 :
The number of iterations is 15
The value of the marginal posterior function is -156,6342
Optimized range parameters are 36,30341 161,9859
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,7148183 0,7099916
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -156,6343
Optimized range parameters are 36,30117 161,8561
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 231,5375 229,9741
The initial values of range parameters are 4,63075 4,599482
Start of the optimization 1 :
The number of iterations is 27
The value of the marginal posterior function is -156,3683
Optimized range parameters are 36,27212 162,5847
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,7020536 0,6973131
Start of the optimization 2 :
The number of iterations is 30
The value of the marginal posterior function is -156,3684
Optimized range parameters are 36,25962 162,5355
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 237,6502 236,0455
The initial values of range parameters are 4,753005 4,720911
Start of the optimization 1 :
The number of iterations is 14
The value of the marginal posterior function is -156,014
Optimized range parameters are 36,19577 162,4854
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,6897369 0,6850796
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -156,014
Optimized range parameters are 36,19621 162,4877
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 221,4001 219,9051
The initial values of range parameters are 4,428002 4,398102
Start of the optimization 1 :
The number of iterations is 36
The value of the marginal posterior function is -155,8456
Optimized range parameters are 36,93092 166,708
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,6778449 0,6732679
Start of the optimization 2 :
The number of iterations is 19
The value of the marginal posterior function is -155,8455
Optimized range parameters are 37,00954 166,8438
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 207,5545 206,153
The initial values of range parameters are 4,15109 4,123061
Start of the optimization 1 :
The number of iterations is 36
The value of the marginal posterior function is -155,2611
Optimized range parameters are 37,54574 171,0122
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,666356 0,6618565
Start of the optimization 2 :
The number of iterations is 26
The value of the marginal posterior function is -155,2611
Optimized range parameters are 37,52707 170,9901
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 211,1368 209,7111
The initial values of range parameters are 4,222736 4,194222
Start of the optimization 1 :
The number of iterations is 38
The value of the marginal posterior function is -154,6105
Optimized range parameters are 37,27271 173,4402
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,6552501 0,6508256
Start of the optimization 2 :
The number of iterations is 38
The value of the marginal posterior function is -154,6104
Optimized range parameters are 37,31152 173,4271
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 229,1987 227,651
The initial values of range parameters are 4,583973 4,553021
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -153,8922
Optimized range parameters are 37,52317 173,0118
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,6445083 0,6401563
Start of the optimization 2 :
The number of iterations is 29
The value of the marginal posterior function is -153,8923
Optimized range parameters are 37,52772 173,0381
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 200,215 198,8631
The initial values of range parameters are 4,004301 3,977262
Start of the optimization 1 :
The number of iterations is 29
The value of the marginal posterior function is -153,0737
Optimized range parameters are 37,6626 173,7851
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,634113 0,6298312
Start of the optimization 2 :
The number of iterations is 38
The value of the marginal posterior function is -153,0733
Optimized range parameters are 37,57977 172,9914
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 208,2931 206,8866
The initial values of range parameters are 4,165862 4,137733
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -152,171
Optimized range parameters are 37,55899 172,4488
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,6240477 0,6198339
Start of the optimization 2 :
The number of iterations is 34
The value of the marginal posterior function is -152,1711
Optimized range parameters are 37,55216 172,3431
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 190,9562 189,6668
The initial values of range parameters are 3,819124 3,793336
Start of the optimization 1 :
The number of iterations is 26
The value of the marginal posterior function is -151,4191
Optimized range parameters are 37,87024 177,6757
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,6142969 0,610149
Start of the optimization 2 :
The number of iterations is 31
The value of the marginal posterior function is -151,4191
Optimized range parameters are 37,8678 177,4358
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 187,3139 186,0491
The initial values of range parameters are 3,746279 3,720983
Start of the optimization 1 :
The number of iterations is 27
The value of the marginal posterior function is -150,3521
Optimized range parameters are 37,93228 177,0193
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,6048462 0,6007621
Start of the optimization 2 :
The number of iterations is 15
The value of the marginal posterior function is -150,3521
Optimized range parameters are 37,92683 176,9819
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 201,0467 199,6892
The initial values of range parameters are 4,020934 3,993783
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -149,229
Optimized range parameters are 37,73923 179,7933
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5956819 0,5916596
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -149,2295
Optimized range parameters are 37,68347 179,3098
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 208,1589 206,7533
The initial values of range parameters are 4,163178 4,135067
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -148,0364
Optimized range parameters are 37,10463 181,452
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,5867911 0,5828289
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -148,0353
Optimized range parameters are 37,10023 181,8802
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 207,5545 206,153
The initial values of range parameters are 4,15109 4,123061
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -147,2549
Optimized range parameters are 36,89036 184,8898
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,5781618 0,5742579
Start of the optimization 2 :
The number of iterations is 18
The value of the marginal posterior function is -147,2555
Optimized range parameters are 36,87001 184,82
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 166,9279 165,8008
The initial values of range parameters are 3,338559 3,316015
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -146,0908
Optimized range parameters are 34,10173 165,8008
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5697827 0,5659353
Start of the optimization 2 :
The number of iterations is 12
The value of the marginal posterior function is -146,091
Optimized range parameters are 34,11888 165,8008
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 164,9351 163,8214
The initial values of range parameters are 3,298703 3,276429
Start of the optimization 1 :
The number of iterations is 29
The value of the marginal posterior function is -144,8362
Optimized range parameters are 33,02363 163,8214
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5616429 0,5578505
Start of the optimization 2 :
The number of iterations is 15
The value of the marginal posterior function is -144,8366
Optimized range parameters are 32,96846 163,8214
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 161,9899 160,8961
The initial values of range parameters are 3,239799 3,217922
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -143,1679
Optimized range parameters are 32,60853 160,8961
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5537325 0,5499935
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -143,1673
Optimized range parameters are 32,60514 160,8961
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 167,4923 166,3613
The initial values of range parameters are 3,349846 3,327227
Start of the optimization 1 :
The number of iterations is 31
The value of the marginal posterior function is -141,4524
Optimized range parameters are 32,84554 166,3613
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5460417 0,5423547
Start of the optimization 2 :
The number of iterations is 10
The value of the marginal posterior function is -141,4531
Optimized range parameters are 32,87053 166,3613
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 158,1908 157,1226
The initial values of range parameters are 3,163815 3,142452
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -140,039
Optimized range parameters are 31,62214 157,1226
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5385617 0,5349251
Start of the optimization 2 :
The number of iterations is 16
The value of the marginal posterior function is -140,0387
Optimized range parameters are 31,62433 157,1226
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 174,0066 172,8316
The initial values of range parameters are 3,480131 3,456632
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -138,1222
Optimized range parameters are 33,48823 172,8316
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5312838 0,5276964
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -138,1223
Optimized range parameters are 33,40525 172,8316
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 150,4312 149,4154
The initial values of range parameters are 3,008624 2,988309
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -139,2082
Optimized range parameters are 30,85041 149,4154
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,5242001 0,5206605
Start of the optimization 2 :
The number of iterations is 25
The value of the marginal posterior function is -139,2085
Optimized range parameters are 30,93585 149,4154
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 163,2096 162,1076
The initial values of range parameters are 3,264193 3,242152
Start of the optimization 1 :
The number of iterations is 30
The value of the marginal posterior function is -136,829
Optimized range parameters are 31,64889 162,1076
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5173027 0,5138097
Start of the optimization 2 :
The number of iterations is 29
The value of the marginal posterior function is -136,8292
Optimized range parameters are 31,73272 162,1076
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 155,3651 154,316
The initial values of range parameters are 3,107301 3,08632
Start of the optimization 1 :
The number of iterations is 38
The value of the marginal posterior function is -137,2507
Optimized range parameters are 31,11748 154,316
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5105845 0,5071368
Start of the optimization 2 :
The number of iterations is 11
The value of the marginal posterior function is -137,2507
Optimized range parameters are 31,11466 154,316
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 159,0334 157,9595
The initial values of range parameters are 3,180667 3,15919
Start of the optimization 1 :
The number of iterations is 19
The value of the marginal posterior function is -134,9592
Optimized range parameters are 30,6283 157,9595
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,5040385 0,5006351
Start of the optimization 2 :
The number of iterations is 20
The value of the marginal posterior function is -134,9589
Optimized range parameters are 30,62948 157,9595
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 150,6829 149,6655
The initial values of range parameters are 3,013659 2,993309
Start of the optimization 1 :
The number of iterations is 9
The value of the marginal posterior function is -133,0322
Optimized range parameters are 28,62106 149,6655
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4976583 0,4942979
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -133,0311
Optimized range parameters are 28,6202 149,6655
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 143,0785 142,1124
The initial values of range parameters are 2,86157 2,842247
Start of the optimization 1 :
The number of iterations is 27
The value of the marginal posterior function is -131,5134
Optimized range parameters are 27,05986 142,1124
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,4914376 0,4881192
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -131,5142
Optimized range parameters are 27,09861 142,1124
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 149,9901 148,9774
The initial values of range parameters are 2,999803 2,979547
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -128,3138
Optimized range parameters are 26,94379 148,9774
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4853704 0,482093
Start of the optimization 2 :
The number of iterations is 11
The value of the marginal posterior function is -128,3136
Optimized range parameters are 26,93153 148,9774
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 141,9733 141,0146
The initial values of range parameters are 2,839465 2,820292
Start of the optimization 1 :
The number of iterations is 15
The value of the marginal posterior function is -126,565
Optimized range parameters are 25,84325 141,0146
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4794513 0,4762139
Start of the optimization 2 :
The number of iterations is 30
The value of the marginal posterior function is -126,5658
Optimized range parameters are 25,84843 141,0146
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 148,4124 147,4103
The initial values of range parameters are 2,968249 2,948206
Start of the optimization 1 :
The number of iterations is 14
The value of the marginal posterior function is -123,2249
Optimized range parameters are 26,04536 147,4103
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4736748 0,4704763
Start of the optimization 2 :
The number of iterations is 37
The value of the marginal posterior function is -123,2255
Optimized range parameters are 26,11794 147,4103
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 141,3136 140,3594
The initial values of range parameters are 2,826273 2,807189
Start of the optimization 1 :
The number of iterations is 24
The value of the marginal posterior function is -121,373
Optimized range parameters are 25,34355 140,3594
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,4680358 0,4648754
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -121,3719
Optimized range parameters are 25,3222 140,3594
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 146,0605 145,0743
The initial values of range parameters are 2,921211 2,901486
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -119,1731
Optimized range parameters are 25,38398 145,0743
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4625295 0,4594063
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -119,1742
Optimized range parameters are 25,27086 145,0743
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 136,1432 135,2239
The initial values of range parameters are 2,722864 2,704479
Start of the optimization 1 :
The number of iterations is 16
The value of the marginal posterior function is -117,4765
Optimized range parameters are 23,8631 135,2239
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4571512 0,4540644
Start of the optimization 2 :
The number of iterations is 31
The value of the marginal posterior function is -117,4779
Optimized range parameters are 23,87798 135,2239
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 121,8399 121,0172
The initial values of range parameters are 2,436797 2,420343
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -117,2996
Optimized range parameters are 22,38443 121,0172
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,4518966 0,4488452
Start of the optimization 2 :
The number of iterations is 19
The value of the marginal posterior function is -117,2989
Optimized range parameters are 22,37563 121,0172
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 123,1404 122,309
The initial values of range parameters are 2,462809 2,446179
Start of the optimization 1 :
The number of iterations is 18
The value of the marginal posterior function is -114,6525
Optimized range parameters are 22,32619 122,309
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4467614 0,4437447
Start of the optimization 2 :
The number of iterations is 38
The value of the marginal posterior function is -114,6523
Optimized range parameters are 22,33099 122,309
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 117,4477 116,6546
The initial values of range parameters are 2,348953 2,333092
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -113,3472
Optimized range parameters are 21,62668 116,6546
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,4417416 0,4387588
Start of the optimization 2 :
The number of iterations is 28
The value of the marginal posterior function is -113,3475
Optimized range parameters are 21,65104 116,6546
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 115,2499 114,4717
The initial values of range parameters are 2,304998 2,289433
Start of the optimization 1 :
The number of iterations is 12
The value of the marginal posterior function is -110,7155
Optimized range parameters are 20,98512 114,4717
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4368334 0,4338837
Start of the optimization 2 :
The number of iterations is 30
The value of the marginal posterior function is -110,7149
Optimized range parameters are 21,00228 114,4717
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 111,8946 111,139
The initial values of range parameters are 2,237892 2,222781
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -108,5065
Optimized range parameters are 20,44681 111,139
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,432033 0,4291158
Start of the optimization 2 :
The number of iterations is 19
The value of the marginal posterior function is -108,507
Optimized range parameters are 20,44983 111,139
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 93,38518 92,75461
The initial values of range parameters are 1,867704 1,855092
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -111,367
Optimized range parameters are 18,37032 92,75461
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,427337 0,4244515
Start of the optimization 2 :
The number of iterations is 18
The value of the marginal posterior function is -111,3671
Optimized range parameters are 18,37078 92,75461
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 99,64494 98,97211
The initial values of range parameters are 1,992899 1,979442
Start of the optimization 1 :
The number of iterations is 23
The value of the marginal posterior function is -106,2562
Optimized range parameters are 18,89163 98,97211
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,422742 0,4198875
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -106,2561
Optimized range parameters are 18,88065 98,97211
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 99,75194 99,07838
The initial values of range parameters are 1,995039 1,981568
Start of the optimization 1 :
The number of iterations is 17
The value of the marginal posterior function is -103,2099
Optimized range parameters are 18,65492 99,07838
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4182447 0,4154206
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -103,2113
Optimized range parameters are 18,65549 99,07838
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 104,3536 103,649
The initial values of range parameters are 2,087073 2,07298
Start of the optimization 1 :
The number of iterations is 20
The value of the marginal posterior function is -98,42375
Optimized range parameters are 18,87395 103,649
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4138422 0,4110477
Start of the optimization 2 :
The number of iterations is 27
The value of the marginal posterior function is -98,42241
Optimized range parameters are 18,87764 103,649
Optimized nugget parameter is 0
Convergence: TRUE
The upper bounds of the range parameters are 103,1118 102,4155
The initial values of range parameters are 2,062236 2,048311
Start of the optimization 1 :
The number of iterations is 20
The value of the marginal posterior function is -95,30729
Optimized range parameters are 18,51323 102,4155
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,4095313 0,406766
Start of the optimization 2 :
The number of iterations is 29
The value of the marginal posterior function is -95,30714
Optimized range parameters are 18,54824 102,4155
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 95,06543 94,42352
The initial values of range parameters are 1,901309 1,88847
Start of the optimization 1 :
The number of iterations is 14
The value of the marginal posterior function is -95,1497
Optimized range parameters are 17,52307 94,42352
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,4053093 0,4025725
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -95,14898
Optimized range parameters are 17,55379 94,42352
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 91,10625 90,49107
The initial values of range parameters are 1,822125 1,809821
Start of the optimization 1 :
The number of iterations is 21
The value of the marginal posterior function is -93,31858
Optimized range parameters are 16,88561 90,49107
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,4011735 0,3984647
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -93,31829
Optimized range parameters are 16,83654 90,49107
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 95,06911 94,42717
The initial values of range parameters are 1,901382 1,888543
Start of the optimization 1 :
The number of iterations is 11
The value of the marginal posterior function is -88,62383
Optimized range parameters are 17,19113 94,42717
Optimized nugget parameter is 0
Convergence: TRUE
The initial values of range parameters are 0,3971213 0,3944398
Start of the optimization 2 :
The number of iterations is 23
The value of the marginal posterior function is -88,62286
Optimized range parameters are 17,21816 94,42717
Optimized nugget parameter is 0
Convergence: FALSE
The upper bounds of the range parameters are 84,25512 83,6862
The initial values of range parameters are 1,685102 1,673724
Start of the optimization 1 :
The number of iterations is 20
The value of the marginal posterior function is -90,56369
Optimized range parameters are 16,01921 83,6862
Optimized nugget parameter is 0
Convergence: FALSE
The initial values of range parameters are 0,39315 0,3904954
Start of the optimization 2 :
The number of iterations is 24
The value of the marginal posterior function is -90,56328
Optimized range parameters are 16,0189 83,6862
Optimized nugget parameter is 0
Convergence: FALSE
Once the active learning process is finished, we obtain the final emulator for the
logarithm of the unnormalized posterior, which is given by the
ActiveLearning.approx_ln_pxl()
method.
We can then estimate the posterior using grid estimation or Metropolis Hastings estimation based on the emulator. An example is as follows. The contour plot shows the estimated posterior.
from psimpy.inference import GridEstimation
import matplotlib.pyplot as plt
grid_estimator = GridEstimation(ndim, bounds, ln_pxl=active_learner.approx_ln_pxl)
posterior, x_ndim = grid_estimator.run(nbins=50)
fig, ax = plt.subplots(1,1,figsize=(5,4))
# initial training points
ax.scatter(init_var_samples[:,0], init_var_samples[:,1], s=10, c='r', marker='o',
zorder=1, alpha=0.8, label='initial training points')
# actively picked training points
ax.scatter(var_samples[n0:,0], var_samples[n0:,1], s=15, c='k', marker='+',
zorder=2, alpha=0.8, label='iterative training points')
# estimated posterior based on the final emulator
posterior = np.where(posterior < 1e-10, None, posterior)
contour = ax.contour(x_ndim[0], x_ndim[1], np.transpose(posterior), levels=10, zorder=0)
plt.colorbar(contour, ax=ax)
ax.legend()
ax.set_title('Active learning')
ax.set_xlabel('x1')
ax.set_ylabel('x2')
ax.set_xlim([-5,5])
ax.set_ylim([-5,5])
plt.tight_layout()

Total running time of the script: ( 2 minutes 12.653 seconds)