Page 1 of 1

How many domains at most can run simultaneously?

PostPosted: Mon Jan 28, 2019 7:58 pm
by jzhang
Hello!

I am running a 12-domain simulation. There is no problem to run 11 domains. But when I added the last domain, WPS ran successfully, while real.exe failed at the very beginning with the error "invalid parent id for domain 2". I am wondering if 11 is the max domain number we can run simultaneously. Domain part namelist is attached below:
&domains
eta_levels = 1.000, 0.996, 0.99, 0.98, 0.97,
0.96, 0.95, 0.94, 0.93, 0.92,
0.91, 0.895, 0.88, 0.865, 0.85,
0.825, 0.8, 0.775, 0.75, 0.72,
0.69, 0.66, 0.63, 0.6, 0.57,
0.54, 0.51, 0.475, 0.44, 0.405,
0.37, 0.33, 0.29, 0.25, 0.21,
0.175, 0.145, 0.115, 0.09, 0.065,
0.045, 0.025, 0.01, 0.000,

time_step = 60,
time_step_fract_num = 0,
time_step_fract_den = 1,
max_dom = 12,
e_we = 130,73,73,61,61,61,61,61,76,76,100,76
e_sn = 100,73,73,73,61,61,61,61,76,76, 76,76
e_vert = 44, 44,44,44,44,44,44,44,44,44, 44,44
p_top_requested = 1000,
num_metgrid_levels = 27,
num_metgrid_soil_levels = 4,
dx = 36000,12000,12000,12000,4000,4000,4000,4000,1333.333,1333.333,1333.333,1333.333
dy = 36000,12000,12000,12000,4000,4000,4000,4000,1333.333,1333.333,1333.333,1333.333
grid_id = 1,2,3,4,5,6,7,8,9,10,11,12
parent_id = 1,1,1,1,2,3,4,4,5,6,7,8
i_parent_start = 1,66,77,105,30,30,24,16,21,21,13,19
j_parent_start = 1,57,14,40,17,32,39,8,21,25,21,11
parent_grid_ratio = 1,3,3,3,3,3,3,3,3,3,3,3
parent_time_step_ratio = 1,3,3,3,3,3,3,3,3,3,3,3
feedback = 1,
smooth_option = 0

I appreciate any help.
Thank you a lot!

Jay

Re: How many domains at most can run simultaneously?

PostPosted: Wed Jan 30, 2019 6:22 pm
by kwthomas
Jay...

WRF 4.0.3 release.

frame/module_driver_constants.F:
max_domains = (MAX_DOMAINS_F - 1) / 2 + 1

arch/postamble (part of the compile process):
-DMAX_DOMAINS_F=$(MAX_DOMAINS)

arch/preamble (part of the compile process):
MAX_DOMAINS = 21

(21-1) / 2 + 1 is 11, which is what you found is the last number that works.

In theory, changing "MAX_DOMAINS" in arch/preamble is what you want, though no guarantees.

After changing, do a "clean -a", otherwise, you are inviting chaos.

Re: How many domains at most can run simultaneously?

PostPosted: Wed Jan 30, 2019 10:12 pm
by jzhang
Thank you so much, Kevin. That is really helpful.