Optmum number of nodes

Discussions centered around methodologies for high performance computing.

Optmum number of nodes

Postby chawas » Wed Oct 05, 2016 2:05 am

What will be the optimum number of nodes for running wrf.exe for two domains as shown by part of the namelist below.


&domains
time_step = 60,
time_step_fract_num = 0,
time_step_fract_den = 1,
max_dom = 2,
e_we = 508, 340,
e_sn = 463, 301,
e_vert = 30, 30,
p_top_requested = 5000,
num_metgrid_levels = 32,
num_metgrid_soil_levels = 4,
dx = 12000, 4000,
dy = 12000, 4000,
grid_id = 1, 2,
chawas
 
Posts: 9
Joined: Thu Sep 15, 2016 6:21 am

Re: Optmum number of nodes

Postby kwthomas » Thu Oct 06, 2016 1:53 pm

There are no rules for the number of nodes, nor will there every be. All supercomputers are different. They have different core counts per node. They have different amounts of memory per node.

Here are some general rules:

o If WRF dies and doesn't say anything useful on why it failed, it is likely that the job OOM'd
(Out Of Memory). In this case, you obviously need more nodes.

o Use all the cores per node, if possible. Idle cores mean slower performance. WRF wll
do all the right things if nproc_x/nproc_y don't divide evenly into your domain, though
that doesn't work for me. The CAPS ARPS postprocess software requires they divide
evenly.

o For best performance, nproc_x should be small. We're talking 4, 5, 6. 2 is too small. 3
will work, but may not be as efficient. This is based on testing on many different
supercomputers.
Kevin W. Thomas
Center for Analysis and Prediction of Storms
University of Oklahoma
kwthomas
 
Posts: 249
Joined: Thu Aug 07, 2008 6:53 pm

Re: Optmum number of nodes

Postby ArjunKap » Fri Feb 23, 2018 12:05 pm

Have you ever run a test on multiple core types on clusters of different sizes to test processor performance and scalability with node count?

If not, I would recommend trying a public cloud provider; running a few quick tests would be cheap and could go a long way. There are some good tools to do this easily. Let me know if you want any references - happy to point you in the right direction!
ArjunKap
 
Posts: 1
Joined: Fri Feb 23, 2018 12:01 pm

Re: Optmum number of nodes

Postby sarasamani » Wed Sep 26, 2018 8:45 pm

hello.if I have been understood well you are saying that we could use multiple laptops instead of an HPC, right?
so please tell more about this trend.

thanks
sarasamani
 
Posts: 1
Joined: Wed Sep 26, 2018 8:37 pm


Return to High Performance Computing

Who is online

Users browsing this forum: No registered users and 1 guest