SIGSEGV, segmentation fault without CFL

Any issues with the actual running of the WRF.

SIGSEGV, segmentation fault without CFL

Postby yuting » Wed Aug 20, 2014 5:48 pm

Dear all,

I'm using WRFV3.6 and have little issue when running wrf.exe. I have searched online about the problem but it seemed still couldn't work. I am running a nested simulation(4 domains), resolutions are 2000,666.6,222.2,74.074 respectively and the time step is 5.

It always stops with the " SIGSEGV, segmentation fault" when it runs for 30 min without any CFL warning or other errors.

But it works well when I use the same namelist and set max_dom from 4 to 3 ( 2000,666.6,222.2). So there might be some problem with domain 4 where I might have ignored.

By the way, the command "ulimit -s unlimited" has already used in my wrf.csh and since my domain is on the ocean, I thought it might not be very useful to add smooth_cg_topo = .true. . Below is my namelist and the last few lines of my rsl.error.0000.

Thank you in advance for any suggestions!

(I used sst_update and nudging at the same time and it works well when running for 3 domains.)


&time_control
run_days = 0,
run_hours = 15,
run_minutes = 0,
run_seconds = 0,
start_year = 2012, 2012, 2012, 2012, 2012,
start_month = 08, 08, 08, 08, 08,
start_day = 27, 27, 27, 27, 28,
start_hour = 21, 21, 21, 21, 00,
start_minute = 00, 00, 00, 00, 00,
start_second = 00, 00, 00, 00, 00,
end_year = 2012, 2012, 2012, 2012, 2012,
end_month = 08, 08, 08, 08, 08,
end_day = 28, 28, 28, 28, 28,
end_hour = 12, 12, 12, 12, 01,
end_minute = 00, 00, 00, 00, 00,
end_second = 00, 00, 00, 00, 00,
interval_seconds = 3600
input_from_file = .true.,.false.,.false., .false., .false.,
history_interval = 60, 60,60, 1, 1, 1,
HISTORY_OUTNAME = "/home/rainny/wrfout_d<domain>_<date>"
frames_per_outfile = 1, 1, 1, 1, 1,
restart = .false.,
restart_interval = 5000,
io_form_history = 2
io_form_restart = 2
io_form_input = 2
io_form_boundary = 2
debug_level = 0
auxinput1_inname = "met_em.d<domain>.<date>"
io_form_auxinput2 = 2,
auxinput4_inname = "wrflowinp_d<domain>",
auxinput4_interval = 60, 60, 60, 60, 60,
io_form_auxinput4 = 2,
! auxinput11_interval_s = 60, 60, 60, 60, 60,
! auxinput11_end_h = 12, 12, 12, 12, 12,
/

&domains
time_step = 5,
time_step_fract_num = 0,
time_step_fract_den = 1,
max_dom = 4,
e_we = 401, 400, 400, 301,
e_sn = 401, 400, 400, 301,
e_vert = 78, 78, 78, 78, 78,
p_top_requested = 1000,
num_metgrid_levels = 27,
num_metgrid_soil_levels = 4,
dx = 2000, 666.6666, 222.2222, 74.074,
dy = 2000, 666.6666, 222.2222, 74.074,
grid_id = 1, 2, 3, 4, 5,
parent_id = 0, 1, 2, 3, 4,
i_parent_start = 118, 145, 110, 150,
j_parent_start = 170, 160, 140, 150,
parent_grid_ratio = 1, 3, 3, 3,
parent_time_step_ratio = 1, 3, 3, 3,
feedback = 1,
smooth_option = 0
use_surface = .true.
! vert_refine_fact = 2,
/

&physics
mp_physics = 10, 10, 10, 10, 10,
ra_lw_physics = 1, 1, 1, 1, 1,
ra_sw_physics = 1, 1, 1, 1, 1,
radt = 30, 30, 30, 30, 30,
sf_sfclay_physics = 1, 1, 1, 1, 1,
sf_surface_physics = 1, 1, 1, 1, 1,
bl_pbl_physics = 1, 1, 0, 0, 0,
bldt = 0, 0, 0, 0, 0,
cu_physics = 0, 0, 0, 0, 0,
cudt = 5, 5, 5, 5, 5,
isfflx = 1,
isftcflx = 2,
ifsnow = 0,
icloud = 1,
surface_input_source = 1,
num_soil_layers = 5,
sf_urban_physics = 0,
maxiens = 1,
maxens = 3,
maxens2 = 3,
maxens3 = 16,
ensdim = 144,
sst_update = 1,
/

&fdda
grid_fdda = 1, 0, 0, 0, 0,
gfdda_inname = "wrffdda_d<domain>",
gfdda_end_h = 15, 12, 12, 12,
gfdda_interval_m = 60, 60, 60, 60, 60,
fgdt = 0, 0, 0, 0, 0,
fgdtzero = 0, 0, 0, 0, 0,
if_no_pbl_nudging_uv = 0, 1, 0, 0, 0,
if_no_pbl_nudging_t = 0, 1, 0, 0, 0,
if_no_pbl_nudging_ph = 0, 1, 0, 0, 0,
if_no_pbl_nudging_q = 0, 1, 0, 0, 0,
if_zfac_uv = 0, 1, 0, 0, 0,
k_zfac_uv = 10, 10, 10, 10, 10,
if_zfac_t = 0, 0, 0, 0, 0,
k_zfac_t = 10, 10, 10, 10, 10,
if_zfac_ph = 0, 0, 0, 0, 0,
k_zfac_ph = 10, 10, 10, 10, 10,
guv = 0.0003, 0.0003, 0.0003, 0.0003, 0.0003,
gt = 0.0003, 0.0003, 0.0003, 0.0003, 0.0003,
gph = 0.0003, 0.0003, 0.0003, 0.0003, 0.0003,
if_ramping = 1,
dtramp_min = 60.0,
io_form_gfdda = 2,
/


&dynamics
w_damping = 1,
diff_opt = 2,
km_opt = 2,
diff_6th_opt = 1, 1, 1, 1, 1,
diff_6th_factor = 0.12, 0.12, 0.12, 0.12, 0.12,
base_temp = 290.
damp_opt = 3,
zdamp = 5000., 5000., 5000., 5000., 5000.,
dampcoef = 0.2, 0.2, 0.2, 0.2, 0.2
khdif = 0, 0, 0, 0, 0,
kvdif = 0, 0, 0, 0, 0,
non_hydrostatic = .true., .true., .true., .true., .true.,
moist_adv_opt = 1, 1, 1, 1,
scalar_adv_opt = 1, 1, 1, 1,
! mix_isotropic = 1, 1, 1, 1,
! sfs_opt = 1, 1, 1, 1,
/

&bdy_control
spec_bdy_width = 5,
spec_zone = 1,
relax_zone = 4,
specified = .true., .false.,.false.,.false.,.false.,
nested = .false., .true., .true., .true., .true.,
/

&grib2
/

&namelist_quilt
nio_tasks_per_group = 0,
nio_groups = 1,
/




rsl.error.0000

Timing for main: time 2012-08-27_21:29:59 on domain 3: 5.33299 elapsed seconds
Timing for main: time 2012-08-27_21:29:59 on domain 4: 0.98908 elapsed seconds
Timing for main: time 2012-08-27_21:29:59 on domain 4: 1.01029 elapsed seconds
Timing for main: time 2012-08-27_21:30:00 on domain 4: 0.97735 elapsed seconds
Timing for main: time 2012-08-27_21:30:00 on domain 3: 5.36541 elapsed seconds
Timing for main: time 2012-08-27_21:30:00 on domain 2: 18.45094 elapsed seconds
Timing for main: time 2012-08-27_21:30:00 on domain 1: 57.31909 elapsed seconds
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
wrf.exe 00000000017ACE84 Unknown Unknown Unknown
wrf.exe 00000000017A619E Unknown Unknown Unknown
wrf.exe 00000000017A1A1F Unknown Unknown Unknown
wrf.exe 00000000017A029F Unknown Unknown Unknown
wrf.exe 00000000014013F7 Unknown Unknown Unknown
wrf.exe 00000000014CEF58 Unknown Unknown Unknown
wrf.exe 0000000000FCB460 Unknown Unknown Unknown
wrf.exe 0000000000EBF5B7 Unknown Unknown Unknown
wrf.exe 000000000055C8EB Unknown Unknown Unknown
wrf.exe 00000000004C9123 Unknown Unknown Unknown
wrf.exe 00000000004C90D7 Unknown Unknown Unknown
wrf.exe 00000000004C906C Unknown Unknown Unknown
libc.so.6 0000003849E1D994 Unknown Unknown Unknown
wrf.exe 00000000004C8F79 Unknown Unknown Unknown
yuting
 
Posts: 2
Joined: Wed Aug 20, 2014 5:40 pm

Re: SIGSEGV, segmentation fault without CFL

Postby mpalmer » Fri Aug 22, 2014 9:28 am

Not sure how much this has to do with it, if at all, but it seems that the seg fault comes after or with the radiation call (You radt=30 and the seg fault happened on 21:30) As a test you could try reducing radt to 2 (match radt to dx/1000)

You could also set ulimt -c unlimted, and run the resulting core through something like gdb and do a backtrace to find out what call is giving you the seg fault.
Michael Palmer, M.Sc.
Predictive Analytics Researcher | Green Power Labs
www.greenpowerlabs.com
mpalmer
 
Posts: 54
Joined: Fri Aug 30, 2013 12:08 pm

Re: SIGSEGV, segmentation fault without CFL

Postby yuting » Fri Aug 22, 2014 12:32 pm

Hi,dear mpalmer
Thank you so much for your kindly reply!

I have tested the problem and found it might be the unsuited microphysics option (or other physics options) which created NaN in my data and transfer the wrong data to the radiation processes. I am changing some of my physics options to see if it could avoid the problem.

I would update if I have any progress:D


mpalmer wrote:Not sure how much this has to do with it, if at all, but it seems that the seg fault comes after or with the radiation call (You radt=30 and the seg fault happened on 21:30) As a test you could try reducing radt to 2 (match radt to dx/1000)

You could also set ulimt -c unlimted, and run the resulting core through something like gdb and do a backtrace to find out what call is giving you the seg fault.
yuting
 
Posts: 2
Joined: Wed Aug 20, 2014 5:40 pm

Re: SIGSEGV, segmentation fault without CFL

Postby ran252 » Wed Jul 29, 2015 1:21 am

I got similar problem. I solved the problem by defining the "lib" path of INTEL compiler.

Just I add a line into my bashrc file:
export LD_LIBRARY_PATH="/usr/local/intel/2012.0/lib/intel64"

Cheers!! :)
ran252
 
Posts: 3
Joined: Mon Aug 11, 2014 12:42 am

Re: SIGSEGV, segmentation fault without CFL

Postby ricardofaria » Sat Jan 14, 2017 12:21 pm

Hi

I have the same problem, did you manage to solve it?
I tried everything, but no success :oops:

Thanks, Ricardo Faria
ricardofaria
 
Posts: 6
Joined: Tue Nov 29, 2016 11:09 am


Return to Runtime Problems

Who is online

Users browsing this forum: No registered users and 2 guests