Modern Compiling and Linking strategies

Ideas, problems or suggestions regarding the WRF software.

Modern Compiling and Linking strategies

Postby arango » Wed Dec 05, 2018 6:41 pm

I was quite surprised that WRF being a modern state-of-the-art atmospheric model has such an archaic configuration and compiling strategy and procedures. Nowadays, gmake provides such powerful tools to do compact managing. I highly recommend Robert Mecklenburg's book Managing Projects with GNU Make: The Power of GNU Make for Building Anything.

The annoying part is that the object, modules, c-preprocessing, and libraries files are hardwired to be located at the source directory $(WRF_SRC_ROOT_DIR)/external. The same applies to the location of the executables. Optimally, these files should be located at the user specified project directory that it is different than the WRF root directory. It is always wise to separate source code from projects. For example, If one has ten different WRF projects, we will need ten copies of the source code if one wants to preserve all the files used during compilation for later reference or for debugging with tools like the TotalView parallel debugger. We cannot copy the files to the desired directory because the full path is registered in the object files and debugging will fail. It is a good idea to have the source code centralized than manage ten copies if the user wants to have version control via svn or git.

We have done this with ROMS. We need just a single, and powerful makefile and every source code directory will need a make configuration file with simple rules. For example, we can have:

Code: Select all
local_sub  := WRF/phys

local_lib  := libwrfphys.a
local_src  := $(wildcard $(local_sub)/*.F)

$(eval $(call make-library,$(local_lib),$(local_src)))

$(eval $(compile-rules))

That is, a library is created for each WRF source directory. Similarly, we can add c-routines to the wildcard function. It is compact and elegant.

Also, we have individual make configuration files containing all the flags and rules for particular computer architecture and compiler. For example, one can have for IFORT compiler on Darwin located in the $(COMPILERS) directory. The configuration file is generated automatically and included into the makefile:

Code: Select all
OS := $(shell uname -s | sed 's/[\/ ]/-/g')

ifneq ($(MAKECMDGOALS),clean)
  include $(COMPILERS)/$(OS)-$(strip $(FORT)).mk

The tricky part is adding the compiling/linking destination directory, $(SCRATCH_DIR), to the makefile:

Code: Select all
#  Make functions for putting the temporary files in $(SCRATCH_DIR)
#  DO NOT modify this section; spaces and blank lines are needed.

# $(call source-dir-to-binary-dir, directory-list)
source-dir-to-binary-dir = $(addprefix $(SCRATCH_DIR)/, $(notdir $1))

# $(call source-to-object, source-file-list)
source-to-object = $(call source-dir-to-binary-dir,   \
                   $(subst .F,.o,$1))

# $(call make-library, library-name, source-file-list)
define make-library
   libraries += $(SCRATCH_DIR)/$1
   sources   += $2

   $(SCRATCH_DIR)/$1: $(call source-dir-to-binary-dir,    \
                      $(subst .F,.o,$2))
        $(AR) $(ARFLAGS) $$@ $$^
        $(RANLIB) $$@

# $(call f90-source, source-file-list)
f90-source = $(call source-dir-to-binary-dir,     \
                   $(subst .F,.f90,$1))

# $(compile-rules)
define compile-rules
  $(foreach f, $(local_src),       \
    $(call one-compile-rule,$(call source-to-object,$f), \
    $(call f90-source,$f),$f))

# $(call one-compile-rule, binary-file, f90-file, source-files)
define one-compile-rule
  $1: $2 $3
        cd $$(SCRATCH_DIR); $$(FC) -c $$(FFLAGS) $(notdir $2)

  $2: $3
        $$(CPP) $$(CPPFLAGS) $$(MY_CPP_FLAGS) $$< > $$@
        $$(CLEAN) $$@


Let me know if you are interested and I can give you examples of what we do in ROMS.

Cheers, Hernan
Posts: 5
Joined: Wed Dec 05, 2018 5:13 pm

Re: Modern Compiling and Linking strategies

Postby swara31 » Thu Mar 28, 2019 6:05 am

I am interested ..!
Posts: 2
Joined: Thu Mar 28, 2019 5:42 am

Return to Software Engineering

Who is online

Users browsing this forum: No registered users and 2 guests