These are my notes for how to build R, and the ncdf/ncdf4 packages, under Windows. The information here is for my own personal use, but I've put it here in case it is helpful to someone else trying to build these, or other, packages under Windows. I *do* *not* support any users in attempting to build ncdf or ncdf4 under windows. USE THIS INFORMATION AT YOUR OWN RISK!!!

I use a 64-bit installation of Windows 7.

Building R for Windows 64-bit

2013-05-09: Starting from scratch on my Lenovo T420 laptop at work

R is compiled using the mingw (mininum gnu for Windows) environment. Therefore, the libraries that contribute to ncdf/ncdf4 and ncdf/ncdf4 itself must also be compiled under that same environment. There are 32- and 64-bit windows ports of the required libraries available, but most of them are for either the Visual Studio or Cygwin environments. Since R itself is compiled with the mingw environment, these existing libraries will not work.

The main idea of the approach I took is to cross compile the required libraries (zip, hdf5, netcdf) on Linux using the Windows 64 platform as the target. Although this seems awkward, too many of the compilation tools for the required libraries rely on a Unix-like environment. I was not successful it trying to compile these libraries under Windows.

First step, then, is to install the mingw64 bit cross compiler on the Linux box. I use Ubuntu 12.04 LTS, so install the required packages as follows:

sudo apt-get install mingw-w64 mingw-w64-dev

The process may be different on your Linux machine. The result of this step is that there will be new compilers and tools on your machine. For me, they are located in /usr/bin and have the previx "x86_64-w64-mingw32", for example:

/usr/bin/x86_64-w64-mingw32-gcc

is the compiler that runs on my Linux box but produces output code that can be run on Windows 64-bit.

There are also headers and libraries installed. On my machine, the directories are here:

/usr/x86_64-w64-mingw32/include
/usr/x86_64-w64-mingw32/lib

This is important because you have to make sure to use these headers and libraries, not the standard Linux ones. More information on how to do this is below. Also, here is a resource that describes a different (old, no longer supported) method, but is helpful for understanding this issue.

1) Cross compile szip (needed for hdf5)

Download source code from the hdf5 web site. I used version 2.1. Unzip and untar into directory szip-2.1_win64. In tcsh, do the following:

setenv PREFIX x86_64-w64-mingw32
setenv CC $PREFIX-gcc
setenv CXX $PREFIX-g++
setenv CPP $PREFIX-cpp
setenv RANLIB $PREFIX-ranlib

./configure --prefix=/home/pierce/src/packages/data/szip/szip-2.1_win64/install --host=x86_64-w64-mingw32  --enable-shared
make
make install

2) Cross compile zlib (needed for hdf5)

See here, here, and here for information.

Download the source from the HDF web site. I used version 1.2.8. Unzip and unter into directory zlib-1.2.8_win64. In tcsh, do the following:

setenv PREFIX x86_64-w64-mingw32
setenv CC $PREFIX-gcc
setenv CXX $PREFIX-g++
setenv CPP $PREFIX-cpp
setenv RANLIB $PREFIX-ranlib

./configure --prefix=/home/pierce/src/packages/system/zlib/zlib-1.2.8_win64/install 

make -f win32/Makefile.gcc 

*** NOTE *** Make install seems to fail, but the library is still produced

3) Cross compile hdf

Happily, someone else already figured out how to do this. Followed his instructions on how to eidt the Makefile.in's and configure script before proceeding. I used version 1.8.8, just as in that web site linked above.

Then:

setenv CC x86_64-w64-mingw32-gcc
setenv CFLAGS "-I/usr/x86_64-w64-mingw32/include -I/home/pierce/src/packages/system/zlib/zlib-1.2.8_win64 -L/usr/x86_64-w64-mingw32/lib -L/home/pierce/src/packages/system/zlib/zlib-1.2.8_win64"
setenv LD_LIBRARY_PATH ".:/usr/x86_64-w64-mingw32/lib"
setenv CXX x86_64-w64-mingw32-g++
setenv CPP x86_64-w64-mingw32-cpp
setenv RANLIB x86_64-w64-mingw32-ranlib
setenv AR x86_64-w64-mingw32-ar
setenv AS x86_64-w64-mingw32-as
setenv DLLTOOL x86_64-w64-mingw32-dlltool
setenv DLLWRAP x86_64-w64-mingw32-dllwrap

mkdir build-win64

cd build-win64

../configure --host=x86_64-w64-mingw32 --prefix=/home/pierce/src/packages/data/hdf/hdf5/hdf5-1.8.8_win64/build-win64/install --with-zlib=/home/pierce/src/packages/system/zlib/zlib-1.2.8_win64 --enable-cxx --enable-shared=no

Note that if you set --enable-shared=yes, then the H5make_libsettings.exe program will not work.

Now type "make", and follow the instructions on the website linked above for copying the intermediate executables (H5make_libsettings.exe, H5detect.exe) to a Windows 64-bit machine, running those files there, and copying the results back to the Linux box you are cross-compiling on. Finally, "make install".

4) Cross compile netcdf

OK this has changed since the last time I tried it. For some reason the old method of cross-compilation under Linux did not work, giving subtle errors in the resultant library. I've switched to using MXE to cross compile the library under Linux, and now it seems to work.

First, add the MXE installation dir to your $PATH: /home/pierce/src/packages/compilers/mxe/mxe/usr/bin. Make sure it is the first thing on the path. Make sure that . is not on the path.

I tried making curl work but could not ... in any event this is what I finally used (with the extraneous CPPFLAGS and LDFLAGS reflecting the unworking curl attempt):

setenv CPPFLAGS -I/home/pierce/src/packages/compilers/mxe/mxe/usr/x86_64-w64-mingw32/include
setenv LDFLAGS "-L/home/pierce/src/packages/compilers/mxe/mxe/usr/x86_64-w64-mingw32/lib -lcurl"
./configure --host=x86_64-w64-mingw32 --enable-static --disable-shared --disable-dap \ 
        --prefix=/home/pierce/src/packages/data/netcdf/netcdf-4.3.1.1_mxe_win64/install_dir
make
make install

5) Compile R on the Windows 64-bit machine

Unlike the other steps, don't cross-compile R, just compile it directly on the Windows-64 machine. I used R version 3.0.1.

a) If you have previously installed/compiled R on your machine, edit the system path to remove the old Rtools and/or R bin directories. I use Rapid Environment Editor for editing the system path, since Window's built-in utility is so hard to use.

b) Download and extract the Rtools package to some new directory. I used c:R/2013-05-09. Have the installation package add the new bin directories to your path, or alternatively use the Rapid Environmental Editor to do it by hand.

c) Do the following:

	cd c:\R\2013-05-09\Rtools
	mkdir tmp

d) Download and extact the R source tarball. I put it in c:/R/2013-05-09/R-3.0.1. Call this "RSRC". Change directories to $RSRC/src/gnuwin32. Edit file MkRules.dist as follows:

change this: "MULTI =" to this: "MULTI = 64"
change this: "BINPREF64 = x86_64-w64-mingw32-" to this: "BINPREF64 = c:/R/2013-05-09/Rtools/gcc-4.6.3/i686-w64-mingw32/bin/"
change this: "WIN = 32" to this: "WIN = 64"

e) Type "make" and wait for it to finish, which can take a while.

f) Got a problem with Tcl include file tcl.h not being found. Fix it as follows:

cd R/2013-05-09/R-3.0.0 
cp -r ../R64/Tcl .

and then go back to the gnuwin32 directory and type 'make' again.

g) Finally, add the R bin directory (c:/R/2013-05-09/bin/x64) to your system path.

6) Copy all the cross-compiled libraries from the Linux to the Windows box

On the Windows 64-bit box, make directories to hold the include files and libraries you cross compiled on Linux. I put the netcdf include files and libraries in c:/R/2013-05-09/netcdf. Use standard Windows drag-and-drop (or ftp) to copy all the netcdf library and include files from the Linux box to this directory on the Windows box.

Likewise, make directories to hold all the hdf libraries on the Windows-64 box. Because there are more of these than netcdf includes/libraries, I split these into two directores: c:/R/2013-05-09/hdf5/lib and c:/R/2013-05-09/hdf5/include. Copy all the cross-compiled library files from the Linux box to the first directory, and all the header files to the second directory. I also put the zlib library files in here as well (I didn't bother making a separate directory for them since there's just one, "libz.a").

7) Build and install ncdf4

a) Start by copying the ncdf4_1.9.tar.gz file to the Windows-64 box. I put it in directory c:/R/2013-05-09/my_packages.

b) If a previous version of ncdf4 is already installed, then at a command prompt on the windows-64 box type the following:

R CMD REMOVE ncdf4

c) In directory my_packages:

gzip --de ncdf4_1.9.tar.gz 
tar xvf ncdf4_1.9.tar

d) Move the original tar file to some other name:

mkdir orig 
mv ncdf4_1.9.tar orig/.

e) cd ncdf4/R and copy load.R.in to load.R; edit it so that the library always loads (i.e., get rid of the "if" and "else" lines, and only include the library.dynam line).

f) cd ../src, and put this into file Makevars.win:

BASE=e:/R/2014-06-15
PKG_CPPFLAGS=-I$(BASE)/netcdf -I$(BASE)/hdf5
PKG_LIBS=-L$(BASE)/netcdf -L$(BASE)/hdf5 -lnetcdf -lhdf5_hl -lhdf5 -lz

g) Make a gzipped tar file of this modified ncdf directory so we can install it:

cd ../.. 
tar cvf ncdf4_1.9.tar ncdf4 
gzip -v ncdf4_1.9.tar

h) Back in the my_packages directory:

R CMD INSTALL --build ncdf4_1.9.tar.gz

This should produce a "ncdf4_1.9.zip" file that can be installed into R on a 64-bit Windows machine. All done!

2) Delete any previous R directories from your system's PATH variable. I use RapidEE (Rapid Environmental Editor) for this task, but you can also go to control panel -> system -> advanced system settings (look in the upper left) -> Environmental Variables, click on 'Path', then click 'Edit'. Open new MS-DOS command windows after this step so they have the proper path.

3) Download R toolset from CRAN's "Building R for Windows" page. I downloaded "Rtools30.exe" and ran it as administrator, selecting "R\2013-05-09" as the directory to install into. I chose both the 32-bit and 64-bit build, and included the extra tools necessary for a R build (Tcl). Allow the installer to set the system PATH, since you removed old entries in step 2.

4) Download the R src gzip file from CRAN. Put it in c:R\2013-05-09, then:

	gzip --de *gz
	tar xvf *tar

4) Do the following:

	cd c:\R\2013-05-09\Rtools
	mkdir tmp

5) cd to "c:\R\2013-05-09\R-3.3.0\src\gnuwin32"

6) Edit file Mkrules.dist. 1) Change the line that says "MULTI = " to "MULTI = 64". 2) Change the line that says "BINPREF64 = x86_64-w64-mingw32-" to "BINPREF64 = c:/R/2013-05-09/Rtools/gcc-4.6.3/i686-w64-mingw32/bin/".

7) Type 'make'.

8) Got a problem with Tcl include file tcl.h not being found. While in directory R/2013-05-09/R-3.0.0 had to do this:
cp -r ../R64/Tcl .
and then 'make again.

Building the ncdf/ncdf4 package for Windows, 64-bit

1) if ncdf is already installed:
R CMD REMOVE ncdf4

2) Make a new temporary directory my_packages, for example, "C:\R\2013-05-09\my_packages"

2) Copy ncdf4_1.9.tar.gz (the source package) to my_packages

3) In directory my_packages:

	gzip --de ncdf4_1.9.tar.gz 
	tar xvf ncdf4_1.9.tar

3.1) cd ncdf4/R and copy load.R.in to load.R; edit it so that the library always loads (i.e., get rid of the "if" and "else" lines, and only include the library.dynam line).

3.5) cd ../src, and put this into file Makevars.win:
PKG_LIBS=-Lc:/R/netcdf_win32_dlls_from_unidata -lnetcdf

4) Make a gzipped tar file of this modified ncdf directory so we can install it:
cd ../.. ; tar cvf ncdf4_1.4.tar ncdf4 ; gzip -v ncdf4_1.4.tar

OLD NOTES

1) Download R toolset from CRAN's "Building R for Windows" page. I downloaded "Rtools215.exe" and ran it, selecting "R\2012-02-09" as the directory to install into. I chose a 64-bit build, and included the extra tools necessary for a R build (Tcl).

2) Set the Windows path to use the toolset in this order: Rtools\bin; Rtools\gcc-4.6.3\bin. Put these BEFORE the other elements in the path (otherwise you get errors regarding "comm: file 1 not in sorted order"), and make sure to set these into the SYSTEM path var, not the user's path var. To check the current path, open a regular MS-DOS command line window and type "PATH". To set the path in windows 7, go to control panel -> system -> advanced system settings (look in the upper left) -> Environmental Variables, click on 'Path', then click 'Edit'. Open new MS-DOS command windows after this step so they have the proper path.

3) Download the R src gzip file from CRAN. Put it in c:R\2012-02, then:
gzip --de *gz
tar xvf *tar
I got a lot of "file ownership" errors on this step but it doesn't seem to have hurt it any.

4) cd to "c:\R\R-2.14.1\src\gnuwin32" and type "make".

5) I had *HUGE* problems with errors such as "cannot create directory /tmp/R2240: directory nonexistant". I think the problem is that directory /tmp is not "found" as a valid directory even though c:\tmp is there. Note that R2240 is a temporary file that is supposed to contain all the .R source code, which is then moved to a file called "all.R". My work around is to edit file "..share/make/basepkg.mk", and change the long script in the target for mkR1: from this:

	@(f=$${TMPDIR:-/tmp}/Rtmpfile; \
	  echo "**b $${f}"; \
	    cat $(RSRC) > "$${f}"; \
	  $(SHELL) $(top_srcdir)/tools/move-if-change "$${f}" all.R)
	f=$${TMPDIR:-/tmp}/Rtmpfile
	cat $(RSRC) > all.R
	echo "done with long script"
	@$(SHELL) $(top_srcdir)/tools/copy-if-change all.R \
	...etc...

to just the super-simplified:

	cat $(RSRC) > all.R

Had to do a similar thing to target mkR2, but include a package header:

	$(ECHO) ".packageName <- \"$(pkg)\"" >  all.R
	cat `LC_COLLATE=C ls $(srcdir)/R/*.R` >> all.R

*** SEE MY MODIFIED FILE ...share\make\basepkg.mk ***

6) Got a problem with Tcl include file tcl.h not being found. While in directory R/2012-02/R-2.14.1 had to do this:
cp -r ../R64/Tcl .
Actually that still didn't work. For some reason it seems to be building the 32 bit version rather than the 64 bit version, so I downloaded the 32 bit Rtools and copied the Tcl library over.

7) Final bin is in R-2.14.1/bin/R.exe. Add R-2.14.1/bin to your windows path, so it can be started up in any DOS command line window just by typing "R".

Building the ncdf/ncdf4 package for Windows

0) if ncdf is already installed:
R CMD REMOVE ncdf4

1) Make a new temporary directory my_packages, for example, "C:\R\2011-07\my_packages"

2) Copy ncdf4_1.4.tar.gz (the source package) to my_packages

3) In directory my_packages:
gzip --de ncdf4_1.4.tar.gz and then tar xvf ncdf4_1.4.tar
Move the original tar file to some other name: "mkdir orig ; mv ncdf4_1.4.tar orig/."

3.1) cd ncdf4/R and copy load.R.in to load.R; edit it so that the library always loads (i.e., get rid of the "if" and "else" lines, and only include the library.dynam line).

3.5) cd ../src, and put this into file Makevars.win:
PKG_LIBS=-Lc:/R/netcdf_win32_dlls_from_unidata -lnetcdf

4) Make a gzipped tar file of this modified ncdf directory so we can install it:
cd ../.. ; tar cvf ncdf4_1.4.tar ncdf4 ; gzip -v ncdf4_1.4.tar

4b) Download Unidata's precompiled netcdf .dlls. THESE ARE HARD TO FIND. They are *not* under the download link. Instead they are in the FAQ, in the question: How can I use netCDF-4.1 with Windows? NOTE that there is also a question How can I use netCDF-4 with Windows? that links to an OLDER VERSION (4.0.1), and should not be used. Man they make it difficult!! Here is a direct link: Download netcdf-4.1.1 windows-32 dlls

5) Put a copy of netcdf.h from the netcdf 4.1.1 package source, directory "libsrc4" (note: NOT "libsrc") into the R/2012-02-09/R-2.14.1/include directory. Get these from a download of the netcdf 4.1.1 src package, not from the windows dll zip downloaded in the prevous step. For example:

cd C:R\2012-02-09\R-2.14.1\include
cp /where/ever/libsrc4/netcdf.h .

6) Copy all NCAR's distributed netcdf dll's to R:
cd C:R\2012-02-09\R-2.14.1\bin\i386
cp /where/ever/netcdf.dll .
cp /where/ever/hdf5_hldll.dll .
cp /where/ever/hdf5dll.dll .
cp /where/ever/szip.dll .
cp /where/ever/zlib1.dll .

7) Back in the my_packages directory, type "R CMD INSTALL --build ncdf4_1.4.tar.gz". This should produce a "ncdf4_1.4.zip" file.

8) This new zip file does not have the necessary hdf5 and netcdf dll's (NCAR's dll's) in it. We need to put the dll's into the zip file. For some reason I found this tricky, many things that I thought would work seemed to, but then R choked when trying to install the zip file (saying it could not 'read' the zip file?!?). It seems to care about the NAME of the zip file; nothing but "ncdf4_1.4.zip" will apparently work. The following procedure seemed to work:

	a) rm -r ncdf4
	b) unzip ncdf4_1.4.zip  (using the MS-DOS command line unzip program, not windows XP unzip feature)
	c) mv ncdf4_1.4.zip orig/.
	d) go into ncdf4/libs/i386 ; copy all the hdf5 and netcdf dll's from step 6 to this directory, so that
	   it has ncdf4.dll AND netcdf.dll, hdf4_hldll.dll, szip.dll, zlib1.dll, etc. in it.
	e) go back out to my_packages
	f) zip -r ncdf4_1.4.zip ncdf4 (again using the MS-DOS command line 'zip' program)

You now have a file ncdf4_1.4.zip that you can install from the windows R gui using menu item Packages... | Install packages from local zip files.