[You can also view the single-page version of this document.]
Greg Stein wrote a custom build system for Subversion, which had been using `automake' and recursive Makefiles. Now it uses a single, top-level Makefile, generated from Makefile.in (which is kept under revision control). `Makefile.in' in turn includes `build-outputs.mk', which is automatically generated from `build.conf' by the `gen-make.py' script. Thus, the latter two are under revision control, but `build-outputs.mk' is not.
Here is Greg's original mail describing the system, followed by some advice about hacking it:
From: Greg Stein <firstname.lastname@example.org> Subject: new build system (was: Re: CVS update: MODIFIED: ac-helpers ...) To: email@example.com Date: Thu, 24 May 2001 07:20:55 -0700 Message-ID: <20010524072055.F5402@lyra.org> On Thu, May 24, 2001 at 01:40:17PM -0000, firstname.lastname@example.org wrote: > User: gstein > Date: 01/05/24 06:40:17 > > Modified: ac-helpers .cvsignore svn-apache.m4 > Added: . Makefile.in > Log: > Switch over to the new non-recursive build system. >... Okay... this is it. We're now on the build system. "It works on my machine." I suspect there may be some tweaks to make on different OSs. I'd be interested to hear if Ben can really build with normal BSD make. It should be possible. The code supports building, installation, checking, and dependencies. It does *NOT* yet deal with the doc/ subdirectory. That is next; I figured this could be rolled out and get the kinks worked out while I do the doc/ stuff. Oh, it doesn't build Neon or APR yet either. I also saw a problem where libsvn_fs wasn't getting built before linking one of the test proggies (see below). Basic operation: same as before. $ ./autogen.sh $ ./configure OPTIONS $ make $ make check $ make install There are some "make check" scripts that need to be fixed up. That'll happen RSN. Some of them create their own log, rather than spewing to stdout (where the top-level make will place the output into [TOP]/tests.log). The old Makefile.am files are still around, but I'll be tossing those along with a bunch of tweaks to all the .cvsignore files. There are a few other cleanups, too. But that can happen as a step two. [ $ cvs rm -f `find . -name Makefile.rm` See the mistake in that line? I didn't when I typed it. The find returned nothing, so cvs rm -f proceeded to delete my entire tree. And the -f made sure to delete all my source files, too. Good fugging thing that I had my mods in some Emacs buffers, or I'd be bitching. I am *so* glad that Ben coded SVN to *not* delete locally modified files *and* that we have an "undel" command. I had to go and tweak a bazillion Entries files to undo the delete... ] The top-level make has a number of shortcuts in it (well, actually in build-outputs.mk): $ make subversion/libsvn_fs/libsvn_fs.la or $ make libsvn_fs The two are the same. So... when your test proggie fails to link because libsvn_fs isn't around, just run "make libsvn_fs" to build it immediately, then go back to the regular "make". Note that the system still conditionally builds the FS stuff based on whether DB (See 'Building on Unix' below) is available, and mod_dav_svn if Apache is available. Handy hint: if you don't like dependencies, then you can do: $ ./autogen.sh -s That will skip the dependency generation that goes into build-outputs.mk. It makes the script run quite a bit faster (48 secs vs 2 secs on my poor little Pentium 120). Note that if you change build.conf, you can simply run: $ ./gen-make.py build.conf to regen build-outputs.mk. You don't have to go back through the whole autogen.sh / configure process. You should also note that autogen.sh and configure run much faster now that we don't have the automake crap. Oh, and our makefiles never re-run configure on you out of the blue (gawd, I hated when automake did that to me). Obviously, there are going to be some tweaky things going on. I also think that the "shadow" builds or whatever they're called (different source and build dirs) are totally broken. Something tweaky will have to happen there. But, thankfully, we only have one Makefile to deal with. Note that I arrange things so that we have one generated file (build-outputs.mk), and one autoconf-generated file (Makefile from .in). I also tried to shove as much logic/rules into Makefile.in. Keeping build-outputs.mk devoid of rules (thus, implying gen-make.py devoid of rules in its output generation) manes that tweaking rules in Makefile.in is much more approachable to people. I think that is about it. Send problems to the dev@ list and/or feel free to dig in and fix them yourself. My next steps are mostly cleanup. After that, I'm going to toss out our use of libtool and rely on APR's libtool setup (no need for us to replicate what APR already did). Cheers, -g -- Greg Stein, http://www.lyra.org/
And here is some advice for those changing or testing the configuration/build system:
From: Karl Fogel <email@example.com> To: firstname.lastname@example.org Subject: when changing build/config stuff, always do this first Date: Wed 28 Nov 2001 Yo everyone: if you change part of the configuration/build system, please make sure to clean out any old installed Subversion libs *before* you try building with your changes. If you don't do this, your changes may appear to work fine, when in fact they would fail if run on a truly pristine system. This script demonstrates what I mean by "clean out". This is `/usr/local/cleanup.sh' on my system. It cleans out the Subversion libs (and the installed httpd-2.0 libs, since I'm often reinstalling that too): #!/bin/sh # Take care of libs cd /usr/local/lib || exit 1 rm -f APRVARS rm -f libapr* rm -f libexpat* rm -f libneon* rm -f libsvn* # Take care of headers cd /usr/local/include || exit 1 rm -f apr* rm -f svn* rm -f neon/* # Take care of headers cd /usr/local/apache2/lib || exit 1 rm -f * When someone reports a configuration bug and you're trying to reproduce it, run this first. :-) The voice of experience, -Karl
The build system is a vital tool for all developers working on trunk. Sometimes, changes made to the build system work perfectly fine for one developer, but inadvertently break the build system for another.
To prevent loss of productivity, any committer (full or partial) can immediately revert any build system change that breaks their ability to effectively do development on their platform of choice, as a matter of ordinary routing, without fear of accusations of an over-reaction. The log message of the commit reverting the change should contain an explanatory note saying why the change is being reverted, containing sufficient detail to be suitable to start off a discussion of the problem on dev@, should someone chose to reply to the commit mail.
However, care should be taken not go into "default revert mode". If you can quickly fix the problem, then please do so. If not, then stop and think about it for a minute. After you've thought about it, and you still have no solution, go ahead and revert the change, and bring the discussion to the list.
Once the change has been reverted, it is up to the original committer of the reverted change to either recommit a fixed version of their original change, if, based on the reverting committer's rationale, they feel very certain that their new version definitely is fixed, or, to submit the revised version for testing to the reverting committer before committing it again.
For more information about build services, head over to ci.apache.org.
If you'd like to receive notifications about buildbot build and test failures, please subscribe to the notifications@ mailing list.
From: Karl Fogel <email@example.com> Subject: writing test cases To: firstname.lastname@example.org Date: Mon, 5 Mar 2001 15:58:46 -0600 Many of us implementing the filesystem interface have now gotten into the habit of writing the test cases (see fs-test.c) *before* writing the actual code. It's really helping us out a lot -- for one thing, it forces one to define the task precisely in advance, and also it speedily reveals the bugs in one's first try (and second, and third...). I'd like to recommend this practice to everyone. If you're implementing an interface, or adding an entirely new feature, or even just fixing a bug, a test for it is a good idea. And if you're going to write the test anyway, you might as well write it first. :-) Yoshiki Hayashi's been sending test cases with all his patches lately, which is what inspired me to write this mail to encourage everyone to do the same. Having those test cases makes patches easier to examine, because they show the patch's purpose very clearly. It's like having a second log message, one whose accuracy is verified at run-time. That said, I don't think we want a rigid policy about this, at least not yet. If you encounter a bug somewhere in the code, but you only have time to write a patch with no test case, that's okay -- having the patch is still useful; someone else can write the test case. As Subversion gets more complex, though, the automated test suite gets more crucial, so let's all get in the habit of using it early. -K