English
资源说明：Grand Unified Builder
GUB the Grand Unified Builder
• Home
• Basics
• Applications
• History
• Links
GUB -- Grand Unified Builder
WHAT IS GUB
Grand Unified Builder (GUB, http://lilypond.org/gub) is a mini source-based
distribution and packaging system. It cross-compiles several packages with
their dependencies and assembles them into a single installation package.
GUB aims at correctness ie, be dependable and reproducible, while avoiding
gratituous rebuilds. It was originally developed for turnkey building of all
LilyPond binary installers. Currently, GUB supports GNU/Linux, FreeBSD, MacOS-X
and Microsoft Windows installation packages. All packages are fully relocatable
and can be installed with user privileges.
GUB uses plain Python classes to describe package build specifics. For
reasonably well behaved packages, individual build specifications can stay very
simple by extending the generic build specification classes.
USAGE
Run these programs with --help argument for instructions
bin/gpkg - Gub Package Manager
bin/gub - the Gub Universal Builder
bin/gib - Gub (platform dependent) Installer Builder
bin/gub-tester - build tester; integrates with
Bazaar,CVS,Darcs,Git,SVN and gub
bin/gupdate - check for and show or update new upstream releases
bin/cygwin-packager - roll cygwin package
BUGS
Send bug reports and patches to lilypond-devel@gnu.org.
Supported installer packages
GNU Denemo
Inkscape
GNU LilyPond
OpenOffice.org
with all essential dependencies. Currently, 207 package build specifications
are included in GUB.
Supported installer platforms
darwin-ppc
tar.bz2 file for Darwin 7 (MacOS 10.3)/PowerPC
darwin-x86
tar.bz2 file for Darwin 8 (MacOS 10.4)/x86
mingw
mingw executable for Windows32
linux-x86
shar archive for Linux/x86
linux-64
shar archive for Linux/x86_64
linux-ppc
shar archive for Linux/PowerPC
freebsd-x86
shar archive for FreeBSD 4/x86
freebsd-64
shar archive for FreeBSD 6/x86_64
cygwin
.tar.bz2 packages for Cygwin/Windows32
arm
shar archive for Linux/ARM (largely untested)
debian
shar archive for Debian (largely untested)
LICENSE
GUB is distributed under the terms of the GNU General Public License, version 2
or later. Gub comes with NO WARRANTY.
ACQUIRING
The Git repository is at
http://github.com/janneke/gub
You can get it by doing
git clone git://github.com/janneke/gub.git
or download the latest tarball doing
wget http://github.com/janneke/gub/tarball/master
GUB home page
http://lilypond.org/gub
INSTALLING
* You need
- about 9 GB of free space (for all platforms)
- standard unix shell utilities: cat, cp, install, mv, rm, sed, ...
- a standard unix development environment with GCC and G++
- Python 2.4 or newer (2.5, 2.6, 3.0 are known to work)
* Try
bin/gub --help
make help
Examples
* Prepare a cross build environment for darwin-x86
bin/gub darwin-x86::cross/gcc
* Cross build bison for FreeBSD x86_64
bin/gub freebsd-64::ftp://ftp.gnu.org/pub/gnu/bison/bison-2.3.tar.gz
* Make a LilyPond installer for your platform
bin/gub lilypond-installer
this leaves the installer in uploads/
* Build Inkscape for your platform
bin/gub inkscape-installer
this leaves the installer in uploads/
* Cross build OpenOffice (go-oo.org) for Windows (mingw32)
make openoffice
this leaves the Windows installer in uploads/
* Cross build Denemo (http://denemo.org) for Windows (mingw32)
make denemo
this leaves the Windows installer in uploads/
GUB the Grand Unified Builder
• Home
• Basics
• Applications
• History
• Links
GUB BASICS
ROADMAP
directory contents and use
. (GNUmakefile, *.make) make drivers (try: make help)
bin/ programs (gub, gpkg, gib, ...; try: bin/gub --help)
gub/ python library
gub/specs/ build specifications (gub/specs/
hello.py,bison.py,denemo.py,lilypond.py...)
patches/ patch files used in build specifications
target/ build trees
log/ (gub.log, ...) (build) log files
uploads/ installers
target// working tree for
target/linux-x86/src/ linux-x86 unpacked sources
target/linux-64/build/ linux-64 build output
target/darwin-x86/log/ darwin-x86 most recent lilypond build log
lilypond.log
target/mingw/packages/ .gup packages for mingw
target/darwin-ppc/log/ status of partial darwin-ppc builds
status/
target/freebsd-x86/root/ freebsd-x86 build and install root
target/tools/root/usr/ native tools used for build
bin/
sourcefiles/ non-patch files used in build specifications
nsis/ nsis windows installer scripts
BASIC WORKINGS
Checksums and rebuilds
GUB seeks to minimise unnecessary rebuilds while being strict about important
changes. Before starting the actual building of a package, the build recipe is
executed as a dry-run and a checksum is calculated from these build commands.
This checksum includes the environment variables DISPLAY, FTP_PROXY,
GUB_TOOLS_PREFIX, HOME, HTTP_PROXY, LIBRESTRICT, PATH [no other environment
variables are allowed through].
The packages' build checksum is matched with the previous build and if it
changes, the package is rebuilt. Otherwise, only the binary package is
reinstalled, if necessary. This means that the .py build specs can be changed
and refactored at will and as long as no actual build commands change, nothing
will be rebuilt.
This makes that forcing rebuilds can can be tricky: gub refuses to do so if it
deems rebuilding unnecessary.
LIBRESTRICT
A common problem with [making reproducible cross] builds is using files or
features that are not controlled by the [GUB] build process, but just happen to
be on the build host and happen to differ between most build hosts.
GUB uses the LD_PRELOAD feature with the gub/specs/librestrict.py package and
code from ./librestrict/restrict.c to disallow reading from build host files.
This makes for much better reprocucibility, especially between different build
hosts.
The downside to this LIBRESTRICT feature is that quit a few packages are not
prepared for cross building. LIBRESTRICT will catch these errors and they have
to be fixed in the package's spec file.
This is what such a LIBRESTRICT assertion could like
librestrict:error:/home/janneke/vc/gub/target/mingw/root/usr/cross/libexec/gcc/i686-mingw32/4.1.1/cc1plus: tried to open () file /home/janneke/vc/lilypond/lily/main.cc
librestrict:allowed:
/home/janneke/vc/gub/target/mingw
/tmp
/dev/null
/dev/urandom
/proc/self
cc1plus: internal compiler error: Aborted
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
[1]10:42:08 janneke@peder:~/vc/gub
$
you often find these in config.log files when package-dependant code blondly
looks in /usr/include or /usr/lib.
stat-restriction
LIBRESTRICT has an experimental strict mode enabled by setting the environment
variable
LIBRESTRICT=open:stat
This will even disallow any STATting outside the target's build tree. It would
be nice to get this to work and some effort has been done. However, this is
potentially a lot of work as not only upstream packages but also build system
authors [autoconf, automake, libtool etc.] look at the build host root file
system while cross building.
Simplifying package builds or avoiding code duplication
Another idea of GUB is to identify, fix and generalise build quircks and move
them from a package's spec file into one of the generic build classes. This
increases maintainability by avoiding code duplication, makes spec files
smaller, simpler and also increases the chances of a new package's spec file to
only need a source url to specify the build.
BASIC USAGE
Build package hello for platform linux-x86
bin/gub linux-x86::hello
Examine why package hello wants to rebuild
less target/linux-x86/log/build.log
search from end back to `mismatch:' [type: FC-c?mismatch:RET]
Force a rebuild of package hello for platform linux-x86
rm -rf target/linux-x86/*/hello-*
Force a rebuild of package hello for all platforms
rm -rf target/*/*/hello-*
Prepare to rebuild entire linux-x86 platform
rm -rf target/linux-x86
Prepare to rebuild everything, all platforms
rm -rf target
SPEC FILES
Plain url builds
<--no spec-->
Well behaved packages can be built without spec file, straight from an url. For
example, to build bison-2.2 as a build tool, do
bin/gub tools::http://ftp.gnu.org/pub/gnu/bison/bison-2.2.tar.gz
This produces
target/tools/packages/bison-2.2.tools.gup
target/tools/packages/bison.tools.hdr
target/tools/packages/bison.checksum
Check files installed under target/tools/root/ by doing
bin/gpkg -p tools files bison
To build the very latest findutils for your architecture, do
bin/gub git://git.savannah.gnu.org/findutils.git
Simple spec
gub/specs/hello.py
from gub import target
class Hello (target.AutoBuild):
source = 'http://lilypond.org/download/gub-sources/hello-1.0.tar.gz'
build it for mingw by doing
bin/gub mingw::hello
__TOOLS package with dependencies
gub/specs/autoconf.py
from gub import tools
class Autoconf__tools (tools.AutoBuild):
source = 'http://ftp.gnu.org/pub/gnu/autoconf/autoconf-2.63.tar.gz'
parallel_build_broken = True
dependencies = [
'm4',
'perl',
]
build it by doing
bin/gub tools::autoconf
configure flags
gub/specs/icoutils.py
from gub import tools
class Icoutils__tools (tools.AutoBuild):
dependencies = ['libpng-devel']
configure_flags = (tools.AutoBuild.configure_flags
+ ' --with-libintl-prefix=%(system_prefix)s'
+ ' --disable-nls')
____-specific instructions
For platform specific instructions, append "__" to the class name.
For platform+architecture-specific instructions, "____" to the
class name.
gub/specs/pixman.py
from gub import target
class Glib (target.AutoBuild):
...
class Glib__freebsd (Glib):
dependencies = Glib.dependencies + ['libiconv-devel']
configure_variables = Glib.configure_variables + ' CFLAGS=-pthread'
class Glib__freebsd__x86 (Glib__freebsd):
# Must include -pthread in lib flags, because our most beloved
# libtool (2.2.6a) thinks it knows best and blondly strips -pthread
# if it thinks it's a compile flag.
# FIXME: should add fixup to update_libtool ()
compile_flags = ' G_THREAD_LIBS=-pthread G_THREAD_LIBS_FOR_GTHREAD=-pthread '
String expansion
In all build commands, standard Python string expansion
%(string-name)s
can be used. These strings are taken from class Settings in gub/settings.py and
from the package's build class and its parent classes.
Adding string expansion variables to a spec build class can be done in several
ways
• class variable
class Foo (target.Autobuild):
common_configure_flags = ' --without-java'
• instance variable
class Foo__mingw (Foo):
def __init__ (self, settings, source):
Foo.__init__ (self, settings, source)
self.target_gcc_flags = '-mms-bitfields'
• decorated funtion returning a string
class Bar__mingw (target.Autobuild):
@context.subst_method
def target_os (self):
return 'windows'
• Manipulating the substitution_dict using gub/build.py's class Change_dict
or overriding the get_substitution_dict method.
Pre-defined strings
Typical variables defined by settings are
build_architecture=x86_64-linux
build_os=linux
build_platform=linux-64
platform=target
target_architecture=x86_64-linux
target_bits=64
target_cpu=x86_64
target_os=linux
target_platform=target
Typical variables defined by the build class
compile_flags=
builddir=.../target/build/...
config_cache_overrides=
configure_binary= .../configure
configure_command= sh .../configure ...
configure_flags= --prefix=... --enable-shared --enable-static
configure_variables= CFLAGS=-I.../target/root/usr/include
install_command=
install_flags=
install_prefix=
install_root=
make_flags=
srcdir=.../target/src/...
More examples
See gub/specs/*.py for some more examples.
GUB -- LilyPond
INSTALLING
Get GUB
git clone git://github.com/janneke/gub.git
or
wget http://github.com/janneke/gub/tarball/master
In addition to the requirements for GUB, you need
• A TeX installation
Build everything
make lilypond
this leaves installers and documentation in
uploads/lilypond-VERSION-RELEASE.darwin-ppc.tar.bz2
uploads/lilypond-VERSION-RELEASE.darwin-x86.tar.bz2
uploads/lilypond-VERSION-RELEASE.freebsd-64.shar.sh
uploads/lilypond-VERSION-RELEASE.freebsd-x86.shar.sh
uploads/lilypond-VERSION-RELEASE.linux-64.shar.sh
uploads/lilypond-VERSION-RELEASE.linux-ppc.shar.sh
uploads/lilypond-VERSION-RELEASE.linux-x86.shar.sh
uploads/lilypond-VERSION-RELEASE.mingw.exe
uploads/lilypond-VERSION-RELEASE.documentation.tar.bz2
uploads/lilypond-VERSION-RELEASE.webdoc.tar.bz2
uploads/webdoc/VERSION
uploads/lilypond-VERSION-RELEASE.test-output.tar.bz2
uploads/wbtest/VERSION
More examples
* Make a LilyPond installer for your platform
bin/gub lilypond-installer
OR (old style)
bin/gub lilypond # package with dependencies
bin/gib --branch=lilypond=git.sv.gnu.org--lilypond.git-master lilypond #installer
* Build all LilyPond installers
make lilypond
this leaves installers for GNU/Linux (x86, x86_64, powerpc),
Windows, MacOSX (x86 and powerpc), FreeBSD (x86, x86_64) and
documentation in uploads/
* Optional: Prepare build environment
make -f lilypond.make bootstrap
TESTING
The test-lily directory contains scripts for running various tests. Run without
arguments for instructions
cron-builder.py - entry point from continuous crontab testing
test-binary.py - test binary builds
with-lock.py - run a command with a lockfile
rsync-lily-doc.py
- prepare documentation for upload
dist-check.py - check tarball for anomalies
cygwin-packager.py
- repackage GUB packages into Cygwin tarballs.
CRON USAGE
I have the following in my crontab
• Hourly build of LilyPond subpackage for all platforms except native (native
is darwin-pcc?)
test-lily/cron-builder.py --package\
--test-options "--to lilypond-cvs@gnu.org --quiet --from hanwen@lilypond.org
--smtp smtp.xs4all.nl --repository downloads/lilypond.git"\
darwin-x86 freebsd-x86 freebsd-64 mingw linux-x86 linux-64
• Hourly build of native installer, to run documentation and dist check.
test-lily/cron-builder.py\
--package --installer --tarball --docs\
--test-options "--to lilypond-cvs@gnu.org --dependent --quiet\
--from hanwen@lilypond.org --smtp smtp.xs4all.nl --repository downloads/lilypond.git"\
--unversioned\
darwin-ppc
• Nightly build of all installers, to be ready for shipping binaries in the
morning.
test-lily/cron-builder.py\
--installer\
--test-options "--to lilypond-cvs@gnu.org --quiet --from hanwen@lilypond.org\
--smtp smtp.xs4all.nl --repository downloads/lilypond.git"\
darwin-x86 freebsd-x86 freebsd-64 mingw linux-x86 linux-64 darwin-ppc
GUB -- GNU Denemo
INSTALLING
Get GUB
git clone git://github.com/janneke/gub.git
or
wget http://github.com/janneke/gub/tarball/master
Build everything
make denemo
this produces a windows installer in
uploads/denemo-x.y.z-b.mingw.exe
More examples
* Build main Denemo installers
make PLATFORMS='mingw linux-x86 linux-64' denemo
* Build all Denemo installers
make 'PLATFORMS=mingw linux-x86 linux-64 linux-ppc darwin-ppc darwin-x86 freebsd-x86 freebsd-64' denemo
this leaves installers for GNU/Linux (x86, x86_64, powerpc),
FreeBSD (x86, x86_64) and Windows, and documentation in uploads/
OR (old style)
bin/gub denemo
bin/gib denemo --branch=lilypond=git.sv.gnu.org--lilypond.git-master --branch=denemo=git.savannah.gnu.org--denemo.git-master
* Optional: Prepare build environment
make -f lilypond.make bootstrap
GUB -- Inkscape
INSTALLING
Optional: Prepare build environment
make -f inkscape.make bootstrap
Build everything
make inkscape
this leaves installers in uploads/
More examples
* Build all Inkscape installers
make inkscape
this leaves installers for GNU/Linux (x86, x86_64, powerpc),
FreeBSD (x86, x86_64) and Windows, and documentation in uploads/
OR (old style)
bin/gub inkscape
bin/gib inkscape --branch=inkscape=trunk inkscape
GUB -- OpenOffice.org
INSTALLING
Get GUB
git clone git://github.com/janneke/gub.git
or
wget http://github.com/janneke/gub/tarball/master
In addition to the requirements for GUB, you need
• wine: to run regcomp.exe and lngconvex.exe and possibly other cross-build
ignorant build tools that are compile-time tied to the build platform
Ongoing work
The cross building of OpenOffice.org for mingw32 is still a work in progress
(WIP). The build in GUB will succeeed and an installer can be created, but
services.rdb may not build correctly.
GSoC 2009
Luckily, Jesús Corrius has accepted the challenge to make this mingw32 Go-Oo
OpenOffice.org almost-prove-concept hack into a real port, provide clean
solutions for all the hackweek mess leftovers, make it work as expected, in
other words: actually usable and run on Windows.
Jesús is working together with Fridrich Strba to clean up and update some of
the cross build and mingw32 patches, while implementing and porting the missing
bits of the mingw32 cross build to openSUSE's osc build system.
When jcorrius announces it works, his fixes will be incorporated in GUB to
provide turnkey OpenOffice.org Windows binaries and installers.
Build everything
make openoffice
this produces a windows installer in
uploads/openoffice-COMMITTISH.mingw.exe
More examples
* Cross build OpenOffice for Windows:
bin/gub mingw::openoffice
* Make the nsis installer
bin/gib mingw::openoffice
* Optional: Prepare build environment
make -f lilypond.make bootstrap
The package
bin/gub tools::openoffice
is a work in progress and attempts to build a tiny fraction of openoffice for
the build tools.
See also
http://github.com/janneke/gub/tree/master/patches
http://lilypond.org/blog/janneke/gub3-ooo-mingw-cross-build
GUB the Grand Unified Builder
• Home
• Basics
• Applications
• History
• Links
GUB -- Grand Unified Builder -- HISTORY
HISTORY
The story starts June 1999 with a crazy guy with an itch to run LilyPond on
Windows. To get a feel for the times, this was LilyPond-1.1.47, requiring Egcs
1.1, Python 1.5, Guile 1.3, discussing on help-gnu-music@gnu.org. Two years
later Cygwin cross, our first cross build environment is released.
This is evolved into a patch set for mknetrel. Spring 2005: In a bold move a
set of patches for MinGW are created to build LilyPond and all its [17]
dependencies. Simultaniously, a cross build environment and installer for
MacOSX is created.
Creating a cross build environment is not too difficult, esp. not if you know
how to use Google. The problem is that they always break and never work on
other machines. GUB starts as an effort to unify the Windows and MacOS builders
and to make the process somewhat dependable and reproducible, i.e.: even more
free time for the developers (if there is such a thing) as the builds always
succeed. It uses Python instead of make and bash and adds checksums for the
package build specs. The unification of cross builds greatly facilitates adding
of new architectures and before long GUB supports cross building lilypond
binaries for 9 different platforms.
GUB2 adds serialization of the Python build code before execution. This makes
the build checksumming less coarse, ie, less gratuitous rebuilds and more
reliable. Also, all Python code is executed before the build process starts,
which means that there is less chance of silly run time errors. Lots of
packages are added in an effort to replace the OpenEmbedded-based build system
for the Samco 3G video phone at SES, Philips Research Eindhoven.
In Novell's Hackweek-iii GUB enters into its Third Life with GUB3, adding
cross-platform build dependencies and patches for an OpenOffice.org MinGW cross
build, including all essential libraries. Later, Inkscape is added, exploring
and implementing x.org dependencies. After that, building a windows installer
for GNU Denemo took only one day, including all dependencies.
GUB the Grand Unified Builder
• Home
• Basics
• Applications
• History
• Links
Links
autopackage.org
openSUSE build service
codesourcery.com
pre-GUB LilyPond installers
emdebian.org
GUB build specifications
GUB patches
lilypond.org/blog/janneke/gub3-ooo-mingw-cross-build
openembedded.org
scratchbox.org
Scratbox2; sb2 Dan Kegel's crosstool
本源码包内暂不包含可直接显示的源代码文件，请下载源码包。
