Update makefile and improve CI:
Add clean rule in makefile
Change dockcross/ to /
Add dev branch in CI
Signed-off-by: Bensuperpc <bensuperpc@gmail.com>
WebAssembly has a better mindshare than asm.js. In the future, it will
be used in much more than browsers. Also, newer versions of Emscripten
default to WebAssembly.
Tag with the pattern:
<imagename>:YYYYMMDD-SHA{N} where YYYYMMDD is the date of the build and
SHA{N} the output of `git rev-parse --short HEAD`
along with `latest` as discussed in Issue #223.
This helps clients use a fixed, reproducible image that will be
available for a long period of time.
Add a mechanism to construct a full cross-compiler environment using the
"crosstool-ng" cross-compiler building utility. This is implemented in
the new "common.crosstool" include, and augments the "dockcross/base"
base image.
Update Makefile to consolidate Dockerfile generation, notably the
"sed"-based inclusion directives.
Finally, employ all of this to generate a "linux-mips" 32-bit hard-float
MIPS cross-compiler Dockcross image.
This commit updates the "browser-asmjs" images to avoid systematic
rebuild of emscripten sdk. Instead, it introduces dependency on the
image maintained by @trzeci
Note also:
* emscripten sdk updated from 1.36.7 to 1.36.14
* symlinking of "/bin/bash" as "/bin/sh" is reverted. This ensures other
images including file like "common.debian" and relying on standard "sh"
behavior will work as expected.
* workaround associated with CMakeForceCompiler has been removed from
Dockerfile. Emscripten as been fixed.
See https://github.com/kripken/emscripten/pull/4477
* we will revisit once official images are available.
See https://github.com/kripken/emscripten/issues/4682
To accommodate the requirements associated with x86 and x64 images, the
command building OpenSSL and CMake became overly complex and hard to
maintain.
This commit has multiple purposes:
(1) simplify common.docker
(2) fix the building of 64-bit shared libraries against the static openssl
libraries by passing the -fPIC flag.
(3) ensure [many]linux-x86 and [many]linux-x64 images have an up-to-date
OpenSSL install. Openssl static libraries are installed in /usr
(4) simplify and speedup CMake build avoiding the second build with
explicit -DCMAKE_USE_OPENSSL:BOOL=ON. Indeed, configuring CMake on Linux
already looks for OpenSSL.
(5) speedup download of CMake source directly downloading the archive
corresponding to the revision.
(6) test CMake by:
- running CMake.FileDownload test
- trying to download a file served over https
This commit build each images with the following arguments:
* IMAGE: Name of the image (e.g dockcross/base, dockcross/manylinux-x64, ...)
* VCS_REF: dockcross/dockcross commit from which this image is built
* VCS_URL: this repository obtained reading remote.origin.url
* BUILD_DATE: Date and time when the build was initiated
Then, within the Dockerfile, the metadata are associated with the image
using the "LABEL" instruction.
See https://docs.docker.com/engine/reference/builder/#/label
The corresponding labels can be found here:
http://label-schema.org/rc1/#build-time-labels
See #28
This commit fixes a regression [1] introduced in 606d0b3 (Makefile: Make sure
BIN directory exist when running any test)
Reported below the dry-run summary before and after, we can confirm the
bin directory is now created when building the `base.test` target.
Before:
$ make base.test --dry-run
docker build -t dockcross/base .
docker run --rm dockcross/base > bin/dockcross-base && chmod +x bin/dockcross-base
After:
$ make base.test --dry-run
docker build -t dockcross/base .
mkdir -p bin
docker run --rm dockcross/base > bin/dockcross-base && chmod +x bin/dockcross-base
[1] Error fixed by this commit:
Successfully built e6388fe49824
docker run --rm dockcross/base > bin/dockcross-base && chmod +x bin/dockcross-base
/bin/sh: 1: cannot create bin/dockcross-base: Directory nonexistent
make: *** [base.test] Error 2
make base.test returned exit code 2
This commit also ensures that:
* all image targets are declared as PHONY
* "test" and not "tests" is declared as PHONY
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared.
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
make browser-asmjs.test --dry-run > ../dockcross-make-browser-asmjs-test-baseline
make linux-ppc64le.test --dry-run > ../dockcross-make-linux-ppc64le-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
make browser-asmjs.test --dry-run > ../dockcross-make-browser-asmjs-test-current
make linux-ppc64le.test --dry-run > ../dockcross-make-linux-ppc64le-test-current
for target in make make-test make-browser-asmjs-test make-linux-ppc64le-test; do
diff --ignore-trailing-space ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared:
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff --ignore-trailing-space ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared.
The only difference was due to the extra space in the command line
associated with windows-x86
```
< bin/dockcross-windows-x86 python test/run.py --exe-suffix ".exe"
---
> bin/dockcross-windows-x86 python test/run.py --exe-suffix ".exe"
```
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff --ignore-trailing-space ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared:
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
Adding a new images with the usual build rule:
$(DOCKER) build -t $(ORG)/$@ $@
is now just a matter of adding an item to the "STANDARD_IMAGES" list.
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared.
Note that since this patch changes the order in which the images are
listed (manylinux is now last), the baseline and current files are
different. After inspection, we can confirm the difference is explained
by the order of the commands.
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
Since image targets are all "phony", they are always consider out-of-date
and will always be rebuilt. This means there is no need to add an explicit
dependency on non-generated files like "Dockerfile", "settings.mk" and
"test/run.py".
The only required dependencies on Dockerfile are the one for the "base"
and "manylinux" images.
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared:
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
Since image targets are all "phony", they are always consider out-of-date
and will always be rebuilt. This means there is no need to add an explicit
dependency on the "Toolchain.cmake" file.
Note also that the dependency on "Toolchain.cmake" was not consistent
for all images. For example, "linux-x64" was missing the dependency
whereas it was specified for "linux-x86".
This commit ensure all image targets are consistent removing the
dependency on "Toolchain.cmake".
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared:
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
See https://www.gnu.org/software/make/manual/html_node/File-Name-Functions.html#File-Name-Functions
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared:
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after is compared:
Before:
```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```
After:
```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current
for target in make make-test; do
diff ../dockcross-$target-current ../dockcross-$target-baseline > /dev/null 2>&1
[[ $? == 1 ]] && \
echo "" && \
echo "Error: Problem with '${target}' target: Dry-run output before and after this commit do not match."
done
```
This commits add test targets for all images, this will facilitate local
testing.
In addition to the usual targets:
make android-arm
make browser-asmjs
[...]
the following targets are now available:
make tests
make android-arm.test
make browser-asmjs.test
[...]
Note that the target "all" has been renamed into "images"
