Commit Graph

1480 Commits

Author SHA1 Message Date
Matt McCormick
2254494748
doc: Add Articles section 2016-11-17 01:12:10 -05:00
Matt McCormick
8dffb1d6d8 base: Use Debian package repository CDN
For improved reliability and performance
2016-11-17 00:59:31 -05:00
Matt McCormick
6b71b08cbd doc: We support Docker for Mac and Docker for Windows
No longer support boot2docker on these platforms
2016-11-16 19:08:04 -08:00
Matt McCormick
d9d65ffcc9 dockcross: Add support for Windows
Tested on Git Bash and Bash on Ubuntu for Windows.

There are some issue with building the Docker image itself, but using the
images works great.
2016-11-16 19:02:55 -08:00
Matt McCormick
0ee005a0ca Merge pull request #67 from thewtex/AndroidToolchainFile
Android toolchain file
2016-11-16 21:14:25 -05:00
Matt McCormick
a138f60ab6 android-arm: Update NDK to 13b and API to 16
API level 16 still covers the majority of systems:

  https://developer.android.com/about/dashboards/index.html

  https://developer.android.com/ndk/guides/stable_apis.html
2016-11-16 16:23:33 -05:00
Matt McCormick
0d3333a3e1 android-arm: Add CMake 3.7 toolchain file support
See:

  https://cmake.org/cmake/help/v3.7/manual/cmake-toolchains.7.html#cross-compiling-for-android-with-a-standalone-toolchain
2016-11-16 15:14:17 -05:00
Matt McCormick
46e4b18ebf Merge pull request #65 from thewtex/build-cmake-with-ssl
Build cmake with ssl
2016-11-06 15:20:25 -05:00
Matt McCormick
b54dbfd9cf base: Improve openssl build for i686
Avoid bash-isms.

Also return true from the sub-shell.
2016-11-06 10:13:08 -05:00
Jean-Christophe Fillion-Robin
3b6e09b520
manylinux-x86: Fix build of Openssl and CMake
This commit fixes the error reported below by ensuring openssl and CMake
are configured and build as x86 binaries.

It also removes the download of CMake binaries that was introduced
by mistake in 1354fe2 (manylinux-x86: Initial addition).

Error:

```
Configured for linux-x86_64.
making all in crypto...
make[1]: Entering directory `/usr/src/openssl-1.0.2j/crypto'
/usr/bin/perl ../util/mkbuildinf.pl "gcc -I. -I.. -I../include  -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -m64 -DL_ENDIAN -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM" "linux-x86_64" >buildinf.h
gcc -I. -I.. -I../include  -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -m64 -DL_ENDIAN -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM   -c -o cryptlib.o cryptlib.c
cryptlib.c:1:0: sorry, unimplemented: 64-bit mode not compiled in
 /* crypto/cryptlib.c */
 ^
make[1]: *** [cryptlib.o] Error 1
```
2016-11-05 17:55:58 -04:00
Jean-Christophe Fillion-Robin
6f7e7b05fd
common.docker: Fix CMake https download building it against openssl 1.0.2j
See #63
2016-11-05 11:25:42 -04:00
Matt McCormick
39c5f29733 Merge pull request #62 from jcfr/manylinux-ownership-of-python-install
manylinux.common: Fix warning changing ownership of python install
2016-11-05 09:27:03 -04:00
Jean-Christophe Fillion-Robin
8923c6a3c7
manylinux.common: Fix warning changing ownership of python install
While the use of sudo (made possible by 53cf084) allows to install
additional packages, the warning copied below was still reported.

To avoid this warning and streamline the installation of new packages,
this commit (1) introduces the concept of "pre_exec" entrypoint hook
and (2) adds such a hook to change the ownership of python "bin" and
"site-packages" directories for the manylinux images.

Warning reported are similar to this one:

```
The directory '/home/jcfr/.cache/pip/http' or its parent directory is
not owned by the current user and the cache has been disabled. Please
check the permissions and owner of that directory. If executing pip
with sudo, you may want sudo's -H flag.
```

Note that the sudo "-H" flag suggested in the warning is not available
in centos.
2016-11-04 22:58:31 -04:00
Matt McCormick
4b7265bde0 Merge pull request #60 from thewtex/no-tty-sudo
No tty sudo
2016-11-04 10:17:18 -04:00
Matt McCormick
f3654d8dce
manylinux: Run visudo -c
Verify /etc/sudoers validity

Tested-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
2016-11-03 21:50:42 -04:00
Matt McCormick
53cf0849fe
manylinux: Do not require a tty for sudo
This enabling installation of Python packages with on CI services.

Issue #57
2016-11-03 21:50:41 -04:00
Matt McCormick
26c5447d29
manylinux: Consolidate common content into common.manylinux 2016-11-03 21:50:41 -04:00
Matt McCormick
d41d7f919e Merge pull request #59 from jcfr/manylinux-upgrade-pip
Manylinux upgrade pip
2016-11-03 21:22:18 -04:00
Matt McCormick
a257df34e8 Merge pull request #58 from thewtex/cmake-add_custom_command-built-emulator
base: Bump CMake for improved add_custom_command
2016-11-03 17:17:53 -04:00
Jean-Christophe Fillion-Robin
667b5c1ca7
manylinux-common/install-python-packages: Ensure latest pip is installed 2016-11-03 13:11:47 -04:00
Jean-Christophe Fillion-Robin
ab0ab22c67
manylinux: Deduplicate install-skbuild.sh and rename to install-python-packages.sh 2016-11-03 13:10:06 -04:00
Matt McCormick
b3feda218b
base: Bump CMake for improved add_custom_command
Improves cross-compilation support in add_custom_command by integrating

  https://gitlab.kitware.com/cmake/cmake/merge_requests/217

CMake Git master from 2016-11-03 (3.8-master).

Addresses #54
2016-11-03 12:38:25 -04:00
Matt McCormick
ee2c1618c3 Merge pull request #56 from thewtex/microbadger-badges
doc: Use MicroBadger badges
2016-11-01 10:27:40 -04:00
Matt McCormick
e43de27275 Merge pull request #55 from thewtex/doc-variables
doc: variable -> variables
2016-11-01 10:01:15 -04:00
Matt McCormick
a7c894ab50
doc: Use MicroBadger badges 2016-11-01 10:00:15 -04:00
Matt McCormick
fdb3a52a01
doc: variable -> variables 2016-11-01 09:51:20 -04:00
Matt McCormick
d7ac739c2e Merge pull request #48 from jcfr/keep-track-buildtime-metadata
dockerfile: Keep track of buildtime metadata
2016-11-01 07:28:35 -04:00
Matt McCormick
95f2462567 Merge pull request #53 from thewtex/scikit-build-0.4.0
manylinux-x{64,86}: Bump scikit-build to 0.4.0
2016-10-31 21:21:32 -04:00
Matt McCormick
623148e6e5
manylinux-x{64,86}: Bump scikit-build to 0.4.0 2016-10-31 18:40:36 -04:00
Matt McCormick
cb95cf3c94 Merge pull request #52 from jcfr/dockcross-ignore-rm-error-when-running-in-unprivileged-lxc-container-no-tty-support
dockcross: Prevent dockcross from blocking when used with Python subprocess
2016-10-31 00:12:45 -04:00
Jean-Christophe Fillion-Robin
cfcc7d6700
dockerfile: Keep track of buildtime metadata
This commit build each images with the following arguments:

* IMAGE: Name of the image (e.g dockcross/base, dockcross/manylinux-x64, ...)
* VCS_REF: dockcross/dockcross commit from which this image is built
* VCS_URL: this repository obtained reading remote.origin.url
* BUILD_DATE: Date and time when the build was initiated

Then, within the Dockerfile, the metadata are associated with the image
using the "LABEL" instruction.
See https://docs.docker.com/engine/reference/builder/#/label

The corresponding labels can be found here:
http://label-schema.org/rc1/#build-time-labels

See #28
2016-10-30 23:57:04 -04:00
Jean-Christophe Fillion-Robin
f31e582f80
dockcross: Simplify container name generation using $RANDOM
This commit removes the complex pipeline leveraging output from
/dev/urandom and simply uses $RANDOM variable. For the purpose
of generating an available container name, it will works well.

Suggested-by: Matt McCormick <matt.mccormick@kitware.com>
2016-10-30 23:46:01 -04:00
Jean-Christophe Fillion-Robin
bd234af0a9
dockcross: Prevent dockcross from blocking when used with Python subprocess
As explained in [1], when using dockcross from python subprocess, the
interactive mode is disabled and output of "cat /dev/urandom" is block
buffered instead of being line buffered.

Workaround to this problem is to simply read a fixed amount of characters
from urandom.

The following two snippets illustrates the problem and the
the implemented  solution:

Works

```python
import subprocess as sp
sp.check_call("var=$(head -c 500 /dev/urandom | base64 | tr -dc 'a-zA-Z0-9' | fold -w 7 | head -n 1); echo $var", shell=True)
```

=> output random string


Fail:

```python
import subprocess as sp
sp.check_call("var=$(cat /dev/urandom | base64 | tr -dc 'a-zA-Z0-9' | fold -w 7 | head -n 1); echo $var", shell=True)
```

=> Hang

[1] http://stackoverflow.com/questions/16805827/unable-to-read-stdout-from-a-running-process#16806506
2016-10-30 03:27:10 -04:00
Matt McCormick
e378a9b336 Merge pull request #51 from jcfr/dockcross-ignore-rm-error-when-running-in-unprivileged-lxc-container
dockcross: Ignore deletion error when running in unprivileged LXC container
2016-10-27 13:34:45 -04:00
Jean-Christophe Fillion-Robin
3288c877e5
dockcross: Fix "tr: Illegal byte sequence" when generating container name on MacOSx
See #50
2016-10-27 13:20:31 -04:00
Jean-Christophe Fillion-Robin
c5edcb3a74
dockcross: Ensure exit code associated with run command is always returned 2016-10-27 11:37:07 -04:00
Jean-Christophe Fillion-Robin
904cb09858
dockcross: Change name of container from "dockcross" to "dockcross_RAND"
where RAND is a 7 characters long randomly generated alphanumeric string.

Note that container name can NOT start with "_",
only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed

See #50
2016-10-27 11:37:03 -04:00
Jean-Christophe Fillion-Robin
7a8f4fab0e
dockcross: Do not output any text after container is removed
See #50
2016-10-27 11:19:37 -04:00
Matt McCormick
195827d40d Merge pull request #49 from thewtex/mxe-update
windows-x{64,86}: Bump MXE to 2016-10-25
2016-10-27 11:12:17 -04:00
Jean-Christophe Fillion-Robin
2e71db2234
dockcross: Ignore deletion error when running in unprivileged LXC container
This commit workarounds the problem described in [1] and [2] by ignoring error
happening when deleting container in unprivileged LXC container.

Fixes #50

[1] https://circleci.com/docs/docker-btrfs-error/
[2] https://discuss.circleci.com/t/docker-error-removing-intermediate-container/70
2016-10-26 23:24:25 -04:00
Matt McCormick
80993ad985 windows-x{64,86}: Bump MXE to 2016-10-25 2016-10-26 17:12:50 -04:00
Matt McCormick
32704fb4e4 Merge pull request #47 from jcfr/fix-circle-ci
Fix circle ci
2016-09-26 13:57:13 -04:00
Jean-Christophe Fillion-Robin
97385c1356
Makefile: Make sure base.test also create bin directory
This commit fixes a regression [1] introduced in 606d0b3 (Makefile: Make sure
BIN directory exist when running any test)

Reported below the dry-run summary before and after, we can confirm the
bin directory is now created when building the `base.test` target.

Before:

$ make base.test --dry-run
docker build -t dockcross/base .
docker run --rm dockcross/base > bin/dockcross-base && chmod +x bin/dockcross-base

After:

$ make base.test --dry-run
docker build -t dockcross/base .
mkdir -p bin
docker run --rm dockcross/base > bin/dockcross-base && chmod +x bin/dockcross-base


[1] Error fixed by this commit:

Successfully built e6388fe49824
docker run --rm dockcross/base > bin/dockcross-base && chmod +x bin/dockcross-base
/bin/sh: 1: cannot create bin/dockcross-base: Directory nonexistent
make: *** [base.test] Error 2

make base.test returned exit code 2
2016-09-26 11:38:43 -04:00
Jean-Christophe Fillion-Robin
cf3cec8819
circle: Fix error in dependencies/override - Add missing 'done' 2016-09-26 10:14:27 -04:00
Matt McCormick
89b49dea22 Merge pull request #45 from jcfr/tweak-makefile
Tweak makefile
2016-09-26 10:01:31 -04:00
Jean-Christophe Fillion-Robin
606d0b3ed9
Makefile: Make sure BIN directory exist when running any test
This commit allows to run any test directly after checking out
the source code.
2016-09-26 00:21:16 -04:00
Jean-Christophe Fillion-Robin
79938b37e5
circle.yml: Simplify reusing 'display_images' target 2016-09-25 23:55:17 -04:00
Jean-Christophe Fillion-Robin
8a4e0845da
Makefile: All images are tested. Remove ALL_IMAGES and DEFAULT_IMAGES vars.
To confirm that the Makefiles works as expected after this patch, the
list of executed commands before and after has been compared.

Before:

```
make --dry-run > ../dockcross-make-baseline
make test --dry-run > ../dockcross-make-test-baseline
```

After:

```
make --dry-run > ../dockcross-make-current;
make test --dry-run > ../dockcross-make-test-current

for target in make make-test; do
  diff --ignore-trailing-space ../dockcross-$target-baseline ../dockcross-$target-current> /dev/null 2>&1
  [[ $? == 1 ]] && \
    echo "" && \
    echo "-------------------------------" && \
    echo "Here is output of:" && \
    echo "  diff --ignore-trailing-space ../dockcross-$target-baseline ../dockcross-$target-current" && \
    echo "" && \
    diff --ignore-trailing-space ../dockcross-$target-baseline ../dockcross-$target-current
done
```

Output:

```
-------------------------------
Here is output of:
  diff --ignore-trailing-space ../dockcross-make-baseline ../dockcross-make-current

8a9
> docker build -t dockcross/linux-ppc64le linux-ppc64le
10a12,14
> cp -r test browser-asmjs/
> docker build -t dockcross/browser-asmjs browser-asmjs
> rm -rf browser-asmjs/test

-------------------------------
Here is output of:
  diff --ignore-trailing-space ../dockcross-make-test-baseline ../dockcross-make-test-current

24a25,27
> docker build -t dockcross/linux-ppc64le linux-ppc64le
> docker run --rm dockcross/linux-ppc64le > bin/dockcross-linux-ppc64le && chmod +x bin/dockcross-linux-ppc64le
> bin/dockcross-linux-ppc64le python test/run.py --languages C
30a34,38
> cp -r test browser-asmjs/
> docker build -t dockcross/browser-asmjs browser-asmjs
> rm -rf browser-asmjs/test
> docker run --rm dockcross/browser-asmjs > bin/dockcross-browser-asmjs && chmod +x bin/dockcross-browser-asmjs
> bin/dockcross-browser-asmjs python test/run.py --exe-suffix ".js"

```
2016-09-25 23:28:23 -04:00
Jean-Christophe Fillion-Robin
4caa5a367f
README: Add "Download all images" and "+Install all dockcross scripts" section 2016-09-25 19:47:23 -04:00
Jean-Christophe Fillion-Robin
aa4735ddf2
Makefile: Add "display_default_images" and "display_all_images" targets 2016-09-25 19:46:49 -04:00