Commit Graph

717 Commits

Author SHA1 Message Date
Jean-Christophe Fillion-Robin
286e6b5c95
Dockerfile: Explicitly depend on "dockcross/base:latest"
As recommended in by CircleCI, explicitly specify the "latest"
tag. This will be important when we will start versioning our images.

Copied from CircleCI documentation [1]:

"Make sure you tag the image you use in the FROM command in your
Dockerfile, even if it is the default “latest” tag. Otherwise, Docker
will pull down all tags for the image you specify. This will create
significant performance problems because the save/load strategy described
above only caches the image layers (and thus tags) that you specify in
the docker save command, so other tags will be re-pulled on every build
if a tag is not specified in the FROM command."

[1] https://circleci.com/docs/docker/#caching-docker-layers
2016-11-26 18:25:08 -05:00
Jean-Christophe Fillion-Robin
1560f4c3b6
makefile/browser-asmjs: Remove now unneeded dependency to "base"
Following 09868d5 (browser-asmjs: Update to depend on existing image from
"trzeci/emscripten"), browser-asmjs does NOT depend anymore on base image.
2016-11-26 18:20:35 -05:00
Jean-Christophe Fillion-Robin
abc164e7c6
circleci-matrix: Push images as part of the test step
To workaround, CircleCI limitation [1]:

"Please note that since we do not support parallel deployment, specifying
‘parallel:true’ in the deployment phase will cause an error."

This commit moves the deployment step as the last action of the test
step. Doing so will ensure, every container will take care of pushing
images it was responsible to build.

[1] https://circleci.com/docs/parallel-manual-setup/#simple-example
2016-11-26 17:54:17 -05:00
Jean-Christophe Fillion-Robin
68393166b9
circleci-matrix: Increase timeout associated with deployment step 2016-11-26 12:50:33 -05:00
Jean-Christophe Fillion-Robin
029b402646
circle-ci-matrix: Fix deployment 2016-11-26 12:15:50 -05:00
Matt McCormick
b4e2a02e0c Merge pull request #84 from jcfr/reenable-parallel-build
Reenable parallel build
2016-11-26 11:29:40 -05:00
Jean-Christophe Fillion-Robin
7fb4b89937
ci: Update circleci-matrix to display when each sub-step starts and finishes 2016-11-25 18:18:17 -05:00
Jean-Christophe Fillion-Robin
6018cc6241
ci: Split building and waiting for base image into its own command
This should help reduce the overall time associated with the test step
and avoid reaching the 120mins cutoff time for a given command.
2016-11-25 18:12:00 -05:00
Jean-Christophe Fillion-Robin
1e4de86ca6
ci: Optimize parallel build
* style: Rename TASK to STEP
* do not pull base image after it was copied from node0 and loaded.
  It caused the current Dockerfile to be out-of-sync with the current image.
* do not pull images after loading the downloaded and up-to-date
  base.tar. It was overwriting the base.tar downloaded and loaded.
* save debian:jessie into base.tar
* use "time" command to help identify long running operation
* rename marker file from "BASE_READY" to "BASE_AVAILABLE"
* build base image only if needed.
2016-11-25 18:12:00 -05:00
Jean-Christophe Fillion-Robin
b5c2221288
Revert "Merge pull request #80 from jcfr/revert-circleci-parallel"
This reverts commit 2f31a84e65, reversing
changes made to e120e36824.
2016-11-25 18:11:21 -05:00
Jean-Christophe Fillion-Robin
e3c3859b0f Merge pull request #83 from jcfr/improve-browser-asmjs
Improve browser asmjs
2016-11-25 18:07:44 -05:00
Jean-Christophe Fillion-Robin
09868d5289
browser-asmjs: Update to depend on existing image from "trzeci/emscripten"
This commit updates the "browser-asmjs" images to avoid systematic
rebuild of emscripten sdk. Instead, it introduces dependency on the
image maintained by @trzeci

Note also:

* emscripten sdk updated from 1.36.7 to 1.36.14

* symlinking of "/bin/bash" as "/bin/sh" is reverted. This ensures other
  images including file like "common.debian" and relying on standard "sh"
  behavior will work as expected.

* workaround associated with CMakeForceCompiler has been removed from
  Dockerfile. Emscripten as been fixed.
  See https://github.com/kripken/emscripten/pull/4477

* we will revisit once official images are available.
  See https://github.com/kripken/emscripten/issues/4682
2016-11-25 15:51:42 -05:00
Jean-Christophe Fillion-Robin
46cef1a33a
common: Introduce "common.debian"
This will facilitate the configuration of other images based on debian
but *NOT* depending on the dockcross/base image.
2016-11-25 15:51:42 -05:00
Jean-Christophe Fillion-Robin
8077008a10
common: Introduce "install-ninja.sh" 2016-11-25 15:51:42 -05:00
Jean-Christophe Fillion-Robin
191d773e01
makefile: Ensure imagefiles are available in build image context 2016-11-25 15:51:38 -05:00
Jean-Christophe Fillion-Robin
593d6a5c07 Merge pull request #81 from jcfr/avoid-rebuild-of-base
base: Optimize image build adding only image and vcs_url metatdata
2016-11-24 18:17:41 -05:00
Jean-Christophe Fillion-Robin
a68f2b6aa4
base: Optimize image build adding only image and vcs_url metatdata
To avoid dependent images to systematically rebuild, this commit
adds only "static" metatdata to the base image
2016-11-24 13:48:58 -05:00
Jean-Christophe Fillion-Robin
2f31a84e65 Merge pull request #80 from jcfr/revert-circleci-parallel
Revert "Merge pull request #75 from jcfr/circleci-parallel-build"
2016-11-24 13:46:37 -05:00
Jean-Christophe Fillion-Robin
fd9f2719e3
Revert "Merge pull request #75 from jcfr/circleci-parallel-build"
Waiting we address the timeout of command after 120min when using
parallel containers, this reverts commit e120e36824, reversing
changes made to 6a4826a8df.

Issue has been reported to CircleCI support as
https://circleci.zendesk.com/hc/en-us/requests/13273
2016-11-23 22:02:35 -05:00
Matt McCormick
e120e36824 Merge pull request #75 from jcfr/circleci-parallel-build
ci: Enable parallel build using circleci-matrix
2016-11-21 11:23:40 -05:00
Matt McCormick
6a4826a8df Merge pull request #77 from jcfr/simplify-and-improve-openssl-and-cmake-install
common.docker: Improve OpenSSL and CMake install introducing helper scripts
2016-11-21 11:20:00 -05:00
Jean-Christophe Fillion-Robin
0552c37fea
common.docker: Improve OpenSSL and CMake install introducing helper scripts
To accommodate the requirements associated with x86 and x64 images, the
command building OpenSSL and CMake became overly complex and hard to
maintain.

This commit has multiple purposes:

(1) simplify common.docker

(2) fix the building of 64-bit shared libraries against the static openssl
libraries by passing the -fPIC flag.

(3) ensure [many]linux-x86 and [many]linux-x64 images have an up-to-date
OpenSSL install. Openssl static libraries are installed in /usr

(4) simplify and speedup CMake build avoiding the second build with
explicit -DCMAKE_USE_OPENSSL:BOOL=ON. Indeed, configuring CMake on Linux
already looks for OpenSSL.

(5) speedup download of CMake source directly downloading the archive
corresponding to the revision.

(6) test CMake by:
  - running CMake.FileDownload test
  - trying to download a file served over https
2016-11-21 06:08:30 -05:00
Jean-Christophe Fillion-Robin
b370ceb3f5 README: Fix "Docker for Windows" link
[ci skip]
2016-11-21 02:24:56 -05:00
Jean-Christophe Fillion-Robin
70de0f9856 Merge pull request #74 from jcfr/base-image-fix-install-openssl
base: Fix building of OpenSSL. See #73
2016-11-20 14:10:32 -05:00
Jean-Christophe Fillion-Robin
9639405d67 Merge pull request #76 from jcfr/linux-toolchain-fix-asm-compiler
[many]linux-(x86|x64)/: toolchain: Fix ASM compiler
2016-11-20 03:01:13 -05:00
Jean-Christophe Fillion-Robin
ed7872edee
[many]linux-(x86|x64)/: toolchain: Fix ASM compiler
Similarly to the linux-x86 toolchain, gcc is used to compile assembler
file.
2016-11-19 22:30:33 -05:00
Jean-Christophe Fillion-Robin
cfd1921127
ci: Enable parallel build using circleci-matrix
The base image is built on the container 0. In the mean time, the other
containers wait for the base image to be ready.
2016-11-19 18:03:16 -05:00
Jean-Christophe Fillion-Robin
75a7b29fb8
common.docker: Delete both openssl-1.0.2j build directory and archive 2016-11-19 14:39:25 -05:00
Jean-Christophe Fillion-Robin
30a73bcd24
base: Fix building of OpenSSL. See #73
This commit fixes the following error reported when building the base
image.

sh: 1: [: =: unexpected operator
sh: 1: [: =: unexpected operator

It turns out that (1) DEFAULT_DOCKCROSS_IMAGE is not set when building the
base image and (2) latest openssl will not be available in linux-x64 and
linux-x86 images.
2016-11-19 14:32:20 -05:00
Matt McCormick
0bc5941bc8 Merge pull request #72 from jcfr/install-openssl-in-linux-based-images
common.docker: Ensure openssl-1.0.2j is available in manylinux and linux images
2016-11-18 20:09:00 -05:00
Jean-Christophe Fillion-Robin
4bb8f97ffc
common.docker: Ensure openssl-1.0.2j is available in manylinux and linux images
If images are any of manylinux-x86, manylinux-x64, linux-x86 or linux-x64,
OpenSSL libraries are installed in /usr .
2016-11-18 15:30:03 -05:00
Jean-Christophe Fillion-Robin
2e9a5f7ed7 Merge pull request #71 from chadawagner/patch-1
quote HOST_PWD in docker cmd to allow spaces
2016-11-18 15:24:31 -05:00
chadawagner
9373f029f4 quote HOST_PWD in docker cmd to allow spaces 2016-11-17 21:19:54 -08:00
Matt McCormick
3a0f84f977 Merge pull request #68 from thewtex/DebianCDN
base: Use Debian package repository CDN
2016-11-17 09:19:27 -05:00
Matt McCormick
9a8d19a52c Merge pull request #69 from thewtex/WindowsSupport
Windows support
2016-11-17 09:18:33 -05:00
Matt McCormick
5e8b3e422a doc: Features formatting 2016-11-17 01:16:55 -05:00
Matt McCormick
192d14ccda Merge pull request #70 from thewtex/Articles
doc: Add Articles section
2016-11-17 01:15:01 -05:00
Matt McCormick
2254494748
doc: Add Articles section 2016-11-17 01:12:10 -05:00
Matt McCormick
8dffb1d6d8 base: Use Debian package repository CDN
For improved reliability and performance
2016-11-17 00:59:31 -05:00
Matt McCormick
6b71b08cbd doc: We support Docker for Mac and Docker for Windows
No longer support boot2docker on these platforms
2016-11-16 19:08:04 -08:00
Matt McCormick
d9d65ffcc9 dockcross: Add support for Windows
Tested on Git Bash and Bash on Ubuntu for Windows.

There are some issue with building the Docker image itself, but using the
images works great.
2016-11-16 19:02:55 -08:00
Matt McCormick
0ee005a0ca Merge pull request #67 from thewtex/AndroidToolchainFile
Android toolchain file
2016-11-16 21:14:25 -05:00
Matt McCormick
a138f60ab6 android-arm: Update NDK to 13b and API to 16
API level 16 still covers the majority of systems:

  https://developer.android.com/about/dashboards/index.html

  https://developer.android.com/ndk/guides/stable_apis.html
2016-11-16 16:23:33 -05:00
Matt McCormick
0d3333a3e1 android-arm: Add CMake 3.7 toolchain file support
See:

  https://cmake.org/cmake/help/v3.7/manual/cmake-toolchains.7.html#cross-compiling-for-android-with-a-standalone-toolchain
2016-11-16 15:14:17 -05:00
Matt McCormick
46e4b18ebf Merge pull request #65 from thewtex/build-cmake-with-ssl
Build cmake with ssl
2016-11-06 15:20:25 -05:00
Matt McCormick
b54dbfd9cf base: Improve openssl build for i686
Avoid bash-isms.

Also return true from the sub-shell.
2016-11-06 10:13:08 -05:00
Jean-Christophe Fillion-Robin
3b6e09b520
manylinux-x86: Fix build of Openssl and CMake
This commit fixes the error reported below by ensuring openssl and CMake
are configured and build as x86 binaries.

It also removes the download of CMake binaries that was introduced
by mistake in 1354fe2 (manylinux-x86: Initial addition).

Error:

```
Configured for linux-x86_64.
making all in crypto...
make[1]: Entering directory `/usr/src/openssl-1.0.2j/crypto'
/usr/bin/perl ../util/mkbuildinf.pl "gcc -I. -I.. -I../include  -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -m64 -DL_ENDIAN -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM" "linux-x86_64" >buildinf.h
gcc -I. -I.. -I../include  -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -Wa,--noexecstack -m64 -DL_ENDIAN -O3 -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM   -c -o cryptlib.o cryptlib.c
cryptlib.c:1:0: sorry, unimplemented: 64-bit mode not compiled in
 /* crypto/cryptlib.c */
 ^
make[1]: *** [cryptlib.o] Error 1
```
2016-11-05 17:55:58 -04:00
Jean-Christophe Fillion-Robin
6f7e7b05fd
common.docker: Fix CMake https download building it against openssl 1.0.2j
See #63
2016-11-05 11:25:42 -04:00
Matt McCormick
39c5f29733 Merge pull request #62 from jcfr/manylinux-ownership-of-python-install
manylinux.common: Fix warning changing ownership of python install
2016-11-05 09:27:03 -04:00
Jean-Christophe Fillion-Robin
8923c6a3c7
manylinux.common: Fix warning changing ownership of python install
While the use of sudo (made possible by 53cf084) allows to install
additional packages, the warning copied below was still reported.

To avoid this warning and streamline the installation of new packages,
this commit (1) introduces the concept of "pre_exec" entrypoint hook
and (2) adds such a hook to change the ownership of python "bin" and
"site-packages" directories for the manylinux images.

Warning reported are similar to this one:

```
The directory '/home/jcfr/.cache/pip/http' or its parent directory is
not owned by the current user and the cache has been disabled. Please
check the permissions and owner of that directory. If executing pip
with sudo, you may want sudo's -H flag.
```

Note that the sudo "-H" flag suggested in the warning is not available
in centos.
2016-11-04 22:58:31 -04:00