Skip to content

ansible: install devtoolset-8 on centos7 #2262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 9, 2020
Merged

ansible: install devtoolset-8 on centos7 #2262

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
6a3f4b0
tools: more host categories
sam-github Apr 7, 2020
049ed10
ansible: install devtoolset-8 on centos7
sam-github Apr 3, 2020
9072ef8
ansible: name for git package differs on centos7
sam-github Apr 3, 2020
dbf36f6
ansible: no arm devtoolset-6-libatomic-devel
sam-github Apr 3, 2020
34ae193
Select devtoolset-8 for centos7-ppc64
sam-github Apr 6, 2020
0ec6786
Select devtoolset-8 for centos7-64/centos7-arm64
sam-github Apr 6, 2020
3753b6b
Select devtoolset-8 for rhel7-s390x
sam-github Apr 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 4 additions & 11 deletions ansible/roles/baselayout/tasks/partials/repo/centos7.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,12 @@
state: present


- name: centos7 | install scl for devtoolset-6
when: "arch != 'arm64' and arch != 'ppc64'"
- name: centos7 | install scl for devtoolset-6, devtoolset-8
when: "arch != 'arm64'"
yum:
name: centos-release-scl
state: present

- name: centos7 | ppc64 | install sclo7 devtoolset-6
when: "arch == 'ppc64'"
yum_repository:
name: sclo7-devtoolset-6
description: cbs.centos.org sclo7-devtoolset-6-rh-release
baseurl: https://cbs.centos.org/repos/sclo7-devtoolset-6-rh-release/$basearch/os/
gpgcheck: no

- name: centos7 | aarch64 | install sclo repo
when: "arch == 'arm64'"
copy:
Expand All @@ -40,7 +32,8 @@
dest: "/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-SCLo"
mode: 0644

- stat: path=/usr/local/bin/python3.7
- stat:
path: /usr/local/bin/python3.7
register: build_python37

- name: centos7 | ppc64 | download python 3.7
Expand Down
9 changes: 6 additions & 3 deletions ansible/roles/baselayout/vars/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,14 @@ packages: {
'ccache,git2u,gcc-c++,devtoolset-6,sudo', # even need gcc on centos6 so ccache has symlinks
],

# centos-release-scl is required to enable SCLo but we do it manually in
# partials/repo/centos7.yml for arm64
centos7_arm64: ['git'], # git2u not available for aarch64 (yet)
centos7_x64: ['git2u','centos-release-scl',], # centos-release-scl is required to enable SCLo
# but we do it manually in partials/repo/centos7.yml for arm64
centos7_x64: ['devtoolset-6-libatomic-devel,git2u,centos-release-scl'],
centos7_ppc64: ['devtoolset-6-libatomic-devel,git'],

centos7: [
'bzip2-devel,openssl-devel,ccache,gcc-c++,devtoolset-6,sudo,git,devtoolset-6-libatomic-devel,zlib-devel,libffi-devel',
'bzip2-devel,openssl-devel,ccache,gcc-c++,devtoolset-6,sudo,zlib-devel,libffi-devel,devtoolset-8,devtoolset-8-libatomic-devel',
],

aix: [
Expand Down
10 changes: 7 additions & 3 deletions jenkins/scripts/VersionSelectorScript.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,11 @@ def buildExclusions = [
// Linux -------------------------------------------------
[ /^centos6/, releaseType, lt(8) ],
[ /^centos6/, anyType, gte(12) ],
[ /centos[67]-(arm)?(64|32)-gcc48/, anyType, gte(10) ],
[ /centos[67]-(arm)?(64|32)-gcc6/, anyType, lt(10) ],
[ /centos6-32-gcc6/, releaseType, gte(10) ], // 32-bit linux for <10 only
[ /^centos[67]-(arm)?(64|32)-gcc48/,anyType, gte(10) ],
[ /^centos[67]-(arm)?(64|32)-gcc6/, anyType, lt(10) ],
Comment on lines +24 to +25
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just added the ^ for consistency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, but are you sure there are no labels used anywhere with some cruft in front? I don't think there but we've been caught with overly strict regexes in the past (in select-compler.sh trying to identify all the benchmark machines iirc was the latest).

[ /^centos[67]-(arm)?(64|32)-gcc6/, anyType, gte(14) ], // 14.x: gcc6 builds stop
[ /^centos7-(arm)?(64)-gcc8/, anyType, lt(14) ], // 14.x: gcc8 builds start
Comment on lines +26 to +27
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are stopping using one set of labels after 14.x, and switching to a new set. I think its intentional to do anyType, its not intended that we test with gcc6/aka devtoolset-6, is it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack
Yes let's just leave it as a clean break even for testing. We can expand later if we see a need but we had had a problem with exploding permutation complexity just because we can in the past, and let's try to resist that urge.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume we still have some linux platforms building/testing on gcc6 thought right ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For 14.x, specifically? I don't know. For earlier releases, yes.

If you want I can take can try to find the compiler levels for all machines 14.x are tested on, but I spent 10 minutes looking, and its not documented anywhere AFAICT. Figuring it out would take a combination of finding all linux builds in https://ci.nodejs.org/view/Node.js%20Daily/job/node-daily-master/1899/, reading the console logs, then sshing into the machines (if they don't explicitly set the version, many don't, I have to look... for example ubuntu 18.04 is using gcc7, I just checked). If nodejs/node#32715 (comment) lands, then the ssh won't be needed, just reading the top of all the console logs.

AIX builds on gcc6.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the CI for nodejs/node#32715 (https://ci.nodejs.org/job/node-test-pull-request/30565/) the following Linux jobs are using gcc6:

[ /^centos6-32-gcc6/, releaseType, gte(10) ], // 32-bit linux for <10 only
[ /^centos7-64/, releaseType, lt(12) ],
[ /debian8-x86/, anyType, gte(10) ], // 32-bit linux for <10 only
[ /debian8/, anyType, gte(13) ],
Expand All @@ -35,6 +37,7 @@ def buildExclusions = [
// Linux PPC LE ------------------------------------------
[ /^centos7-ppcle/, anyType, lt(10) ],
[ /^ppcle-ubuntu/, releaseType, gte(10) ],
[ /^ppcle-ubuntu/, anyType, gte(14) ],

// Linux S390X --------------------------------------------
[ /s390x/, anyType, lt(6) ],
Expand All @@ -48,6 +51,7 @@ def buildExclusions = [
[ /^cross-compiler-armv[67]-gcc-4.9/, anyType, lt(10) ],
[ /^cross-compiler-armv[67]-gcc-4.9/, anyType, gte(12) ],
[ /^cross-compiler-armv[67]-gcc-6/, anyType, lt(12) ],
[ /^ubuntu1604-arm64/, anyType, gte(14) ],

// Windows -----------------------------------------------
// https://github.com/nodejs/build/blob/master/doc/windows-visualstudio-supported-versions.md
Expand Down
28 changes: 27 additions & 1 deletion jenkins/scripts/select-compiler.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ if [ "$SELECT_ARCH" = "PPC64LE" ]; then
echo "Setting compiler for Node version $NODEJS_MAJOR_VERSION on ppc64le"

case $NODE_NAME in
*centos7* )
if [ "$NODEJS_MAJOR_VERSION" -gt "13" ]; then
# Setup devtoolset-8, sets LD_LIBRARY_PATH, PATH, etc.
. /opt/rh/devtoolset-8/enable
export CC="ccache ppc64le-redhat-linux-gcc"
export CXX="ccache ppc64le-redhat-linux-g++"
export LINK="ppc64le-redhat-linux-g++"
echo "Compiler set to devtoolset-8"
return
fi
;;
*centos7* )
if [ "$NODEJS_MAJOR_VERSION" -gt "9" ]; then
# Setup devtoolset-6, sets LD_LIBRARY_PATH, PATH, etc.
Expand Down Expand Up @@ -75,7 +86,14 @@ elif [ "$SELECT_ARCH" = "S390X" ]; then

echo "Setting compiler for Node version $NODEJS_MAJOR_VERSION on s390x"

if [ "$NODEJS_MAJOR_VERSION" -gt "9" ]; then
if [ "$NODEJS_MAJOR_VERSION" -gt "13" ]; then
# Setup devtoolset-8, sets LD_LIBRARY_PATH, PATH, etc.
. /opt/rh/devtoolset-8/enable
export CC="ccache s390x-redhat-linux-gcc"
export CXX="ccache s390x-redhat-linux-g++"
export LINK="s390x-redhat-linux-g++"
echo "Compiler set to devtoolset-8"
elif [ "$NODEJS_MAJOR_VERSION" -gt "9" ]; then
# Setup devtoolset-6, sets LD_LIBRARY_PATH, PATH, etc.
. /opt/rh/devtoolset-6/enable
export CC="ccache s390x-redhat-linux-gcc"
Expand Down Expand Up @@ -143,6 +161,10 @@ elif [ "$SELECT_ARCH" = "X64" ]; then
echo "Setting compiler for Node version $NODEJS_MAJOR_VERSION on x64"

case $nodes in
centos7-64-gcc8 )
. /opt/rh/devtoolset-8/enable
echo "Compiler set to devtoolset-8"
;;
centos6-64-gcc48 )
. /opt/rh/devtoolset-2/enable
echo "Compiler set to devtoolset-2"
Expand All @@ -166,6 +188,10 @@ elif [ "$SELECT_ARCH" = "ARM64" ]; then


case $nodes in
centos7-arm64-gcc8 )
. /opt/rh/devtoolset-8/enable
echo "Compiler set to devtoolset-8"
;;
centos[67]-arm64-gcc6 )
. /opt/rh/devtoolset-6/enable
echo "Compiler set to devtoolset-6"
Expand Down
2 changes: 2 additions & 0 deletions tools/build-p-ssh-hosts.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,13 @@ grep -- 'armv7l_pi2' hosts/all > hosts/armv7l_pi2
grep -- 'centos6-x64' hosts/all > hosts/centos6-x64
grep -- 'centos7-ppc64_le' hosts/all > hosts/centos7-ppc64_le
grep -- 'centos7-ppc64_le' hosts/all > hosts/centos7-ppc64_le
grep -- 'centos7' hosts/all > hosts/centos7
grep -- 'digitalocean' hosts/all > hosts/digitalocean
grep -- 'infra-' hosts/all > hosts/infra
grep -- 'joyent' hosts/all > hosts/joyent
grep -- 'release-' hosts/all > hosts/release
grep -- 'rhel7-s390x' hosts/all > hosts/rhel7-s390x
grep -- 'rhel7.*-s390x' hosts/all > hosts/rhel7x-s390x
grep -- 'smartos' hosts/all > hosts/smartos
grep -- 'softlayer' hosts/all > hosts/softlayer
grep -- 'test-' hosts/all > hosts/test
Expand Down