CircleCI start running steps in parallel after number of steps
I have a config.yaml for my CirleCI workflow.
I use PHPUnit to run my tests but I would like to start running these in parallel using Circle to speed up the deploy process.
So BuildJob looks like this:
jobs:
# build-job
build-job:
<<: *defaults
resource_class: large
steps:
- checkout
- run: sudo apt install -y libsqlite3-dev zlib1g-dev libpng-dev libxml2-dev
- run: sudo docker-php-ext-install zip
- run: sudo docker-php-ext-install gd
- run: sudo docker-php-ext-install soap
- run: sudo docker-php-ext-install bcmath
- run: sudo composer self-update
- restore_cache:
keys:
- composer-v1- checksum "composer.lock"
- composer-v1-
- run: composer install -n --prefer-dist
- save_cache:
key: composer-v1- checksum "composer.lock"
paths:
- vendor
- restore_cache:
keys:
- node-v1- checksum "package.json"
- node-v1-
- run:
name: Create sqlite db
command: touch database/database.sqlite
- run:
name: Run tests
command: ./vendor/bin/phpunit -d memory_limit=512M
- persist_to_workspace:
root: ~/laravel
paths: .
So I know I need to add parallelism: 4 into there before the steps to run them in parallel, as documented here but I need the steps up to Run tests in order. Then I will have multiple run tests steps to hit each folder of tests.
Is there anyway to achieve this?
phpunit circleci circleci-2.0
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
add a comment |
I have a config.yaml for my CirleCI workflow.
I use PHPUnit to run my tests but I would like to start running these in parallel using Circle to speed up the deploy process.
So BuildJob looks like this:
jobs:
# build-job
build-job:
<<: *defaults
resource_class: large
steps:
- checkout
- run: sudo apt install -y libsqlite3-dev zlib1g-dev libpng-dev libxml2-dev
- run: sudo docker-php-ext-install zip
- run: sudo docker-php-ext-install gd
- run: sudo docker-php-ext-install soap
- run: sudo docker-php-ext-install bcmath
- run: sudo composer self-update
- restore_cache:
keys:
- composer-v1- checksum "composer.lock"
- composer-v1-
- run: composer install -n --prefer-dist
- save_cache:
key: composer-v1- checksum "composer.lock"
paths:
- vendor
- restore_cache:
keys:
- node-v1- checksum "package.json"
- node-v1-
- run:
name: Create sqlite db
command: touch database/database.sqlite
- run:
name: Run tests
command: ./vendor/bin/phpunit -d memory_limit=512M
- persist_to_workspace:
root: ~/laravel
paths: .
So I know I need to add parallelism: 4 into there before the steps to run them in parallel, as documented here but I need the steps up to Run tests in order. Then I will have multiple run tests steps to hit each folder of tests.
Is there anyway to achieve this?
phpunit circleci circleci-2.0
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
1
First off,parallelismis not necessary. You just need to define multiple jobs (you only have the onebuild-job) and split up the testing scope of phpunit.
– matfax
Nov 15 '18 at 23:21
add a comment |
I have a config.yaml for my CirleCI workflow.
I use PHPUnit to run my tests but I would like to start running these in parallel using Circle to speed up the deploy process.
So BuildJob looks like this:
jobs:
# build-job
build-job:
<<: *defaults
resource_class: large
steps:
- checkout
- run: sudo apt install -y libsqlite3-dev zlib1g-dev libpng-dev libxml2-dev
- run: sudo docker-php-ext-install zip
- run: sudo docker-php-ext-install gd
- run: sudo docker-php-ext-install soap
- run: sudo docker-php-ext-install bcmath
- run: sudo composer self-update
- restore_cache:
keys:
- composer-v1- checksum "composer.lock"
- composer-v1-
- run: composer install -n --prefer-dist
- save_cache:
key: composer-v1- checksum "composer.lock"
paths:
- vendor
- restore_cache:
keys:
- node-v1- checksum "package.json"
- node-v1-
- run:
name: Create sqlite db
command: touch database/database.sqlite
- run:
name: Run tests
command: ./vendor/bin/phpunit -d memory_limit=512M
- persist_to_workspace:
root: ~/laravel
paths: .
So I know I need to add parallelism: 4 into there before the steps to run them in parallel, as documented here but I need the steps up to Run tests in order. Then I will have multiple run tests steps to hit each folder of tests.
Is there anyway to achieve this?
phpunit circleci circleci-2.0
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
I have a config.yaml for my CirleCI workflow.
I use PHPUnit to run my tests but I would like to start running these in parallel using Circle to speed up the deploy process.
So BuildJob looks like this:
jobs:
# build-job
build-job:
<<: *defaults
resource_class: large
steps:
- checkout
- run: sudo apt install -y libsqlite3-dev zlib1g-dev libpng-dev libxml2-dev
- run: sudo docker-php-ext-install zip
- run: sudo docker-php-ext-install gd
- run: sudo docker-php-ext-install soap
- run: sudo docker-php-ext-install bcmath
- run: sudo composer self-update
- restore_cache:
keys:
- composer-v1- checksum "composer.lock"
- composer-v1-
- run: composer install -n --prefer-dist
- save_cache:
key: composer-v1- checksum "composer.lock"
paths:
- vendor
- restore_cache:
keys:
- node-v1- checksum "package.json"
- node-v1-
- run:
name: Create sqlite db
command: touch database/database.sqlite
- run:
name: Run tests
command: ./vendor/bin/phpunit -d memory_limit=512M
- persist_to_workspace:
root: ~/laravel
paths: .
So I know I need to add parallelism: 4 into there before the steps to run them in parallel, as documented here but I need the steps up to Run tests in order. Then I will have multiple run tests steps to hit each folder of tests.
Is there anyway to achieve this?
phpunit circleci circleci-2.0
phpunit circleci circleci-2.0
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
asked Nov 13 '18 at 11:22
Lewis SmithLewis Smith
168112
168112
1
First off,parallelismis not necessary. You just need to define multiple jobs (you only have the onebuild-job) and split up the testing scope of phpunit.
– matfax
Nov 15 '18 at 23:21
add a comment |
1
First off,parallelismis not necessary. You just need to define multiple jobs (you only have the onebuild-job) and split up the testing scope of phpunit.
– matfax
Nov 15 '18 at 23:21
1
1
First off,
parallelism is not necessary. You just need to define multiple jobs (you only have the one build-job) and split up the testing scope of phpunit.– matfax
Nov 15 '18 at 23:21
First off,
parallelism is not necessary. You just need to define multiple jobs (you only have the one build-job) and split up the testing scope of phpunit.– matfax
Nov 15 '18 at 23:21
add a comment |
1 Answer
1
active
oldest
votes
The usual approach of using the CircleCI CLI command circleci tests split to split the tests up for parallel containers does not work with phpunit, according to this discussion (https://discuss.circleci.com/t/how-to-execute-phpunit-tests-in-parallel/25622). However, as @matfax is saying, can you run the tests in several logical groups? Something like
./vendor/bin/phpunit tests/ThisGroup
./vendor/bin/phpunit tests/ThatGroup
If so, you could then run each of those commands in a parallel job within a CircleCI workflow.
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53279951%2fcircleci-start-running-steps-in-parallel-after-number-of-steps%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The usual approach of using the CircleCI CLI command circleci tests split to split the tests up for parallel containers does not work with phpunit, according to this discussion (https://discuss.circleci.com/t/how-to-execute-phpunit-tests-in-parallel/25622). However, as @matfax is saying, can you run the tests in several logical groups? Something like
./vendor/bin/phpunit tests/ThisGroup
./vendor/bin/phpunit tests/ThatGroup
If so, you could then run each of those commands in a parallel job within a CircleCI workflow.
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
add a comment |
The usual approach of using the CircleCI CLI command circleci tests split to split the tests up for parallel containers does not work with phpunit, according to this discussion (https://discuss.circleci.com/t/how-to-execute-phpunit-tests-in-parallel/25622). However, as @matfax is saying, can you run the tests in several logical groups? Something like
./vendor/bin/phpunit tests/ThisGroup
./vendor/bin/phpunit tests/ThatGroup
If so, you could then run each of those commands in a parallel job within a CircleCI workflow.
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
add a comment |
The usual approach of using the CircleCI CLI command circleci tests split to split the tests up for parallel containers does not work with phpunit, according to this discussion (https://discuss.circleci.com/t/how-to-execute-phpunit-tests-in-parallel/25622). However, as @matfax is saying, can you run the tests in several logical groups? Something like
./vendor/bin/phpunit tests/ThisGroup
./vendor/bin/phpunit tests/ThatGroup
If so, you could then run each of those commands in a parallel job within a CircleCI workflow.
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
The usual approach of using the CircleCI CLI command circleci tests split to split the tests up for parallel containers does not work with phpunit, according to this discussion (https://discuss.circleci.com/t/how-to-execute-phpunit-tests-in-parallel/25622). However, as @matfax is saying, can you run the tests in several logical groups? Something like
./vendor/bin/phpunit tests/ThisGroup
./vendor/bin/phpunit tests/ThatGroup
If so, you could then run each of those commands in a parallel job within a CircleCI workflow.
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
answered Nov 30 '18 at 23:20
macbeccammacbeccam
343
343
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
add a comment |
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
Trouble is running those first parts, I think I need a couple jobs, one to set up and one to run the tests in parallel
– Lewis Smith
Dec 3 '18 at 11:07
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
One to set up (and save workspace), then multiple to run tests in parallel (with attached workspace) I think.
– macbeccam
Dec 3 '18 at 16:10
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
Yep sounds about right.
– Lewis Smith
Dec 3 '18 at 16:23
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
You should not attempt to use workspace for tools installed with apt-get, etc. Instead consider a single job that adds these to a custom docker image that downstream jobs can use. To make builds even faster, consider a separate project/repo to build your PHP image as needed, and reference that in you jobs.
– Eddie
Dec 6 '18 at 15:54
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53279951%2fcircleci-start-running-steps-in-parallel-after-number-of-steps%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
First off,
parallelismis not necessary. You just need to define multiple jobs (you only have the onebuild-job) and split up the testing scope of phpunit.– matfax
Nov 15 '18 at 23:21