diff --git a/features/execute.feature b/features/execute.feature index a6966171..b5d8ad74 100644 --- a/features/execute.feature +++ b/features/execute.feature @@ -15,41 +15,23 @@ Feature: execute Then the stdout should not contain "Cloning from 'file://" @no-clobber - Scenario: When command run fails, fail fast if option defined - When I run `msync exec --verbose --fail-fast -- /bin/false` + Scenario: When command run fails and using default skip_broken value + When I run `msync exec --verbose -- /bin/false` Then the exit status should be 1 And the stderr should contain: """ - Command execution failed + puppet-test: Command execution failed """ @no-clobber - Scenario: When command run fails, run all and summarize errors if option fail-fast is not set - When I run `msync exec --verbose --no-fail-fast -- /bin/false` + Scenario: When command run fails and using skip_broken + When I run `msync exec --verbose -s -- /bin/false` Then the exit status should be 1 And the stderr should contain: """ - Error(s) during `execute` command: - * + Skipping 'puppet-test' as execute process failed """ - Scenario: Show fail-fast default value in help - When I successfully run `msync help exec` - Then the stdout should contain: - """ - [--fail-fast], [--no-fail-fast], [--skip-fail-fast] # Abort the run after a command execution failure - # Default: true - """ - - Scenario: Override fail-fast default value using config file - Given the global option "fail_fast" sets to "false" - When I successfully run `msync help exec` - Then the stdout should contain: - """ - [--fail-fast], [--no-fail-fast], [--skip-fail-fast] # Abort the run after a command execution failure - """ - # NOTE: It seems there is a Thor bug here: default value is missing in help when sets to 'false' - Scenario: Show preserved bundle env var. Given a basic setup with a puppet module "puppet-test" from "awesome" And I set the environment variables to: @@ -60,3 +42,46 @@ Feature: execute """ BUNLE_PATH=/opt/msync/bundle """ + + @no-clobber + Scenario: Adding a new test file + Given a basic setup with a puppet module "puppet-test" from "fakenamespace" + When I successfully run `msync exec --noop -- touch test.yaml` + Then the output should match: + """ + Files added: + test.yaml + """ + And the puppet module "puppet-test" from "fakenamespace" should have no commits made by "Aruba" + + Scenario: Modifying an existing file without commiting the change + Given a basic setup with a puppet module "puppet-test" from "fakenamespace" + And the puppet module "puppet-test" from "fakenamespace" has a file named "README.md" with: + """ + Hello world! + """ + When I successfully run `msync exec --noop -- sed -i "$ a some text" README.md` + Then the output should match: + """ + Files changed: + +diff --git a/README.md b/README.md + """ + And the output should contain: + """ + some text + """ + And the puppet module "puppet-test" from "fakenamespace" should have no commits made by "Aruba" + + Scenario: Modifying an existing file and committing the change + Given a basic setup with a puppet module "puppet-test" from "fakenamespace" + And the puppet module "puppet-test" from "fakenamespace" has a file named "Gemfile" with: + """ + source 'https://rubygems.org' + """ + When I successfully run `msync exec -m "Update Gemfile" -r test -- sed -i "$ a some source" Gemfile` + Then the puppet module "puppet-test" from "fakenamespace" should have only 1 commit made by "Aruba" + And the puppet module "puppet-test" from "fakenamespace" should have 1 commit made by "Aruba" in branch "test" + And the puppet module "puppet-test" from "fakenamespace" should have a branch "test" with a file named "Gemfile" which contains: + """ + some source + """ diff --git a/features/execute/pull_request.feature b/features/execute/pull_request.feature new file mode 100644 index 00000000..3090a9c7 --- /dev/null +++ b/features/execute/pull_request.feature @@ -0,0 +1,68 @@ +Feature: Create a pull-request/merge-request after execute + + Scenario: Ask for PR without credentials + Given a basic setup with a puppet module "puppet-test" from "fakenamespace" + And a file named "managed_modules.yml" with: + """ + --- + puppet-test: + gitlab: + base_url: https://gitlab.example.com + """ + When I run `msync exec --message "Add testfile in test" --pr -- touch test.yaml` + Then the stderr should contain "A token is required to use services from gitlab" + And the exit status should be 1 + And the puppet module "puppet-test" from "fakenamespace" should have no commits made by "Aruba" + + Scenario: Ask for PR/MR with modules from GitHub and from GitLab + Given a basic setup with a puppet module "puppet-github" from "fakenamespace" + And a basic setup with a puppet module "puppet-gitlab" from "fakenamespace" + And a file named "managed_modules.yml" with: + """ + --- + puppet-github: + github: + base_url: https://github.example.com + token: 'secret' + puppet-gitlab: + gitlab: + base_url: https://gitlab.example.com + token: 'secret' + """ + When I successfully run `msync exec --noop --branch managed_update --pr -- touch test.yaml` + Then the output should contain "Would submit PR " + And the output should contain "Would submit MR " + And the puppet module "puppet-github" from "fakenamespace" should have no commits made by "Aruba" + And the puppet module "puppet-gitlab" from "fakenamespace" should have no commits made by "Aruba" + + Scenario: Ask for PR with same source and target branch + Given a basic setup with a puppet module "puppet-test" from "fakenamespace" + And a file named "managed_modules.yml" with: + """ + --- + puppet-test: + gitlab: + token: 'secret' + base_url: 'https://gitlab.example.com' + """ + When I run `msync exec --noop --branch managed_update --pr --pr-target-branch managed_update -- touch test.yaml` + Then the stderr should contain "Unable to open a pull request with the same source and target branch: 'managed_update'" + And the exit status should be 1 + And the puppet module "puppet-test" from "fakenamespace" should have no commits made by "Aruba" + + Scenario: Ask for PR with the default branch as source and target + Given a basic setup with a puppet module "puppet-test" from "fakenamespace" + And the puppet module "puppet-test" from "fakenamespace" has the default branch named "custom_default_branch" + And a file named "managed_modules.yml" with: + """ + --- + puppet-test: + github: + token: 'secret' + base_url: 'https://gitlab.example.com' + """ + And a directory named "moduleroot" + When I run `msync exec --noop --pr -- touch test.yaml` + Then the stderr should contain "Unable to open a pull request with the same source and target branch: 'custom_default_branch'" + And the exit status should be 1 + And the puppet module "puppet-test" from "fakenamespace" should have no commits made by "Aruba" diff --git a/lib/modulesync.rb b/lib/modulesync.rb index 99fac0a5..6ea7b351 100644 --- a/lib/modulesync.rb +++ b/lib/modulesync.rb @@ -159,6 +159,36 @@ def self.manage_module(puppet_module, module_files, defaults) end end + def self.execute_in_module(puppet_module, options) + puts "Executing in '#{puppet_module.given_name}'" + puppet_module.repository.prepare_workspace(branch: options[:branch], operate_offline: false, rebase: options[:rebase]) + + command_args = options[:command_args].dup + env_whitelist = (options[:env] || '').split(',').map(&:strip) + command_args[0] = File.expand_path command_args[0] if File.exist?(File.expand_path(command_args[0])) + + # Remove bundler-related env vars to allow the subprocess to run `bundle` unless explicitly whitelisted. + command_env = ENV.reject { |k, _v| k.match?(/(^BUNDLE|^SOURCE_DATE_EPOCH$|^GEM_|RUBY)/) && !env_whitelist.include?(k) } + + success = system(command_env, *command_args, unsetenv_others: true, chdir: puppet_module.working_directory) + unless success + message = "Command execution failed ('#{command_args.join ' '}': #{$CHILD_STATUS})" + raise ModuleSync::Error, message + end + + if options[:noop] + puts "Using no-op. Files in '#{puppet_module.given_name}' may be changed but will not be committed." + changed = puppet_module.repository.show_changes(options) + changed && options[:pr] && puppet_module.open_pull_request + else + status = puppet_module.repository.git.status + changed_files = [status.changed, status.added, status.deleted, puppet_module.repository.untracked_unignored_files].flat_map(&:keys).uniq + pushed = puppet_module.repository.submit_changes(changed_files, options) + + options[:pr] && (pushed || puppet_module.pull_request_branch_ready?) && puppet_module.open_pull_request + end + end + def self.config_path(file, options) return file if Pathname.new(file).absolute? @@ -211,45 +241,25 @@ def self.clone(cli_options) def self.execute(cli_options) @options = config_defaults.merge(cli_options) - errors = {} - managed_modules.each do |puppet_module| - $stdout.puts "#{puppet_module.given_name}:" - - puppet_module.repository.clone unless puppet_module.repository.cloned? - if @options[:default_branch] - puppet_module.repository.switch branch: false - else - puppet_module.repository.switch branch: @options[:branch] - end - - command_args = cli_options[:command_args] - env_whitelist = (@options[:env] || '').split(',') - local_script = File.expand_path command_args[0] - command_args[0] = local_script if File.exist?(local_script) - - # Remove bundler-related env vars to allow the subprocess to run `bundle` unless explicitly whitelisted. - command_env = ENV.reject { |k, _v| k.match?(/(^BUNDLE|^SOURCE_DATE_EPOCH$|^GEM_|RUBY)/) && !env_whitelist.include?(k) } - - result = system command_env, *command_args, unsetenv_others: true, chdir: puppet_module.working_directory - unless result - message = "Command execution failed ('#{@options[:command_args].join ' '}': #{$CHILD_STATUS})" - raise Thor::Error, message if @options[:fail_fast] - - errors[puppet_module.given_name] = message - warn message - end + initialized_modules = managed_modules + initialized_modules.each(&:git_service) if options[:pr] - $stdout.puts '' - end + errors = false + initialized_modules.each do |puppet_module| + execute_in_module(puppet_module, options) + rescue ModuleSync::Error, Git::Error => e + message = e.message || 'Error during `execute`' + warn "#{puppet_module.given_name}: #{message}" + exit 1 unless options[:skip_broken] + errors = true + warn "Skipping '#{puppet_module.given_name}' as execute process failed" + rescue StandardError + raise unless options[:skip_broken] - unless errors.empty? - raise Thor::Error, <<~MSG - Error(s) during `execute` command: - #{errors.map { |name, message| " * #{name}: #{message}" }.join "\n"} - MSG + errors = true + warn "Skipping '#{puppet_module.given_name}' as execute process failed" end - - exit 1 unless errors.empty? + exit 1 if errors end def self.reset(cli_options) diff --git a/lib/modulesync/cli.rb b/lib/modulesync/cli.rb index 4c28ffec..a51fb7ee 100644 --- a/lib/modulesync/cli.rb +++ b/lib/modulesync/cli.rb @@ -177,25 +177,67 @@ def update \x5 `msync exec true` will run "$PWD/true", not `/bin/true` if "$PWD/true" exists. DESC + option :message, + aliases: '-m', + desc: 'Commit message to apply to changed modules.', + default: CLI.defaults[:message] option :configs, aliases: '-c', desc: 'The local directory or remote repository to define the list of managed modules, ' \ 'the file templates, and the default values for template variables.' option :managed_modules_conf, desc: 'The file name to define the list of managed modules' + option :remote_branch, + aliases: '-r', + desc: 'Remote branch name to push the changes to. Defaults to the branch name.', + default: CLI.defaults[:remote_branch] + option :skip_broken, + type: :boolean, + aliases: '-s', + desc: 'Process remaining modules if an error is found', + default: false + option :amend, + type: :boolean, + desc: 'Amend previous commit', + default: false + option :sign, + type: :boolean, + desc: 'Cryptographically sign commits and tags', + default: false + option :signoff, + type: :boolean, + desc: 'Add a Signed-off-by trailer to commits', + default: false + option :force, + type: :boolean, + desc: 'Force push amended commit', + default: false + option :noop, + type: :boolean, + desc: 'No-op mode', + default: false + option :pr, + type: :boolean, + desc: 'Submit pull/merge request', + default: false + option :pr_title, + desc: 'Title of pull/merge request', + default: CLI.defaults[:pr_title] || 'Update to module template files' + option :pr_labels, + type: :array, + desc: 'Labels to add to the pull/merge request', + default: CLI.defaults[:pr_labels] || [] + option :pr_target_branch, + desc: 'Target branch for the pull/merge request', + default: CLI.defaults[:pr_target_branch] + option :rebase, + type: :boolean, + desc: 'Rebase the local branch onto the latest remote default branch before syncing.', + default: false option :branch, aliases: '-b', desc: 'Branch name to make the changes in.', default: CLI.defaults[:branch] - option :default_branch, - aliases: '-B', - type: :boolean, - desc: 'Work on the default branch (take precedence over --branch).', - default: false - option :fail_fast, - type: :boolean, - desc: 'Abort the run after a command execution failure', - default: CLI.defaults[:fail_fast].nil? || CLI.defaults[:fail_fast] option :env, aliases: '-e', desc: 'Comma-separated list of environment variables to preserve.'