You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

533 lines
23 KiB

Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
Sign merges, CRUD, Wiki and Repository initialisation with gpg key (#7631) This PR fixes #7598 by providing a configurable way of signing commits across the Gitea instance. Per repository configurability and import/generation of trusted secure keys is not provided by this PR - from a security PoV that's probably impossible to do properly. Similarly web-signing, that is asking the user to sign something, is not implemented - this could be done at a later stage however. ## Features - [x] If commit.gpgsign is set in .gitconfig sign commits and files created through repofiles. (merges should already have been signed.) - [x] Verify commits signed with the default gpg as valid - [x] Signer, Committer and Author can all be different - [x] Allow signer to be arbitrarily different - We still require the key to have an activated email on Gitea. A more complete implementation would be to use a keyserver and mark external-or-unactivated with an "unknown" trust level icon. - [x] Add a signing-key.gpg endpoint to get the default gpg pub key if available - Rather than add a fake web-flow user I've added this as an endpoint on /api/v1/signing-key.gpg - [x] Try to match the default key with a user on gitea - this is done at verification time - [x] Make things configurable? - app.ini configuration done - [x] when checking commits are signed need to check if they're actually verifiable too - [x] Add documentation I have decided that adjusting the docker to create a default gpg key is not the correct thing to do and therefore have not implemented this.
5 years ago
  1. // Copyright 2019 The Gitea Authors.
  2. // All rights reserved.
  3. // Use of this source code is governed by a MIT-style
  4. // license that can be found in the LICENSE file.
  5. package pull
  6. import (
  7. "bufio"
  8. "bytes"
  9. "fmt"
  10. "io/ioutil"
  11. "os"
  12. "path/filepath"
  13. "regexp"
  14. "strings"
  15. "time"
  16. "code.gitea.io/gitea/models"
  17. "code.gitea.io/gitea/modules/cache"
  18. "code.gitea.io/gitea/modules/git"
  19. "code.gitea.io/gitea/modules/log"
  20. "code.gitea.io/gitea/modules/notification"
  21. "code.gitea.io/gitea/modules/setting"
  22. "code.gitea.io/gitea/modules/timeutil"
  23. "github.com/mcuadros/go-version"
  24. )
  25. // Merge merges pull request to base repository.
  26. // FIXME: add repoWorkingPull make sure two merges does not happen at same time.
  27. func Merge(pr *models.PullRequest, doer *models.User, baseGitRepo *git.Repository, mergeStyle models.MergeStyle, message string) (err error) {
  28. binVersion, err := git.BinVersion()
  29. if err != nil {
  30. log.Error("git.BinVersion: %v", err)
  31. return fmt.Errorf("Unable to get git version: %v", err)
  32. }
  33. if err = pr.GetHeadRepo(); err != nil {
  34. log.Error("GetHeadRepo: %v", err)
  35. return fmt.Errorf("GetHeadRepo: %v", err)
  36. } else if err = pr.GetBaseRepo(); err != nil {
  37. log.Error("GetBaseRepo: %v", err)
  38. return fmt.Errorf("GetBaseRepo: %v", err)
  39. }
  40. prUnit, err := pr.BaseRepo.GetUnit(models.UnitTypePullRequests)
  41. if err != nil {
  42. log.Error("pr.BaseRepo.GetUnit(models.UnitTypePullRequests): %v", err)
  43. return err
  44. }
  45. prConfig := prUnit.PullRequestsConfig()
  46. if err := pr.CheckUserAllowedToMerge(doer); err != nil {
  47. log.Error("CheckUserAllowedToMerge(%v): %v", doer, err)
  48. return fmt.Errorf("CheckUserAllowedToMerge: %v", err)
  49. }
  50. // Check if merge style is correct and allowed
  51. if !prConfig.IsMergeStyleAllowed(mergeStyle) {
  52. return models.ErrInvalidMergeStyle{ID: pr.BaseRepo.ID, Style: mergeStyle}
  53. }
  54. defer func() {
  55. go AddTestPullRequestTask(doer, pr.BaseRepo.ID, pr.BaseBranch, false)
  56. }()
  57. // Clone base repo.
  58. tmpBasePath, err := models.CreateTemporaryPath("merge")
  59. if err != nil {
  60. log.Error("CreateTemporaryPath: %v", err)
  61. return err
  62. }
  63. defer func() {
  64. if err := models.RemoveTemporaryPath(tmpBasePath); err != nil {
  65. log.Error("Merge: RemoveTemporaryPath: %s", err)
  66. }
  67. }()
  68. headRepoPath := pr.HeadRepo.RepoPath()
  69. if err := git.InitRepository(tmpBasePath, false); err != nil {
  70. log.Error("git init tmpBasePath: %v", err)
  71. return err
  72. }
  73. remoteRepoName := "head_repo"
  74. baseBranch := "base"
  75. // Add head repo remote.
  76. addCacheRepo := func(staging, cache string) error {
  77. p := filepath.Join(staging, ".git", "objects", "info", "alternates")
  78. f, err := os.OpenFile(p, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
  79. if err != nil {
  80. log.Error("Could not create .git/objects/info/alternates file in %s: %v", staging, err)
  81. return err
  82. }
  83. defer f.Close()
  84. data := filepath.Join(cache, "objects")
  85. if _, err := fmt.Fprintln(f, data); err != nil {
  86. log.Error("Could not write to .git/objects/info/alternates file in %s: %v", staging, err)
  87. return err
  88. }
  89. return nil
  90. }
  91. if err := addCacheRepo(tmpBasePath, baseGitRepo.Path); err != nil {
  92. log.Error("Unable to add base repository to temporary repo [%s -> %s]: %v", pr.BaseRepo.FullName(), tmpBasePath, err)
  93. return fmt.Errorf("Unable to add base repository to temporary repo [%s -> tmpBasePath]: %v", pr.BaseRepo.FullName(), err)
  94. }
  95. var outbuf, errbuf strings.Builder
  96. if err := git.NewCommand("remote", "add", "-t", pr.BaseBranch, "-m", pr.BaseBranch, "origin", baseGitRepo.Path).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  97. log.Error("Unable to add base repository as origin [%s -> %s]: %v\n%s\n%s", pr.BaseRepo.FullName(), tmpBasePath, err, outbuf.String(), errbuf.String())
  98. return fmt.Errorf("Unable to add base repository as origin [%s -> tmpBasePath]: %v\n%s\n%s", pr.BaseRepo.FullName(), err, outbuf.String(), errbuf.String())
  99. }
  100. outbuf.Reset()
  101. errbuf.Reset()
  102. if err := git.NewCommand("fetch", "origin", "--no-tags", pr.BaseBranch+":"+baseBranch, pr.BaseBranch+":original_"+baseBranch).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  103. log.Error("Unable to fetch origin base branch [%s:%s -> base, original_base in %s]: %v:\n%s\n%s", pr.BaseRepo.FullName(), pr.BaseBranch, tmpBasePath, err, outbuf.String(), errbuf.String())
  104. return fmt.Errorf("Unable to fetch origin base branch [%s:%s -> base, original_base in tmpBasePath]: %v\n%s\n%s", pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  105. }
  106. outbuf.Reset()
  107. errbuf.Reset()
  108. if err := git.NewCommand("symbolic-ref", "HEAD", git.BranchPrefix+baseBranch).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  109. log.Error("Unable to set HEAD as base branch [%s]: %v\n%s\n%s", tmpBasePath, err, outbuf.String(), errbuf.String())
  110. return fmt.Errorf("Unable to set HEAD as base branch [tmpBasePath]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  111. }
  112. outbuf.Reset()
  113. errbuf.Reset()
  114. if err := addCacheRepo(tmpBasePath, headRepoPath); err != nil {
  115. log.Error("Unable to add head repository to temporary repo [%s -> %s]: %v", pr.HeadRepo.FullName(), tmpBasePath, err)
  116. return fmt.Errorf("Unable to head base repository to temporary repo [%s -> tmpBasePath]: %v", pr.HeadRepo.FullName(), err)
  117. }
  118. if err := git.NewCommand("remote", "add", remoteRepoName, headRepoPath).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  119. log.Error("Unable to add head repository as head_repo [%s -> %s]: %v\n%s\n%s", pr.HeadRepo.FullName(), tmpBasePath, err, outbuf.String(), errbuf.String())
  120. return fmt.Errorf("Unable to add head repository as head_repo [%s -> tmpBasePath]: %v\n%s\n%s", pr.HeadRepo.FullName(), err, outbuf.String(), errbuf.String())
  121. }
  122. outbuf.Reset()
  123. errbuf.Reset()
  124. trackingBranch := "tracking"
  125. // Fetch head branch
  126. if err := git.NewCommand("fetch", "--no-tags", remoteRepoName, pr.HeadBranch+":"+trackingBranch).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  127. log.Error("Unable to fetch head_repo head branch [%s:%s -> tracking in %s]: %v:\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, tmpBasePath, err, outbuf.String(), errbuf.String())
  128. return fmt.Errorf("Unable to fetch head_repo head branch [%s:%s -> tracking in tmpBasePath]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, err, outbuf.String(), errbuf.String())
  129. }
  130. outbuf.Reset()
  131. errbuf.Reset()
  132. stagingBranch := "staging"
  133. // Enable sparse-checkout
  134. sparseCheckoutList, err := getDiffTree(tmpBasePath, baseBranch, trackingBranch)
  135. if err != nil {
  136. log.Error("getDiffTree(%s, %s, %s): %v", tmpBasePath, baseBranch, trackingBranch, err)
  137. return fmt.Errorf("getDiffTree: %v", err)
  138. }
  139. infoPath := filepath.Join(tmpBasePath, ".git", "info")
  140. if err := os.MkdirAll(infoPath, 0700); err != nil {
  141. log.Error("Unable to create .git/info in %s: %v", tmpBasePath, err)
  142. return fmt.Errorf("Unable to create .git/info in tmpBasePath: %v", err)
  143. }
  144. sparseCheckoutListPath := filepath.Join(infoPath, "sparse-checkout")
  145. if err := ioutil.WriteFile(sparseCheckoutListPath, []byte(sparseCheckoutList), 0600); err != nil {
  146. log.Error("Unable to write .git/info/sparse-checkout file in %s: %v", tmpBasePath, err)
  147. return fmt.Errorf("Unable to write .git/info/sparse-checkout file in tmpBasePath: %v", err)
  148. }
  149. var gitConfigCommand func() *git.Command
  150. if version.Compare(binVersion, "1.8.0", ">=") {
  151. gitConfigCommand = func() *git.Command {
  152. return git.NewCommand("config", "--local")
  153. }
  154. } else {
  155. gitConfigCommand = func() *git.Command {
  156. return git.NewCommand("config")
  157. }
  158. }
  159. // Switch off LFS process (set required, clean and smudge here also)
  160. if err := gitConfigCommand().AddArguments("filter.lfs.process", "").RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  161. log.Error("git config [filter.lfs.process -> <> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  162. return fmt.Errorf("git config [filter.lfs.process -> <> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  163. }
  164. outbuf.Reset()
  165. errbuf.Reset()
  166. if err := gitConfigCommand().AddArguments("filter.lfs.required", "false").RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  167. log.Error("git config [filter.lfs.required -> <false> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  168. return fmt.Errorf("git config [filter.lfs.required -> <false> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  169. }
  170. outbuf.Reset()
  171. errbuf.Reset()
  172. if err := gitConfigCommand().AddArguments("filter.lfs.clean", "").RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  173. log.Error("git config [filter.lfs.clean -> <> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  174. return fmt.Errorf("git config [filter.lfs.clean -> <> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  175. }
  176. outbuf.Reset()
  177. errbuf.Reset()
  178. if err := gitConfigCommand().AddArguments("filter.lfs.smudge", "").RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  179. log.Error("git config [filter.lfs.smudge -> <> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  180. return fmt.Errorf("git config [filter.lfs.smudge -> <> ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  181. }
  182. outbuf.Reset()
  183. errbuf.Reset()
  184. if err := gitConfigCommand().AddArguments("core.sparseCheckout", "true").RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  185. log.Error("git config [core.sparseCheckout -> true ]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  186. return fmt.Errorf("git config [core.sparsecheckout -> true]: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  187. }
  188. outbuf.Reset()
  189. errbuf.Reset()
  190. // Read base branch index
  191. if err := git.NewCommand("read-tree", "HEAD").RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  192. log.Error("git read-tree HEAD: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  193. return fmt.Errorf("Unable to read base branch in to the index: %v\n%s\n%s", err, outbuf.String(), errbuf.String())
  194. }
  195. outbuf.Reset()
  196. errbuf.Reset()
  197. // Determine if we should sign
  198. signArg := ""
  199. if version.Compare(binVersion, "1.7.9", ">=") {
  200. sign, keyID := pr.BaseRepo.SignMerge(doer, tmpBasePath, "HEAD", trackingBranch)
  201. if sign {
  202. signArg = "-S" + keyID
  203. } else if version.Compare(binVersion, "2.0.0", ">=") {
  204. signArg = "--no-gpg-sign"
  205. }
  206. }
  207. sig := doer.NewGitSig()
  208. commitTimeStr := time.Now().Format(time.RFC3339)
  209. // Because this may call hooks we should pass in the environment
  210. env := append(os.Environ(),
  211. "GIT_AUTHOR_NAME="+sig.Name,
  212. "GIT_AUTHOR_EMAIL="+sig.Email,
  213. "GIT_AUTHOR_DATE="+commitTimeStr,
  214. "GIT_COMMITTER_NAME="+sig.Name,
  215. "GIT_COMMITTER_EMAIL="+sig.Email,
  216. "GIT_COMMITTER_DATE="+commitTimeStr,
  217. )
  218. // Merge commits.
  219. switch mergeStyle {
  220. case models.MergeStyleMerge:
  221. cmd := git.NewCommand("merge", "--no-ff", "--no-commit", trackingBranch)
  222. if err := runMergeCommand(pr, mergeStyle, cmd, tmpBasePath); err != nil {
  223. log.Error("Unable to merge tracking into base: %v", err)
  224. return err
  225. }
  226. if err := commitAndSignNoAuthor(pr, message, signArg, tmpBasePath, env); err != nil {
  227. log.Error("Unable to make final commit: %v", err)
  228. return err
  229. }
  230. case models.MergeStyleRebase:
  231. fallthrough
  232. case models.MergeStyleRebaseMerge:
  233. // Checkout head branch
  234. if err := git.NewCommand("checkout", "-b", stagingBranch, trackingBranch).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  235. log.Error("git checkout base prior to merge post staging rebase [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  236. return fmt.Errorf("git checkout base prior to merge post staging rebase [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  237. }
  238. outbuf.Reset()
  239. errbuf.Reset()
  240. // Rebase before merging
  241. if err := git.NewCommand("rebase", baseBranch).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  242. // Rebase will leave a REBASE_HEAD file in .git if there is a conflict
  243. if _, statErr := os.Stat(filepath.Join(tmpBasePath, ".git", "REBASE_HEAD")); statErr == nil {
  244. // The original commit SHA1 that is failing will be in .git/rebase-apply/original-commit
  245. commitShaBytes, readErr := ioutil.ReadFile(filepath.Join(tmpBasePath, ".git", "rebase-apply", "original-commit"))
  246. if readErr != nil {
  247. // Abandon this attempt to handle the error
  248. log.Error("git rebase staging on to base [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  249. return fmt.Errorf("git rebase staging on to base [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  250. }
  251. log.Debug("RebaseConflict at %s [%s:%s -> %s:%s]: %v\n%s\n%s", strings.TrimSpace(string(commitShaBytes)), pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  252. return models.ErrRebaseConflicts{
  253. Style: mergeStyle,
  254. CommitSHA: strings.TrimSpace(string(commitShaBytes)),
  255. StdOut: outbuf.String(),
  256. StdErr: errbuf.String(),
  257. Err: err,
  258. }
  259. }
  260. log.Error("git rebase staging on to base [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  261. return fmt.Errorf("git rebase staging on to base [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  262. }
  263. outbuf.Reset()
  264. errbuf.Reset()
  265. // Checkout base branch again
  266. if err := git.NewCommand("checkout", baseBranch).RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  267. log.Error("git checkout base prior to merge post staging rebase [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  268. return fmt.Errorf("git checkout base prior to merge post staging rebase [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  269. }
  270. outbuf.Reset()
  271. errbuf.Reset()
  272. cmd := git.NewCommand("merge")
  273. if mergeStyle == models.MergeStyleRebase {
  274. cmd.AddArguments("--ff-only")
  275. } else {
  276. cmd.AddArguments("--no-ff", "--no-commit")
  277. }
  278. cmd.AddArguments(stagingBranch)
  279. // Prepare merge with commit
  280. if err := runMergeCommand(pr, mergeStyle, cmd, tmpBasePath); err != nil {
  281. log.Error("Unable to merge staging into base: %v", err)
  282. return err
  283. }
  284. if mergeStyle == models.MergeStyleRebaseMerge {
  285. if err := commitAndSignNoAuthor(pr, message, signArg, tmpBasePath, env); err != nil {
  286. log.Error("Unable to make final commit: %v", err)
  287. return err
  288. }
  289. }
  290. case models.MergeStyleSquash:
  291. // Merge with squash
  292. cmd := git.NewCommand("merge", "--squash", trackingBranch)
  293. if err := runMergeCommand(pr, mergeStyle, cmd, tmpBasePath); err != nil {
  294. log.Error("Unable to merge --squash tracking into base: %v", err)
  295. return err
  296. }
  297. sig := pr.Issue.Poster.NewGitSig()
  298. if signArg == "" {
  299. if err := git.NewCommand("commit", fmt.Sprintf("--author='%s <%s>'", sig.Name, sig.Email), "-m", message).RunInDirTimeoutEnvPipeline(env, -1, tmpBasePath, &outbuf, &errbuf); err != nil {
  300. log.Error("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  301. return fmt.Errorf("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  302. }
  303. } else {
  304. if err := git.NewCommand("commit", signArg, fmt.Sprintf("--author='%s <%s>'", sig.Name, sig.Email), "-m", message).RunInDirTimeoutEnvPipeline(env, -1, tmpBasePath, &outbuf, &errbuf); err != nil {
  305. log.Error("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  306. return fmt.Errorf("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  307. }
  308. }
  309. outbuf.Reset()
  310. errbuf.Reset()
  311. default:
  312. return models.ErrInvalidMergeStyle{ID: pr.BaseRepo.ID, Style: mergeStyle}
  313. }
  314. // OK we should cache our current head and origin/headbranch
  315. mergeHeadSHA, err := git.GetFullCommitID(tmpBasePath, "HEAD")
  316. if err != nil {
  317. return fmt.Errorf("Failed to get full commit id for HEAD: %v", err)
  318. }
  319. mergeBaseSHA, err := git.GetFullCommitID(tmpBasePath, "original_"+baseBranch)
  320. if err != nil {
  321. return fmt.Errorf("Failed to get full commit id for origin/%s: %v", pr.BaseBranch, err)
  322. }
  323. // Now it's questionable about where this should go - either after or before the push
  324. // I think in the interests of data safety - failures to push to the lfs should prevent
  325. // the merge as you can always remerge.
  326. if setting.LFS.StartServer {
  327. if err := LFSPush(tmpBasePath, mergeHeadSHA, mergeBaseSHA, pr); err != nil {
  328. return err
  329. }
  330. }
  331. var headUser *models.User
  332. err = pr.HeadRepo.GetOwner()
  333. if err != nil {
  334. if !models.IsErrUserNotExist(err) {
  335. log.Error("Can't find user: %d for head repository - %v", pr.HeadRepo.OwnerID, err)
  336. return err
  337. }
  338. log.Error("Can't find user: %d for head repository - defaulting to doer: %s - %v", pr.HeadRepo.OwnerID, doer.Name, err)
  339. headUser = doer
  340. } else {
  341. headUser = pr.HeadRepo.Owner
  342. }
  343. env = models.FullPushingEnvironment(
  344. headUser,
  345. doer,
  346. pr.BaseRepo,
  347. pr.BaseRepo.Name,
  348. pr.ID,
  349. )
  350. // Push back to upstream.
  351. if err := git.NewCommand("push", "origin", baseBranch+":"+pr.BaseBranch).RunInDirTimeoutEnvPipeline(env, -1, tmpBasePath, &outbuf, &errbuf); err != nil {
  352. if strings.Contains(errbuf.String(), "non-fast-forward") {
  353. return models.ErrMergePushOutOfDate{
  354. Style: mergeStyle,
  355. StdOut: outbuf.String(),
  356. StdErr: errbuf.String(),
  357. Err: err,
  358. }
  359. }
  360. return fmt.Errorf("git push: %s", errbuf.String())
  361. }
  362. outbuf.Reset()
  363. errbuf.Reset()
  364. pr.MergedCommitID, err = baseGitRepo.GetBranchCommitID(pr.BaseBranch)
  365. if err != nil {
  366. return fmt.Errorf("GetBranchCommit: %v", err)
  367. }
  368. pr.MergedUnix = timeutil.TimeStampNow()
  369. pr.Merger = doer
  370. pr.MergerID = doer.ID
  371. if err = pr.SetMerged(); err != nil {
  372. log.Error("setMerged [%d]: %v", pr.ID, err)
  373. }
  374. if err = models.MergePullRequestAction(doer, pr.Issue.Repo, pr.Issue); err != nil {
  375. log.Error("MergePullRequestAction [%d]: %v", pr.ID, err)
  376. }
  377. // Reset cached commit count
  378. cache.Remove(pr.Issue.Repo.GetCommitsCountCacheKey(pr.BaseBranch, true))
  379. // Reload pull request information.
  380. if err = pr.LoadAttributes(); err != nil {
  381. log.Error("LoadAttributes: %v", err)
  382. return nil
  383. }
  384. notification.NotifyIssueChangeStatus(doer, pr.Issue, true)
  385. return nil
  386. }
  387. func commitAndSignNoAuthor(pr *models.PullRequest, message, signArg, tmpBasePath string, env []string) error {
  388. var outbuf, errbuf strings.Builder
  389. if signArg == "" {
  390. if err := git.NewCommand("commit", "-m", message).RunInDirTimeoutEnvPipeline(env, -1, tmpBasePath, &outbuf, &errbuf); err != nil {
  391. log.Error("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  392. return fmt.Errorf("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  393. }
  394. } else {
  395. if err := git.NewCommand("commit", signArg, "-m", message).RunInDirTimeoutEnvPipeline(env, -1, tmpBasePath, &outbuf, &errbuf); err != nil {
  396. log.Error("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  397. return fmt.Errorf("git commit [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  398. }
  399. }
  400. return nil
  401. }
  402. func runMergeCommand(pr *models.PullRequest, mergeStyle models.MergeStyle, cmd *git.Command, tmpBasePath string) error {
  403. var outbuf, errbuf strings.Builder
  404. if err := cmd.RunInDirPipeline(tmpBasePath, &outbuf, &errbuf); err != nil {
  405. // Merge will leave a MERGE_HEAD file in the .git folder if there is a conflict
  406. if _, statErr := os.Stat(filepath.Join(tmpBasePath, ".git", "MERGE_HEAD")); statErr == nil {
  407. // We have a merge conflict error
  408. log.Debug("MergeConflict [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  409. return models.ErrMergeConflicts{
  410. Style: mergeStyle,
  411. StdOut: outbuf.String(),
  412. StdErr: errbuf.String(),
  413. Err: err,
  414. }
  415. } else if strings.Contains(errbuf.String(), "refusing to merge unrelated histories") {
  416. log.Debug("MergeUnrelatedHistories [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  417. return models.ErrMergeUnrelatedHistories{
  418. Style: mergeStyle,
  419. StdOut: outbuf.String(),
  420. StdErr: errbuf.String(),
  421. Err: err,
  422. }
  423. }
  424. log.Error("git merge [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  425. return fmt.Errorf("git merge [%s:%s -> %s:%s]: %v\n%s\n%s", pr.HeadRepo.FullName(), pr.HeadBranch, pr.BaseRepo.FullName(), pr.BaseBranch, err, outbuf.String(), errbuf.String())
  426. }
  427. return nil
  428. }
  429. var escapedSymbols = regexp.MustCompile(`([*[?! \\])`)
  430. func getDiffTree(repoPath, baseBranch, headBranch string) (string, error) {
  431. getDiffTreeFromBranch := func(repoPath, baseBranch, headBranch string) (string, error) {
  432. var outbuf, errbuf strings.Builder
  433. // Compute the diff-tree for sparse-checkout
  434. if err := git.NewCommand("diff-tree", "--no-commit-id", "--name-only", "-r", "-z", "--root", baseBranch, headBranch, "--").RunInDirPipeline(repoPath, &outbuf, &errbuf); err != nil {
  435. return "", fmt.Errorf("git diff-tree [%s base:%s head:%s]: %s", repoPath, baseBranch, headBranch, errbuf.String())
  436. }
  437. return outbuf.String(), nil
  438. }
  439. scanNullTerminatedStrings := func(data []byte, atEOF bool) (advance int, token []byte, err error) {
  440. if atEOF && len(data) == 0 {
  441. return 0, nil, nil
  442. }
  443. if i := bytes.IndexByte(data, '\x00'); i >= 0 {
  444. return i + 1, data[0:i], nil
  445. }
  446. if atEOF {
  447. return len(data), data, nil
  448. }
  449. return 0, nil, nil
  450. }
  451. list, err := getDiffTreeFromBranch(repoPath, baseBranch, headBranch)
  452. if err != nil {
  453. return "", err
  454. }
  455. // Prefixing '/' for each entry, otherwise all files with the same name in subdirectories would be matched.
  456. out := bytes.Buffer{}
  457. scanner := bufio.NewScanner(strings.NewReader(list))
  458. scanner.Split(scanNullTerminatedStrings)
  459. for scanner.Scan() {
  460. filepath := scanner.Text()
  461. // escape '*', '?', '[', spaces and '!' prefix
  462. filepath = escapedSymbols.ReplaceAllString(filepath, `\$1`)
  463. // no necessary to escape the first '#' symbol because the first symbol is '/'
  464. fmt.Fprintf(&out, "/%s\n", filepath)
  465. }
  466. return out.String(), nil
  467. }