You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

301 lines
8.9 KiB

API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
Better logging (#6038) (#6095) * Panic don't fatal on create new logger Fixes #5854 Signed-off-by: Andrew Thornton <art27@cantab.net> * partial broken * Update the logging infrastrcture Signed-off-by: Andrew Thornton <art27@cantab.net> * Reset the skip levels for Fatal and Error Signed-off-by: Andrew Thornton <art27@cantab.net> * broken ncsa * More log.Error fixes Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove nal * set log-levels to lowercase * Make console_test test all levels * switch to lowercased levels * OK now working * Fix vetting issues * Fix lint * Fix tests * change default logging to match current gitea * Improve log testing Signed-off-by: Andrew Thornton <art27@cantab.net> * reset error skip levels to 0 * Update documentation and access logger configuration * Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE * Fix broken level caching * Refactor the router log * Add Router logger * Add colorizing options * Adjust router colors * Only create logger if they will be used * update app.ini.sample * rename Attribute ColorAttribute * Change from white to green for function * Set fatal/error levels * Restore initial trace logger * Fix Trace arguments in modules/auth/auth.go * Properly handle XORMLogger * Improve admin/config page * fix fmt * Add auto-compression of old logs * Update error log levels * Remove the unnecessary skip argument from Error, Fatal and Critical * Add stacktrace support * Fix tests * Remove x/sync from vendors? * Add stderr option to console logger * Use filepath.ToSlash to protect against Windows in tests * Remove prefixed underscores from names in colors.go * Remove not implemented database logger This was removed from Gogs on 4 Mar 2016 but left in the configuration since then. * Ensure that log paths are relative to ROOT_PATH * use path.Join * rename jsonConfig to logConfig * Rename "config" to "jsonConfig" to make it clearer * Requested changes * Requested changes: XormLogger * Try to color the windows terminal If successful default to colorizing the console logs * fixup * Colorize initially too * update vendor * Colorize logs on default and remove if this is not a colorizing logger * Fix documentation * fix test * Use go-isatty to detect if on windows we are on msys or cygwin * Fix spelling mistake * Add missing vendors * More changes * Rationalise the ANSI writer protection * Adjust colors on advice from @0x5c * Make Flags a comma separated list * Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING * Ensure matching is done on the non-colored message - to simpify EXPRESSION
5 years ago
  1. // Copyright 2017 Gitea. All rights reserved.
  2. // Use of this source code is governed by a MIT-style
  3. // license that can be found in the LICENSE file.
  4. package models
  5. import (
  6. "container/list"
  7. "crypto/sha1"
  8. "fmt"
  9. "strings"
  10. "time"
  11. "code.gitea.io/gitea/modules/log"
  12. "code.gitea.io/gitea/modules/setting"
  13. api "code.gitea.io/gitea/modules/structs"
  14. "code.gitea.io/gitea/modules/timeutil"
  15. "xorm.io/xorm"
  16. )
  17. // CommitStatus holds a single Status of a single Commit
  18. type CommitStatus struct {
  19. ID int64 `xorm:"pk autoincr"`
  20. Index int64 `xorm:"INDEX UNIQUE(repo_sha_index)"`
  21. RepoID int64 `xorm:"INDEX UNIQUE(repo_sha_index)"`
  22. Repo *Repository `xorm:"-"`
  23. State api.CommitStatusState `xorm:"VARCHAR(7) NOT NULL"`
  24. SHA string `xorm:"VARCHAR(64) NOT NULL INDEX UNIQUE(repo_sha_index)"`
  25. TargetURL string `xorm:"TEXT"`
  26. Description string `xorm:"TEXT"`
  27. ContextHash string `xorm:"char(40) index"`
  28. Context string `xorm:"TEXT"`
  29. Creator *User `xorm:"-"`
  30. CreatorID int64
  31. CreatedUnix timeutil.TimeStamp `xorm:"INDEX created"`
  32. UpdatedUnix timeutil.TimeStamp `xorm:"INDEX updated"`
  33. }
  34. func (status *CommitStatus) loadRepo(e Engine) (err error) {
  35. if status.Repo == nil {
  36. status.Repo, err = getRepositoryByID(e, status.RepoID)
  37. if err != nil {
  38. return fmt.Errorf("getRepositoryByID [%d]: %v", status.RepoID, err)
  39. }
  40. }
  41. if status.Creator == nil && status.CreatorID > 0 {
  42. status.Creator, err = getUserByID(e, status.CreatorID)
  43. if err != nil {
  44. return fmt.Errorf("getUserByID [%d]: %v", status.CreatorID, err)
  45. }
  46. }
  47. return nil
  48. }
  49. // APIURL returns the absolute APIURL to this commit-status.
  50. func (status *CommitStatus) APIURL() string {
  51. _ = status.loadRepo(x)
  52. return fmt.Sprintf("%sapi/v1/repos/%s/statuses/%s",
  53. setting.AppURL, status.Repo.FullName(), status.SHA)
  54. }
  55. // APIFormat assumes some fields assigned with values:
  56. // Required - Repo, Creator
  57. func (status *CommitStatus) APIFormat() *api.Status {
  58. _ = status.loadRepo(x)
  59. apiStatus := &api.Status{
  60. Created: status.CreatedUnix.AsTime(),
  61. Updated: status.CreatedUnix.AsTime(),
  62. State: api.StatusState(status.State),
  63. TargetURL: status.TargetURL,
  64. Description: status.Description,
  65. ID: status.Index,
  66. URL: status.APIURL(),
  67. Context: status.Context,
  68. }
  69. if status.Creator != nil {
  70. apiStatus.Creator = status.Creator.APIFormat()
  71. }
  72. return apiStatus
  73. }
  74. // CalcCommitStatus returns commit status state via some status, the commit statues should order by id desc
  75. func CalcCommitStatus(statuses []*CommitStatus) *CommitStatus {
  76. var lastStatus *CommitStatus
  77. var state api.CommitStatusState
  78. for _, status := range statuses {
  79. if status.State.NoBetterThan(state) {
  80. state = status.State
  81. lastStatus = status
  82. }
  83. }
  84. if lastStatus == nil {
  85. if len(statuses) > 0 {
  86. lastStatus = statuses[0]
  87. } else {
  88. lastStatus = &CommitStatus{}
  89. }
  90. }
  91. return lastStatus
  92. }
  93. // CommitStatusOptions holds the options for query commit statuses
  94. type CommitStatusOptions struct {
  95. ListOptions
  96. State string
  97. SortType string
  98. }
  99. // GetCommitStatuses returns all statuses for a given commit.
  100. func GetCommitStatuses(repo *Repository, sha string, opts *CommitStatusOptions) ([]*CommitStatus, int64, error) {
  101. if opts.Page <= 0 {
  102. opts.Page = 1
  103. }
  104. if opts.PageSize <= 0 {
  105. opts.Page = ItemsPerPage
  106. }
  107. countSession := listCommitStatusesStatement(repo, sha, opts)
  108. countSession = opts.setSessionPagination(countSession)
  109. maxResults, err := countSession.Count(new(CommitStatus))
  110. if err != nil {
  111. log.Error("Count PRs: %v", err)
  112. return nil, maxResults, err
  113. }
  114. statuses := make([]*CommitStatus, 0, opts.PageSize)
  115. findSession := listCommitStatusesStatement(repo, sha, opts)
  116. findSession = opts.setSessionPagination(findSession)
  117. sortCommitStatusesSession(findSession, opts.SortType)
  118. return statuses, maxResults, findSession.Find(&statuses)
  119. }
  120. func listCommitStatusesStatement(repo *Repository, sha string, opts *CommitStatusOptions) *xorm.Session {
  121. sess := x.Where("repo_id = ?", repo.ID).And("sha = ?", sha)
  122. switch opts.State {
  123. case "pending", "success", "error", "failure", "warning":
  124. sess.And("state = ?", opts.State)
  125. }
  126. return sess
  127. }
  128. func sortCommitStatusesSession(sess *xorm.Session, sortType string) {
  129. switch sortType {
  130. case "oldest":
  131. sess.Asc("created_unix")
  132. case "recentupdate":
  133. sess.Desc("updated_unix")
  134. case "leastupdate":
  135. sess.Asc("updated_unix")
  136. case "leastindex":
  137. sess.Desc("index")
  138. case "highestindex":
  139. sess.Asc("index")
  140. default:
  141. sess.Desc("created_unix")
  142. }
  143. }
  144. // GetLatestCommitStatus returns all statuses with a unique context for a given commit.
  145. func GetLatestCommitStatus(repo *Repository, sha string, page int) ([]*CommitStatus, error) {
  146. ids := make([]int64, 0, 10)
  147. err := x.Limit(10, page*10).
  148. Table(&CommitStatus{}).
  149. Where("repo_id = ?", repo.ID).And("sha = ?", sha).
  150. Select("max( id ) as id").
  151. GroupBy("context_hash").OrderBy("max( id ) desc").Find(&ids)
  152. if err != nil {
  153. return nil, err
  154. }
  155. statuses := make([]*CommitStatus, 0, len(ids))
  156. if len(ids) == 0 {
  157. return statuses, nil
  158. }
  159. return statuses, x.In("id", ids).Find(&statuses)
  160. }
  161. // FindRepoRecentCommitStatusContexts returns repository's recent commit status contexts
  162. func FindRepoRecentCommitStatusContexts(repoID int64, before time.Duration) ([]string, error) {
  163. start := timeutil.TimeStampNow().AddDuration(-before)
  164. ids := make([]int64, 0, 10)
  165. if err := x.Table("commit_status").
  166. Where("repo_id = ?", repoID).
  167. And("updated_unix >= ?", start).
  168. Select("max( id ) as id").
  169. GroupBy("context_hash").OrderBy("max( id ) desc").
  170. Find(&ids); err != nil {
  171. return nil, err
  172. }
  173. var contexts = make([]string, 0, len(ids))
  174. if len(ids) == 0 {
  175. return contexts, nil
  176. }
  177. return contexts, x.Select("context").Table("commit_status").In("id", ids).Find(&contexts)
  178. }
  179. // NewCommitStatusOptions holds options for creating a CommitStatus
  180. type NewCommitStatusOptions struct {
  181. Repo *Repository
  182. Creator *User
  183. SHA string
  184. CommitStatus *CommitStatus
  185. }
  186. // NewCommitStatus save commit statuses into database
  187. func NewCommitStatus(opts NewCommitStatusOptions) error {
  188. if opts.Repo == nil {
  189. return fmt.Errorf("NewCommitStatus[nil, %s]: no repository specified", opts.SHA)
  190. }
  191. repoPath := opts.Repo.RepoPath()
  192. if opts.Creator == nil {
  193. return fmt.Errorf("NewCommitStatus[%s, %s]: no user specified", repoPath, opts.SHA)
  194. }
  195. sess := x.NewSession()
  196. defer sess.Close()
  197. if err := sess.Begin(); err != nil {
  198. return fmt.Errorf("NewCommitStatus[repo_id: %d, user_id: %d, sha: %s]: %v", opts.Repo.ID, opts.Creator.ID, opts.SHA, err)
  199. }
  200. opts.CommitStatus.Description = strings.TrimSpace(opts.CommitStatus.Description)
  201. opts.CommitStatus.Context = strings.TrimSpace(opts.CommitStatus.Context)
  202. opts.CommitStatus.TargetURL = strings.TrimSpace(opts.CommitStatus.TargetURL)
  203. opts.CommitStatus.SHA = opts.SHA
  204. opts.CommitStatus.CreatorID = opts.Creator.ID
  205. opts.CommitStatus.RepoID = opts.Repo.ID
  206. // Get the next Status Index
  207. var nextIndex int64
  208. lastCommitStatus := &CommitStatus{
  209. SHA: opts.SHA,
  210. RepoID: opts.Repo.ID,
  211. }
  212. has, err := sess.Desc("index").Limit(1).Get(lastCommitStatus)
  213. if err != nil {
  214. if err := sess.Rollback(); err != nil {
  215. log.Error("NewCommitStatus: sess.Rollback: %v", err)
  216. }
  217. return fmt.Errorf("NewCommitStatus[%s, %s]: %v", repoPath, opts.SHA, err)
  218. }
  219. if has {
  220. log.Debug("NewCommitStatus[%s, %s]: found", repoPath, opts.SHA)
  221. nextIndex = lastCommitStatus.Index
  222. }
  223. opts.CommitStatus.Index = nextIndex + 1
  224. log.Debug("NewCommitStatus[%s, %s]: %d", repoPath, opts.SHA, opts.CommitStatus.Index)
  225. opts.CommitStatus.ContextHash = hashCommitStatusContext(opts.CommitStatus.Context)
  226. // Insert new CommitStatus
  227. if _, err = sess.Insert(opts.CommitStatus); err != nil {
  228. if err := sess.Rollback(); err != nil {
  229. log.Error("Insert CommitStatus: sess.Rollback: %v", err)
  230. }
  231. return fmt.Errorf("Insert CommitStatus[%s, %s]: %v", repoPath, opts.SHA, err)
  232. }
  233. return sess.Commit()
  234. }
  235. // SignCommitWithStatuses represents a commit with validation of signature and status state.
  236. type SignCommitWithStatuses struct {
  237. Status *CommitStatus
  238. *SignCommit
  239. }
  240. // ParseCommitsWithStatus checks commits latest statuses and calculates its worst status state
  241. func ParseCommitsWithStatus(oldCommits *list.List, repo *Repository) *list.List {
  242. var (
  243. newCommits = list.New()
  244. e = oldCommits.Front()
  245. )
  246. for e != nil {
  247. c := e.Value.(SignCommit)
  248. commit := SignCommitWithStatuses{
  249. SignCommit: &c,
  250. }
  251. statuses, err := GetLatestCommitStatus(repo, commit.ID.String(), 0)
  252. if err != nil {
  253. log.Error("GetLatestCommitStatus: %v", err)
  254. } else {
  255. commit.Status = CalcCommitStatus(statuses)
  256. }
  257. newCommits.PushBack(commit)
  258. e = e.Next()
  259. }
  260. return newCommits
  261. }
  262. // hashCommitStatusContext hash context
  263. func hashCommitStatusContext(context string) string {
  264. return fmt.Sprintf("%x", sha1.Sum([]byte(context)))
  265. }