You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

165 lines
6.3 KiB

Graceful Queues: Issue Indexing and Tasks (#9363) * Queue: Add generic graceful queues with settings * Queue & Setting: Add worker pool implementation * Queue: Add worker settings * Queue: Make resizing worker pools * Queue: Add name variable to queues * Queue: Add monitoring * Queue: Improve logging * Issues: Gracefulise the issues indexer Remove the old now unused specific queues * Task: Move to generic queue and gracefulise * Issues: Standardise the issues indexer queue settings * Fix test * Queue: Allow Redis to connect to unix * Prevent deadlock during early shutdown of issue indexer * Add MaxWorker settings to queues * Merge branch 'master' into graceful-queues * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_disk.go * Update modules/queue/queue_disk_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Rename queue.Description to queue.ManagedQueue as per @guillep2k * Cancel pool workers when removed * Remove dependency on queue from setting * Update modules/queue/queue_redis.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * As per @guillep2k add mutex locks on shutdown/terminate * move unlocking out of setInternal * Add warning if number of workers < 0 * Small changes as per @guillep2k * No redis host specified not found * Clean up documentation for queues * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md * Update modules/indexer/issues/indexer_test.go * Ensure that persistable channel queue is added to manager * Rename QUEUE_NAME REDIS_QUEUE_NAME * Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME" This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df. Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
4 years ago
Graceful Queues: Issue Indexing and Tasks (#9363) * Queue: Add generic graceful queues with settings * Queue & Setting: Add worker pool implementation * Queue: Add worker settings * Queue: Make resizing worker pools * Queue: Add name variable to queues * Queue: Add monitoring * Queue: Improve logging * Issues: Gracefulise the issues indexer Remove the old now unused specific queues * Task: Move to generic queue and gracefulise * Issues: Standardise the issues indexer queue settings * Fix test * Queue: Allow Redis to connect to unix * Prevent deadlock during early shutdown of issue indexer * Add MaxWorker settings to queues * Merge branch 'master' into graceful-queues * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_disk.go * Update modules/queue/queue_disk_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Rename queue.Description to queue.ManagedQueue as per @guillep2k * Cancel pool workers when removed * Remove dependency on queue from setting * Update modules/queue/queue_redis.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * As per @guillep2k add mutex locks on shutdown/terminate * move unlocking out of setInternal * Add warning if number of workers < 0 * Small changes as per @guillep2k * No redis host specified not found * Clean up documentation for queues * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md * Update modules/indexer/issues/indexer_test.go * Ensure that persistable channel queue is added to manager * Rename QUEUE_NAME REDIS_QUEUE_NAME * Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME" This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df. Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
4 years ago
Graceful Queues: Issue Indexing and Tasks (#9363) * Queue: Add generic graceful queues with settings * Queue & Setting: Add worker pool implementation * Queue: Add worker settings * Queue: Make resizing worker pools * Queue: Add name variable to queues * Queue: Add monitoring * Queue: Improve logging * Issues: Gracefulise the issues indexer Remove the old now unused specific queues * Task: Move to generic queue and gracefulise * Issues: Standardise the issues indexer queue settings * Fix test * Queue: Allow Redis to connect to unix * Prevent deadlock during early shutdown of issue indexer * Add MaxWorker settings to queues * Merge branch 'master' into graceful-queues * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/indexer/issues/indexer.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Update modules/queue/queue_disk.go * Update modules/queue/queue_disk_channel.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * Rename queue.Description to queue.ManagedQueue as per @guillep2k * Cancel pool workers when removed * Remove dependency on queue from setting * Update modules/queue/queue_redis.go Co-Authored-By: guillep2k <18600385+guillep2k@users.noreply.github.com> * As per @guillep2k add mutex locks on shutdown/terminate * move unlocking out of setInternal * Add warning if number of workers < 0 * Small changes as per @guillep2k * No redis host specified not found * Clean up documentation for queues * Update docs/content/doc/advanced/config-cheat-sheet.en-us.md * Update modules/indexer/issues/indexer_test.go * Ensure that persistable channel queue is added to manager * Rename QUEUE_NAME REDIS_QUEUE_NAME * Revert "Rename QUEUE_NAME REDIS_QUEUE_NAME" This reverts commit 1f83b4fc9b9dabda186257b38c265fe7012f90df. Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
4 years ago
  1. {{template "base/head" .}}
  2. <div class="admin monitor">
  3. {{template "admin/navbar" .}}
  4. <div class="ui container">
  5. {{template "base/alert" .}}
  6. <h4 class="ui top attached header">
  7. {{.i18n.Tr "admin.monitor.queue" .Queue.Name}}
  8. </h4>
  9. <div class="ui attached table segment">
  10. <table class="ui very basic striped table">
  11. <thead>
  12. <tr>
  13. <th>{{.i18n.Tr "admin.monitor.queue.name"}}</th>
  14. <th>{{.i18n.Tr "admin.monitor.queue.type"}}</th>
  15. <th>{{.i18n.Tr "admin.monitor.queue.exemplar"}}</th>
  16. <th>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</th>
  17. <th>{{.i18n.Tr "admin.monitor.queue.maxnumberworkers"}}</th>
  18. </tr>
  19. </thead>
  20. <tbody>
  21. <tr>
  22. <td>{{.Queue.Name}}</td>
  23. <td>{{.Queue.Type}}</td>
  24. <td>{{.Queue.ExemplarType}}</td>
  25. <td>{{$sum := .Queue.NumberOfWorkers}}{{if lt $sum 0}}-{{else}}{{$sum}}{{end}}</td>
  26. <td>{{if lt $sum 0}}-{{else}}{{.Queue.MaxNumberOfWorkers}}{{end}}</td>
  27. </tr>
  28. </tbody>
  29. </table>
  30. </div>
  31. {{if lt $sum 0 }}
  32. <h4 class="ui top attached header">
  33. {{.i18n.Tr "admin.monitor.queue.nopool.title"}}
  34. </h4>
  35. <div class="ui attached segment">
  36. {{if eq .Queue.Type "wrapped" }}
  37. <p>{{.i18n.Tr "admin.monitor.queue.wrapped.desc"}}</p>
  38. {{else if eq .Queue.Type "persistable-channel"}}
  39. <p>{{.i18n.Tr "admin.monitor.queue.persistable-channel.desc"}}</p>
  40. {{else}}
  41. <p>{{.i18n.Tr "admin.monitor.queue.nopool.desc"}}</p>
  42. {{end}}
  43. </div>
  44. {{else}}
  45. <h4 class="ui top attached header">
  46. {{.i18n.Tr "admin.monitor.queue.settings.title"}}
  47. </h4>
  48. <div class="ui attached segment">
  49. <p>{{.i18n.Tr "admin.monitor.queue.settings.desc"}}</p>
  50. <form method="POST" action="{{.Link}}/set">
  51. {{$.CsrfTokenHtml}}
  52. <div class="ui form">
  53. <div class="inline field">
  54. <label for="max-number">{{.i18n.Tr "admin.monitor.queue.settings.maxnumberworkers"}}</label>
  55. <input name="max-number" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.settings.maxnumberworkers.placeholder" .Queue.MaxNumberOfWorkers}}">
  56. </div>
  57. <div class="inline field">
  58. <label for="timeout">{{.i18n.Tr "admin.monitor.queue.settings.timeout"}}</label>
  59. <input name="timeout" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.settings.timeout.placeholder" .Queue.BoostTimeout }}">
  60. </div>
  61. <div class="inline field">
  62. <label for="number">{{.i18n.Tr "admin.monitor.queue.settings.numberworkers"}}</label>
  63. <input name="number" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.settings.numberworkers.placeholder" .Queue.BoostWorkers}}">
  64. </div>
  65. <div class="inline field">
  66. <label>{{.i18n.Tr "admin.monitor.queue.settings.blocktimeout"}}</label>
  67. <span>{{.i18n.Tr "admin.monitor.queue.settings.blocktimeout.value" .Queue.BlockTimeout}}</span>
  68. </div>
  69. <button class="ui submit button">{{.i18n.Tr "admin.monitor.queue.settings.submit"}}</button>
  70. </div>
  71. </form>
  72. </div>
  73. <h4 class="ui top attached header">
  74. {{.i18n.Tr "admin.monitor.queue.pool.addworkers.title"}}
  75. </h4>
  76. <div class="ui attached segment">
  77. <p>{{.i18n.Tr "admin.monitor.queue.pool.addworkers.desc"}}</p>
  78. <form method="POST" action="{{.Link}}/add">
  79. {{$.CsrfTokenHtml}}
  80. <div class="ui form">
  81. <div class="fields">
  82. <div class="field">
  83. <label>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</label>
  84. <input name="number" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.pool.addworkers.numberworkers.placeholder"}}">
  85. </div>
  86. <div class="field">
  87. <label>{{.i18n.Tr "admin.monitor.queue.pool.timeout"}}</label>
  88. <input name="timeout" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.pool.addworkers.timeout.placeholder"}}">
  89. </div>
  90. </div>
  91. <button class="ui submit button">{{.i18n.Tr "admin.monitor.queue.pool.addworkers.submit"}}</button>
  92. </div>
  93. </form>
  94. </div>
  95. <h4 class="ui top attached header">
  96. {{.i18n.Tr "admin.monitor.queue.pool.flush.title"}}
  97. </h4>
  98. <div class="ui attached segment">
  99. <p>{{.i18n.Tr "admin.monitor.queue.pool.flush.desc"}}</p>
  100. <form method="POST" action="{{.Link}}/flush">
  101. {{$.CsrfTokenHtml}}
  102. <div class="ui form">
  103. <div class="fields">
  104. <div class="field">
  105. <label>{{.i18n.Tr "admin.monitor.queue.pool.timeout"}}</label>
  106. <input name="timeout" type="text" placeholder="{{.i18n.Tr "admin.monitor.queue.pool.addworkers.timeout.placeholder"}}">
  107. </div>
  108. </div>
  109. <button class="ui submit button">{{.i18n.Tr "admin.monitor.queue.pool.flush.submit"}}</button>
  110. </div>
  111. </form>
  112. </div>
  113. <h4 class="ui top attached header">
  114. {{.i18n.Tr "admin.monitor.queue.pool.workers.title"}}
  115. </h4>
  116. <div class="ui attached table segment">
  117. <table class="ui very basic striped table">
  118. <thead>
  119. <tr>
  120. <th>{{.i18n.Tr "admin.monitor.queue.numberworkers"}}</th>
  121. <th>{{.i18n.Tr "admin.monitor.start"}}</th>
  122. <th>{{.i18n.Tr "admin.monitor.queue.pool.timeout"}}</th>
  123. <th></th>
  124. </tr>
  125. </thead>
  126. <tbody>
  127. {{range .Queue.Workers}}
  128. <tr>
  129. <td>{{.Workers}}{{if .IsFlusher}}<i class="icon sync-alternate" title="{{.i18n.Tr "admin.monitor.queue.flush"}}"></i>{{end}}</td>
  130. <td>{{DateFmtLong .Start}}</td>
  131. <td>{{if .HasTimeout}}{{DateFmtLong .Timeout}}{{else}}-{{end}}</td>
  132. <td>
  133. <a class="delete-button" href="" data-url="{{$.Link}}/cancel/{{.PID}}" data-id="{{.PID}}" data-name="{{.Workers}}"><i class="close icon text red" title="{{$.i18n.Tr "remove"}}"></i></a>
  134. </td>
  135. </tr>
  136. {{else}}
  137. <tr>
  138. <td colspan="4">{{.i18n.Tr "admin.monitor.queue.pool.workers.none" }}
  139. </tr>
  140. {{end}}
  141. </tbody>
  142. </table>
  143. </div>
  144. {{end}}
  145. <h4 class="ui top attached header">
  146. {{.i18n.Tr "admin.monitor.queue.configuration"}}
  147. </h4>
  148. <div class="ui attached segment">
  149. <pre>{{.Queue.Configuration | JsonPrettyPrint}}
  150. </div>
  151. </div>
  152. </div>
  153. <div class="ui small basic delete modal">
  154. <div class="ui icon header">
  155. <i class="close icon"></i>
  156. {{.i18n.Tr "admin.monitor.queue.pool.cancel"}}
  157. </div>
  158. <div class="content">
  159. <p>{{$.i18n.Tr "admin.monitor.queue.pool.cancel_notices" `<span class="name"></span>` | Safe}}</p>
  160. <p>{{$.i18n.Tr "admin.monitor.queue.pool.cancel_desc"}}</p>
  161. </div>
  162. {{template "base/delete_modal_actions" .}}
  163. </div>
  164. {{template "base/footer" .}}