You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1011 lines
28 KiB

10 years ago
10 years ago
10 years ago
7 years ago
7 years ago
9 years ago
9 years ago
Better logging (#6038) (#6095) * Panic don't fatal on create new logger Fixes #5854 Signed-off-by: Andrew Thornton <art27@cantab.net> * partial broken * Update the logging infrastrcture Signed-off-by: Andrew Thornton <art27@cantab.net> * Reset the skip levels for Fatal and Error Signed-off-by: Andrew Thornton <art27@cantab.net> * broken ncsa * More log.Error fixes Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove nal * set log-levels to lowercase * Make console_test test all levels * switch to lowercased levels * OK now working * Fix vetting issues * Fix lint * Fix tests * change default logging to match current gitea * Improve log testing Signed-off-by: Andrew Thornton <art27@cantab.net> * reset error skip levels to 0 * Update documentation and access logger configuration * Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE * Fix broken level caching * Refactor the router log * Add Router logger * Add colorizing options * Adjust router colors * Only create logger if they will be used * update app.ini.sample * rename Attribute ColorAttribute * Change from white to green for function * Set fatal/error levels * Restore initial trace logger * Fix Trace arguments in modules/auth/auth.go * Properly handle XORMLogger * Improve admin/config page * fix fmt * Add auto-compression of old logs * Update error log levels * Remove the unnecessary skip argument from Error, Fatal and Critical * Add stacktrace support * Fix tests * Remove x/sync from vendors? * Add stderr option to console logger * Use filepath.ToSlash to protect against Windows in tests * Remove prefixed underscores from names in colors.go * Remove not implemented database logger This was removed from Gogs on 4 Mar 2016 but left in the configuration since then. * Ensure that log paths are relative to ROOT_PATH * use path.Join * rename jsonConfig to logConfig * Rename "config" to "jsonConfig" to make it clearer * Requested changes * Requested changes: XormLogger * Try to color the windows terminal If successful default to colorizing the console logs * fixup * Colorize initially too * update vendor * Colorize logs on default and remove if this is not a colorizing logger * Fix documentation * fix test * Use go-isatty to detect if on windows we are on msys or cygwin * Fix spelling mistake * Add missing vendors * More changes * Rationalise the ANSI writer protection * Adjust colors on advice from @0x5c * Make Flags a comma separated list * Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING * Ensure matching is done on the non-colored message - to simpify EXPRESSION
5 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
8 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
  1. // Copyright 2014 The Gogs Authors. All rights reserved.
  2. // Copyright 2019 The Gitea Authors. All rights reserved.
  3. // Use of this source code is governed by a MIT-style
  4. // license that can be found in the LICENSE file.
  5. package models
  6. import (
  7. "bufio"
  8. "crypto/rsa"
  9. "crypto/x509"
  10. "encoding/asn1"
  11. "encoding/base64"
  12. "encoding/binary"
  13. "encoding/pem"
  14. "errors"
  15. "fmt"
  16. "io/ioutil"
  17. "math/big"
  18. "os"
  19. "path/filepath"
  20. "strings"
  21. "sync"
  22. "time"
  23. "code.gitea.io/gitea/modules/log"
  24. "code.gitea.io/gitea/modules/process"
  25. "code.gitea.io/gitea/modules/setting"
  26. "code.gitea.io/gitea/modules/timeutil"
  27. "github.com/unknwon/com"
  28. "golang.org/x/crypto/ssh"
  29. "xorm.io/builder"
  30. "xorm.io/xorm"
  31. )
  32. const (
  33. tplCommentPrefix = `# gitea public key`
  34. tplCommand = "%s --config=%q serv key-%d"
  35. tplPublicKey = tplCommentPrefix + "\n" + `command=%q,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty %s` + "\n"
  36. )
  37. var sshOpLocker sync.Mutex
  38. // KeyType specifies the key type
  39. type KeyType int
  40. const (
  41. // KeyTypeUser specifies the user key
  42. KeyTypeUser = iota + 1
  43. // KeyTypeDeploy specifies the deploy key
  44. KeyTypeDeploy
  45. )
  46. // PublicKey represents a user or deploy SSH public key.
  47. type PublicKey struct {
  48. ID int64 `xorm:"pk autoincr"`
  49. OwnerID int64 `xorm:"INDEX NOT NULL"`
  50. Name string `xorm:"NOT NULL"`
  51. Fingerprint string `xorm:"INDEX NOT NULL"`
  52. Content string `xorm:"TEXT NOT NULL"`
  53. Mode AccessMode `xorm:"NOT NULL DEFAULT 2"`
  54. Type KeyType `xorm:"NOT NULL DEFAULT 1"`
  55. LoginSourceID int64 `xorm:"NOT NULL DEFAULT 0"`
  56. CreatedUnix timeutil.TimeStamp `xorm:"created"`
  57. UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
  58. HasRecentActivity bool `xorm:"-"`
  59. HasUsed bool `xorm:"-"`
  60. }
  61. // AfterLoad is invoked from XORM after setting the values of all fields of this object.
  62. func (key *PublicKey) AfterLoad() {
  63. key.HasUsed = key.UpdatedUnix > key.CreatedUnix
  64. key.HasRecentActivity = key.UpdatedUnix.AddDuration(7*24*time.Hour) > timeutil.TimeStampNow()
  65. }
  66. // OmitEmail returns content of public key without email address.
  67. func (key *PublicKey) OmitEmail() string {
  68. return strings.Join(strings.Split(key.Content, " ")[:2], " ")
  69. }
  70. // AuthorizedString returns formatted public key string for authorized_keys file.
  71. func (key *PublicKey) AuthorizedString() string {
  72. return fmt.Sprintf(tplPublicKey, fmt.Sprintf(tplCommand, setting.AppPath, setting.CustomConf, key.ID), key.Content)
  73. }
  74. func extractTypeFromBase64Key(key string) (string, error) {
  75. b, err := base64.StdEncoding.DecodeString(key)
  76. if err != nil || len(b) < 4 {
  77. return "", fmt.Errorf("invalid key format: %v", err)
  78. }
  79. keyLength := int(binary.BigEndian.Uint32(b))
  80. if len(b) < 4+keyLength {
  81. return "", fmt.Errorf("invalid key format: not enough length %d", keyLength)
  82. }
  83. return string(b[4 : 4+keyLength]), nil
  84. }
  85. const ssh2keyStart = "---- BEGIN SSH2 PUBLIC KEY ----"
  86. // parseKeyString parses any key string in OpenSSH or SSH2 format to clean OpenSSH string (RFC4253).
  87. func parseKeyString(content string) (string, error) {
  88. // remove whitespace at start and end
  89. content = strings.TrimSpace(content)
  90. var keyType, keyContent, keyComment string
  91. if strings.HasPrefix(content, ssh2keyStart) {
  92. // Parse SSH2 file format.
  93. // Transform all legal line endings to a single "\n".
  94. content = strings.NewReplacer("\r\n", "\n", "\r", "\n").Replace(content)
  95. lines := strings.Split(content, "\n")
  96. continuationLine := false
  97. for _, line := range lines {
  98. // Skip lines that:
  99. // 1) are a continuation of the previous line,
  100. // 2) contain ":" as that are comment lines
  101. // 3) contain "-" as that are begin and end tags
  102. if continuationLine || strings.ContainsAny(line, ":-") {
  103. continuationLine = strings.HasSuffix(line, "\\")
  104. } else {
  105. keyContent += line
  106. }
  107. }
  108. t, err := extractTypeFromBase64Key(keyContent)
  109. if err != nil {
  110. return "", fmt.Errorf("extractTypeFromBase64Key: %v", err)
  111. }
  112. keyType = t
  113. } else {
  114. if strings.Contains(content, "-----BEGIN") {
  115. // Convert PEM Keys to OpenSSH format
  116. // Transform all legal line endings to a single "\n".
  117. content = strings.NewReplacer("\r\n", "\n", "\r", "\n").Replace(content)
  118. block, _ := pem.Decode([]byte(content))
  119. if block == nil {
  120. return "", fmt.Errorf("failed to parse PEM block containing the public key")
  121. }
  122. pub, err := x509.ParsePKIXPublicKey(block.Bytes)
  123. if err != nil {
  124. var pk rsa.PublicKey
  125. _, err2 := asn1.Unmarshal(block.Bytes, &pk)
  126. if err2 != nil {
  127. return "", fmt.Errorf("failed to parse DER encoded public key as either PKIX or PEM RSA Key: %v %v", err, err2)
  128. }
  129. pub = &pk
  130. }
  131. sshKey, err := ssh.NewPublicKey(pub)
  132. if err != nil {
  133. return "", fmt.Errorf("unable to convert to ssh public key: %v", err)
  134. }
  135. content = string(ssh.MarshalAuthorizedKey(sshKey))
  136. }
  137. // Parse OpenSSH format.
  138. // Remove all newlines
  139. content = strings.NewReplacer("\r\n", "", "\n", "").Replace(content)
  140. parts := strings.SplitN(content, " ", 3)
  141. switch len(parts) {
  142. case 0:
  143. return "", errors.New("empty key")
  144. case 1:
  145. keyContent = parts[0]
  146. case 2:
  147. keyType = parts[0]
  148. keyContent = parts[1]
  149. default:
  150. keyType = parts[0]
  151. keyContent = parts[1]
  152. keyComment = parts[2]
  153. }
  154. // If keyType is not given, extract it from content. If given, validate it.
  155. t, err := extractTypeFromBase64Key(keyContent)
  156. if err != nil {
  157. return "", fmt.Errorf("extractTypeFromBase64Key: %v", err)
  158. }
  159. if len(keyType) == 0 {
  160. keyType = t
  161. } else if keyType != t {
  162. return "", fmt.Errorf("key type and content does not match: %s - %s", keyType, t)
  163. }
  164. }
  165. // Finally we need to check whether we can actually read the proposed key:
  166. _, _, _, _, err := ssh.ParseAuthorizedKey([]byte(keyType + " " + keyContent + " " + keyComment))
  167. if err != nil {
  168. return "", fmt.Errorf("invalid ssh public key: %v", err)
  169. }
  170. return keyType + " " + keyContent + " " + keyComment, nil
  171. }
  172. // writeTmpKeyFile writes key content to a temporary file
  173. // and returns the name of that file, along with any possible errors.
  174. func writeTmpKeyFile(content string) (string, error) {
  175. tmpFile, err := ioutil.TempFile(setting.SSH.KeyTestPath, "gitea_keytest")
  176. if err != nil {
  177. return "", fmt.Errorf("TempFile: %v", err)
  178. }
  179. defer tmpFile.Close()
  180. if _, err = tmpFile.WriteString(content); err != nil {
  181. return "", fmt.Errorf("WriteString: %v", err)
  182. }
  183. return tmpFile.Name(), nil
  184. }
  185. // SSHKeyGenParsePublicKey extracts key type and length using ssh-keygen.
  186. func SSHKeyGenParsePublicKey(key string) (string, int, error) {
  187. // The ssh-keygen in Windows does not print key type, so no need go further.
  188. if setting.IsWindows {
  189. return "", 0, nil
  190. }
  191. tmpName, err := writeTmpKeyFile(key)
  192. if err != nil {
  193. return "", 0, fmt.Errorf("writeTmpKeyFile: %v", err)
  194. }
  195. defer os.Remove(tmpName)
  196. stdout, stderr, err := process.GetManager().Exec("SSHKeyGenParsePublicKey", setting.SSH.KeygenPath, "-lf", tmpName)
  197. if err != nil {
  198. return "", 0, fmt.Errorf("fail to parse public key: %s - %s", err, stderr)
  199. }
  200. if strings.Contains(stdout, "is not a public key file") {
  201. return "", 0, ErrKeyUnableVerify{stdout}
  202. }
  203. fields := strings.Split(stdout, " ")
  204. if len(fields) < 4 {
  205. return "", 0, fmt.Errorf("invalid public key line: %s", stdout)
  206. }
  207. keyType := strings.Trim(fields[len(fields)-1], "()\r\n")
  208. return strings.ToLower(keyType), com.StrTo(fields[0]).MustInt(), nil
  209. }
  210. // SSHNativeParsePublicKey extracts the key type and length using the golang SSH library.
  211. func SSHNativeParsePublicKey(keyLine string) (string, int, error) {
  212. fields := strings.Fields(keyLine)
  213. if len(fields) < 2 {
  214. return "", 0, fmt.Errorf("not enough fields in public key line: %s", keyLine)
  215. }
  216. raw, err := base64.StdEncoding.DecodeString(fields[1])
  217. if err != nil {
  218. return "", 0, err
  219. }
  220. pkey, err := ssh.ParsePublicKey(raw)
  221. if err != nil {
  222. if strings.Contains(err.Error(), "ssh: unknown key algorithm") {
  223. return "", 0, ErrKeyUnableVerify{err.Error()}
  224. }
  225. return "", 0, fmt.Errorf("ParsePublicKey: %v", err)
  226. }
  227. // The ssh library can parse the key, so next we find out what key exactly we have.
  228. switch pkey.Type() {
  229. case ssh.KeyAlgoDSA:
  230. rawPub := struct {
  231. Name string
  232. P, Q, G, Y *big.Int
  233. }{}
  234. if err := ssh.Unmarshal(pkey.Marshal(), &rawPub); err != nil {
  235. return "", 0, err
  236. }
  237. // as per https://bugzilla.mindrot.org/show_bug.cgi?id=1647 we should never
  238. // see dsa keys != 1024 bit, but as it seems to work, we will not check here
  239. return "dsa", rawPub.P.BitLen(), nil // use P as per crypto/dsa/dsa.go (is L)
  240. case ssh.KeyAlgoRSA:
  241. rawPub := struct {
  242. Name string
  243. E *big.Int
  244. N *big.Int
  245. }{}
  246. if err := ssh.Unmarshal(pkey.Marshal(), &rawPub); err != nil {
  247. return "", 0, err
  248. }
  249. return "rsa", rawPub.N.BitLen(), nil // use N as per crypto/rsa/rsa.go (is bits)
  250. case ssh.KeyAlgoECDSA256:
  251. return "ecdsa", 256, nil
  252. case ssh.KeyAlgoECDSA384:
  253. return "ecdsa", 384, nil
  254. case ssh.KeyAlgoECDSA521:
  255. return "ecdsa", 521, nil
  256. case ssh.KeyAlgoED25519:
  257. return "ed25519", 256, nil
  258. }
  259. return "", 0, fmt.Errorf("unsupported key length detection for type: %s", pkey.Type())
  260. }
  261. // CheckPublicKeyString checks if the given public key string is recognized by SSH.
  262. // It returns the actual public key line on success.
  263. func CheckPublicKeyString(content string) (_ string, err error) {
  264. if setting.SSH.Disabled {
  265. return "", ErrSSHDisabled{}
  266. }
  267. content, err = parseKeyString(content)
  268. if err != nil {
  269. return "", err
  270. }
  271. content = strings.TrimRight(content, "\n\r")
  272. if strings.ContainsAny(content, "\n\r") {
  273. return "", errors.New("only a single line with a single key please")
  274. }
  275. // remove any unnecessary whitespace now
  276. content = strings.TrimSpace(content)
  277. if !setting.SSH.MinimumKeySizeCheck {
  278. return content, nil
  279. }
  280. var (
  281. fnName string
  282. keyType string
  283. length int
  284. )
  285. if setting.SSH.StartBuiltinServer {
  286. fnName = "SSHNativeParsePublicKey"
  287. keyType, length, err = SSHNativeParsePublicKey(content)
  288. } else {
  289. fnName = "SSHKeyGenParsePublicKey"
  290. keyType, length, err = SSHKeyGenParsePublicKey(content)
  291. }
  292. if err != nil {
  293. return "", fmt.Errorf("%s: %v", fnName, err)
  294. }
  295. log.Trace("Key info [native: %v]: %s-%d", setting.SSH.StartBuiltinServer, keyType, length)
  296. if minLen, found := setting.SSH.MinimumKeySizes[keyType]; found && length >= minLen {
  297. return content, nil
  298. } else if found && length < minLen {
  299. return "", fmt.Errorf("key length is not enough: got %d, needs %d", length, minLen)
  300. }
  301. return "", fmt.Errorf("key type is not allowed: %s", keyType)
  302. }
  303. // appendAuthorizedKeysToFile appends new SSH keys' content to authorized_keys file.
  304. func appendAuthorizedKeysToFile(keys ...*PublicKey) error {
  305. // Don't need to rewrite this file if builtin SSH server is enabled.
  306. if setting.SSH.StartBuiltinServer {
  307. return nil
  308. }
  309. sshOpLocker.Lock()
  310. defer sshOpLocker.Unlock()
  311. if setting.SSH.RootPath != "" {
  312. // First of ensure that the RootPath is present, and if not make it with 0700 permissions
  313. // This of course doesn't guarantee that this is the right directory for authorized_keys
  314. // but at least if it's supposed to be this directory and it doesn't exist and we're the
  315. // right user it will at least be created properly.
  316. err := os.MkdirAll(setting.SSH.RootPath, 0700)
  317. if err != nil {
  318. log.Error("Unable to MkdirAll(%s): %v", setting.SSH.RootPath, err)
  319. return err
  320. }
  321. }
  322. fPath := filepath.Join(setting.SSH.RootPath, "authorized_keys")
  323. f, err := os.OpenFile(fPath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600)
  324. if err != nil {
  325. return err
  326. }
  327. defer f.Close()
  328. // Note: chmod command does not support in Windows.
  329. if !setting.IsWindows {
  330. fi, err := f.Stat()
  331. if err != nil {
  332. return err
  333. }
  334. // .ssh directory should have mode 700, and authorized_keys file should have mode 600.
  335. if fi.Mode().Perm() > 0600 {
  336. log.Error("authorized_keys file has unusual permission flags: %s - setting to -rw-------", fi.Mode().Perm().String())
  337. if err = f.Chmod(0600); err != nil {
  338. return err
  339. }
  340. }
  341. }
  342. for _, key := range keys {
  343. if _, err = f.WriteString(key.AuthorizedString()); err != nil {
  344. return err
  345. }
  346. }
  347. return nil
  348. }
  349. // checkKeyFingerprint only checks if key fingerprint has been used as public key,
  350. // it is OK to use same key as deploy key for multiple repositories/users.
  351. func checkKeyFingerprint(e Engine, fingerprint string) error {
  352. has, err := e.Get(&PublicKey{
  353. Fingerprint: fingerprint,
  354. })
  355. if err != nil {
  356. return err
  357. } else if has {
  358. return ErrKeyAlreadyExist{0, fingerprint, ""}
  359. }
  360. return nil
  361. }
  362. func calcFingerprintSSHKeygen(publicKeyContent string) (string, error) {
  363. // Calculate fingerprint.
  364. tmpPath, err := writeTmpKeyFile(publicKeyContent)
  365. if err != nil {
  366. return "", err
  367. }
  368. defer os.Remove(tmpPath)
  369. stdout, stderr, err := process.GetManager().Exec("AddPublicKey", "ssh-keygen", "-lf", tmpPath)
  370. if err != nil {
  371. return "", fmt.Errorf("'ssh-keygen -lf %s' failed with error '%s': %s", tmpPath, err, stderr)
  372. } else if len(stdout) < 2 {
  373. return "", errors.New("not enough output for calculating fingerprint: " + stdout)
  374. }
  375. return strings.Split(stdout, " ")[1], nil
  376. }
  377. func calcFingerprintNative(publicKeyContent string) (string, error) {
  378. // Calculate fingerprint.
  379. pk, _, _, _, err := ssh.ParseAuthorizedKey([]byte(publicKeyContent))
  380. if err != nil {
  381. return "", err
  382. }
  383. return ssh.FingerprintSHA256(pk), nil
  384. }
  385. func calcFingerprint(publicKeyContent string) (string, error) {
  386. //Call the method based on configuration
  387. var (
  388. fnName, fp string
  389. err error
  390. )
  391. if setting.SSH.StartBuiltinServer {
  392. fnName = "calcFingerprintNative"
  393. fp, err = calcFingerprintNative(publicKeyContent)
  394. } else {
  395. fnName = "calcFingerprintSSHKeygen"
  396. fp, err = calcFingerprintSSHKeygen(publicKeyContent)
  397. }
  398. if err != nil {
  399. return "", fmt.Errorf("%s: %v", fnName, err)
  400. }
  401. return fp, nil
  402. }
  403. func addKey(e Engine, key *PublicKey) (err error) {
  404. if len(key.Fingerprint) == 0 {
  405. key.Fingerprint, err = calcFingerprint(key.Content)
  406. if err != nil {
  407. return err
  408. }
  409. }
  410. // Save SSH key.
  411. if _, err = e.Insert(key); err != nil {
  412. return err
  413. }
  414. return appendAuthorizedKeysToFile(key)
  415. }
  416. // AddPublicKey adds new public key to database and authorized_keys file.
  417. func AddPublicKey(ownerID int64, name, content string, loginSourceID int64) (*PublicKey, error) {
  418. log.Trace(content)
  419. fingerprint, err := calcFingerprint(content)
  420. if err != nil {
  421. return nil, err
  422. }
  423. sess := x.NewSession()
  424. defer sess.Close()
  425. if err = sess.Begin(); err != nil {
  426. return nil, err
  427. }
  428. if err := checkKeyFingerprint(sess, fingerprint); err != nil {
  429. return nil, err
  430. }
  431. // Key name of same user cannot be duplicated.
  432. has, err := sess.
  433. Where("owner_id = ? AND name = ?", ownerID, name).
  434. Get(new(PublicKey))
  435. if err != nil {
  436. return nil, err
  437. } else if has {
  438. return nil, ErrKeyNameAlreadyUsed{ownerID, name}
  439. }
  440. key := &PublicKey{
  441. OwnerID: ownerID,
  442. Name: name,
  443. Fingerprint: fingerprint,
  444. Content: content,
  445. Mode: AccessModeWrite,
  446. Type: KeyTypeUser,
  447. LoginSourceID: loginSourceID,
  448. }
  449. if err = addKey(sess, key); err != nil {
  450. return nil, fmt.Errorf("addKey: %v", err)
  451. }
  452. return key, sess.Commit()
  453. }
  454. // GetPublicKeyByID returns public key by given ID.
  455. func GetPublicKeyByID(keyID int64) (*PublicKey, error) {
  456. key := new(PublicKey)
  457. has, err := x.
  458. ID(keyID).
  459. Get(key)
  460. if err != nil {
  461. return nil, err
  462. } else if !has {
  463. return nil, ErrKeyNotExist{keyID}
  464. }
  465. return key, nil
  466. }
  467. func searchPublicKeyByContentWithEngine(e Engine, content string) (*PublicKey, error) {
  468. key := new(PublicKey)
  469. has, err := e.
  470. Where("content like ?", content+"%").
  471. Get(key)
  472. if err != nil {
  473. return nil, err
  474. } else if !has {
  475. return nil, ErrKeyNotExist{}
  476. }
  477. return key, nil
  478. }
  479. // SearchPublicKeyByContent searches content as prefix (leak e-mail part)
  480. // and returns public key found.
  481. func SearchPublicKeyByContent(content string) (*PublicKey, error) {
  482. return searchPublicKeyByContentWithEngine(x, content)
  483. }
  484. // SearchPublicKey returns a list of public keys matching the provided arguments.
  485. func SearchPublicKey(uid int64, fingerprint string) ([]*PublicKey, error) {
  486. keys := make([]*PublicKey, 0, 5)
  487. cond := builder.NewCond()
  488. if uid != 0 {
  489. cond = cond.And(builder.Eq{"owner_id": uid})
  490. }
  491. if fingerprint != "" {
  492. cond = cond.And(builder.Eq{"fingerprint": fingerprint})
  493. }
  494. return keys, x.Where(cond).Find(&keys)
  495. }
  496. // ListPublicKeys returns a list of public keys belongs to given user.
  497. func ListPublicKeys(uid int64, listOptions ListOptions) ([]*PublicKey, error) {
  498. sess := x.Where("owner_id = ?", uid)
  499. if listOptions.Page != 0 {
  500. sess = listOptions.setSessionPagination(sess)
  501. keys := make([]*PublicKey, 0, listOptions.PageSize)
  502. return keys, sess.Find(&keys)
  503. }
  504. keys := make([]*PublicKey, 0, 5)
  505. return keys, sess.Find(&keys)
  506. }
  507. // ListPublicLdapSSHKeys returns a list of synchronized public ldap ssh keys belongs to given user and login source.
  508. func ListPublicLdapSSHKeys(uid int64, loginSourceID int64) ([]*PublicKey, error) {
  509. keys := make([]*PublicKey, 0, 5)
  510. return keys, x.
  511. Where("owner_id = ? AND login_source_id = ?", uid, loginSourceID).
  512. Find(&keys)
  513. }
  514. // UpdatePublicKeyUpdated updates public key use time.
  515. func UpdatePublicKeyUpdated(id int64) error {
  516. // Check if key exists before update as affected rows count is unreliable
  517. // and will return 0 affected rows if two updates are made at the same time
  518. if cnt, err := x.ID(id).Count(&PublicKey{}); err != nil {
  519. return err
  520. } else if cnt != 1 {
  521. return ErrKeyNotExist{id}
  522. }
  523. _, err := x.ID(id).Cols("updated_unix").Update(&PublicKey{
  524. UpdatedUnix: timeutil.TimeStampNow(),
  525. })
  526. if err != nil {
  527. return err
  528. }
  529. return nil
  530. }
  531. // deletePublicKeys does the actual key deletion but does not update authorized_keys file.
  532. func deletePublicKeys(e Engine, keyIDs ...int64) error {
  533. if len(keyIDs) == 0 {
  534. return nil
  535. }
  536. _, err := e.In("id", keyIDs).Delete(new(PublicKey))
  537. return err
  538. }
  539. // DeletePublicKey deletes SSH key information both in database and authorized_keys file.
  540. func DeletePublicKey(doer *User, id int64) (err error) {
  541. key, err := GetPublicKeyByID(id)
  542. if err != nil {
  543. return err
  544. }
  545. // Check if user has access to delete this key.
  546. if !doer.IsAdmin && doer.ID != key.OwnerID {
  547. return ErrKeyAccessDenied{doer.ID, key.ID, "public"}
  548. }
  549. sess := x.NewSession()
  550. defer sess.Close()
  551. if err = sess.Begin(); err != nil {
  552. return err
  553. }
  554. if err = deletePublicKeys(sess, id); err != nil {
  555. return err
  556. }
  557. if err = sess.Commit(); err != nil {
  558. return err
  559. }
  560. sess.Close()
  561. return RewriteAllPublicKeys()
  562. }
  563. // RewriteAllPublicKeys removes any authorized key and rewrite all keys from database again.
  564. // Note: x.Iterate does not get latest data after insert/delete, so we have to call this function
  565. // outside any session scope independently.
  566. func RewriteAllPublicKeys() error {
  567. return rewriteAllPublicKeys(x)
  568. }
  569. func rewriteAllPublicKeys(e Engine) error {
  570. //Don't rewrite key if internal server
  571. if setting.SSH.StartBuiltinServer || !setting.SSH.CreateAuthorizedKeysFile {
  572. return nil
  573. }
  574. sshOpLocker.Lock()
  575. defer sshOpLocker.Unlock()
  576. if setting.SSH.RootPath != "" {
  577. // First of ensure that the RootPath is present, and if not make it with 0700 permissions
  578. // This of course doesn't guarantee that this is the right directory for authorized_keys
  579. // but at least if it's supposed to be this directory and it doesn't exist and we're the
  580. // right user it will at least be created properly.
  581. err := os.MkdirAll(setting.SSH.RootPath, 0700)
  582. if err != nil {
  583. log.Error("Unable to MkdirAll(%s): %v", setting.SSH.RootPath, err)
  584. return err
  585. }
  586. }
  587. fPath := filepath.Join(setting.SSH.RootPath, "authorized_keys")
  588. tmpPath := fPath + ".tmp"
  589. t, err := os.OpenFile(tmpPath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
  590. if err != nil {
  591. return err
  592. }
  593. defer func() {
  594. t.Close()
  595. os.Remove(tmpPath)
  596. }()
  597. if setting.SSH.AuthorizedKeysBackup && com.IsExist(fPath) {
  598. bakPath := fmt.Sprintf("%s_%d.gitea_bak", fPath, time.Now().Unix())
  599. if err = com.Copy(fPath, bakPath); err != nil {
  600. return err
  601. }
  602. }
  603. err = e.Iterate(new(PublicKey), func(idx int, bean interface{}) (err error) {
  604. _, err = t.WriteString((bean.(*PublicKey)).AuthorizedString())
  605. return err
  606. })
  607. if err != nil {
  608. return err
  609. }
  610. if com.IsExist(fPath) {
  611. f, err := os.Open(fPath)
  612. if err != nil {
  613. return err
  614. }
  615. scanner := bufio.NewScanner(f)
  616. for scanner.Scan() {
  617. line := scanner.Text()
  618. if strings.HasPrefix(line, tplCommentPrefix) {
  619. scanner.Scan()
  620. continue
  621. }
  622. _, err = t.WriteString(line + "\n")
  623. if err != nil {
  624. f.Close()
  625. return err
  626. }
  627. }
  628. f.Close()
  629. }
  630. t.Close()
  631. return os.Rename(tmpPath, fPath)
  632. }
  633. // ________ .__ ____ __.
  634. // \______ \ ____ ______ | | ____ ___.__.| |/ _|____ ___.__.
  635. // | | \_/ __ \\____ \| | / _ < | || <_/ __ < | |
  636. // | ` \ ___/| |_> > |_( <_> )___ || | \ ___/\___ |
  637. // /_______ /\___ > __/|____/\____// ____||____|__ \___ > ____|
  638. // \/ \/|__| \/ \/ \/\/
  639. // DeployKey represents deploy key information and its relation with repository.
  640. type DeployKey struct {
  641. ID int64 `xorm:"pk autoincr"`
  642. KeyID int64 `xorm:"UNIQUE(s) INDEX"`
  643. RepoID int64 `xorm:"UNIQUE(s) INDEX"`
  644. Name string
  645. Fingerprint string
  646. Content string `xorm:"-"`
  647. Mode AccessMode `xorm:"NOT NULL DEFAULT 1"`
  648. CreatedUnix timeutil.TimeStamp `xorm:"created"`
  649. UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
  650. HasRecentActivity bool `xorm:"-"`
  651. HasUsed bool `xorm:"-"`
  652. }
  653. // AfterLoad is invoked from XORM after setting the values of all fields of this object.
  654. func (key *DeployKey) AfterLoad() {
  655. key.HasUsed = key.UpdatedUnix > key.CreatedUnix
  656. key.HasRecentActivity = key.UpdatedUnix.AddDuration(7*24*time.Hour) > timeutil.TimeStampNow()
  657. }
  658. // GetContent gets associated public key content.
  659. func (key *DeployKey) GetContent() error {
  660. pkey, err := GetPublicKeyByID(key.KeyID)
  661. if err != nil {
  662. return err
  663. }
  664. key.Content = pkey.Content
  665. return nil
  666. }
  667. // IsReadOnly checks if the key can only be used for read operations
  668. func (key *DeployKey) IsReadOnly() bool {
  669. return key.Mode == AccessModeRead
  670. }
  671. func checkDeployKey(e Engine, keyID, repoID int64, name string) error {
  672. // Note: We want error detail, not just true or false here.
  673. has, err := e.
  674. Where("key_id = ? AND repo_id = ?", keyID, repoID).
  675. Get(new(DeployKey))
  676. if err != nil {
  677. return err
  678. } else if has {
  679. return ErrDeployKeyAlreadyExist{keyID, repoID}
  680. }
  681. has, err = e.
  682. Where("repo_id = ? AND name = ?", repoID, name).
  683. Get(new(DeployKey))
  684. if err != nil {
  685. return err
  686. } else if has {
  687. return ErrDeployKeyNameAlreadyUsed{repoID, name}
  688. }
  689. return nil
  690. }
  691. // addDeployKey adds new key-repo relation.
  692. func addDeployKey(e *xorm.Session, keyID, repoID int64, name, fingerprint string, mode AccessMode) (*DeployKey, error) {
  693. if err := checkDeployKey(e, keyID, repoID, name); err != nil {
  694. return nil, err
  695. }
  696. key := &DeployKey{
  697. KeyID: keyID,
  698. RepoID: repoID,
  699. Name: name,
  700. Fingerprint: fingerprint,
  701. Mode: mode,
  702. }
  703. _, err := e.Insert(key)
  704. return key, err
  705. }
  706. // HasDeployKey returns true if public key is a deploy key of given repository.
  707. func HasDeployKey(keyID, repoID int64) bool {
  708. has, _ := x.
  709. Where("key_id = ? AND repo_id = ?", keyID, repoID).
  710. Get(new(DeployKey))
  711. return has
  712. }
  713. // AddDeployKey add new deploy key to database and authorized_keys file.
  714. func AddDeployKey(repoID int64, name, content string, readOnly bool) (*DeployKey, error) {
  715. fingerprint, err := calcFingerprint(content)
  716. if err != nil {
  717. return nil, err
  718. }
  719. accessMode := AccessModeRead
  720. if !readOnly {
  721. accessMode = AccessModeWrite
  722. }
  723. sess := x.NewSession()
  724. defer sess.Close()
  725. if err = sess.Begin(); err != nil {
  726. return nil, err
  727. }
  728. pkey := &PublicKey{
  729. Fingerprint: fingerprint,
  730. }
  731. has, err := sess.Get(pkey)
  732. if err != nil {
  733. return nil, err
  734. }
  735. if has {
  736. if pkey.Type != KeyTypeDeploy {
  737. return nil, ErrKeyAlreadyExist{0, fingerprint, ""}
  738. }
  739. } else {
  740. // First time use this deploy key.
  741. pkey.Mode = accessMode
  742. pkey.Type = KeyTypeDeploy
  743. pkey.Content = content
  744. pkey.Name = name
  745. if err = addKey(sess, pkey); err != nil {
  746. return nil, fmt.Errorf("addKey: %v", err)
  747. }
  748. }
  749. key, err := addDeployKey(sess, pkey.ID, repoID, name, pkey.Fingerprint, accessMode)
  750. if err != nil {
  751. return nil, err
  752. }
  753. return key, sess.Commit()
  754. }
  755. // GetDeployKeyByID returns deploy key by given ID.
  756. func GetDeployKeyByID(id int64) (*DeployKey, error) {
  757. return getDeployKeyByID(x, id)
  758. }
  759. func getDeployKeyByID(e Engine, id int64) (*DeployKey, error) {
  760. key := new(DeployKey)
  761. has, err := e.ID(id).Get(key)
  762. if err != nil {
  763. return nil, err
  764. } else if !has {
  765. return nil, ErrDeployKeyNotExist{id, 0, 0}
  766. }
  767. return key, nil
  768. }
  769. // GetDeployKeyByRepo returns deploy key by given public key ID and repository ID.
  770. func GetDeployKeyByRepo(keyID, repoID int64) (*DeployKey, error) {
  771. return getDeployKeyByRepo(x, keyID, repoID)
  772. }
  773. func getDeployKeyByRepo(e Engine, keyID, repoID int64) (*DeployKey, error) {
  774. key := &DeployKey{
  775. KeyID: keyID,
  776. RepoID: repoID,
  777. }
  778. has, err := e.Get(key)
  779. if err != nil {
  780. return nil, err
  781. } else if !has {
  782. return nil, ErrDeployKeyNotExist{0, keyID, repoID}
  783. }
  784. return key, nil
  785. }
  786. // UpdateDeployKeyCols updates deploy key information in the specified columns.
  787. func UpdateDeployKeyCols(key *DeployKey, cols ...string) error {
  788. _, err := x.ID(key.ID).Cols(cols...).Update(key)
  789. return err
  790. }
  791. // UpdateDeployKey updates deploy key information.
  792. func UpdateDeployKey(key *DeployKey) error {
  793. _, err := x.ID(key.ID).AllCols().Update(key)
  794. return err
  795. }
  796. // DeleteDeployKey deletes deploy key from its repository authorized_keys file if needed.
  797. func DeleteDeployKey(doer *User, id int64) error {
  798. sess := x.NewSession()
  799. defer sess.Close()
  800. if err := sess.Begin(); err != nil {
  801. return err
  802. }
  803. if err := deleteDeployKey(sess, doer, id); err != nil {
  804. return err
  805. }
  806. return sess.Commit()
  807. }
  808. func deleteDeployKey(sess Engine, doer *User, id int64) error {
  809. key, err := getDeployKeyByID(sess, id)
  810. if err != nil {
  811. if IsErrDeployKeyNotExist(err) {
  812. return nil
  813. }
  814. return fmt.Errorf("GetDeployKeyByID: %v", err)
  815. }
  816. // Check if user has access to delete this key.
  817. if !doer.IsAdmin {
  818. repo, err := getRepositoryByID(sess, key.RepoID)
  819. if err != nil {
  820. return fmt.Errorf("GetRepositoryByID: %v", err)
  821. }
  822. has, err := isUserRepoAdmin(sess, repo, doer)
  823. if err != nil {
  824. return fmt.Errorf("GetUserRepoPermission: %v", err)
  825. } else if !has {
  826. return ErrKeyAccessDenied{doer.ID, key.ID, "deploy"}
  827. }
  828. }
  829. if _, err = sess.ID(key.ID).Delete(new(DeployKey)); err != nil {
  830. return fmt.Errorf("delete deploy key [%d]: %v", key.ID, err)
  831. }
  832. // Check if this is the last reference to same key content.
  833. has, err := sess.
  834. Where("key_id = ?", key.KeyID).
  835. Get(new(DeployKey))
  836. if err != nil {
  837. return err
  838. } else if !has {
  839. if err = deletePublicKeys(sess, key.KeyID); err != nil {
  840. return err
  841. }
  842. // after deleted the public keys, should rewrite the public keys file
  843. if err = rewriteAllPublicKeys(sess); err != nil {
  844. return err
  845. }
  846. }
  847. return nil
  848. }
  849. // ListDeployKeys returns all deploy keys by given repository ID.
  850. func ListDeployKeys(repoID int64, listOptions ListOptions) ([]*DeployKey, error) {
  851. return listDeployKeys(x, repoID, listOptions)
  852. }
  853. func listDeployKeys(e Engine, repoID int64, listOptions ListOptions) ([]*DeployKey, error) {
  854. sess := e.Where("repo_id = ?", repoID)
  855. if listOptions.Page != 0 {
  856. sess = listOptions.setSessionPagination(sess)
  857. keys := make([]*DeployKey, 0, listOptions.PageSize)
  858. return keys, sess.Find(&keys)
  859. }
  860. keys := make([]*DeployKey, 0, 5)
  861. return keys, sess.Find(&keys)
  862. }
  863. // SearchDeployKeys returns a list of deploy keys matching the provided arguments.
  864. func SearchDeployKeys(repoID int64, keyID int64, fingerprint string) ([]*DeployKey, error) {
  865. keys := make([]*DeployKey, 0, 5)
  866. cond := builder.NewCond()
  867. if repoID != 0 {
  868. cond = cond.And(builder.Eq{"repo_id": repoID})
  869. }
  870. if keyID != 0 {
  871. cond = cond.And(builder.Eq{"key_id": keyID})
  872. }
  873. if fingerprint != "" {
  874. cond = cond.And(builder.Eq{"fingerprint": fingerprint})
  875. }
  876. return keys, x.Where(cond).Find(&keys)
  877. }