You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1032 lines
28 KiB

10 years ago
10 years ago
10 years ago
8 years ago
8 years ago
10 years ago
10 years ago
Better logging (#6038) (#6095) * Panic don't fatal on create new logger Fixes #5854 Signed-off-by: Andrew Thornton <art27@cantab.net> * partial broken * Update the logging infrastrcture Signed-off-by: Andrew Thornton <art27@cantab.net> * Reset the skip levels for Fatal and Error Signed-off-by: Andrew Thornton <art27@cantab.net> * broken ncsa * More log.Error fixes Signed-off-by: Andrew Thornton <art27@cantab.net> * Remove nal * set log-levels to lowercase * Make console_test test all levels * switch to lowercased levels * OK now working * Fix vetting issues * Fix lint * Fix tests * change default logging to match current gitea * Improve log testing Signed-off-by: Andrew Thornton <art27@cantab.net> * reset error skip levels to 0 * Update documentation and access logger configuration * Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE * Fix broken level caching * Refactor the router log * Add Router logger * Add colorizing options * Adjust router colors * Only create logger if they will be used * update app.ini.sample * rename Attribute ColorAttribute * Change from white to green for function * Set fatal/error levels * Restore initial trace logger * Fix Trace arguments in modules/auth/auth.go * Properly handle XORMLogger * Improve admin/config page * fix fmt * Add auto-compression of old logs * Update error log levels * Remove the unnecessary skip argument from Error, Fatal and Critical * Add stacktrace support * Fix tests * Remove x/sync from vendors? * Add stderr option to console logger * Use filepath.ToSlash to protect against Windows in tests * Remove prefixed underscores from names in colors.go * Remove not implemented database logger This was removed from Gogs on 4 Mar 2016 but left in the configuration since then. * Ensure that log paths are relative to ROOT_PATH * use path.Join * rename jsonConfig to logConfig * Rename "config" to "jsonConfig" to make it clearer * Requested changes * Requested changes: XormLogger * Try to color the windows terminal If successful default to colorizing the console logs * fixup * Colorize initially too * update vendor * Colorize logs on default and remove if this is not a colorizing logger * Fix documentation * fix test * Use go-isatty to detect if on windows we are on msys or cygwin * Fix spelling mistake * Add missing vendors * More changes * Rationalise the ANSI writer protection * Adjust colors on advice from @0x5c * Make Flags a comma separated list * Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING * Ensure matching is done on the non-colored message - to simpify EXPRESSION
5 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
API add/generalize pagination (#9452) * paginate results * fixed deadlock * prevented breaking change * updated swagger * go fmt * fixed find topic * go mod tidy * go mod vendor with go1.13.5 * fixed repo find topics * fixed unit test * added Limit method to Engine struct; use engine variable when provided; fixed gitignore * use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors * set Page value on Home route * improved memory allocations * fixed response headers * removed logfiles * fixed import order * import order * improved swagger * added function to get models.ListOptions from context * removed pagesize diff on unit test * fixed imports * removed unnecessary struct field * fixed go fmt * scoped PR * code improvements * code improvements * go mod tidy * fixed import order * fixed commit statuses session * fixed files headers * fixed headers; added pagination for notifications * go mod tidy * go fmt * removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list * Apply suggestions from code review Co-Authored-By: 6543 <6543@obermui.de> Co-Authored-By: zeripath <art27@cantab.net> * fixed build error * CI.restart() * fixed merge conflicts resolve * fixed conflicts resolve * improved FindTrackedTimesOptions.ToOptions() method * added backwards compatibility on ListReleases request; fixed issue tracked time ToSession * fixed build error; fixed swagger template * fixed swagger template * fixed ListReleases backwards compatibility * added page to user search route Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: zeripath <art27@cantab.net>
4 years ago
  1. // Copyright 2014 The Gogs Authors. All rights reserved.
  2. // Copyright 2019 The Gitea Authors. All rights reserved.
  3. // Use of this source code is governed by a MIT-style
  4. // license that can be found in the LICENSE file.
  5. package models
  6. import (
  7. "bufio"
  8. "crypto/rsa"
  9. "crypto/x509"
  10. "encoding/asn1"
  11. "encoding/base64"
  12. "encoding/binary"
  13. "encoding/pem"
  14. "errors"
  15. "fmt"
  16. "io"
  17. "io/ioutil"
  18. "math/big"
  19. "os"
  20. "path/filepath"
  21. "strings"
  22. "sync"
  23. "time"
  24. "code.gitea.io/gitea/modules/log"
  25. "code.gitea.io/gitea/modules/process"
  26. "code.gitea.io/gitea/modules/setting"
  27. "code.gitea.io/gitea/modules/timeutil"
  28. "github.com/unknwon/com"
  29. "golang.org/x/crypto/ssh"
  30. "xorm.io/builder"
  31. "xorm.io/xorm"
  32. )
  33. const (
  34. tplCommentPrefix = `# gitea public key`
  35. tplCommand = "%s --config=%q serv key-%d"
  36. tplPublicKey = tplCommentPrefix + "\n" + `command=%q,no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty %s` + "\n"
  37. )
  38. var sshOpLocker sync.Mutex
  39. // KeyType specifies the key type
  40. type KeyType int
  41. const (
  42. // KeyTypeUser specifies the user key
  43. KeyTypeUser = iota + 1
  44. // KeyTypeDeploy specifies the deploy key
  45. KeyTypeDeploy
  46. )
  47. // PublicKey represents a user or deploy SSH public key.
  48. type PublicKey struct {
  49. ID int64 `xorm:"pk autoincr"`
  50. OwnerID int64 `xorm:"INDEX NOT NULL"`
  51. Name string `xorm:"NOT NULL"`
  52. Fingerprint string `xorm:"INDEX NOT NULL"`
  53. Content string `xorm:"TEXT NOT NULL"`
  54. Mode AccessMode `xorm:"NOT NULL DEFAULT 2"`
  55. Type KeyType `xorm:"NOT NULL DEFAULT 1"`
  56. LoginSourceID int64 `xorm:"NOT NULL DEFAULT 0"`
  57. CreatedUnix timeutil.TimeStamp `xorm:"created"`
  58. UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
  59. HasRecentActivity bool `xorm:"-"`
  60. HasUsed bool `xorm:"-"`
  61. }
  62. // AfterLoad is invoked from XORM after setting the values of all fields of this object.
  63. func (key *PublicKey) AfterLoad() {
  64. key.HasUsed = key.UpdatedUnix > key.CreatedUnix
  65. key.HasRecentActivity = key.UpdatedUnix.AddDuration(7*24*time.Hour) > timeutil.TimeStampNow()
  66. }
  67. // OmitEmail returns content of public key without email address.
  68. func (key *PublicKey) OmitEmail() string {
  69. return strings.Join(strings.Split(key.Content, " ")[:2], " ")
  70. }
  71. // AuthorizedString returns formatted public key string for authorized_keys file.
  72. func (key *PublicKey) AuthorizedString() string {
  73. return fmt.Sprintf(tplPublicKey, fmt.Sprintf(tplCommand, setting.AppPath, setting.CustomConf, key.ID), key.Content)
  74. }
  75. func extractTypeFromBase64Key(key string) (string, error) {
  76. b, err := base64.StdEncoding.DecodeString(key)
  77. if err != nil || len(b) < 4 {
  78. return "", fmt.Errorf("invalid key format: %v", err)
  79. }
  80. keyLength := int(binary.BigEndian.Uint32(b))
  81. if len(b) < 4+keyLength {
  82. return "", fmt.Errorf("invalid key format: not enough length %d", keyLength)
  83. }
  84. return string(b[4 : 4+keyLength]), nil
  85. }
  86. const ssh2keyStart = "---- BEGIN SSH2 PUBLIC KEY ----"
  87. // parseKeyString parses any key string in OpenSSH or SSH2 format to clean OpenSSH string (RFC4253).
  88. func parseKeyString(content string) (string, error) {
  89. // remove whitespace at start and end
  90. content = strings.TrimSpace(content)
  91. var keyType, keyContent, keyComment string
  92. if strings.HasPrefix(content, ssh2keyStart) {
  93. // Parse SSH2 file format.
  94. // Transform all legal line endings to a single "\n".
  95. content = strings.NewReplacer("\r\n", "\n", "\r", "\n").Replace(content)
  96. lines := strings.Split(content, "\n")
  97. continuationLine := false
  98. for _, line := range lines {
  99. // Skip lines that:
  100. // 1) are a continuation of the previous line,
  101. // 2) contain ":" as that are comment lines
  102. // 3) contain "-" as that are begin and end tags
  103. if continuationLine || strings.ContainsAny(line, ":-") {
  104. continuationLine = strings.HasSuffix(line, "\\")
  105. } else {
  106. keyContent += line
  107. }
  108. }
  109. t, err := extractTypeFromBase64Key(keyContent)
  110. if err != nil {
  111. return "", fmt.Errorf("extractTypeFromBase64Key: %v", err)
  112. }
  113. keyType = t
  114. } else {
  115. if strings.Contains(content, "-----BEGIN") {
  116. // Convert PEM Keys to OpenSSH format
  117. // Transform all legal line endings to a single "\n".
  118. content = strings.NewReplacer("\r\n", "\n", "\r", "\n").Replace(content)
  119. block, _ := pem.Decode([]byte(content))
  120. if block == nil {
  121. return "", fmt.Errorf("failed to parse PEM block containing the public key")
  122. }
  123. pub, err := x509.ParsePKIXPublicKey(block.Bytes)
  124. if err != nil {
  125. var pk rsa.PublicKey
  126. _, err2 := asn1.Unmarshal(block.Bytes, &pk)
  127. if err2 != nil {
  128. return "", fmt.Errorf("failed to parse DER encoded public key as either PKIX or PEM RSA Key: %v %v", err, err2)
  129. }
  130. pub = &pk
  131. }
  132. sshKey, err := ssh.NewPublicKey(pub)
  133. if err != nil {
  134. return "", fmt.Errorf("unable to convert to ssh public key: %v", err)
  135. }
  136. content = string(ssh.MarshalAuthorizedKey(sshKey))
  137. }
  138. // Parse OpenSSH format.
  139. // Remove all newlines
  140. content = strings.NewReplacer("\r\n", "", "\n", "").Replace(content)
  141. parts := strings.SplitN(content, " ", 3)
  142. switch len(parts) {
  143. case 0:
  144. return "", errors.New("empty key")
  145. case 1:
  146. keyContent = parts[0]
  147. case 2:
  148. keyType = parts[0]
  149. keyContent = parts[1]
  150. default:
  151. keyType = parts[0]
  152. keyContent = parts[1]
  153. keyComment = parts[2]
  154. }
  155. // If keyType is not given, extract it from content. If given, validate it.
  156. t, err := extractTypeFromBase64Key(keyContent)
  157. if err != nil {
  158. return "", fmt.Errorf("extractTypeFromBase64Key: %v", err)
  159. }
  160. if len(keyType) == 0 {
  161. keyType = t
  162. } else if keyType != t {
  163. return "", fmt.Errorf("key type and content does not match: %s - %s", keyType, t)
  164. }
  165. }
  166. // Finally we need to check whether we can actually read the proposed key:
  167. _, _, _, _, err := ssh.ParseAuthorizedKey([]byte(keyType + " " + keyContent + " " + keyComment))
  168. if err != nil {
  169. return "", fmt.Errorf("invalid ssh public key: %v", err)
  170. }
  171. return keyType + " " + keyContent + " " + keyComment, nil
  172. }
  173. // writeTmpKeyFile writes key content to a temporary file
  174. // and returns the name of that file, along with any possible errors.
  175. func writeTmpKeyFile(content string) (string, error) {
  176. tmpFile, err := ioutil.TempFile(setting.SSH.KeyTestPath, "gitea_keytest")
  177. if err != nil {
  178. return "", fmt.Errorf("TempFile: %v", err)
  179. }
  180. defer tmpFile.Close()
  181. if _, err = tmpFile.WriteString(content); err != nil {
  182. return "", fmt.Errorf("WriteString: %v", err)
  183. }
  184. return tmpFile.Name(), nil
  185. }
  186. // SSHKeyGenParsePublicKey extracts key type and length using ssh-keygen.
  187. func SSHKeyGenParsePublicKey(key string) (string, int, error) {
  188. // The ssh-keygen in Windows does not print key type, so no need go further.
  189. if setting.IsWindows {
  190. return "", 0, nil
  191. }
  192. tmpName, err := writeTmpKeyFile(key)
  193. if err != nil {
  194. return "", 0, fmt.Errorf("writeTmpKeyFile: %v", err)
  195. }
  196. defer os.Remove(tmpName)
  197. stdout, stderr, err := process.GetManager().Exec("SSHKeyGenParsePublicKey", setting.SSH.KeygenPath, "-lf", tmpName)
  198. if err != nil {
  199. return "", 0, fmt.Errorf("fail to parse public key: %s - %s", err, stderr)
  200. }
  201. if strings.Contains(stdout, "is not a public key file") {
  202. return "", 0, ErrKeyUnableVerify{stdout}
  203. }
  204. fields := strings.Split(stdout, " ")
  205. if len(fields) < 4 {
  206. return "", 0, fmt.Errorf("invalid public key line: %s", stdout)
  207. }
  208. keyType := strings.Trim(fields[len(fields)-1], "()\r\n")
  209. return strings.ToLower(keyType), com.StrTo(fields[0]).MustInt(), nil
  210. }
  211. // SSHNativeParsePublicKey extracts the key type and length using the golang SSH library.
  212. func SSHNativeParsePublicKey(keyLine string) (string, int, error) {
  213. fields := strings.Fields(keyLine)
  214. if len(fields) < 2 {
  215. return "", 0, fmt.Errorf("not enough fields in public key line: %s", keyLine)
  216. }
  217. raw, err := base64.StdEncoding.DecodeString(fields[1])
  218. if err != nil {
  219. return "", 0, err
  220. }
  221. pkey, err := ssh.ParsePublicKey(raw)
  222. if err != nil {
  223. if strings.Contains(err.Error(), "ssh: unknown key algorithm") {
  224. return "", 0, ErrKeyUnableVerify{err.Error()}
  225. }
  226. return "", 0, fmt.Errorf("ParsePublicKey: %v", err)
  227. }
  228. // The ssh library can parse the key, so next we find out what key exactly we have.
  229. switch pkey.Type() {
  230. case ssh.KeyAlgoDSA:
  231. rawPub := struct {
  232. Name string
  233. P, Q, G, Y *big.Int
  234. }{}
  235. if err := ssh.Unmarshal(pkey.Marshal(), &rawPub); err != nil {
  236. return "", 0, err
  237. }
  238. // as per https://bugzilla.mindrot.org/show_bug.cgi?id=1647 we should never
  239. // see dsa keys != 1024 bit, but as it seems to work, we will not check here
  240. return "dsa", rawPub.P.BitLen(), nil // use P as per crypto/dsa/dsa.go (is L)
  241. case ssh.KeyAlgoRSA:
  242. rawPub := struct {
  243. Name string
  244. E *big.Int
  245. N *big.Int
  246. }{}
  247. if err := ssh.Unmarshal(pkey.Marshal(), &rawPub); err != nil {
  248. return "", 0, err
  249. }
  250. return "rsa", rawPub.N.BitLen(), nil // use N as per crypto/rsa/rsa.go (is bits)
  251. case ssh.KeyAlgoECDSA256:
  252. return "ecdsa", 256, nil
  253. case ssh.KeyAlgoECDSA384:
  254. return "ecdsa", 384, nil
  255. case ssh.KeyAlgoECDSA521:
  256. return "ecdsa", 521, nil
  257. case ssh.KeyAlgoED25519:
  258. return "ed25519", 256, nil
  259. }
  260. return "", 0, fmt.Errorf("unsupported key length detection for type: %s", pkey.Type())
  261. }
  262. // CheckPublicKeyString checks if the given public key string is recognized by SSH.
  263. // It returns the actual public key line on success.
  264. func CheckPublicKeyString(content string) (_ string, err error) {
  265. if setting.SSH.Disabled {
  266. return "", ErrSSHDisabled{}
  267. }
  268. content, err = parseKeyString(content)
  269. if err != nil {
  270. return "", err
  271. }
  272. content = strings.TrimRight(content, "\n\r")
  273. if strings.ContainsAny(content, "\n\r") {
  274. return "", errors.New("only a single line with a single key please")
  275. }
  276. // remove any unnecessary whitespace now
  277. content = strings.TrimSpace(content)
  278. if !setting.SSH.MinimumKeySizeCheck {
  279. return content, nil
  280. }
  281. var (
  282. fnName string
  283. keyType string
  284. length int
  285. )
  286. if setting.SSH.StartBuiltinServer {
  287. fnName = "SSHNativeParsePublicKey"
  288. keyType, length, err = SSHNativeParsePublicKey(content)
  289. } else {
  290. fnName = "SSHKeyGenParsePublicKey"
  291. keyType, length, err = SSHKeyGenParsePublicKey(content)
  292. }
  293. if err != nil {
  294. return "", fmt.Errorf("%s: %v", fnName, err)
  295. }
  296. log.Trace("Key info [native: %v]: %s-%d", setting.SSH.StartBuiltinServer, keyType, length)
  297. if minLen, found := setting.SSH.MinimumKeySizes[keyType]; found && length >= minLen {
  298. return content, nil
  299. } else if found && length < minLen {
  300. return "", fmt.Errorf("key length is not enough: got %d, needs %d", length, minLen)
  301. }
  302. return "", fmt.Errorf("key type is not allowed: %s", keyType)
  303. }
  304. // appendAuthorizedKeysToFile appends new SSH keys' content to authorized_keys file.
  305. func appendAuthorizedKeysToFile(keys ...*PublicKey) error {
  306. // Don't need to rewrite this file if builtin SSH server is enabled.
  307. if setting.SSH.StartBuiltinServer {
  308. return nil
  309. }
  310. sshOpLocker.Lock()
  311. defer sshOpLocker.Unlock()
  312. if setting.SSH.RootPath != "" {
  313. // First of ensure that the RootPath is present, and if not make it with 0700 permissions
  314. // This of course doesn't guarantee that this is the right directory for authorized_keys
  315. // but at least if it's supposed to be this directory and it doesn't exist and we're the
  316. // right user it will at least be created properly.
  317. err := os.MkdirAll(setting.SSH.RootPath, 0700)
  318. if err != nil {
  319. log.Error("Unable to MkdirAll(%s): %v", setting.SSH.RootPath, err)
  320. return err
  321. }
  322. }
  323. fPath := filepath.Join(setting.SSH.RootPath, "authorized_keys")
  324. f, err := os.OpenFile(fPath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0600)
  325. if err != nil {
  326. return err
  327. }
  328. defer f.Close()
  329. // Note: chmod command does not support in Windows.
  330. if !setting.IsWindows {
  331. fi, err := f.Stat()
  332. if err != nil {
  333. return err
  334. }
  335. // .ssh directory should have mode 700, and authorized_keys file should have mode 600.
  336. if fi.Mode().Perm() > 0600 {
  337. log.Error("authorized_keys file has unusual permission flags: %s - setting to -rw-------", fi.Mode().Perm().String())
  338. if err = f.Chmod(0600); err != nil {
  339. return err
  340. }
  341. }
  342. }
  343. for _, key := range keys {
  344. if _, err = f.WriteString(key.AuthorizedString()); err != nil {
  345. return err
  346. }
  347. }
  348. return nil
  349. }
  350. // checkKeyFingerprint only checks if key fingerprint has been used as public key,
  351. // it is OK to use same key as deploy key for multiple repositories/users.
  352. func checkKeyFingerprint(e Engine, fingerprint string) error {
  353. has, err := e.Get(&PublicKey{
  354. Fingerprint: fingerprint,
  355. })
  356. if err != nil {
  357. return err
  358. } else if has {
  359. return ErrKeyAlreadyExist{0, fingerprint, ""}
  360. }
  361. return nil
  362. }
  363. func calcFingerprintSSHKeygen(publicKeyContent string) (string, error) {
  364. // Calculate fingerprint.
  365. tmpPath, err := writeTmpKeyFile(publicKeyContent)
  366. if err != nil {
  367. return "", err
  368. }
  369. defer os.Remove(tmpPath)
  370. stdout, stderr, err := process.GetManager().Exec("AddPublicKey", "ssh-keygen", "-lf", tmpPath)
  371. if err != nil {
  372. if strings.Contains(stderr, "is not a public key file") {
  373. return "", ErrKeyUnableVerify{stderr}
  374. }
  375. return "", fmt.Errorf("'ssh-keygen -lf %s' failed with error '%s': %s", tmpPath, err, stderr)
  376. } else if len(stdout) < 2 {
  377. return "", errors.New("not enough output for calculating fingerprint: " + stdout)
  378. }
  379. return strings.Split(stdout, " ")[1], nil
  380. }
  381. func calcFingerprintNative(publicKeyContent string) (string, error) {
  382. // Calculate fingerprint.
  383. pk, _, _, _, err := ssh.ParseAuthorizedKey([]byte(publicKeyContent))
  384. if err != nil {
  385. return "", err
  386. }
  387. return ssh.FingerprintSHA256(pk), nil
  388. }
  389. func calcFingerprint(publicKeyContent string) (string, error) {
  390. //Call the method based on configuration
  391. var (
  392. fnName, fp string
  393. err error
  394. )
  395. if setting.SSH.StartBuiltinServer {
  396. fnName = "calcFingerprintNative"
  397. fp, err = calcFingerprintNative(publicKeyContent)
  398. } else {
  399. fnName = "calcFingerprintSSHKeygen"
  400. fp, err = calcFingerprintSSHKeygen(publicKeyContent)
  401. }
  402. if err != nil {
  403. if IsErrKeyUnableVerify(err) {
  404. log.Info("%s", publicKeyContent)
  405. return "", err
  406. }
  407. return "", fmt.Errorf("%s: %v", fnName, err)
  408. }
  409. return fp, nil
  410. }
  411. func addKey(e Engine, key *PublicKey) (err error) {
  412. if len(key.Fingerprint) == 0 {
  413. key.Fingerprint, err = calcFingerprint(key.Content)
  414. if err != nil {
  415. return err
  416. }
  417. }
  418. // Save SSH key.
  419. if _, err = e.Insert(key); err != nil {
  420. return err
  421. }
  422. return appendAuthorizedKeysToFile(key)
  423. }
  424. // AddPublicKey adds new public key to database and authorized_keys file.
  425. func AddPublicKey(ownerID int64, name, content string, loginSourceID int64) (*PublicKey, error) {
  426. log.Trace(content)
  427. fingerprint, err := calcFingerprint(content)
  428. if err != nil {
  429. return nil, err
  430. }
  431. sess := x.NewSession()
  432. defer sess.Close()
  433. if err = sess.Begin(); err != nil {
  434. return nil, err
  435. }
  436. if err := checkKeyFingerprint(sess, fingerprint); err != nil {
  437. return nil, err
  438. }
  439. // Key name of same user cannot be duplicated.
  440. has, err := sess.
  441. Where("owner_id = ? AND name = ?", ownerID, name).
  442. Get(new(PublicKey))
  443. if err != nil {
  444. return nil, err
  445. } else if has {
  446. return nil, ErrKeyNameAlreadyUsed{ownerID, name}
  447. }
  448. key := &PublicKey{
  449. OwnerID: ownerID,
  450. Name: name,
  451. Fingerprint: fingerprint,
  452. Content: content,
  453. Mode: AccessModeWrite,
  454. Type: KeyTypeUser,
  455. LoginSourceID: loginSourceID,
  456. }
  457. if err = addKey(sess, key); err != nil {
  458. return nil, fmt.Errorf("addKey: %v", err)
  459. }
  460. return key, sess.Commit()
  461. }
  462. // GetPublicKeyByID returns public key by given ID.
  463. func GetPublicKeyByID(keyID int64) (*PublicKey, error) {
  464. key := new(PublicKey)
  465. has, err := x.
  466. ID(keyID).
  467. Get(key)
  468. if err != nil {
  469. return nil, err
  470. } else if !has {
  471. return nil, ErrKeyNotExist{keyID}
  472. }
  473. return key, nil
  474. }
  475. func searchPublicKeyByContentWithEngine(e Engine, content string) (*PublicKey, error) {
  476. key := new(PublicKey)
  477. has, err := e.
  478. Where("content like ?", content+"%").
  479. Get(key)
  480. if err != nil {
  481. return nil, err
  482. } else if !has {
  483. return nil, ErrKeyNotExist{}
  484. }
  485. return key, nil
  486. }
  487. // SearchPublicKeyByContent searches content as prefix (leak e-mail part)
  488. // and returns public key found.
  489. func SearchPublicKeyByContent(content string) (*PublicKey, error) {
  490. return searchPublicKeyByContentWithEngine(x, content)
  491. }
  492. // SearchPublicKey returns a list of public keys matching the provided arguments.
  493. func SearchPublicKey(uid int64, fingerprint string) ([]*PublicKey, error) {
  494. keys := make([]*PublicKey, 0, 5)
  495. cond := builder.NewCond()
  496. if uid != 0 {
  497. cond = cond.And(builder.Eq{"owner_id": uid})
  498. }
  499. if fingerprint != "" {
  500. cond = cond.And(builder.Eq{"fingerprint": fingerprint})
  501. }
  502. return keys, x.Where(cond).Find(&keys)
  503. }
  504. // ListPublicKeys returns a list of public keys belongs to given user.
  505. func ListPublicKeys(uid int64, listOptions ListOptions) ([]*PublicKey, error) {
  506. sess := x.Where("owner_id = ?", uid)
  507. if listOptions.Page != 0 {
  508. sess = listOptions.setSessionPagination(sess)
  509. keys := make([]*PublicKey, 0, listOptions.PageSize)
  510. return keys, sess.Find(&keys)
  511. }
  512. keys := make([]*PublicKey, 0, 5)
  513. return keys, sess.Find(&keys)
  514. }
  515. // ListPublicLdapSSHKeys returns a list of synchronized public ldap ssh keys belongs to given user and login source.
  516. func ListPublicLdapSSHKeys(uid int64, loginSourceID int64) ([]*PublicKey, error) {
  517. keys := make([]*PublicKey, 0, 5)
  518. return keys, x.
  519. Where("owner_id = ? AND login_source_id = ?", uid, loginSourceID).
  520. Find(&keys)
  521. }
  522. // UpdatePublicKeyUpdated updates public key use time.
  523. func UpdatePublicKeyUpdated(id int64) error {
  524. // Check if key exists before update as affected rows count is unreliable
  525. // and will return 0 affected rows if two updates are made at the same time
  526. if cnt, err := x.ID(id).Count(&PublicKey{}); err != nil {
  527. return err
  528. } else if cnt != 1 {
  529. return ErrKeyNotExist{id}
  530. }
  531. _, err := x.ID(id).Cols("updated_unix").Update(&PublicKey{
  532. UpdatedUnix: timeutil.TimeStampNow(),
  533. })
  534. if err != nil {
  535. return err
  536. }
  537. return nil
  538. }
  539. // deletePublicKeys does the actual key deletion but does not update authorized_keys file.
  540. func deletePublicKeys(e Engine, keyIDs ...int64) error {
  541. if len(keyIDs) == 0 {
  542. return nil
  543. }
  544. _, err := e.In("id", keyIDs).Delete(new(PublicKey))
  545. return err
  546. }
  547. // DeletePublicKey deletes SSH key information both in database and authorized_keys file.
  548. func DeletePublicKey(doer *User, id int64) (err error) {
  549. key, err := GetPublicKeyByID(id)
  550. if err != nil {
  551. return err
  552. }
  553. // Check if user has access to delete this key.
  554. if !doer.IsAdmin && doer.ID != key.OwnerID {
  555. return ErrKeyAccessDenied{doer.ID, key.ID, "public"}
  556. }
  557. sess := x.NewSession()
  558. defer sess.Close()
  559. if err = sess.Begin(); err != nil {
  560. return err
  561. }
  562. if err = deletePublicKeys(sess, id); err != nil {
  563. return err
  564. }
  565. if err = sess.Commit(); err != nil {
  566. return err
  567. }
  568. sess.Close()
  569. return RewriteAllPublicKeys()
  570. }
  571. // RewriteAllPublicKeys removes any authorized key and rewrite all keys from database again.
  572. // Note: x.Iterate does not get latest data after insert/delete, so we have to call this function
  573. // outside any session scope independently.
  574. func RewriteAllPublicKeys() error {
  575. return rewriteAllPublicKeys(x)
  576. }
  577. func rewriteAllPublicKeys(e Engine) error {
  578. //Don't rewrite key if internal server
  579. if setting.SSH.StartBuiltinServer || !setting.SSH.CreateAuthorizedKeysFile {
  580. return nil
  581. }
  582. sshOpLocker.Lock()
  583. defer sshOpLocker.Unlock()
  584. if setting.SSH.RootPath != "" {
  585. // First of ensure that the RootPath is present, and if not make it with 0700 permissions
  586. // This of course doesn't guarantee that this is the right directory for authorized_keys
  587. // but at least if it's supposed to be this directory and it doesn't exist and we're the
  588. // right user it will at least be created properly.
  589. err := os.MkdirAll(setting.SSH.RootPath, 0700)
  590. if err != nil {
  591. log.Error("Unable to MkdirAll(%s): %v", setting.SSH.RootPath, err)
  592. return err
  593. }
  594. }
  595. fPath := filepath.Join(setting.SSH.RootPath, "authorized_keys")
  596. tmpPath := fPath + ".tmp"
  597. t, err := os.OpenFile(tmpPath, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0600)
  598. if err != nil {
  599. return err
  600. }
  601. defer func() {
  602. t.Close()
  603. os.Remove(tmpPath)
  604. }()
  605. if setting.SSH.AuthorizedKeysBackup && com.IsExist(fPath) {
  606. bakPath := fmt.Sprintf("%s_%d.gitea_bak", fPath, time.Now().Unix())
  607. if err = com.Copy(fPath, bakPath); err != nil {
  608. return err
  609. }
  610. }
  611. if err := regeneratePublicKeys(e, t); err != nil {
  612. return err
  613. }
  614. t.Close()
  615. return os.Rename(tmpPath, fPath)
  616. }
  617. // RegeneratePublicKeys regenerates the authorized_keys file
  618. func RegeneratePublicKeys(t io.StringWriter) error {
  619. return regeneratePublicKeys(x, t)
  620. }
  621. func regeneratePublicKeys(e Engine, t io.StringWriter) error {
  622. err := e.Iterate(new(PublicKey), func(idx int, bean interface{}) (err error) {
  623. _, err = t.WriteString((bean.(*PublicKey)).AuthorizedString())
  624. return err
  625. })
  626. if err != nil {
  627. return err
  628. }
  629. fPath := filepath.Join(setting.SSH.RootPath, "authorized_keys")
  630. if com.IsExist(fPath) {
  631. f, err := os.Open(fPath)
  632. if err != nil {
  633. return err
  634. }
  635. scanner := bufio.NewScanner(f)
  636. for scanner.Scan() {
  637. line := scanner.Text()
  638. if strings.HasPrefix(line, tplCommentPrefix) {
  639. scanner.Scan()
  640. continue
  641. }
  642. _, err = t.WriteString(line + "\n")
  643. if err != nil {
  644. f.Close()
  645. return err
  646. }
  647. }
  648. f.Close()
  649. }
  650. return nil
  651. }
  652. // ________ .__ ____ __.
  653. // \______ \ ____ ______ | | ____ ___.__.| |/ _|____ ___.__.
  654. // | | \_/ __ \\____ \| | / _ < | || <_/ __ < | |
  655. // | ` \ ___/| |_> > |_( <_> )___ || | \ ___/\___ |
  656. // /_______ /\___ > __/|____/\____// ____||____|__ \___ > ____|
  657. // \/ \/|__| \/ \/ \/\/
  658. // DeployKey represents deploy key information and its relation with repository.
  659. type DeployKey struct {
  660. ID int64 `xorm:"pk autoincr"`
  661. KeyID int64 `xorm:"UNIQUE(s) INDEX"`
  662. RepoID int64 `xorm:"UNIQUE(s) INDEX"`
  663. Name string
  664. Fingerprint string
  665. Content string `xorm:"-"`
  666. Mode AccessMode `xorm:"NOT NULL DEFAULT 1"`
  667. CreatedUnix timeutil.TimeStamp `xorm:"created"`
  668. UpdatedUnix timeutil.TimeStamp `xorm:"updated"`
  669. HasRecentActivity bool `xorm:"-"`
  670. HasUsed bool `xorm:"-"`
  671. }
  672. // AfterLoad is invoked from XORM after setting the values of all fields of this object.
  673. func (key *DeployKey) AfterLoad() {
  674. key.HasUsed = key.UpdatedUnix > key.CreatedUnix
  675. key.HasRecentActivity = key.UpdatedUnix.AddDuration(7*24*time.Hour) > timeutil.TimeStampNow()
  676. }
  677. // GetContent gets associated public key content.
  678. func (key *DeployKey) GetContent() error {
  679. pkey, err := GetPublicKeyByID(key.KeyID)
  680. if err != nil {
  681. return err
  682. }
  683. key.Content = pkey.Content
  684. return nil
  685. }
  686. // IsReadOnly checks if the key can only be used for read operations
  687. func (key *DeployKey) IsReadOnly() bool {
  688. return key.Mode == AccessModeRead
  689. }
  690. func checkDeployKey(e Engine, keyID, repoID int64, name string) error {
  691. // Note: We want error detail, not just true or false here.
  692. has, err := e.
  693. Where("key_id = ? AND repo_id = ?", keyID, repoID).
  694. Get(new(DeployKey))
  695. if err != nil {
  696. return err
  697. } else if has {
  698. return ErrDeployKeyAlreadyExist{keyID, repoID}
  699. }
  700. has, err = e.
  701. Where("repo_id = ? AND name = ?", repoID, name).
  702. Get(new(DeployKey))
  703. if err != nil {
  704. return err
  705. } else if has {
  706. return ErrDeployKeyNameAlreadyUsed{repoID, name}
  707. }
  708. return nil
  709. }
  710. // addDeployKey adds new key-repo relation.
  711. func addDeployKey(e *xorm.Session, keyID, repoID int64, name, fingerprint string, mode AccessMode) (*DeployKey, error) {
  712. if err := checkDeployKey(e, keyID, repoID, name); err != nil {
  713. return nil, err
  714. }
  715. key := &DeployKey{
  716. KeyID: keyID,
  717. RepoID: repoID,
  718. Name: name,
  719. Fingerprint: fingerprint,
  720. Mode: mode,
  721. }
  722. _, err := e.Insert(key)
  723. return key, err
  724. }
  725. // HasDeployKey returns true if public key is a deploy key of given repository.
  726. func HasDeployKey(keyID, repoID int64) bool {
  727. has, _ := x.
  728. Where("key_id = ? AND repo_id = ?", keyID, repoID).
  729. Get(new(DeployKey))
  730. return has
  731. }
  732. // AddDeployKey add new deploy key to database and authorized_keys file.
  733. func AddDeployKey(repoID int64, name, content string, readOnly bool) (*DeployKey, error) {
  734. fingerprint, err := calcFingerprint(content)
  735. if err != nil {
  736. return nil, err
  737. }
  738. accessMode := AccessModeRead
  739. if !readOnly {
  740. accessMode = AccessModeWrite
  741. }
  742. sess := x.NewSession()
  743. defer sess.Close()
  744. if err = sess.Begin(); err != nil {
  745. return nil, err
  746. }
  747. pkey := &PublicKey{
  748. Fingerprint: fingerprint,
  749. }
  750. has, err := sess.Get(pkey)
  751. if err != nil {
  752. return nil, err
  753. }
  754. if has {
  755. if pkey.Type != KeyTypeDeploy {
  756. return nil, ErrKeyAlreadyExist{0, fingerprint, ""}
  757. }
  758. } else {
  759. // First time use this deploy key.
  760. pkey.Mode = accessMode
  761. pkey.Type = KeyTypeDeploy
  762. pkey.Content = content
  763. pkey.Name = name
  764. if err = addKey(sess, pkey); err != nil {
  765. return nil, fmt.Errorf("addKey: %v", err)
  766. }
  767. }
  768. key, err := addDeployKey(sess, pkey.ID, repoID, name, pkey.Fingerprint, accessMode)
  769. if err != nil {
  770. return nil, err
  771. }
  772. return key, sess.Commit()
  773. }
  774. // GetDeployKeyByID returns deploy key by given ID.
  775. func GetDeployKeyByID(id int64) (*DeployKey, error) {
  776. return getDeployKeyByID(x, id)
  777. }
  778. func getDeployKeyByID(e Engine, id int64) (*DeployKey, error) {
  779. key := new(DeployKey)
  780. has, err := e.ID(id).Get(key)
  781. if err != nil {
  782. return nil, err
  783. } else if !has {
  784. return nil, ErrDeployKeyNotExist{id, 0, 0}
  785. }
  786. return key, nil
  787. }
  788. // GetDeployKeyByRepo returns deploy key by given public key ID and repository ID.
  789. func GetDeployKeyByRepo(keyID, repoID int64) (*DeployKey, error) {
  790. return getDeployKeyByRepo(x, keyID, repoID)
  791. }
  792. func getDeployKeyByRepo(e Engine, keyID, repoID int64) (*DeployKey, error) {
  793. key := &DeployKey{
  794. KeyID: keyID,
  795. RepoID: repoID,
  796. }
  797. has, err := e.Get(key)
  798. if err != nil {
  799. return nil, err
  800. } else if !has {
  801. return nil, ErrDeployKeyNotExist{0, keyID, repoID}
  802. }
  803. return key, nil
  804. }
  805. // UpdateDeployKeyCols updates deploy key information in the specified columns.
  806. func UpdateDeployKeyCols(key *DeployKey, cols ...string) error {
  807. _, err := x.ID(key.ID).Cols(cols...).Update(key)
  808. return err
  809. }
  810. // UpdateDeployKey updates deploy key information.
  811. func UpdateDeployKey(key *DeployKey) error {
  812. _, err := x.ID(key.ID).AllCols().Update(key)
  813. return err
  814. }
  815. // DeleteDeployKey deletes deploy key from its repository authorized_keys file if needed.
  816. func DeleteDeployKey(doer *User, id int64) error {
  817. sess := x.NewSession()
  818. defer sess.Close()
  819. if err := sess.Begin(); err != nil {
  820. return err
  821. }
  822. if err := deleteDeployKey(sess, doer, id); err != nil {
  823. return err
  824. }
  825. return sess.Commit()
  826. }
  827. func deleteDeployKey(sess Engine, doer *User, id int64) error {
  828. key, err := getDeployKeyByID(sess, id)
  829. if err != nil {
  830. if IsErrDeployKeyNotExist(err) {
  831. return nil
  832. }
  833. return fmt.Errorf("GetDeployKeyByID: %v", err)
  834. }
  835. // Check if user has access to delete this key.
  836. if !doer.IsAdmin {
  837. repo, err := getRepositoryByID(sess, key.RepoID)
  838. if err != nil {
  839. return fmt.Errorf("GetRepositoryByID: %v", err)
  840. }
  841. has, err := isUserRepoAdmin(sess, repo, doer)
  842. if err != nil {
  843. return fmt.Errorf("GetUserRepoPermission: %v", err)
  844. } else if !has {
  845. return ErrKeyAccessDenied{doer.ID, key.ID, "deploy"}
  846. }
  847. }
  848. if _, err = sess.ID(key.ID).Delete(new(DeployKey)); err != nil {
  849. return fmt.Errorf("delete deploy key [%d]: %v", key.ID, err)
  850. }
  851. // Check if this is the last reference to same key content.
  852. has, err := sess.
  853. Where("key_id = ?", key.KeyID).
  854. Get(new(DeployKey))
  855. if err != nil {
  856. return err
  857. } else if !has {
  858. if err = deletePublicKeys(sess, key.KeyID); err != nil {
  859. return err
  860. }
  861. // after deleted the public keys, should rewrite the public keys file
  862. if err = rewriteAllPublicKeys(sess); err != nil {
  863. return err
  864. }
  865. }
  866. return nil
  867. }
  868. // ListDeployKeys returns all deploy keys by given repository ID.
  869. func ListDeployKeys(repoID int64, listOptions ListOptions) ([]*DeployKey, error) {
  870. return listDeployKeys(x, repoID, listOptions)
  871. }
  872. func listDeployKeys(e Engine, repoID int64, listOptions ListOptions) ([]*DeployKey, error) {
  873. sess := e.Where("repo_id = ?", repoID)
  874. if listOptions.Page != 0 {
  875. sess = listOptions.setSessionPagination(sess)
  876. keys := make([]*DeployKey, 0, listOptions.PageSize)
  877. return keys, sess.Find(&keys)
  878. }
  879. keys := make([]*DeployKey, 0, 5)
  880. return keys, sess.Find(&keys)
  881. }
  882. // SearchDeployKeys returns a list of deploy keys matching the provided arguments.
  883. func SearchDeployKeys(repoID int64, keyID int64, fingerprint string) ([]*DeployKey, error) {
  884. keys := make([]*DeployKey, 0, 5)
  885. cond := builder.NewCond()
  886. if repoID != 0 {
  887. cond = cond.And(builder.Eq{"repo_id": repoID})
  888. }
  889. if keyID != 0 {
  890. cond = cond.And(builder.Eq{"key_id": keyID})
  891. }
  892. if fingerprint != "" {
  893. cond = cond.And(builder.Eq{"fingerprint": fingerprint})
  894. }
  895. return keys, x.Where(cond).Find(&keys)
  896. }