Merge remote-tracking branch 'giteaofficial/main'

* giteaofficial/main:
  Prevent NPE whilst migrating if there is a team request review (go-gitea#19855)
  [skip ci] Updated translations via Crowdin
  Add support for rendering terminal output with colors (go-gitea#19497)
  Fix viewed images not loading in a PR (go-gitea#19919)
  Remove out-dated comments (go-gitea#19921)
  Automatically render wiki TOC (go-gitea#19873)
  Improve wording on delete access token modal (go-gitea#19909)
  [skip ci] Updated translations via Crowdin
  Add breaking email restrictions checker in doctor (go-gitea#19903)
  Ensure minimum mirror interval is reported on settings page (go-gitea#19895)
  Improve UX on modal for deleting an access token (go-gitea#19894)
  update discord invite (go-gitea#19907)
  Only log non ErrNotExist errors in git.GetNote  (go-gitea#19884)
  [skip ci] Updated translations via Crowdin
  Update frontend guideline (go-gitea#19901)
  Make AppDataPath absolute against the AppWorkPath if it is not (go-gitea#19815)

Author: zjjhot

.github/ISSUE_TEMPLATE/config.yml

@@ -4,7 +4,7 @@ contact_links:
     url: https://tinyurl.com/security-gitea
     about: For security concerns, please send a mail to [email protected] instead of opening a public issue.
   - name: Discord Server
-    url: https://discord.gg/gitea
+    url: https://discord.gg/Gitea
     about: Please ask questions and discuss configuration or deployment problems here.
   - name: Discourse Forum
     url: https://discourse.gitea.io

docs/content/doc/developers/guidelines-frontend.md

@@ -27,9 +27,9 @@ The HTML pages are rendered by [Go HTML Template](https://pkg.go.dev/html/templa
 
 The source files can be found in the following directories:
 * **Less styles:** `web_src/less/`
-* **Javascript files:** `web_src/js/`
-* **Vue layouts:** `web_src/js/components/`
-* **HTML templates:** `templates/`
+* **JavaScript files:** `web_src/js/`
+* **Vue components:** `web_src/js/components/`
+* **Go HTML templates:** `templates/`
 
 ## General Guidelines
 
@@ -40,24 +40,29 @@ We recommend [Google HTML/CSS Style Guide](https://google.github.io/styleguide/h
 1. Every feature (Fomantic-UI/jQuery module) should be put in separate files/directories.
 2. HTML ids and classes should use kebab-case.
 3. HTML ids and classes used in JavaScript should be unique for the whole project, and should contain 2-3 feature related keywords. We recommend to use the `js-` prefix for classes that are only used in JavaScript.
-4. jQuery events across different features should use their own namespaces.
-5. CSS styling for classes provided by frameworks should not be overwritten. Always use new class-names to overwrite framework styles. We recommend to use the `us-` prefix for user defined styles.  
+4. jQuery events across different features could use their own namespaces if there are potential conflicts.
+5. CSS styling for classes provided by frameworks should not be overwritten. Always use new class-names with 2-3 feature related keywords to overwrite framework styles.  
 6. The backend can pass complex data to the frontend by using `ctx.PageData["myModuleData"] = map[]{}`
 7. Simple pages and SEO-related pages use Go HTML Template render to generate static Fomantic-UI HTML output. Complex pages can use Vue2 (or Vue3 in future).
 
 
 ### Framework Usage
 
-Mixing different frameworks together is highly discouraged. A JavaScript module should follow one major framework and follow the framework's best practice.
+Mixing different frameworks together is discouraged, it makes the code difficult to be maintained.
+A JavaScript module should follow one major framework and follow the framework's best practice.
 
 Recommended implementations:
-* Vue + Native
+* Vue + Vanilla JS
 * Fomantic-UI (jQuery)
-* Native only
+* Vanilla JS
 
 Discouraged implementations:
-* Vue + jQuery
-* jQuery + Native
+* Vue + Fomantic-UI (jQuery)
+* jQuery + Vanilla JS
+
+To make UI consistent, Vue components can use Fomantic-UI CSS classes.
+Although mixing different frameworks is discouraged, 
+it should also work if the mixing is necessary and the code is well-designed and maintainable. 
 
 ### `async` Functions
 
@@ -75,7 +80,8 @@ Some lint rules and IDEs also have warnings if the returned Promise is not handl
 
 ### HTML Attributes and `dataset`
 
-We forbid `dataset` usage, its camel-casing behaviour makes it hard to grep for attributes. However there are still some special cases, so the current guideline is:
+The usage of `dataset` is forbidden, its camel-casing behaviour makes it hard to grep for attributes.
+However, there are still some special cases, so the current guideline is:
 
 * For legacy code:
   * `$.data()` should be refactored to `$.attr()`.
@@ -86,6 +92,10 @@ We forbid `dataset` usage, its camel-casing behaviour makes it hard to grep for
   * never bind any user data to a DOM node, use a suitable design pattern to describe the relation between node and data.
 
 
+### Legacy Code
+
+A lot of legacy code already existed before this document's written. It's recommended to refactor legacy code to follow the guidelines.
+
 ### Vue2/Vue3 and JSX
 
 Gitea is using Vue2 now, we plan to upgrade to Vue3. We decided not to introduce JSX to keep the HTML and the JavaScript code separated.

go.mod

@@ -15,6 +15,7 @@ require (
 	github.com/PuerkitoBio/goquery v1.8.0
 	github.com/alecthomas/chroma v0.10.0
 	github.com/blevesearch/bleve/v2 v2.3.2
+	github.com/buildkite/terminal-to-html/v3 v3.6.1
 	github.com/caddyserver/certmagic v0.16.1
 	github.com/chi-middleware/proxy v1.1.1
 	github.com/denisenkom/go-mssqldb v0.12.0
@@ -42,7 +43,7 @@ require (
 	github.com/gogs/cron v0.0.0-20171120032916-9f6c956d3e14
 	github.com/gogs/go-gogs-client v0.0.0-20210131175652-1d7215cd8d85
 	github.com/golang-jwt/jwt/v4 v4.4.1
-	github.com/google/go-github/v39 v39.2.0
+	github.com/google/go-github/v45 v45.0.0
 	github.com/google/pprof v0.0.0-20220509035851-59ca7ad80af3
 	github.com/google/uuid v1.3.0
 	github.com/gorilla/feeds v1.1.1

go.sum

@@ -265,6 +265,8 @@ github.com/boombuler/barcode v1.0.1/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl
 github.com/bradfitz/gomemcache v0.0.0-20190329173943-551aad21a668/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA=
 github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b h1:L/QXpzIa3pOvUGt1D1lA5KjYhPBAN/3iWdP7xeFS9F0=
 github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b/go.mod h1:H0wQNHz2YrLsuXOZozoeDmnHXkNCRmMW0gwFWDfEZDA=
+github.com/buildkite/terminal-to-html/v3 v3.6.1 h1:yHS+GXsPDXevb67YXjkVwZ4tolDCgPYa9RVOrzHlgGE=
+github.com/buildkite/terminal-to-html/v3 v3.6.1/go.mod h1:g0ME1XqbkBSgXR9YmlIHcJIjzaMyWW+HbsG0rPb5puo=
 github.com/caarlos0/ctrlc v1.0.0/go.mod h1:CdXpj4rmq0q/1Eb44M9zi2nKB0QraNKuRGYGrrHhcQw=
 github.com/caddyserver/certmagic v0.16.1 h1:rdSnjcUVJojmL4M0efJ+yHXErrrijS4YYg3FuwRdJkI=
 github.com/caddyserver/certmagic v0.16.1/go.mod h1:jKQ5n+ViHAr6DbPwEGLTSM2vDwTO6EvCKBblBRUvvuQ=
@@ -733,11 +735,11 @@ github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o=
 github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE=
+github.com/google/go-cmp v0.5.8 h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
 github.com/google/go-github/v28 v28.1.1/go.mod h1:bsqJWQX05omyWVmc00nEUql9mhQyv38lDZ8kPZcQVoM=
-github.com/google/go-github/v39 v39.2.0 h1:rNNM311XtPOz5rDdsJXAp2o8F67X9FnROXTvto3aSnQ=
-github.com/google/go-github/v39 v39.2.0/go.mod h1:C1s8C5aCC9L+JXIYpJM5GYytdX52vC1bLvHEF1IhBrE=
+github.com/google/go-github/v45 v45.0.0 h1:LU0WBjYidxIVyx7PZeWb+FP4JZJ3Wh3FQgdumnGqiLs=
+github.com/google/go-github/v45 v45.0.0/go.mod h1:FObaZJEDSTa/WGCzZ2Z3eoCDXWJKMenWWTrd8jrta28=
 github.com/google/go-licenses v0.0.0-20210329231322-ce1d9163b77d/go.mod h1:+TYOmkVoJOpwnS0wfdsJCV9CoD5nJYsHoFk/0CrTK4M=
 github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
 github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
@@ -1499,6 +1501,7 @@ github.com/urfave/cli v1.22.4/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtX
 github.com/urfave/cli v1.22.5/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
 github.com/urfave/cli v1.22.9 h1:cv3/KhXGBGjEXLC4bH0sLuJ9BewaAbpk5oyMOveu4pw=
 github.com/urfave/cli v1.22.9/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
+github.com/urfave/cli/v2 v2.2.0/go.mod h1:SE9GqnLQmjVa0iPEY0f1w3ygNIYcIJ0OKPMoW2caLfQ=
 github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
 github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8=
 github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw=

main.go

@@ -18,6 +18,7 @@ import (
 	"code.gitea.io/gitea/modules/setting"
 
 	// register supported doc types
+	_ "code.gitea.io/gitea/modules/markup/console"
 	_ "code.gitea.io/gitea/modules/markup/csv"
 	_ "code.gitea.io/gitea/modules/markup/markdown"
 	_ "code.gitea.io/gitea/modules/markup/orgmode"

modules/doctor/breaking.go

@@ -0,0 +1,69 @@
+// Copyright 2022 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package doctor
+
+import (
+	"context"
+	"fmt"
+
+	"code.gitea.io/gitea/models/db"
+	"code.gitea.io/gitea/models/user"
+	"code.gitea.io/gitea/modules/log"
+
+	"xorm.io/builder"
+)
+
+func iterateUserAccounts(ctx context.Context, each func(*user.User) error) error {
+	err := db.Iterate(
+		ctx,
+		new(user.User),
+		builder.Gt{"id": 0},
+		func(idx int, bean interface{}) error {
+			return each(bean.(*user.User))
+		},
+	)
+	return err
+}
+
+// Since 1.16.4 new restrictions has been set on email addresses. However users with invalid email
+// addresses would be currently facing a error due to their invalid email address.
+// Ref: https://github.com/go-gitea/gitea/pull/19085 & https://github.com/go-gitea/gitea/pull/17688
+func checkUserEmail(ctx context.Context, logger log.Logger, _ bool) error {
+	// We could use quirky SQL to get all users that start without a [a-zA-Z0-9], but that would mean
+	// DB provider-specific SQL and only works _now_. So instead we iterate trough all user accounts and
+	// use the user.ValidateEmail function to be future-proof.
+	var invalidUserCount int64
+	if err := iterateUserAccounts(ctx, func(u *user.User) error {
+		// Only check for users, skip
+		if u.Type != user.UserTypeIndividual {
+			return nil
+		}
+
+		if err := user.ValidateEmail(u.Email); err != nil {
+			invalidUserCount++
+			logger.Warn("User[id=%d name=%q] have not a valid e-mail: %v", u.ID, u.Name, err)
+		}
+		return nil
+	}); err != nil {
+		return fmt.Errorf("iterateUserAccounts: %v", err)
+	}
+
+	if invalidUserCount == 0 {
+		logger.Info("All users have a valid e-mail.")
+	} else {
+		logger.Warn("%d user(s) have a non-valid e-mail.", invalidUserCount)
+	}
+	return nil
+}
+
+func init() {
+	Register(&Check{
+		Title:     "Check if users has an valid email address",
+		Name:      "check-user-email",
+		IsDefault: false,
+		Run:       checkUserEmail,
+		Priority:  9,
+	})
+}

modules/git/notes_nogogit.go

@@ -46,7 +46,10 @@ func GetNote(ctx context.Context, repo *Repository, commitID string, note *Note)
 			commitID = commitID[2:]
 		}
 		if err != nil {
-			log.Error("Unable to find git note corresponding to the commit %q. Error: %v", originalCommitID, err)
+			// Err may have been updated by the SubTree we need to recheck if it's again an ErrNotExist
+			if !IsErrNotExist(err) {
+				log.Error("Unable to find git note corresponding to the commit %q. Error: %v", originalCommitID, err)
+			}
 			return err
 		}
 	}

modules/markup/console/console.go

@@ -0,0 +1,95 @@
+// Copyright 2022 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package console
+
+import (
+	"bytes"
+	"io"
+	"path/filepath"
+	"regexp"
+	"strings"
+
+	"code.gitea.io/gitea/modules/markup"
+	"code.gitea.io/gitea/modules/setting"
+
+	trend "github.com/buildkite/terminal-to-html/v3"
+	"github.com/go-enry/go-enry/v2"
+)
+
+// MarkupName describes markup's name
+var MarkupName = "console"
+
+func init() {
+	markup.RegisterRenderer(Renderer{})
+}
+
+// Renderer implements markup.Renderer
+type Renderer struct{}
+
+// Name implements markup.Renderer
+func (Renderer) Name() string {
+	return MarkupName
+}
+
+// NeedPostProcess implements markup.Renderer
+func (Renderer) NeedPostProcess() bool { return false }
+
+// Extensions implements markup.Renderer
+func (Renderer) Extensions() []string {
+	return []string{".sh-session"}
+}
+
+// SanitizerRules implements markup.Renderer
+func (Renderer) SanitizerRules() []setting.MarkupSanitizerRule {
+	return []setting.MarkupSanitizerRule{
+		{Element: "span", AllowAttr: "class", Regexp: regexp.MustCompile(`^term-((fg[ix]?|bg)\d+|container)$`)},
+	}
+}
+
+// SanitizerDisabled disabled sanitize if return true
+func (Renderer) SanitizerDisabled() bool {
+	return false
+}
+
+// CanRender implements markup.RendererContentDetector
+func (Renderer) CanRender(filename string, input io.Reader) bool {
+	buf, err := io.ReadAll(input)
+	if err != nil {
+		return false
+	}
+	if enry.GetLanguage(filepath.Base(filename), buf) != enry.OtherLanguage {
+		return false
+	}
+	return bytes.ContainsRune(buf, '\x1b')
+}
+
+// Render renders terminal colors to HTML with all specific handling stuff.
+func (Renderer) Render(ctx *markup.RenderContext, input io.Reader, output io.Writer) error {
+	buf, err := io.ReadAll(input)
+	if err != nil {
+		return err
+	}
+	buf = trend.Render(buf)
+	buf = bytes.ReplaceAll(buf, []byte("\n"), []byte(`<br>`))
+	_, err = output.Write(buf)
+	return err
+}
+
+// Render renders terminal colors to HTML with all specific handling stuff.
+func Render(ctx *markup.RenderContext, input io.Reader, output io.Writer) error {
+	if ctx.Type == "" {
+		ctx.Type = MarkupName
+	}
+	return markup.Render(ctx, input, output)
+}
+
+// RenderString renders terminal colors in string to HTML with all specific handling stuff and return string
+func RenderString(ctx *markup.RenderContext, content string) (string, error) {
+	var buf strings.Builder
+	if err := Render(ctx, strings.NewReader(content), &buf); err != nil {
+		return "", err
+	}
+	return buf.String(), nil
+}

modules/markup/console/console_test.go

@@ -0,0 +1,31 @@
+// Copyright 2022 The Gitea Authors. All rights reserved.
+// Use of this source code is governed by a MIT-style
+// license that can be found in the LICENSE file.
+
+package console
+
+import (
+	"strings"
+	"testing"
+
+	"code.gitea.io/gitea/modules/markup"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestRenderConsole(t *testing.T) {
+	var render Renderer
+	kases := map[string]string{
+		"\x1b[37m\x1b[40mnpm\x1b[0m \x1b[0m\x1b[32minfo\x1b[0m \x1b[0m\x1b[35mit worked if it ends with\x1b[0m ok": "<span class=\"term-fg37 term-bg40\">npm</span> <span class=\"term-fg32\">info</span> <span class=\"term-fg35\">it worked if it ends with</span> ok",
+	}
+
+	for k, v := range kases {
+		var buf strings.Builder
+		canRender := render.CanRender("test", strings.NewReader(k))
+		assert.True(t, canRender)
+
+		err := render.Render(&markup.RenderContext{}, strings.NewReader(k), &buf)
+		assert.NoError(t, err)
+		assert.EqualValues(t, v, buf.String())
+	}
+}