Fix #9993
Fix #5654

* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( https://github.com/tootsuite/mastodon/issues/9989 )

* Allow unicode characters in URL query strings

Fixes #8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.

Add tests to avoid similar issues in the future

updates some "context" and "it" lines to have clearer explanations
updates "context" lines to properly describe function input, and "it" lines to describe results

Federate custom emojis with accounts

* Don't normalize URLs in toots

URL normalization is ill-defined and may cause certain links to break.

* Change specs since we are not normalizing user-provided URLs

* Sanitize classlist properly

* Actually properly sanitize every class after the first

* Improve Formatter spec to check for multiple classes and non-space whitespace

* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted

* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up

* Whitelist allowed classes for federated statuses

Allowed classes are currently:

 - Any microformats class (h/p/u/dt/e-*)
 - the classes mention, hashtag, ellipses and invisible.

this last one is somewhat suspect, but Mastodon currently uses it to render hidden link text.

resolved #3790

* Fix code style

* Remove some arguments of Formatter.instance.format

* Improve spec for Formatter

Resolve #3270

so that when a reblog parse fails on another instance, it doesn't
look like a misattributed/stolen text

* Fix regressions from #2683

Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses

* Fix lint issue

* Convert spoiler text to plaintext in desktop notifications

* fix character escaping in URL

* add tests

* put a comma after the last item

* add HTML escape test

Try to fix Travis CI setup

affected, replaced onClick handler with onMouseUp/Down to detect text
selection not trigger onClick handler then

to sign in page after sign up instead of root path which redirects to /about

as well as formatting of content