Also, fix some n+1 queries

Resolve #10365

* Backend changes for custom emoji support in poll options

* Serialize poll emojis in REST API

* Render custom emojis in poll options

* Render custom emoji in poll options on public pages

* When serializing polls over OStatus, serialize poll options to text

* Do the same for RSS feeds

* Use “[ ] ” as a prefix for poll options instead of “- ”

Fix #9993
Fix #5654

* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( https://github.com/tootsuite/mastodon/issues/9989 )

* Allow unicode characters in URL query strings

Fixes #8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.

* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification

* Downcase signed_headers string before building the signed string

The HTTP Signatures draft does not mandate the “headers” field to be downcased,
but mandates the header field names to be downcased in the signed string, which
means that prior to this patch, Mastodon could fail to process signatures from
some compliant clients. It also means that it would not actually check the
Digest of non-compliant clients that wouldn't use a lowercased Digest field
name.

Thankfully, I don't know of any such client.

* Revert "Remove dead code (#8919)"

This reverts commit a00ce8c92c.

* Restore time window checking, change it to 12 hours

By checking the Date header, we can prevent replaying old vulnerable
signatures. The focus is to prevent replaying old vulnerable requests
from software that has been fixed in the meantime, so a somewhat long
window should be fine and accounts for timezone misconfiguration.

* Escape users' URLs when formatting them

Fixes possible HTML injection

* Escape all string interpolations in Formatter class

Slightly improve performance by reducing class allocations
from repeated Formatter#encode calls

* Fix code style issues

Add tests to avoid similar issues in the future

* Add animate custom emoji param to embed pages

* Rename param, use it for avatars and gifs

* Fix issues pointed by codeclimate and breaking test

* Ignore brakeman warning

Follow-up to #6124

* Add entity cache

Use a caching layer for mentions and custom emojis that are
dynamically extracted from text.

Reduce duplicate text extractions

* Fix code style issue

* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121

Federate custom emojis with accounts

* Don't normalize URLs in toots

URL normalization is ill-defined and may cause certain links to break.

* Change specs since we are not normalizing user-provided URLs

* Add semi-support for Video/Image objects in ActivityPub

Video and Image objects will create corresponding status records
with manually crafted text contents (title + URL)

* Extract html-url-finding logic into JsonLdHelper

* Fallback to id when url missing, extract supported object types

Use the non-animated version in web UI, but return both in API

* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted

Fixes #4496

* Remove some arguments of Formatter.instance.format

* Improve spec for Formatter

so that when a reblog parse fails on another instance, it doesn't
look like a misattributed/stolen text

* Fix regressions from #2683

Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses

* Fix lint issue

* Convert spoiler text to plaintext in desktop notifications

that caused user pages to break when showing toots with CWs

* services: scan spoiler_text for hashtags (#699)

* views: link hashtags from spoiler_texts

This covers linking hashtags from within the spoiler
text on the server-generated pages.

* services: fix string concat going into hashtag RE

Cleaner Ruby syntax, may handle immutable strings better

* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller

This commit fixes hashtag_html so it correctly handles matches with multiple hash-signs.

Bug located by @over9001, initial fix suggested by @nightpool.

* fix character escaping in URL

* add tests

* put a comma after the last item

* add HTML escape test

Fix username styling regression introduced in #1063
Fix report screen background regression introduced in #1415

When transmitting data in a HTML-encoded element like <content type="html" />,
relying on newlines being preserved is not wise, since HTML by itself
does not care for newlines - it cares for <p> and <br>

Additional fix: reset NSFW toggle after sending toot

* significant improvement in microformats markup

This is a huge improvement and I believe will close #965.

Had these microformats reviewed by others in the community to help
ensure they are at least correct, if not complete.

I did not want to change the structure of the page, and so there it does
not fully mark up the entire ancestry chain, or reply chain, only the
direct decendants and direct ancestors are correctly associated, but
this is likely fine as the most important bit is to have access to the
urls for those toots which are now correctly fetchable.

* improve code climate

* trying to pass code climate tests

* code climate

* fix p-summary for content warning posts

* fix error introduced when merging via github