Fixes #17297

Fix regression from #16896

In order to work around https://github.com/mastodon/mastodon/issues/16895,
add a warning to .env.production.sample, and change the mastodon:setup rake
task to:
- output a warning if a variable will be interpreted differently by dotenv
  and docker-compose
- ensure the printed config is compatible with docker-compose

It appears assets are built during image build, and they shouldn't need
to be rebuilt, since we now have reproducible builds.

* Fix external user creation failing when invite request text is required

Also fixes tootctl-based user creation.

* Add test about invites when invite request text is otherwise required

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Fix a regression introduced in #13928, caused by TTY::Command building
shell commands by chaining string substitutions.

Ditch TTY::Command and use system instead (both do shell out).

When using one of the docker-compose containers, mastodon:setup will use the
existing .env.production rather than the generated one during the setup steps.

This is because dotenv does not overwrite env variables that are alreayd
defined, and the docker-compose.yml file loads the environment variables
from .env.production.

* Add support for special characters in Redis passwords

Fixes #13154

* Refactor

* Move more tasks to tootctl

- tootctl feeds build
- tootctl feeds clear
- tootctl accounts refresh

Clean up exit codes and help messages

* Move user modifying to tootctl

* Improve user modification through CLI, rename commands

add -> create
mod -> modify
del -> delete

To remove ambiguity

* Fix code style issues

* Fix not being able to unset admin/mod role

* rubocop: quit being so picky

* rubocop: miscellany

* rubocop: prefer present to blank

Remove old deprecated rake tasks

Still check for S3_CLOUDFRONT_HOST for existing installs.

./bin/tootctl media remove --days 7 --background

Make the old rake task point to it

* Speed up some rake tasks by moving execution to Sidekiq

mastodon:media:remove_silenced
mastodon:media:remove_remote
mastodon:media:redownload_avatars
mastodon:feeds:build

* Fix code style issue

* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts

Closes #7221

Comparison was downcasing only one side, therefore if previously
existing account had a non-lowercase spelling, it would be ignored
when checking for duplicates.

New rake task `mastodon:maintenance:find_duplicate_usernames` will
help find constraint violations that might have occured from the
presence of this bug.

Bump version to 2.3.3

HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.

* add detailed SMTP settings setup in mastodon:setup

* add localhost SMTP settings setup in mastodon:setup

* SMTP settings setup should exit after successful delivery of test mail

Media attachments are part of the association cache of statuses,
since they are presumed to be immutable. Unless this cache is
cleared manually, the statuses will continue to look like they
have media embedded.

* Some images can cause `convert` to fail, which crashes this whole task

* Add more specific exception

* Ensure the app does not even start if OTP_SECRET is not set

* Remove PAPERCLIP_SECRET (it's not used by anything, actually)

Imports are for internal consumption and the url option isn't even
used correctly, so we can remove the hash stuff from them

* Add better CLI prompt

* Add rake mastodon:setup interactive wizard

* Test db/redis/smtp configurations and add admin user at the end

* Test database connection even when database does not exist yet

* Fix regeneration key not getting expired

* Add rake task to remove old regeneration markers

* Add rake task to check and purge accounts that are missing in origin

* Add progress bar and --force options to mastodon:maintenance:purge_removed_accounts

* Add moderator role and add pundit policies for admin actions

* Add rake task for turning user into mod and revoking it again

* Fix handling of unauthorized exception

* Deliver new report e-mails to staff, not just admins

* Add promote/demote to admin UI, hide some actions conditionally

* Fix unused i18n

* When accessing uncached media attachment, redownload it

* Prevent re-download of rejected media

They are marked as read-only by Rails, but we know what we are doing,
so we are un-marking them as such.

The mastodon:maintenance:update_counter_caches task is not really
supposed to be run anymore (it was a one-time thing during an upgrade)
however, just in case, I have modified it to not touch ActivityPub
accounts.

Also, no point writing to logger from these rake tasks, since they
are not to be run from cron. Better to give stdout feedback.

* Use casecmp() instead of casecmp?() for now

casecmp?() is only available in ruby 2.4.0.  Users running earlier ruby versions
would see errors, e.g., running
RAILS_ENV=production rails mastodon:maintenance:remove_deprecated_preview_cards.

* Correctly check whether casecmp() returns 0

The path template of the attached files must explicitly be defined because
it is contradicting to the name of the class.

* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour

* ActivityPub migration procedure

Once one account is detected as going from OStatus to ActivityPub,
invalidate WebFinger cache for other accounts from the same domain

* Unsubscribe from PuSH updates once we receive an ActivityPub payload

* Re-subscribe to PuSH unless already unsubscribed, regardless of protocol

* Move clean up of unconfirmed users to sidekiq-scheduler

* mastodon:daily is now deprecated