You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

427 lines
14 KiB

Merge branch 'main' into glitch-soc/merge-upstream Conflicts: - `app/lib/activitypub/activity/create.rb`: Upstream refactored how `Create` activities are handled and how values are extracted from `Create`d objects. This conflicted with how glitch-soc supported the `directMessage` flag to explicitly distinguish between limited and direct messages. Ported glitch-soc's changes to latest upstream changes. - `app/services/fan_out_on_write_service.rb`: Upstream largely refactored that file and changed some of the logic. This conflicted with glitch-soc's handling of the direct timeline and the options to allow replies and boosts in public feeds. Ported those glitch-soc changes on top of latest upstream changes. - `app/services/process_mentions_service.rb`: Upstream refactored to move mention-related ActivityPub deliveries to `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check to not send local-only toots to remote mentioned users. Took upstream's version, as the check is not needed anymore, since it is performed at the `ActivityPub::DeliveryWorker` call site already. - `app/workers/feed_insert_worker.rb`: Upstream added support for `update` toot events, while glitch-soc had support for an extra timeline support, `direct`. Ported upstream changes and extended them to the `direct` timeline. Additional changes: - `app/lib/activitypub/parser/status_parser.rb`: Added code to handle the `directMessage` flag and take it into account to compute visibility. - `app/lib/feed_manager.rb`: Extended upstream's support of `update` toot events to glitch-soc's `direct` timeline.
2 years ago
Merge branch 'main' into glitch-soc/merge-upstream Conflicts: - `app/lib/activitypub/activity/create.rb`: Upstream refactored how `Create` activities are handled and how values are extracted from `Create`d objects. This conflicted with how glitch-soc supported the `directMessage` flag to explicitly distinguish between limited and direct messages. Ported glitch-soc's changes to latest upstream changes. - `app/services/fan_out_on_write_service.rb`: Upstream largely refactored that file and changed some of the logic. This conflicted with glitch-soc's handling of the direct timeline and the options to allow replies and boosts in public feeds. Ported those glitch-soc changes on top of latest upstream changes. - `app/services/process_mentions_service.rb`: Upstream refactored to move mention-related ActivityPub deliveries to `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check to not send local-only toots to remote mentioned users. Took upstream's version, as the check is not needed anymore, since it is performed at the `ActivityPub::DeliveryWorker` call site already. - `app/workers/feed_insert_worker.rb`: Upstream added support for `update` toot events, while glitch-soc had support for an extra timeline support, `direct`. Ported upstream changes and extended them to the `direct` timeline. Additional changes: - `app/lib/activitypub/parser/status_parser.rb`: Added code to handle the `directMessage` flag and take it into account to compute visibility. - `app/lib/feed_manager.rb`: Extended upstream's support of `update` toot events to glitch-soc's `direct` timeline.
2 years ago
  1. # frozen_string_literal: true
  2. class ActivityPub::Activity::Create < ActivityPub::Activity
  3. include FormattingHelper
  4. def perform
  5. dereference_object!
  6. case @object['type']
  7. when 'EncryptedMessage'
  8. create_encrypted_message
  9. else
  10. create_status
  11. end
  12. end
  13. private
  14. def create_encrypted_message
  15. return reject_payload! if invalid_origin?(object_uri) || @options[:delivered_to_account_id].blank?
  16. target_account = Account.find(@options[:delivered_to_account_id])
  17. target_device = target_account.devices.find_by(device_id: @object.dig('to', 'deviceId'))
  18. return if target_device.nil?
  19. target_device.encrypted_messages.create!(
  20. from_account: @account,
  21. from_device_id: @object.dig('attributedTo', 'deviceId'),
  22. type: @object['messageType'],
  23. body: @object['cipherText'],
  24. digest: @object.dig('digest', 'digestValue'),
  25. message_franking: message_franking.to_token
  26. )
  27. end
  28. def message_franking
  29. MessageFranking.new(
  30. hmac: @object.dig('digest', 'digestValue'),
  31. original_franking: @object['messageFranking'],
  32. source_account_id: @account.id,
  33. target_account_id: @options[:delivered_to_account_id],
  34. timestamp: Time.now.utc
  35. )
  36. end
  37. def create_status
  38. return reject_payload! if unsupported_object_type? || invalid_origin?(object_uri) || tombstone_exists? || !related_to_local_activity?
  39. with_lock("create:#{object_uri}") do
  40. return if delete_arrived_first?(object_uri) || poll_vote?
  41. @status = find_existing_status
  42. if @status.nil?
  43. process_status
  44. elsif @options[:delivered_to_account_id].present?
  45. postprocess_audience_and_deliver
  46. end
  47. end
  48. @status
  49. end
  50. def audience_to
  51. as_array(@object['to'] || @json['to']).map { |x| value_or_id(x) }
  52. end
  53. def audience_cc
  54. as_array(@object['cc'] || @json['cc']).map { |x| value_or_id(x) }
  55. end
  56. def process_status
  57. @tags = []
  58. @mentions = []
  59. @silenced_account_ids = []
  60. @params = {}
  61. process_status_params
  62. process_tags
  63. process_audience
  64. ApplicationRecord.transaction do
  65. @status = Status.create!(@params)
  66. attach_tags(@status)
  67. end
  68. resolve_thread(@status)
  69. fetch_replies(@status)
  70. distribute
  71. forward_for_reply
  72. end
  73. def distribute
  74. # Spread out crawling randomly to avoid DDoSing the link
  75. LinkCrawlWorker.perform_in(rand(1..59).seconds, @status.id)
  76. # Distribute into home and list feeds and notify mentioned accounts
  77. ::DistributionWorker.perform_async(@status.id, { 'silenced_account_ids' => @silenced_account_ids }) if @options[:override_timestamps] || @status.within_realtime_window?
  78. end
  79. def find_existing_status
  80. status = status_from_uri(object_uri)
  81. status ||= Status.find_by(uri: @object['atomUri']) if @object['atomUri'].present?
  82. status
  83. end
  84. def process_status_params
  85. @status_parser = ActivityPub::Parser::StatusParser.new(@json, followers_collection: @account.followers_url)
  86. @params = {
  87. uri: @status_parser.uri,
  88. url: @status_parser.url || @status_parser.uri,
  89. account: @account,
  90. text: converted_object_type? ? converted_text : (@status_parser.text || ''),
  91. language: @status_parser.language,
  92. spoiler_text: converted_object_type? ? '' : (@status_parser.spoiler_text || ''),
  93. created_at: @status_parser.created_at,
  94. edited_at: @status_parser.edited_at && @status_parser.edited_at != @status_parser.created_at ? @status_parser.edited_at : nil,
  95. override_timestamps: @options[:override_timestamps],
  96. reply: @status_parser.reply,
  97. sensitive: @account.sensitized? || @status_parser.sensitive || false,
  98. visibility: @status_parser.visibility,
  99. thread: replied_to_status,
  100. conversation: conversation_from_uri(@object['conversation']),
  101. media_attachment_ids: process_attachments.take(4).map(&:id),
  102. poll: process_poll,
  103. }
  104. end
  105. def process_audience
  106. # Unlike with tags, there is no point in resolving accounts we don't already
  107. # know here, because silent mentions would only be used for local access control anyway
  108. accounts_in_audience = (audience_to + audience_cc).uniq.filter_map do |audience|
  109. account_from_uri(audience) unless ActivityPub::TagManager.instance.public_collection?(audience)
  110. end
  111. # If the payload was delivered to a specific inbox, the inbox owner must have
  112. # access to it, unless they already have access to it anyway
  113. if @options[:delivered_to_account_id]
  114. accounts_in_audience << delivered_to_account
  115. accounts_in_audience.uniq!
  116. end
  117. accounts_in_audience.each do |account|
  118. # This runs after tags are processed, and those translate into non-silent
  119. # mentions, which take precedence
  120. next if @mentions.any? { |mention| mention.account_id == account.id }
  121. @mentions << Mention.new(account: account, silent: true)
  122. # If there is at least one silent mention, then the status can be considered
  123. # as a limited-audience status, and not strictly a direct message, but only
  124. # if we considered a direct message in the first place
  125. @params[:visibility] = :limited if @params[:visibility] == :direct && !@object['directMessage']
  126. end
  127. # Accounts that are tagged but are not in the audience are not
  128. # supposed to be notified explicitly
  129. @silenced_account_ids = @mentions.map(&:account_id) - accounts_in_audience.map(&:id)
  130. end
  131. def postprocess_audience_and_deliver
  132. return if @status.mentions.find_by(account_id: @options[:delivered_to_account_id])
  133. @status.mentions.create(account: delivered_to_account, silent: true)
  134. @status.update(visibility: :limited) if @status.direct_visibility? && !@object['directMessage']
  135. return unless delivered_to_account.following?(@account)
  136. FeedInsertWorker.perform_async(@status.id, delivered_to_account.id, 'home')
  137. end
  138. def delivered_to_account
  139. @delivered_to_account ||= Account.find(@options[:delivered_to_account_id])
  140. end
  141. def attach_tags(status)
  142. @tags.each do |tag|
  143. status.tags << tag
  144. tag.update(last_status_at: status.created_at) if tag.last_status_at.nil? || (tag.last_status_at < status.created_at && tag.last_status_at < 12.hours.ago)
  145. end
  146. # If we're processing an old status, this may register tags as being used now
  147. # as opposed to when the status was really published, but this is probably
  148. # not a big deal
  149. Trends.tags.register(status)
  150. @mentions.each do |mention|
  151. mention.status = status
  152. mention.save
  153. end
  154. end
  155. def process_tags
  156. return if @object['tag'].nil?
  157. as_array(@object['tag']).each do |tag|
  158. if equals_or_includes?(tag['type'], 'Hashtag')
  159. process_hashtag tag
  160. elsif equals_or_includes?(tag['type'], 'Mention')
  161. process_mention tag
  162. elsif equals_or_includes?(tag['type'], 'Emoji')
  163. process_emoji tag
  164. end
  165. end
  166. end
  167. def process_hashtag(tag)
  168. return if tag['name'].blank?
  169. Tag.find_or_create_by_names(tag['name']) do |hashtag|
  170. @tags << hashtag unless @tags.include?(hashtag) || !hashtag.valid?
  171. end
  172. rescue ActiveRecord::RecordInvalid
  173. nil
  174. end
  175. def process_mention(tag)
  176. return if tag['href'].blank?
  177. account = account_from_uri(tag['href'])
  178. account = ActivityPub::FetchRemoteAccountService.new.call(tag['href'], request_id: @options[:request_id]) if account.nil?
  179. return if account.nil?
  180. @mentions << Mention.new(account: account, silent: false)
  181. end
  182. def process_emoji(tag)
  183. return if skip_download?
  184. custom_emoji_parser = ActivityPub::Parser::CustomEmojiParser.new(tag)
  185. return if custom_emoji_parser.shortcode.blank? || custom_emoji_parser.image_remote_url.blank?
  186. emoji = CustomEmoji.find_by(shortcode: custom_emoji_parser.shortcode, domain: @account.domain)
  187. return unless emoji.nil? || custom_emoji_parser.image_remote_url != emoji.image_remote_url || (custom_emoji_parser.updated_at && custom_emoji_parser.updated_at >= emoji.updated_at)
  188. begin
  189. emoji ||= CustomEmoji.new(domain: @account.domain, shortcode: custom_emoji_parser.shortcode, uri: custom_emoji_parser.uri)
  190. emoji.image_remote_url = custom_emoji_parser.image_remote_url
  191. emoji.save
  192. rescue Seahorse::Client::NetworkingError => e
  193. Rails.logger.warn "Error storing emoji: #{e}"
  194. end
  195. end
  196. def process_attachments
  197. return [] if @object['attachment'].nil?
  198. media_attachments = []
  199. as_array(@object['attachment']).each do |attachment|
  200. media_attachment_parser = ActivityPub::Parser::MediaAttachmentParser.new(attachment)
  201. next if media_attachment_parser.remote_url.blank? || media_attachments.size >= 4
  202. begin
  203. media_attachment = MediaAttachment.create(
  204. account: @account,
  205. remote_url: media_attachment_parser.remote_url,
  206. thumbnail_remote_url: media_attachment_parser.thumbnail_remote_url,
  207. description: media_attachment_parser.description,
  208. focus: media_attachment_parser.focus,
  209. blurhash: media_attachment_parser.blurhash
  210. )
  211. media_attachments << media_attachment
  212. next if unsupported_media_type?(media_attachment_parser.file_content_type) || skip_download?
  213. media_attachment.download_file!
  214. media_attachment.download_thumbnail!
  215. media_attachment.save
  216. rescue Mastodon::UnexpectedResponseError, HTTP::TimeoutError, HTTP::ConnectionError, OpenSSL::SSL::SSLError
  217. RedownloadMediaWorker.perform_in(rand(30..600).seconds, media_attachment.id)
  218. rescue Seahorse::Client::NetworkingError => e
  219. Rails.logger.warn "Error storing media attachment: #{e}"
  220. end
  221. end
  222. media_attachments
  223. rescue Addressable::URI::InvalidURIError => e
  224. Rails.logger.debug { "Invalid URL in attachment: #{e}" }
  225. media_attachments
  226. end
  227. def process_poll
  228. poll_parser = ActivityPub::Parser::PollParser.new(@object)
  229. return unless poll_parser.valid?
  230. @account.polls.new(
  231. multiple: poll_parser.multiple,
  232. expires_at: poll_parser.expires_at,
  233. options: poll_parser.options,
  234. cached_tallies: poll_parser.cached_tallies,
  235. voters_count: poll_parser.voters_count
  236. )
  237. end
  238. def poll_vote?
  239. return false if replied_to_status.nil? || replied_to_status.preloadable_poll.nil? || !replied_to_status.local? || !replied_to_status.preloadable_poll.options.include?(@object['name'])
  240. poll_vote! unless replied_to_status.preloadable_poll.expired?
  241. true
  242. end
  243. def poll_vote!
  244. poll = replied_to_status.preloadable_poll
  245. already_voted = true
  246. with_lock("vote:#{replied_to_status.poll_id}:#{@account.id}") do
  247. already_voted = poll.votes.where(account: @account).exists?
  248. poll.votes.create!(account: @account, choice: poll.options.index(@object['name']), uri: object_uri)
  249. end
  250. increment_voters_count! unless already_voted
  251. ActivityPub::DistributePollUpdateWorker.perform_in(3.minutes, replied_to_status.id) unless replied_to_status.preloadable_poll.hide_totals?
  252. end
  253. def resolve_thread(status)
  254. return unless status.reply? && status.thread.nil? && Request.valid_url?(in_reply_to_uri)
  255. ThreadResolveWorker.perform_async(status.id, in_reply_to_uri, { 'request_id' => @options[:request_id] })
  256. end
  257. def fetch_replies(status)
  258. collection = @object['replies']
  259. return if collection.nil?
  260. replies = ActivityPub::FetchRepliesService.new.call(status, collection, allow_synchronous_requests: false, request_id: @options[:request_id])
  261. return unless replies.nil?
  262. uri = value_or_id(collection)
  263. ActivityPub::FetchRepliesWorker.perform_async(status.id, uri, { 'request_id' => @options[:request_id] }) unless uri.nil?
  264. end
  265. def conversation_from_uri(uri)
  266. return nil if uri.nil?
  267. return Conversation.find_by(id: OStatus::TagManager.instance.unique_tag_to_local_id(uri, 'Conversation')) if OStatus::TagManager.instance.local_id?(uri)
  268. begin
  269. Conversation.find_or_create_by!(uri: uri)
  270. rescue ActiveRecord::RecordInvalid, ActiveRecord::RecordNotUnique
  271. retry
  272. end
  273. end
  274. def replied_to_status
  275. return @replied_to_status if defined?(@replied_to_status)
  276. if in_reply_to_uri.blank?
  277. @replied_to_status = nil
  278. else
  279. @replied_to_status = status_from_uri(in_reply_to_uri)
  280. @replied_to_status ||= status_from_uri(@object['inReplyToAtomUri']) if @object['inReplyToAtomUri'].present?
  281. @replied_to_status
  282. end
  283. end
  284. def in_reply_to_uri
  285. value_or_id(@object['inReplyTo'])
  286. end
  287. def converted_text
  288. linkify([@status_parser.title.presence, @status_parser.spoiler_text.presence, @status_parser.url || @status_parser.uri].compact.join("\n\n"))
  289. end
  290. def unsupported_media_type?(mime_type)
  291. mime_type.present? && !MediaAttachment.supported_mime_types.include?(mime_type)
  292. end
  293. def skip_download?
  294. return @skip_download if defined?(@skip_download)
  295. @skip_download ||= DomainBlock.reject_media?(@account.domain)
  296. end
  297. def reply_to_local?
  298. !replied_to_status.nil? && replied_to_status.account.local?
  299. end
  300. def related_to_local_activity?
  301. fetch? || followed_by_local_accounts? || requested_through_relay? ||
  302. responds_to_followed_account? || addresses_local_accounts?
  303. end
  304. def responds_to_followed_account?
  305. !replied_to_status.nil? && (replied_to_status.account.local? || replied_to_status.account.passive_relationships.exists?)
  306. end
  307. def addresses_local_accounts?
  308. return true if @options[:delivered_to_account_id]
  309. local_usernames = (audience_to + audience_cc).uniq.select { |uri| ActivityPub::TagManager.instance.local_uri?(uri) }.map { |uri| ActivityPub::TagManager.instance.uri_to_local_id(uri, :username) }
  310. return false if local_usernames.empty?
  311. Account.local.where(username: local_usernames).exists?
  312. end
  313. def tombstone_exists?
  314. Tombstone.exists?(uri: object_uri)
  315. end
  316. def forward_for_reply
  317. return unless @status.distributable? && @json['signature'].present? && reply_to_local?
  318. ActivityPub::RawDistributionWorker.perform_async(Oj.dump(@json), replied_to_status.account_id, [@account.preferred_inbox_url])
  319. end
  320. def increment_voters_count!
  321. poll = replied_to_status.preloadable_poll
  322. unless poll.voters_count.nil?
  323. poll.voters_count = poll.voters_count + 1
  324. poll.save
  325. end
  326. rescue ActiveRecord::StaleObjectError
  327. poll.reload
  328. retry
  329. end
  330. end