Revamp post filtering system (#18058)
* Add model for custom filter keywords
* Use CustomFilterKeyword internally
Does not change the API
* Fix /filters/edit and /filters/new
* Add migration tests
* Remove whole_word column from custom_filters (covered by custom_filter_keywords)
* Redesign /filters
Instead of a list, present a card that displays more information and handles
multiple keywords per filter.
* Redesign /filters/new and /filters/edit to add and remove keywords
This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.
* Add /api/v2/filters to edit filter with multiple keywords
Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
`keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`
API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
`keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
`keywords_attributes` can also be passed to edit, delete or add keywords in
one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword
* Change from `irreversible` boolean to `action` enum
* Remove irrelevent `irreversible_must_be_within_context` check
* Fix /filters/new and /filters/edit with update for filter_action
* Fix Rubocop/Codeclimate complaining about task names
* Refactor FeedManager#phrase_filtered?
This moves regexp building and filter caching to the `CustomFilter` class.
This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.
* Perform server-side filtering and output result in REST API
* Fix numerous filters_changed events being sent when editing multiple keywords at once
* Add some tests
* Use the new API in the WebUI
- use client-side logic for filters we have fetched rules for.
This is so that filter changes can be retroactively applied without
reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
(e.g. network error, or initial timeline loading)
* Minor optimizations and refactoring
* Perform server-side filtering on the streaming server
* Change the wording of filter action labels
* Fix issues pointed out by linter
* Change design of “Show anyway” link in accordence to review comments
* Drop “irreversible” filtering behavior
* Move /api/v2/filter_keywords to /api/v1/filters/keywords
* Rename `filter_results` attribute to `filtered`
* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer
* Fix systemChannelId value in streaming server
* Simplify code by removing client-side filtering code
The simplifcation comes at a cost though: filters aren't retroactively
applied anymore. 2 years ago Revamp post filtering system (#18058)
* Add model for custom filter keywords
* Use CustomFilterKeyword internally
Does not change the API
* Fix /filters/edit and /filters/new
* Add migration tests
* Remove whole_word column from custom_filters (covered by custom_filter_keywords)
* Redesign /filters
Instead of a list, present a card that displays more information and handles
multiple keywords per filter.
* Redesign /filters/new and /filters/edit to add and remove keywords
This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.
* Add /api/v2/filters to edit filter with multiple keywords
Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
`keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`
API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
`keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
`keywords_attributes` can also be passed to edit, delete or add keywords in
one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword
* Change from `irreversible` boolean to `action` enum
* Remove irrelevent `irreversible_must_be_within_context` check
* Fix /filters/new and /filters/edit with update for filter_action
* Fix Rubocop/Codeclimate complaining about task names
* Refactor FeedManager#phrase_filtered?
This moves regexp building and filter caching to the `CustomFilter` class.
This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.
* Perform server-side filtering and output result in REST API
* Fix numerous filters_changed events being sent when editing multiple keywords at once
* Add some tests
* Use the new API in the WebUI
- use client-side logic for filters we have fetched rules for.
This is so that filter changes can be retroactively applied without
reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
(e.g. network error, or initial timeline loading)
* Minor optimizations and refactoring
* Perform server-side filtering on the streaming server
* Change the wording of filter action labels
* Fix issues pointed out by linter
* Change design of “Show anyway” link in accordence to review comments
* Drop “irreversible” filtering behavior
* Move /api/v2/filter_keywords to /api/v1/filters/keywords
* Rename `filter_results` attribute to `filtered`
* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer
* Fix systemChannelId value in streaming server
* Simplify code by removing client-side filtering code
The simplifcation comes at a cost though: filters aren't retroactively
applied anymore. 2 years ago Revamp post filtering system (#18058)
* Add model for custom filter keywords
* Use CustomFilterKeyword internally
Does not change the API
* Fix /filters/edit and /filters/new
* Add migration tests
* Remove whole_word column from custom_filters (covered by custom_filter_keywords)
* Redesign /filters
Instead of a list, present a card that displays more information and handles
multiple keywords per filter.
* Redesign /filters/new and /filters/edit to add and remove keywords
This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.
* Add /api/v2/filters to edit filter with multiple keywords
Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
`keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`
API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
`keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
`keywords_attributes` can also be passed to edit, delete or add keywords in
one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword
* Change from `irreversible` boolean to `action` enum
* Remove irrelevent `irreversible_must_be_within_context` check
* Fix /filters/new and /filters/edit with update for filter_action
* Fix Rubocop/Codeclimate complaining about task names
* Refactor FeedManager#phrase_filtered?
This moves regexp building and filter caching to the `CustomFilter` class.
This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.
* Perform server-side filtering and output result in REST API
* Fix numerous filters_changed events being sent when editing multiple keywords at once
* Add some tests
* Use the new API in the WebUI
- use client-side logic for filters we have fetched rules for.
This is so that filter changes can be retroactively applied without
reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
(e.g. network error, or initial timeline loading)
* Minor optimizations and refactoring
* Perform server-side filtering on the streaming server
* Change the wording of filter action labels
* Fix issues pointed out by linter
* Change design of “Show anyway” link in accordence to review comments
* Drop “irreversible” filtering behavior
* Move /api/v2/filter_keywords to /api/v1/filters/keywords
* Rename `filter_results` attribute to `filtered`
* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer
* Fix systemChannelId value in streaming server
* Simplify code by removing client-side filtering code
The simplifcation comes at a cost though: filters aren't retroactively
applied anymore. 2 years ago Revamp post filtering system (#18058)
* Add model for custom filter keywords
* Use CustomFilterKeyword internally
Does not change the API
* Fix /filters/edit and /filters/new
* Add migration tests
* Remove whole_word column from custom_filters (covered by custom_filter_keywords)
* Redesign /filters
Instead of a list, present a card that displays more information and handles
multiple keywords per filter.
* Redesign /filters/new and /filters/edit to add and remove keywords
This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.
* Add /api/v2/filters to edit filter with multiple keywords
Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
`keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`
API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
`keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
`keywords_attributes` can also be passed to edit, delete or add keywords in
one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword
* Change from `irreversible` boolean to `action` enum
* Remove irrelevent `irreversible_must_be_within_context` check
* Fix /filters/new and /filters/edit with update for filter_action
* Fix Rubocop/Codeclimate complaining about task names
* Refactor FeedManager#phrase_filtered?
This moves regexp building and filter caching to the `CustomFilter` class.
This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.
* Perform server-side filtering and output result in REST API
* Fix numerous filters_changed events being sent when editing multiple keywords at once
* Add some tests
* Use the new API in the WebUI
- use client-side logic for filters we have fetched rules for.
This is so that filter changes can be retroactively applied without
reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
(e.g. network error, or initial timeline loading)
* Minor optimizations and refactoring
* Perform server-side filtering on the streaming server
* Change the wording of filter action labels
* Fix issues pointed out by linter
* Change design of “Show anyway” link in accordence to review comments
* Drop “irreversible” filtering behavior
* Move /api/v2/filter_keywords to /api/v1/filters/keywords
* Rename `filter_results` attribute to `filtered`
* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer
* Fix systemChannelId value in streaming server
* Simplify code by removing client-side filtering code
The simplifcation comes at a cost though: filters aren't retroactively
applied anymore. 2 years ago |
|
- version: 2.1
-
- orbs:
- ruby: circleci/ruby@1.4.1
- node: circleci/node@5.0.1
-
- executors:
- default:
- parameters:
- ruby-version:
- type: string
- docker:
- - image: cimg/ruby:<< parameters.ruby-version >>
- environment:
- BUNDLE_JOBS: 3
- BUNDLE_RETRY: 3
- CONTINUOUS_INTEGRATION: true
- DB_HOST: localhost
- DB_USER: root
- DISABLE_SIMPLECOV: true
- RAILS_ENV: test
- - image: cimg/postgres:14.0
- environment:
- POSTGRES_USER: root
- POSTGRES_HOST_AUTH_METHOD: trust
- - image: cimg/redis:6.2
-
- commands:
- install-system-dependencies:
- steps:
- - run:
- name: Install system dependencies
- command: |
- sudo apt-get update
- sudo apt-get install -y libicu-dev libidn11-dev
- install-ruby-dependencies:
- parameters:
- ruby-version:
- type: string
- steps:
- - run:
- command: |
- bundle config clean 'true'
- bundle config frozen 'true'
- bundle config without 'development production'
- name: Set bundler settings
- - ruby/install-deps:
- bundler-version: '2.3.8'
- key: ruby<< parameters.ruby-version >>-gems-v1
- wait-db:
- steps:
- - run:
- command: dockerize -wait tcp://localhost:5432 -wait tcp://localhost:6379 -timeout 1m
- name: Wait for PostgreSQL and Redis
-
- jobs:
- build:
- docker:
- - image: cimg/ruby:3.0-node
- environment:
- RAILS_ENV: test
- steps:
- - checkout
- - install-system-dependencies
- - install-ruby-dependencies:
- ruby-version: '3.0'
- - node/install-packages:
- cache-version: v1
- pkg-manager: yarn
- - run:
- command: ./bin/rails assets:precompile
- name: Precompile assets
- - persist_to_workspace:
- paths:
- - public/assets
- - public/packs-test
- root: .
-
- test:
- parameters:
- ruby-version:
- type: string
- executor:
- name: default
- ruby-version: << parameters.ruby-version >>
- environment:
- ALLOW_NOPAM: true
- PAM_ENABLED: true
- PAM_DEFAULT_SERVICE: pam_test
- PAM_CONTROLLED_SERVICE: pam_test_controlled
- parallelism: 4
- steps:
- - checkout
- - install-system-dependencies
- - run:
- command: sudo apt-get install -y ffmpeg imagemagick libpam-dev
- name: Install additional system dependencies
- - run:
- command: bundle config with 'pam_authentication'
- name: Enable PAM authentication
- - install-ruby-dependencies:
- ruby-version: << parameters.ruby-version >>
- - attach_workspace:
- at: .
- - wait-db
- - run:
- command: ./bin/rails db:create db:schema:load db:seed
- name: Load database schema
- - ruby/rspec-test
-
- test-migrations:
- executor:
- name: default
- ruby-version: '3.0'
- steps:
- - checkout
- - install-system-dependencies
- - install-ruby-dependencies:
- ruby-version: '3.0'
- - wait-db
- - run:
- command: ./bin/rails db:create
- name: Create database
- - run:
- command: ./bin/rails db:migrate VERSION=20171010025614
- name: Run migrations up to v2.0.0
- - run:
- command: ./bin/rails tests:migrations:populate_v2
- name: Populate database with test data
- - run:
- command: ./bin/rails db:migrate VERSION=20180514140000
- name: Run migrations up to v2.4.0
- - run:
- command: ./bin/rails tests:migrations:populate_v2_4
- name: Populate database with test data
- - run:
- command: ./bin/rails db:migrate VERSION=20180707154237
- name: Run migrations up to v2.4.3
- - run:
- command: ./bin/rails tests:migrations:populate_v2_4_3
- name: Populate database with test data
- - run:
- command: ./bin/rails db:migrate
- name: Run all remaining migrations
- - run:
- command: ./bin/rails tests:migrations:check_database
- name: Check migration result
-
- test-two-step-migrations:
- executor:
- name: default
- ruby-version: '3.0'
- steps:
- - checkout
- - install-system-dependencies
- - install-ruby-dependencies:
- ruby-version: '3.0'
- - wait-db
- - run:
- command: ./bin/rails db:create
- name: Create database
- - run:
- command: ./bin/rails db:migrate VERSION=20171010025614
- name: Run migrations up to v2.0.0
- - run:
- command: ./bin/rails tests:migrations:populate_v2
- name: Populate database with test data
- - run:
- command: ./bin/rails db:migrate VERSION=20180514140000
- name: Run pre-deployment migrations up to v2.4.0
- environment:
- SKIP_POST_DEPLOYMENT_MIGRATIONS: true
- - run:
- command: ./bin/rails tests:migrations:populate_v2_4
- name: Populate database with test data
- - run:
- command: ./bin/rails db:migrate VERSION=20180707154237
- name: Run migrations up to v2.4.3
- environment:
- SKIP_POST_DEPLOYMENT_MIGRATIONS: true
- - run:
- command: ./bin/rails tests:migrations:populate_v2_4_3
- name: Populate database with test data
- - run:
- command: ./bin/rails db:migrate
- name: Run all remaining pre-deployment migrations
- environment:
- SKIP_POST_DEPLOYMENT_MIGRATIONS: true
- - run:
- command: ./bin/rails db:migrate
- name: Run all post-deployment migrations
- - run:
- command: ./bin/rails tests:migrations:check_database
- name: Check migration result
-
- workflows:
- version: 2
- build-and-test:
- jobs:
- - build
- - test:
- matrix:
- parameters:
- ruby-version:
- - '2.7'
- - '3.0'
- name: test-ruby<< matrix.ruby-version >>
- requires:
- - build
- - test-migrations:
- requires:
- - build
- - test-two-step-migrations:
- requires:
- - build
- - node/run:
- cache-version: v1
- name: test-webui
- pkg-manager: yarn
- requires:
- - build
- version: lts
- yarn-run: test:jest
|