Commit Graph

1308 Commits

Author SHA1 Message Date
Claire 537afa00f3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/feed_manager.rb`:
  Not a real conflict, glitch-soc-only DM-related method
  too close to changed upstream stuff.
  Ported upstream changes.
- `app/services/batched_remove_status_service.rb`:
  Additional logic in glitch-soc to clear DMs from timelines.
  Ported upstream changes and fixed the DM TL clearing logic.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
  Additional code in glitch-soc to clear DM timelines.
  Ported upstream changes.
2020-12-23 01:47:45 +01:00
ThibG 3249d35bdc
Improve account deletion performances further (#15407)
* Delete status records by batches of 50

* Do not precompute values that are only used once

* Do not generate redis events for removal of public toots older than two weeks

* Filter reported toots a priori for polls and status deletion

* Do not process reblogs when cleaning up public timelines

As in Mastodon proper, reblogs don't appear in public TLs

* Clean the deleted account's own feed in one go

* Refactor Account#clean_feed_manager and List#clean_feed_manager

* Delete instead of destroy a few more associations

* Fix preloading

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 23:57:46 +01:00
Claire e4f8679eae Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/models/form/admin_settings.rb`:
  New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
  New setting added upstream. Ported it.
2020-12-15 14:27:06 +01:00
Eugen Rochko 1045549f85
Add stoplight for object storage failures, return HTTP 503 (#13043) 2020-12-15 12:55:29 +01:00
Claire a16175645d Merge branch 'master' into glitch-soc/merge-upstream 2020-12-09 14:42:33 +01:00
trwnh 127c543a6e
rename replies_policy enumerables (#15304) 2020-12-09 04:34:17 +01:00
Claire f8d867bac4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/services/remove_status_service.rb`:
  Conflict caused by us having a distinc Direct timeline.
  Ported upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
  Conflict between glitch-soc's variable character limit and upstream
  refactoring that part of the code.
  Ported upstream changes.
2020-12-05 17:33:37 +01:00
Eugen Rochko e7e099d1a0
Fix deletes not reaching every server that interacted with status (#15200)
Extract logic for determining ActivityPub inboxes to send deletes
to to its own class and explicitly include the person the status
replied to (even if not mentioned), people who favourited it, and
people who replied to it (though that one is still not recursive)
2020-11-27 15:48:31 +01:00
Thibaut Girka 48f0f3ffee Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.github/ISSUE_TEMPLATE/bug_report.md`:
  Upstream added the `bug` label to bug reports.
  Did the same.
- `app/services/fan_out_on_write_service.rb`:
  Upstream put DMs back into timelines, glitch-soc was already doing it.
  Ignored upstream changes.
2020-11-20 13:27:48 +01:00
ThibG 2f6831f318
Fix sending spurious Rejects when processing remote account deletion (#15104)
* Fix sending spurious Rejects when processing remote account deletion

* Make skip_side_effects imply skip_activitypub
2020-11-19 17:39:47 +01:00
Eugen Rochko df1653174b
Add cache buster feature for media files (#15155)
Nginx can be configured to bypass proxy cache when a special header
is in the request. If the response is cacheable, it will replace
the cache for that request. Proxy caching of media files is
desirable when using object storage as a way of minimizing bandwidth
costs, but has the drawback of leaving deleted media files for
a configured amount of cache time. A cache buster can make those
media files immediately unavailable. This especially makes sense
when suspending and unsuspending an account.
2020-11-19 17:38:06 +01:00
Thibaut Girka de5cc20dd8 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Conflict caused because of glitch-soc's theming system.
  Took upstream's new code and applied the theming system changes on top
  of it.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
  Conflict caused because of glitch-soc's theming system.
  Took upstream's new code and applied the theming system changes on top
  of it.
2020-11-14 00:30:36 +01:00
Eugen Rochko aa10200e58
Fix streaming API allowing connections to persist after access token invalidation (#15111)
Fix #14816
2020-11-12 23:05:24 +01:00
Thibaut Girka ecd83f495c Merge branch 'master' into glitch-soc/merge-upstream 2020-11-09 10:53:53 +01:00
ThibG b0893291ef
Fix Move handler not being triggered when failing to fetch target (#15107)
When failing to fetch the target account, the ProcessingWorker fails
as expected, but since it hasn't cleared the `move_in_progress` flag,
the next attempt at processing skips the `Move` activity altogether.

This commit changes it to clear the flag when encountering any
unexpected error on fetching the target account. This is likely to
occur because, of, e.g., a timeout, when many instances query the
same actor at the same time.
2020-11-09 01:41:52 +01:00
Thibaut Girka 0437d70628 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
  Conflict due to upstream changing suspension logic while
  glitch-soc has an extra option to hide followers count.
  Ported upstream changes.
2020-11-08 14:20:35 +01:00
Eugen Rochko 3134691948
Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
Thibaut Girka 412218af2e Merge branch 'master' into glitch-soc/merge-upstream 2020-11-07 18:23:02 +01:00
Takeshi Umeda d6fe0c94ca
Add account sensitized (#14361)
* Add account sensitized

* Fix i18n normalize

* Fix description and spec

* Fix spec

* Fix wording
2020-11-04 20:45:01 +01:00
Thibaut Girka ec49aa8175 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, we deleted it to not be flooded by Depandabot.
  Kept deleted.
- `Gemfile.lock`:
  Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
  Upstream updated the serializer to support timed mutes, while
  glitch-soc added a custom API ages ago to get information that
  is already available elsewhere.
  Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
  🤷
- `app/models/user.rb`:
  New user setting added upstream while we have glitch-soc-specific
  user settings. Added upstream's user setting.
- `config/settings.yml`:
  Upstream added a new user setting close to a user setting we had
  changed the defaults for. Added the new upstream setting.
- `package.json`:
  Upstream dependency updated “too close” to a glitch-soc-specific
  dependency. No real conflict. Updated the dependency.
2020-10-21 19:10:50 +02:00
ThibG ca56527140
Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
abcang 9649ca0fbe
Removed disabling comments for Style/MethodMissingSuper (#15014)
* Removed disabling comments for Style/MethodMissingSuper

* Update rubocop for codeclimate
2020-10-20 09:13:39 +02:00
Josh Leeb-du Toit 0c24f4dce2
Add support for Gemini urls (#15013)
This PR updates the `valid_url` regex and sanitizer allowlist to provide
support for Gemini urls.

Closes #14991
2020-10-19 17:02:13 +02:00
Takeshi Umeda b4c4af18dc
Fix a bear check when the activity object is nil (#14981) 2020-10-13 22:41:00 +02:00
Thibaut Girka 311f457430 Fix crash when autolinking an invalid URI in Markdown 2020-10-13 14:35:07 +02:00
Eugen Rochko 5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko 7d985f2aac
Remove dependency on goldfinger gem (#14919)
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.

Fix #14091
2020-10-08 00:34:57 +02:00
ThibG 78e45a5285
Add option to disable swiping motions across the WebUI (#13885)
Fixes #13882
2020-09-30 19:31:03 +02:00
Thibaut Girka a7aedebc31 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
  Upstream changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/models/user.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc not preventing moved accounts from logging
  in.
  Ported upstream changes while keeping the ability for moved accounts to log
  in.
- `app/policies/status_policy.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's ability  to hide followers count.
  Ported upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `package.json`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
2020-09-28 14:13:30 +02:00
ThibG d60290044e
Add environment variable to allow requests to some private addresses (#14722) 2020-09-26 20:57:39 +02:00
Eugen Rochko 974b1b79ce
Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
Eugen Rochko ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Thibaut Girka 9748f074a3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb

All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
2020-09-08 16:26:47 +02:00
Eugen Rochko 65760f59df
Refactor feed manager (#14761) 2020-09-08 03:41:16 +02:00
ThibG 517af45e32
Fix multiple boosts of a same toot erroneously appearing in TL (#14759)
* Check for and record reblog info atomically

Instead of using ZREVRANK to determine whether a reblog is a new reblog or not,
use ZADD's NX option to perform the check/addition option atomically.

* Replace ZREVRANK call with ZSCORE key which is more efficient

* Make tests a bit stricter

* Fix off-by-one
2020-09-07 18:00:15 +02:00
Thibaut Girka e5f934ddf0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict caused because we have additional code to make sure pinned
  local-only toots don't get rendered on the ActivityPub endpoints.
  Ported upstream changes.
2020-09-07 09:21:38 +02:00
Eugen Rochko 17340365bb
Add featured hashtags as an ActivityPub collection (#11595) 2020-09-02 02:11:12 +02:00
Thibaut Girka 3fefb7909b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/webpack/shared.js`:
  Upstream has changed how Tesseract.js gets included and dropped a dependency.
  The conflict is caused by glitch-soc having different code due to its
  theming system.
  Ported upstream changes.
- `lib/mastodon/version.rb`:
  Upstream refactor/code style change in a place we replaced upstream's
  repo URL with ours.
  Ported upstram changes, keeping our repo URL.
- `yarn.lock`:
  Upstream dropped dependencies, one of which was textually too close to
  a glitch-soc-specific dependency. Not a real conflict.
2020-09-01 17:02:18 +02:00
ThibG 79305428a7
Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00
Eugen Rochko 1c308af84c
Change own direct-visibility statuses to be in the home feed again (#14711)
And remove highlighting in web UI

Full circle from #8940
2020-09-01 13:30:42 +02:00
dependabot[bot] 8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
Thibaut Girka 8c3c27bf06 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream change too close to a glitch-soc change related to
  instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Minor conflict due to glitch-soc's handling of Direct Messages,
  merged upstream changes.
- `yarn.lock`:
  Not really a conflict, caused by glitch-soc-only dependencies
  being textually too close to updated upstream dependencies.
  Merged upstream changes.
2020-08-30 16:13:08 +02:00
Eugen Rochko 52157fdcba
Add support for dereferencing objects through bearcaps (#14683) 2020-08-30 12:34:20 +02:00
ThibG 92319d7340
Fix dereferencing remote statuses not using the correct account (#14656)
Follow-up to #14359

In the case of limited toots, the receiver may not be explicitly part of the
audience. If a specific user's inbox URI was specified, it makes sense to
dereference the toot from the corresponding user, instead of trying to find
someone in the explicit audience.
2020-08-24 16:56:21 +02:00
ThibG 720214feb0
Add support for inlined objects in activity audience (#14514)
* Add support for inlined objects in activity audience

* Add tests
2020-08-24 14:11:47 +02:00
Fire Demon 3d5b3e357c Add support for direct message assertions 2020-08-04 19:43:19 +02:00
Thibaut Girka 2cfa142961 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `yarn.lock`:
  No real conflict, just a glitch-soc-only dependency textually too close to
  one updated upstream. Ported upstream changes.
2020-08-01 20:24:23 +02:00
ThibG bfd5aea206
Fix handling of Reject Follow when a matching follow relationship exists (#14479)
* Add tests

* Fix handling of Reject Follow when a matching follow relationship exists

Regression from #12199
2020-08-01 18:20:37 +02:00
Thibaut Girka b9f351d845 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
  Not a “real” conflict, but change too close to a change we made to
  fix the vanilla WebUI locally pushing authored local-only toots in the
  public TL view.
2020-07-22 13:38:17 +02:00
ThibG 5d9acc0ce4
Fix not handling Undo on some activity types when they aren't inlined (#14346)
* Fix not handling Undo on some activity types when they aren't inlined

When receiving an Undo for a non-inlined activity, try looking it up in
database using the URI. The queries are ad-hoc because we don't have a global
index of object URIs, and not all activity types are stored in database with
an index on their URI.

Announces are just statuses, and have an index on URIs, so this check can
be done efficiently.

Accepts cannot be handled at all because we don't record their URI at any
point.

Follows don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

Likes don't have an index on URI, they have an index on the issuing account,
but the number of favs per account may be very high, so I decided not to
handle that.

Blocks don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.

In all cases, if an Undo could not be handled properly, we call `delete_later!`
because that does not require us to know more than the URI of the undone
property.

* Add tests

* Make newer blocks overwrite older ones

Allows re-synchronizing block info by re-blocking and un-blocking again
when the original Undo Block has been lost.
2020-07-22 11:45:35 +02:00
ThibG bcf85b5208
Dereference object URIs in Create and Update messages (#14359)
* Dereference object URIs in Create and Update messages

Fixes #14353

Signed-off-by: Thibaut Girka <thib@sitedethib.com>

* Refactor, and perform origin check *before* attempting to fetch object

Co-authored-by: Fire Demon <firedemon@creature.cafe>
2020-07-22 11:43:17 +02:00
Takeshi Umeda fcb3f259e5
Fix to add RedisLock to handle Announce activity (#14365) 2020-07-20 11:25:26 +02:00
Takeshi Umeda 7540e235a2
Fix movie width and frame_rate returning nil (#14357)
* Fix movie width and frame_rate returning nil

* Add StreamValidationError and raise

* Fix code style
2020-07-19 22:28:27 +02:00
ThibG 322d74fc2a
Fix boosted toots from blocked account not being retroactively removed from TL (#14339)
* Fix boosted toots from blocked account not being retroactively removed from TL

Fixes #14301

* Add test for clear_from_timeline
2020-07-17 07:07:54 +02:00
Thibaut Girka 3f60b096b5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/routes.rb`:
  Upstream disabled E2EE routes, which we did earlier, but slightly
  differently. Took upstream's version.
2020-07-15 15:32:40 +02:00
Takeshi Umeda 5c23ec2fa6
Improve group processing (#14279)
* Fix the local group's followers collection

* Fix to accept followed relayed_through_account

* Add local delivery to the group's followers

* Fix code style

* Revert "Add local delivery to the group's followers"

This reverts commit 3237effc199772e4c4d30f19082cbc5633f56196.
2020-07-14 19:05:34 +02:00
abcang 6a96af4d20
Fix rubocop warning (#14288)
* Fix rubocop warning

* use limit variable

* use ContextCreatingMethods option
2020-07-14 19:05:07 +02:00
Thibaut Girka 45deca65b3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Conflict because part of that file has been split to
  `app/javascript/core/settings.js`. Ported those changes
  there.
2020-07-01 19:23:14 +02:00
Eugen Rochko 6d23d40420
Change Redis#exists calls to Redis#exists? to avoid deprecation warning (#14191) 2020-07-01 19:05:21 +02:00
Thibaut Girka 39a0622de7 Add support for li attribute to li element 2020-07-01 01:02:27 +02:00
Thibaut Girka f186e0c91c Add support for “start” and “reversed” attributes to ol element
Fixes #1367
2020-07-01 01:02:27 +02:00
Eugen Rochko 7aaf2b44ec
Fix remote files not using Content-Type header, streaming (#14184) 2020-06-30 23:58:02 +02:00
Thibaut Girka 9c3b9d5ab9 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream deleted it, I decided to keep it.
- `package.json`:
  Upstream updated a dependency textually too close to wavesurfer.js
  which has been deleted from upstream but is kept in glitch-soc for now.
2020-06-30 23:00:20 +02:00
Eugen Rochko 64aac30733
Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Thibaut Girka a01674fb2b Merge branch 'master' into glitch-soc/merge-upstream 2020-06-26 13:02:14 +02:00
Sasha Sorokin f5cea4d2ea
Fix Thai being skipped from language detection (#13989)
Thai does not separate words by spaces, so I figured out it should be
in 'reliable characters regexp' that denotes languages that do the same.

Related #13891.
2020-06-25 22:45:01 +02:00
Thibaut Girka aae60a2366 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes.
2020-06-24 15:33:41 +02:00
ThibG 01a99f7ec7
Fix crash in MergeWorker (#14129)
Similarly to #12324, the code is passing an Account object where an id
is expected.
2020-06-23 16:40:01 +02:00
Thibaut Girka 12c8ac9e14 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict due to glitch-soc having to take care of local-only
  pinned toots in that controller.
  Took upstream's changes and restored the local-only special
  handling.
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflicts due to the theming system, applied upstream
  changes, adapted the following two files for glitch-soc's
  theming system:
  - `app/controllers/concerns/sign_in_token_authentication_concern.rb`
  - `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
  Minor conflict due to glitch-soc having to handle local-only
  toots specially. Applied upstream changes and restored
  the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
  Minor conflict due to the theming system.
- `package.json`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
- `yarn.lock`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
2020-06-09 10:39:20 +02:00
Ben Lubar c66403b257
FIX: filters ignore media descriptions (#13837)
* FIX: filters ignore media descriptions

* remove parentheses to make codeclimate happy

* combine the text and run the regular expression only once.

https://github.com/tootsuite/mastodon/pull/13837#discussion_r431752581

* Fix use of “filter” instead of “compact”, fix coding style issues

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-06-09 00:11:42 +02:00
Eugen Rochko 5d8398c8b8
Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
Ben Lubar ead09f5ddc
Fix exception when trying to serialize posts with <a> tags in them without hrefs (#1334)
* fix exception when trying to serialize posts with <a> tags in them without hrefs

* Add tests

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-05-28 12:47:40 +02:00
Thibaut Girka 27b5143dc4 Merge branch 'master' into glitch-soc/merge-upstream 2020-05-20 18:22:42 +02:00
ThibG a319c1e60f
Add support for `summary` field for media description (#13763) 2020-05-15 17:08:59 +02:00
ThibG 71fce71c94
Fix webfinger returning wrong status code on malformed or missing param (#13759)
Fixes #13757
2020-05-14 23:28:06 +02:00
Thibaut Girka 3baacf6993 Merge branch 'master' into glitch-soc/merge-upstream 2020-05-13 22:11:49 +02:00
Takeshi Umeda 5daae6dc61
Fix keybase config serializer (#13742) 2020-05-12 15:28:29 +02:00
Thibaut Girka 4a70792b4a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, just a glitch-soc-only dependency too close to a
  dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
  Not a real conflict, just a change too close to glitch-soc-changed code
  for optionally showing boosts in public timelines.
  Applied upstream changes.
- `app/views/layouts/application.html.haml`:
  Upstream a new, static CSS file, conflict due to glitch-soc's theming
  system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
  Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
  both files are very different. Removed 'unsafe-inline' as well.
2020-05-10 16:19:56 +02:00
Eugen Rochko e9ecbca70d
Fix error within error when limiting backtrace to 3 lines (#13120)
Fix #13086, close #13113
2020-05-10 10:30:27 +02:00
ThibG a4240fd027
Improve RSS entries for statuses (#13592)
* Improve RSS entries for statuses

- Render polls in both accounts and tags serializers
- Refactor RSS serializers
- Change title preview to include ellipsis when truncated
- Change title preview to show CW instead of toot text
- Add tests

* Remove title from OEmbed serialization

Twitter doesn't serialize title either, and tihs allows us to move the
title formatting code to the RSS serializers.
2020-05-10 09:50:54 +02:00
Thibaut Girka c6ff4c634c Fix crash when invalid addresses are linked in HTML or Markdown toots 2020-05-09 16:01:00 +02:00
Thibaut Girka 81e49ba5c6 Merge branch 'master' into glitch-soc/merge-upstream 2020-04-17 20:16:24 +02:00
Eugen Rochko 5edff32733
Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
Thibaut Girka 9101254d0a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Upstream modified code that we split between multiple files due
  to glitch-soc's theming system.
  Ported those changes.
2020-04-05 18:32:06 +02:00
Eugen Rochko c9efb400b4
Add rate limit for reporting (#13390) 2020-04-05 14:40:08 +02:00
ThibG 89e28c7674
Fix PostgreSQL load when linking in announcements (#13250)
* Fix PostgreSQL load when linking in announcements

Fixes #13245 by caching status lookups

Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.

Since it's only used by announcements so far, there won't
be much statuses to cache.

* Perform status lookup when saving announcements, not when rendering them

* Change EntityCache#status to fetch URLs instead of looking into the database

* Move announcement link lookup to publishing worker

* Address issues pointed out during review
2020-04-05 12:51:22 +02:00
Thibaut Girka 7b435fd9bf Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/services/post_status_service.rb`:
  CW/sensitive logic fixed upstream, but different in glitch-soc.
  Ported the changes accordingly.
2020-03-27 21:54:44 +01:00
Eugen Rochko 0c8945e5ff
Change `tootctl media remove-orphans` to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
2020-03-26 01:56:41 +01:00
Thibaut Girka 02f1c04fab Make sanitizer *not* add no-referrer etc. in local markdown toots if the link is “safe” 2020-03-24 12:07:47 +01:00
Thibaut Girka 2c510ee00a Fix glitch-soc marking every link in toots as a tag
Fixes #1281
2020-03-24 12:07:47 +01:00
Thibaut Girka 9abb227250 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
  Updating rails-ujs, no real conflict, but a comment to close to changed
  code. Various glitch-soc-only files have been updated to match those changes,
  though.
- `package.json`:
  No real conflict, just an additional dependency in glitch-soc that was too
  close to something updated upstream. Took upstream's changes.
2020-03-22 16:10:44 +01:00
dependabot-preview[bot] 7088633ae1
Bump cld3 from 3.2.6 to 3.3.0 (#13107)
* Bump cld3 from 3.2.6 to 3.3.0

Bumps [cld3](https://github.com/akihikodaki/cld3-ruby) from 3.2.6 to 3.3.0.
- [Release notes](https://github.com/akihikodaki/cld3-ruby/releases)
- [Commits](https://github.com/akihikodaki/cld3-ruby/compare/v3.2.6...v3.3.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix compatibility with cld3 3.3.0

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2020-03-09 00:12:52 +01:00
Thibaut Girka c790ecb14d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
  Conflict due to upstream adding a new parameter (with_rate_limit),
  too close to glitch-soc's own additional parameter (content_type).
  Added upstream's parameter.
- `app/services/post_status_service.rb`:
  Conflict due to upstream adding a new parameter (rate_limit),
  too close to glitch-soc's own additional parameter (content_type).
  Added upstream's parameter.
- `app/views/settings/preferences/appearance/show.html.haml`:
  Conflict due to us not exposing theme settings here (as we have
  a different flavour/skin menu).
  Took upstream change, while still not exposing theme settings.
- `config/webpack/shared.js`:
  Coding style fixes for a part we have rewritten.
  Discarded upstream changes.
2020-03-08 19:38:53 +01:00
Eugen Rochko 339ce1c4e9
Add specific rate limits for posting and following (#13172) 2020-03-08 15:17:39 +01:00
Thibaut Girka dae5e446fe Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile`:
  We updated httplog in a separate commit.
  Took upstream's change which updated it further.
- `Gemfile.lock`:
  We updated httplog in a separate commit.
  Took upstream's change which updated it further.
- `app/lib/sanitize_config.rb`:
  Upstream added better unsupported link stripping,
  while we had different sanitizing configs.
  Took only upstream's link stripping code.
- `config/locales/simple_form.pl.yml`:
  Strings unused in glitch-soc had been removed from
  glitch-soc, reintroduced them even if they are not
  useful, to reduce the risk of later merge conflicts.
2020-02-09 12:15:55 +01:00
Eugen Rochko b1349342d2
Fix rendering `<a>` without `href` when scheme unsupported (#13040)
- Disallow links with relative paths
- Disallow iframes with non-http protocols and relative paths

Close #13037
2020-02-08 21:22:38 +01:00
Eugen Rochko a64973aecf
Fix malformed HTML causing uncaught error (#13042)
Fix OEmbed preview API leaking existence of private statuses (see #12930)
2020-02-07 15:24:22 +01:00
Thibaut Girka c56a504d11 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/serializers/rest/account_serializer.rb`:
  Upstream added code too close to glitch-soc-specific followers-hiding code.
  Ported upstream changes.
2020-01-27 15:46:50 +01:00
Eugen Rochko b9d74d4076
Add streaming API updates for announcements being modified or deleted (#12963)
Change `all_day` to be a visual client-side cue only

Publish immediately if `scheduled_at` is in the past

Add `published_at` and `updated_at` to announcements JSON
2020-01-26 20:07:26 +01:00
Thibaut Girka 9adeaf2bfc Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Minor conflict due to theming system
2020-01-24 14:37:06 +01:00
Eugen Rochko f52c988e12
Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
2020-01-23 22:00:13 +01:00
ThibG a8e46cf7a1 Add support for magnet: URIs (#12905) 2020-01-23 21:27:26 +01:00
Thibaut Girka 45709d6987 Merge branch 'master' into glitch-soc/master
Conflicts:
- `README.md`:
  We have different README files. Discarded upstream changes.
- `app/views/layouts/admin.html.haml`:
  Conflict due to glitch-soc theming system.
  Adapted upstream changes.
- `app/views/layouts/embedded.html.haml`:
  Conflict due to glitch-soc theming system.
  Adapted upstream changes.
- `yarn.lock`:
  No real conflict, glitch-specific dependency too close to
  an updated one. Adapted upstream change.
2020-01-20 15:00:22 +01:00
ThibG 345bba4812 Exempt staff members from spam check (#12874)
Consider admins and moderators as trusted, for the purpose of the
spam checker.

Fixes #12872
2020-01-17 07:50:30 +01:00
Thibaut Girka 980c004f06 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  No real conflict, glitch-soc-only dependency (redcarpet) too close to an
  upstream one (rdf-normalize)
- `README.md`:
  we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
  No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
  Upstream added code to keep bookmarked statuses, we were already doing so
  with slightly different code. Discarded upstream's changes.
- `package.json`:
  No real conflict, glitch-soc-only dependency (favico.js) too close to
  an upstream one
2020-01-12 15:57:34 +01:00
ThibG 24cd2126c6 Fix Ruby 2.7 support (#12831)
- update http gem to avoid errors
- update blurhash gem to avoid shared object loading error
- update goldfinger gem so the http gem could be updated
- update json gem to avoid warnings
2020-01-12 15:49:30 +01:00
ThibG ea436b355b Add support for linking XMPP URIs in toots (#12709)
* Fix wrong grouping in Twitter valid_url regex

* Add support for xmpp URIs

Fixes #9776

The difficult part is autolinking, because Twitter-text's extractor does
some pretty ad-hoc stuff to find things that “look like” URLs, and XMPP
URIs do not really match the assumptions of that lib, so it doesn't sound
wise to try to shoehorn it into the existing regex.

This is why I used a specific regex (very close, although slightly more
permissive than the RFC), and a specific scan function (a simplified version
of the generalized one from Twitter).

* Remove leading “xmpp:” from auto-linked text
2020-01-11 02:15:25 +01:00
Daigo 3 Dango e9ea09d173 Suppress backtrace when delivering toots (#12798)
This is to suppress irrelevant backtrace from errors raised when
delivering toots to remote servers. The errors are usually out of
control by the local server and backtraces don't provide much
information.

This is similar to https://github.com/tootsuite/mastodon/pull/5174
and shortens backtraces like below:

```
WARN: Mastodon::UnexpectedResponseError: https://example.com/inbox returned code 523
WARN: app/workers/activitypub/delivery_worker.rb:48:in `block (3 levels) in perform_request'
app/lib/request.rb:75:in `perform'
app/workers/activitypub/delivery_worker.rb:47:in `block (2 levels) in perform_request'
app/lib/request_pool.rb:53:in `use'
app/lib/request_pool.rb:108:in `block (2 levels) in with'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/notifications.rb:170:in `instrument'
app/lib/request_pool.rb:107:in `block in with'
app/lib/connection_pool/shared_connection_pool.rb:21:in `block (2 levels) in with'
app/lib/connection_pool/shared_connection_pool.rb:20:in `handle_interrupt'
app/lib/connection_pool/shared_connection_pool.rb:20:in `block in with'
app/lib/connection_pool/shared_connection_pool.rb:16:in `handle_interrupt'
app/lib/connection_pool/shared_connection_pool.rb:16:in `with'
app/lib/request_pool.rb:106:in `with'
app/workers/activitypub/delivery_worker.rb:46:in `block in perform_request'
vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:51:in `run_code'
vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:42:in `run_yellow'
vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:24:in `run'
app/workers/activitypub/delivery_worker.rb:57:in `perform_request'
app/workers/activitypub/delivery_worker.rb:25:in `perform'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:31:in `block in call'
vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/statsd/publisher.rb:27:in `statsd_time'
vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:30:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
app/lib/sidekiq_error_handler.rb:5:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/scout_apm-2.3.0.pre3/lib/scout_apm/background_job_integrations/sidekiq.rb:69:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/sidekiq-unique-jobs-6.0.18/lib/sidekiq_unique_jobs/server/middleware.rb:29:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:73:in `block in wrap'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:72:in `wrap'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
```

```
WARN: Stoplight::Error::RedLight: https://example.com/inbox
WARN: vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:46:in `run_red'
vendor/bundle/ruby/2.7.0/gems/stoplight-2.2.0/lib/stoplight/light/runnable.rb:25:in `run'
app/workers/activitypub/delivery_worker.rb:57:in `perform_request'
app/workers/activitypub/delivery_worker.rb:25:in `perform'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:31:in `block in call'
vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/statsd/publisher.rb:27:in `statsd_time'
vendor/bundle/ruby/2.7.0/gems/nsa-0.2.7/lib/nsa/collectors/sidekiq.rb:30:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
app/lib/sidekiq_error_handler.rb:5:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/scout_apm-2.3.0.pre3/lib/scout_apm/background_job_integrations/sidekiq.rb:69:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/sidekiq-unique-jobs-6.0.18/lib/sidekiq_unique_jobs/server/middleware.rb:29:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:73:in `block in wrap'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/execution_wrapper.rb:87:in `wrap'
vendor/bundle/ruby/2.7.0/gems/activesupport-5.2.4.1/lib/active_support/reloader.rb:72:in `wrap'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in `call'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
vendor/bundle/ruby/2.7.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
```
2020-01-11 02:15:03 +01:00
Daigo 3 Dango 206dfd7dad Clarify keyword arguments with ** (#12769)
This change is to suppress the warning below on on ruby-2.7.0:

- warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call

https://www.ruby-lang.org/en/news/2019/12/12/separation-of-positional-and-keyword-arguments-in-ruby-3-0/
2020-01-11 05:57:05 +09:00
Thibaut Girka 01eaeab56d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/application_controller.rb`:
  Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
  Conflict due to theming system.
2020-01-04 23:04:42 +01:00
Eugen Rochko bb27ca14ef
Fix error when searching for empty phrase (#12711)
Fix #12619
2019-12-30 18:30:24 +01:00
Thibaut Girka ab5ff638f2 Merge branch 'master' into glitch-soc/master
Conflicts:
- `config/locales/en.yml`
  No real conflict, upstream added a translatable string “too close” to
  one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
  Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
  No real conflict, additional dependency in glitch-soc
2019-12-19 13:52:54 +01:00
ThibG 2ee5a9d9c3 Clean up OStatus-related codepaths (#12173)
* Remove “protocol” argument and return value, as only ActivityPub is supported

* Remove FetchRemoteAccountService, only use ActivityPub::FetchRemoteAccountService

* Fix tests
2019-12-17 13:32:57 +01:00
ThibG da2143b308 Fixes featured hashtag setting page erroring out instead of rejecting invalid tags (#12436)
* Revert "Fix ignoring whole status because of one invalid hashtag (#11621)"

This reverts commit dff46b260b.

* Fix statuses being rejected because of invalid hashtag names

* Add spec for invalid hashtag names in statuses

* Add test for featured tags controller
2019-12-17 13:31:56 +01:00
Thomas Citharel 8094955461 Add Event activity-type support (#12637)
This adds support for Event AP type in Mastodon. Events are converted
into toots by taking their title (AS name) and their URL (AP ID). Event
picture is also brought in if available.

Testable by fetching event content from https://test.mobilizon.org

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2019-12-16 23:55:28 +01:00
Thibaut Girka abcba5c198 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- package.json
  Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
  Not really a conflict, caused by an additional dependency in glitch-soc.
2019-12-07 12:28:59 +01:00
Takeshi Umeda f43f1e0184 Add basic support for group actors (#12071)
* Show badge on group actor in WebUI

* Do not notify in case of  by following group actor

* If you mention group actor, also mention group actor followers

* Relax characters that can be used in username (same as Application)

* Revert "Relax characters that can be used in username (same as Application)"

This reverts commit 7e10a137b878d0db1b5252c52106faef5e09ca4b.

* Delete display_name method
2019-12-04 20:36:33 +01:00
Yamagishi Kazutoshi 6be16d02cb Update ESLint and RuboCop in Code Climate (#12534) 2019-12-02 18:25:43 +01:00
Thibaut Girka 99f1f48741 Merge branch 'master' into glitch-soc/merge-upstream 2019-12-01 12:12:42 +01:00
ThibG f05b0463db Fallback to Create audience when object has no defined audience (#12249)
Fixes #11137
2019-11-30 19:58:00 +01:00
Thibaut Girka ff67385cfb Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
  discarded upstream changes
- app/controllers/api/v1/bookmarks_controller.rb
  finally merged upstream, some code style fixes
  and slightly changed pagination code
- app/controllers/application_controller.rb
  changed upstream to always return HTML error pages
  slight conflict caused by theming code
- app/models/bookmark.rb
  finally merged upstream, no real conflict
- spec/controllers/api/v1/bookmarks_controller_spec.rb
  finally merged upstream, slightly changed pagination code
2019-11-20 15:36:09 +01:00
Darius Kazemi 0092096328 Fix type mismatch (#12324)
This was [causing an issue with feed regeneartion in tootctl](https://github.com/hometown-fork/hometown/issues/24), and @davefp fixed the issue.
2019-11-07 21:51:48 +02:00
Thibaut Girka 0ae7d84273 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`
2019-11-06 14:38:28 +01:00
ThibG e37358b3bc Fix code referencing wrong class (#12263)
For some reason, I have seen this only triggered here:
https://circleci.com/gh/tootsuite/mastodon/98324?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link

But Follow.find_by referenced the ActivityPub::Activity::Follow class
instead of the model class.
2019-11-04 13:02:27 +01:00
Thibaut Girka 41a98b6543 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/helpers/statuses_helper.rb
  Upstream moved account helpers to their own file, we had extra
  helpers there, moved too.
- app/lib/sanitize_config.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- config/locales/simple_form.en.yml
- spec/lib/sanitize_config_spec.rb
2019-10-27 12:14:01 +01:00
Faye Duxovni 48f75b86ae Add setting for whether to crop images in unexpanded toots (#12126) 2019-10-24 22:51:41 +02:00
puckipedia d2919f7e94 Allow Accept/Reject with a non-embedded object (#12199)
Some ActivityPub servers refuse to embed remote objects into their own
output. This is because they are not the authoritative source for these
objects, and as such embedding them is always a waste of space. The
follow request and follow models contain a URI, so this can be used to
match them.
2019-10-24 22:45:43 +02:00
BSKY fccf83e1f2 Add noopener and/or noreferrer (#12202) 2019-10-24 22:44:42 +02:00
Thibaut Girka 10f1478498 Merge branch 'master' into glitch-soc/merge-upstream 2019-10-11 22:41:21 +02:00
Eugen Rochko aa509a3d8a
Fix auto-report string saying the account has been auto-silenced (#12142) 2019-10-10 18:47:24 +02:00
Thibaut Girka 61631f4751 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/javascript/styles/mastodon/components.scss
  conflicts caused by image URLs being different
- app/models/status.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
- app/views/statuses/_simple_status.html.haml
- config/locales/en.yml
  some strings were changed upstream
- spec/models/status_spec.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
2019-10-10 17:26:08 +02:00
Eugen Rochko b5f7e12817
Remove auto-silence behaviour from spam check (#12117)
Fix #12113
2019-10-09 07:11:23 +02:00
Eugen Rochko f665901e3c
Fix performance of home feed regeneration (#12084)
Fetching statuses from all followed accounts at once takes too long
within Postgres. Fetching them one by one and merging in Ruby
could be a lot less resource-intensive

Because the query for dynamically fetching the home timeline is so
heavy, we can no longer offer it when the home timeline is missing
2019-10-06 22:11:17 +02:00
Thibaut Girka 16ff7c5627 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- Gemfile.lock
- app/controllers/about_controller.rb
- app/controllers/auth/sessions_controller.rb
2019-09-30 12:23:57 +02:00
ThibG 3babf8464b Add voters count support (#11917)
* Add voters count to polls

* Add ActivityPub serialization and parsing of voters count

* Add support for voters count in WebUI

* Move incrementation of voters count out of redis lock

* Reword “voters” to “people”
2019-09-29 22:58:01 +02:00
Eugen Rochko 5f69eb89e2
Add a nodeinfo endpoint (#12002)
* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* Add some additional node info stuff

* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* expanding this to include federation info

* codeclimate feedback

* CC feedback

* using activeserializers seems like a good idea...

* get rid of draft 2.1 version

* Reimplement 2.1, also fix metaData -> metadata

* Fix metaData -> metadata here too

* Fix nodeinfo 2.1 tests

* Implement cache for monthly user aggregate

* Useless

* Remove ostatus from the list of supported protocols

* Fix nodeinfo's open_registration reading obsolete setting variable

* Only serialize domain blocks with user-facing limitations

* Do not needlessly list noop severity in nodeinfo

* Only serialize domain blocks info in nodeinfo when they are set to be displayed to everyone

* Enable caching for nodeinfo endpoints

* Fix rendering nodeinfo

* CodeClimate fixes

* Please CodeClimate

* Change InstancePresenter#active_user_count_months for clarity

* Refactor NodeInfoSerializer#metadata

* Remove nodeinfo 2.1 support as the schema doesn't exist

* Clean-up
2019-09-29 21:31:51 +02:00
Eugen Rochko 368a87755b
Fix account migration not affecting followers on origin server (#11980) 2019-09-29 16:23:13 +02:00
ThibG 18b451c0e6 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
Yamagishi Kazutoshi 67bef15e53 Add fallback section ID with ToC (#11941) 2019-09-23 17:25:10 +02:00
Yamagishi Kazutoshi 73a5ef03b2 Respect original ID with ToC (#11895) 2019-09-20 17:13:44 +02:00
Eugen Rochko d930eb88b6
Add table of contents to about page (#11885)
Move public domain blocks information to about page
2019-09-19 11:09:05 +02:00
Thibaut Girka 5cadb47238 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb
  Minor conflict due to glitch-soc's theming code
2019-09-18 17:25:56 +02:00
Eugen Rochko 4f6af87906
Change spam check to apply to local accounts and add a threshold (#11806)
Instead of detecting spam on first duplicate message, add a
threshold of 5 such messages to reduce false positives
2019-09-18 12:53:13 +02:00
Eugen Rochko 38dc51b2d6
Fix Move handler queuing jobs that will fail if account is suspended (#11864)
Don't put Move handler on cooldown if it didn't run. Skip unmerging
from timelines to save unnecessary work.
2019-09-17 08:44:45 +02:00
Thibaut Girka 74c5b2bd08 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
  Conflict because we changed the number of default results to be
  configurable
- app/lib/settings/scoped_settings.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
2019-09-13 18:13:43 +02:00
Eugen Rochko c5d37f18cb
Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Tao Bror Bojlén 4fe127664b add admin setting for default search engine indexing (fix #11750) (#11804) 2019-09-11 08:44:58 +02:00
Eugen Rochko 031ca25014
Add retry for failed media downloads and `tootctl media refresh` (#11775) 2019-09-10 15:29:12 +02:00
ThibG ec2a439a22 Add account bio to account admin view (#11473)
* Add account bio to account admin view

* Change styling to make bio fields / content more readable
2019-09-09 14:15:52 +02:00
Thibaut Girka 5088eb8388 Merge branch 'master' into glitch-soc/merge-upstream 2019-09-05 11:36:41 +02:00
ThibG 1653b58777 Attempt to concurrently connect to remote IP addresses (#11757)
* Attempt to concurrently connect to remote IP addresses

* Reduce code length to please CodeClimate 🤷
2019-09-05 05:32:53 +02:00
ThibG 692c5b439a Fix ActivityPub context not being dynamically computed (#11746)
* Fix contexts not being dynamically included

Fixes #11649

* Refactor Note context in serializer

* Refactor Actor serializer
2019-09-03 22:52:32 +02:00
Eugen Rochko 70ddef2654
Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
Thibaut Girka 9044a2b051 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/directories/index.html.haml
  Upstream has redesigned the profile directory, and we
  had a glitch-soc-specific change to hide follower counts.
  Ported that change to the new design.
2019-08-30 13:38:41 +02:00
Eugen Rochko cb447b28c4
Add profile directory to web UI (#11688)
* Add profile directory to web UI

* Add a line of bio to the directory
2019-08-30 00:14:36 +02:00
Thibaut Girka 48b8a1f414 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb

All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
2019-08-29 12:07:50 +02:00
Eugen Rochko 97192d9a77
Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
Haelwenn (lanodan) Monnier 317fa43d9a Add support for <sup> formatting element
This is based of 3e095cab83

Related: https://git.pleroma.social/pleroma/pleroma/issues/1191
2019-08-20 21:44:09 +02:00
Thibaut Girka 1488be7d96 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/home_controller.rb
- app/controllers/shares_controller.rb
- app/javascript/packs/public.js
- app/models/status.rb
- app/serializers/initial_state_serializer.rb
- app/views/home/index.html.haml
- app/views/layouts/public.html.haml
- app/views/public_timelines/show.html.haml
- app/views/shares/show.html.haml
- app/views/tags/show.html.haml
- config/initializers/content_security_policy.rb
- config/locales/en.yml
- config/webpack/shared.js
- package.json
2019-08-19 21:49:35 +02:00
Eugen Rochko 70da6d6630
Fix accounts search by full/partial display name and others (#11580)
- Restrict followers counts to local users to minimize local advantage
- Fix emoji shortcodes causing error in search
- Fix search syntax parse errors not being caught
2019-08-16 13:00:30 +02:00
Thibaut Girka 597ea5687a Add option to disable emoji replacements
Fixes #647

The option is found in `/settings` (because that was easier to write it this
way) but only affects the glitch-soc front-end.
2019-08-12 23:01:37 +02:00
Thibaut Girka 04534604c6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/views/admin/dashboard/index.html.haml
- config/locales/simple_form.en.yml
2019-08-07 13:56:48 +02:00
Eugen Rochko 5e35aa8280
Fix non-lowercase hashtags not being picked up by the streaming API (#11508)
Regression from f371b32

Fix hashtag links always being lowercase
2019-08-07 10:01:19 +02:00
Eugen Rochko 9072fe5ab6
Add trends UI with admin and user settings (#11502) 2019-08-06 17:57:52 +02:00
Thibaut Girka e50554391a Merge branch 'master' into glitch-soc/merge-upstream 2019-08-05 13:13:28 +02:00
Eugen Rochko e46e9c9a8e
Fix delete regression (#11450)
Regression from ff789a751a
2019-07-31 09:23:30 +02:00
ThibG ff789a751a Fix boosting & unboosting preventing a boost from appearing in the TL (#11405)
* Fix boosting & unboosting preventing a boost from appearing in the TL

* Add tests

* Avoids side effects when aggregate_reblogs isn't true
2019-07-30 13:18:23 +02:00
Eugen Rochko b9b0313c78
Revert "Remove conversation URI (#11423)" (#11424)
This reverts commit 75f7f9930e.
2019-07-28 17:47:37 +02:00
Eugen Rochko 75f7f9930e
Remove conversation URI (#11423)
It is not part of ActivityPub and will free up a lot of space
2019-07-28 17:30:12 +02:00
Thibaut Girka bca3825c17 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-28 16:28:05 +02:00
Eugen Rochko f371b32137
Change hashtags to preserve first-used casing (#11416) 2019-07-28 05:59:51 +02:00
Eugen Rochko b9fbcbfe4e
Add search syntax for operators and phrases (#11411) 2019-07-27 04:42:08 +02:00
Thibaut Girka 444796b69b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/application_controller.rb
- app/controllers/auth/confirmations_controller.rb
- app/controllers/auth/sessions_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
2019-07-23 10:51:07 +02:00
ThibG 7de8c51873 Play animated custom emoji on hover (#11348)
* Play animated custom emoji on hover in status

* Play animated custom emoji on hover in display names

* Play animated custom emoji on hover in bios/bio fields

* Add support for animation on hover on public pages emojis too

* Fix tests

* Code style cleanup
2019-07-21 18:10:40 +02:00
Thibaut Girka 86d4461948 Fix HTML entities being encoded as hashtag links… 2019-07-19 20:15:03 +02:00
Thibaut Girka 249991c498 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile.lock
- app/controllers/accounts_controller.rb
- app/controllers/admin/dashboard_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/following_accounts_controller.rb
- app/controllers/remote_follow_controller.rb
- app/controllers/stream_entries_controller.rb
- app/controllers/tags_controller.rb
- app/javascript/packs/public.js
- app/lib/sanitize_config.rb
- app/models/account.rb
- app/models/form/admin_settings.rb
- app/models/media_attachment.rb
- app/models/stream_entry.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/services/batched_remove_status_service.rb
- app/services/post_status_service.rb
- app/services/process_mentions_service.rb
- app/services/reblog_service.rb
- app/services/remove_status_service.rb
- app/views/admin/settings/edit.html.haml
- config/locales/simple_form.pl.yml
- config/settings.yml
- docker-compose.yml
2019-07-19 18:26:49 +02:00
ThibG fda437a020 Fix sanitizing lists contents (#11354)
* Add test

* Fix code for sanitizing nested lists stripping all tags
2019-07-19 01:44:58 +02:00
ThibG 730c4053d6 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
2019-07-19 01:44:42 +02:00
Eugen Rochko 5bfe1e1f05
Change language detection to include hashtags as words (#11341) 2019-07-18 03:02:15 +02:00
ThibG 7e2b6da57f Add setting to disable the anti-spam (#11296)
* Add environment variable to disable the anti-spam

* Move antispam setting to admin settings

* Fix typo

* antispam → spam_check
2019-07-17 21:09:15 +02:00
Eugen Rochko 729723f857 Add option to disable real-time updates in web UI (#9984)
Fix #9031
Fix #7913
2019-07-16 16:11:50 +02:00
Eugen Rochko 9b1d3e4acb
Add option to disable real-time updates in web UI (#9984)
Fix #9031
Fix #7913
2019-07-16 06:30:47 +02:00
Eugen Rochko 6ff67be0f6
Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
2019-07-13 16:45:50 +02:00
Eugen Rochko 5bf67ca913
Add ActivityPub secure mode (#11269)
* Add HTTP signature requirement for served ActivityPub resources

* Change `SECURE_MODE` to `AUTHORIZED_FETCH`

* Add 'Signature' to 'Vary' header and improve code style

* Improve code style by adding `public_fetch_mode?` method
2019-07-11 20:11:09 +02:00
Eugen Rochko 4e8dcc5dbb
Add HTTP signatures to all outgoing ActivityPub GET requests (#11284) 2019-07-11 14:49:55 +02:00
Eugen Rochko 5d3feed191
Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
Eugen Rochko 4e92183227
Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko 63c7fe8e48
Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
Eugen Rochko b851456139
Remove Atom feeds and old URLs in the form of `GET /:username/updates/:id` (#11247) 2019-07-07 16:16:51 +02:00
ThibG a0b614f10a Fix support for HTTP proxies (#11245)
* Disable incorrect check for hidden services in Socket

Hidden services can only be accessed with an HTTP proxy, in which
case the host seen by the Socket class will be the proxy, not the
target host.

Hidden services are already filtered in `Request#initialize`.

* Use our Socket class to connect to HTTP proxies

Avoid the timeout logic being bypassed

* Add support for IP addresses in Request::Socket

* Refactor a bit, no need to keep the DNS resolver around
2019-07-07 13:53:09 +02:00
Thibaut Girka 7039dca12c Merge commit 'c07cca4727041ea5a5721acbc603d4bfb45a15a6' into glitch-soc/merge-upstream
Unlike upstream, kept the direct timeline endpoint, as it is still of use in
glitch-soc.
2019-07-07 13:49:31 +02:00
Eugen Rochko 406b46395d
Fix URLs appearing twice in errors of ActivityPub::DeliveryWorker (#11231) 2019-07-07 03:37:01 +02:00
ThibG 58276715be Fix support for HTTP proxies (#11245)
* Disable incorrect check for hidden services in Socket

Hidden services can only be accessed with an HTTP proxy, in which
case the host seen by the Socket class will be the proxy, not the
target host.

Hidden services are already filtered in `Request#initialize`.

* Use our Socket class to connect to HTTP proxies

Avoid the timeout logic being bypassed

* Add support for IP addresses in Request::Socket

* Refactor a bit, no need to keep the DNS resolver around
2019-07-07 02:05:38 +02:00
Eugen Rochko 23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
ThibG ed544f3166 Fix HTTP requests to IPv6 hosts (#11240) 2019-07-05 22:03:00 +02:00
Thibaut Girka 6ab84c12a7 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-04 16:21:39 +02:00
Eugen Rochko 1a747f7020
Fix statsd UDP sockets not being cleaned up in Sidekiq (#11230) 2019-07-02 01:01:17 +02:00
Eugen Rochko 0d9ffe56fb
Add request pool to improve delivery performance (#10353)
* Add request pool to improve delivery performance

Fix #7909

* Ensure connection is closed when exception interrupts execution

* Remove Timeout#timeout from socket connection

* Fix infinite retrial loop on HTTP::ConnectionError

* Close sockets on failure, reduce idle time to 90 seconds

* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server

* Use a shared pool size, 512 by default, to stay below open file limit

* Add some tests

* Add more tests

* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds

* Use a shared pool that returns preferred connection but re-purposes other ones when needed

* Fix wrong connection being returned on subsequent calls within the same thread

* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
2019-07-02 00:34:38 +02:00
Thibaut Girka ca17bae904 Use a redis-cached feed for the DM timeline 2019-06-27 16:44:12 +02:00
Thibaut Girka aaec64a500 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/preferences_controller.rb
- app/lib/user_settings_decorator.rb
- app/models/user.rb
- config/locales/simple_form.en.yml
2019-06-26 23:19:22 +02:00
ThibG 3086c645fd Add option to disable blurhash previews (#11188)
* Add option to disable blurhash previews

* Update option text

* Change options order
2019-06-26 19:33:04 +02:00
ThibG 915c619394 Add support for Audio activities (#11189)
Fixes #11127
2019-06-26 19:32:36 +02:00
ThibG 47ef4a6c7a Apply filters to poll options (#11174)
* Apply filters to poll options in WebUI

Fixes #11128

* Apply filters to poll options server-side

* Add poll options to searchable text
2019-06-25 14:45:14 +02:00
Thibaut Girka ddd875ad99 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream added audio attachment support
- app/serializers/initial_state_serializer.rb
  Upstream added audio attachment support and how mimetypes are returned
- app/serializers/rest/instance_serializer.rb
  Upstream added a few fields
- config/application.rb
  Upstream added a different paperclip transcoder
2019-06-24 15:02:59 +02:00
Eugen Rochko aa9b37822b
Fix audio not being downloaded from remote servers (#11145) 2019-06-22 02:50:36 +02:00
Eugen Rochko 707ddf7808
Change domain blocks to automatically support subdomains (#11138)
* Change domain blocks to automatically support subdomains

If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)

* Match subdomains of existing accounts when blocking/unblocking domains

* Improve code style
2019-06-22 00:13:10 +02:00
Thibaut Girka 5ea45351a0 Fix rendering of emoji in public pages 2019-06-18 20:14:08 +02:00
Eugen Rochko 103a9f4466
Fix sanitizer making block level elements unreadable (#10836)
Fix #10834
2019-06-16 21:46:36 +02:00
Thibaut Girka 1b0ff4cd69 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/notifications_controller.rb
- app/javascript/packs/public.js
- app/views/settings/preferences/show.html.haml
- app/views/stream_entries/_simple_status.html.haml
- config/locales/simple_form.en.yml
- config/locales/simple_form.pl.yml
- config/navigation.rb
- config/routes.rb
2019-06-10 18:59:53 +02:00
Eugen Rochko 8746f4d17b
Change priority of delete activity forwards for replies and reblogs (#11002)
Fix #11001
2019-06-09 12:47:33 +02:00
Thibaut Girka 34b8346e7f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
- app/controllers/stream_entries_controller.rb
2019-06-07 17:00:36 +02:00
ThibG 6c464cd424 Do not misattribute inlined boosts if `attributedTo` isn't present (#10967)
* Do not misattribute inlined boosts if `attributedTo` isn't present

Fixes #10950

* Fix tests
2019-06-04 23:24:31 +02:00
Eugen Rochko f2b743e715
Refactor all ActivityPub deliveries to be serialized and signed through one concern (#10966) 2019-06-04 23:11:18 +02:00
Thibaut Girka 02b56c7e1a Merge remote-tracking branch 'upstream/master' into glitch-soc/merge-upstream 2019-06-04 20:40:19 +02:00
ThibG 3a84bacf86 Handle blank poll options more gracefully (#10946)
Pleroma currently allows (erroneously imho) empty poll options, that is,
options with an empty (but existing) `name`.
2019-06-03 20:04:00 +02:00
Thibaut Girka 07d4ecfe5e Truncate long URLs while providing alt text for inline images 2019-05-28 19:33:43 +02:00
Thibaut Girka 18b77224d3 Translate incoming remote img tags by a link 2019-05-28 19:33:43 +02:00
Thibaut Girka 20d01a954e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/preferences_controller.rb
- app/lib/user_settings_decorator.rb
- app/models/user.rb

Conflicts due to the addition of a new preference upstream,
“advanced layout”.
2019-05-26 15:41:40 +02:00
Eugen Rochko 1e5532e693
Add responsive panels to the single-column layout (#10820)
* Add responsive panels to the single-column layout

* Fixes

* Fix not being able to save the preference

* Fix code style issues

* Set max-height on the compose textarea and add a link to relationship manager
2019-05-25 21:27:00 +02:00
ThibG 72bf354e97
Merge pull request #1067 from ThibG/glitch-soc/merge-upstream
Merge upstream changes
2019-05-23 22:01:34 +02:00
Thibaut Girka d6a7d62dc6 Do not share a single Redcarpet::Markdown object across threads
There are apparently thread-safety issues: https://github.com/vmg/redcarpet/pull/672
2019-05-23 21:19:44 +02:00
Thibaut Girka bf988123ba Merge branch 'master' into glitch-soc/merge-upstream 2019-05-23 21:18:27 +02:00
ThibG 9a5561a5b8 Fix possible race condition when processing statuses (#10815) 2019-05-23 20:00:39 +02:00
Thibaut Girka 7c609bdab9 Allow rel=tag in status text
Fixes tag links in local Markdown or HTML-authored statuses
2019-05-23 12:28:33 +02:00
Thibaut Girka 2332b3f146 Fix local text/html toots not being sanitized 2019-05-21 23:12:43 +02:00
Thibaut Girka 9ca21e93cc Minor optimization 2019-05-17 23:51:14 +02:00
Thibaut Girka dd5bf40b97 Properly escape HTML in code blocks 2019-05-17 23:51:14 +02:00
Thibaut Girka a6b7c23f6f Add option for default toot content-type 2019-05-17 23:51:14 +02:00
Thibaut Girka 0be93820f3 Fix autolinking, and newlines in code blocks
Autolinking is now performed *after* the Markdown pass, by replacing HTML
tags with zero-width spaces and running the twitter-text extractor as usual,
except it does not auto-link URLs to avoid links in links…
2019-05-17 23:51:14 +02:00
Thibaut Girka d7520f8175 Add support for HTML-formatted toots 2019-05-17 23:51:14 +02:00
Thibaut Girka 94aef563b9 Add support for markdown-formatted toots 2019-05-17 23:51:14 +02:00
Thibaut Girka 6badf2d252 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-15 17:11:40 +02:00
ThibG c1c8c7b2c8 Minor performance improvements and cleanup in formatter (#10765) 2019-05-15 06:54:06 +02:00
Thibaut Girka 68629f2773 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
  minor conflict because of glitch-soc's theming system
- app/controllers/stream_entries_controller.rb
  minor conflict because of glitch-soc's theming system
2019-05-10 17:09:12 +02:00
ThibG 62f5235b6f Prevent silenced local users from notifying remote users not following them (#10575)
* Prevent silenced local users from notifying remote users not following them

This is an attempt to extend the local restrictions of silenced users to the
federation.

* Add tests

* Add tests for making sure private status don't get sent over OStatus
2019-05-09 22:05:43 +02:00
Thibaut Girka 33c80e0783 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
2019-05-04 16:37:26 +02:00
Thibaut Girka 3e095cab83 Add support for missing formatting tags 2019-04-29 19:31:09 +02:00
Eugen Rochko fba96c808d
Add blurhash (#10630)
* Add blurhash

* Use fallback color for spoiler when blurhash missing

* Federate the blurhash and accept it as long as it's at most 5x5

* Display unknown media attachments as blurhash placeholders

* Improve style of embed actions and spoiler button

* Change blurhash resolution from 3x3 to 4x4

* Improve dependency definitions

* Fix code style issues
2019-04-27 03:24:09 +02:00
Thibaut Girka 27e8354914 Add support for lists in statuses 2019-04-22 23:07:14 +02:00
Thibaut Girka 2e3640ecbb Accept richer text from remote statuses
Support abbr, del, pre, blockquote, code, strong, b, em, i, and h1…h5
HTML elements in remote statuses, add corresponding CSS.
2019-04-22 23:07:14 +02:00
Thibaut Girka a9eaa780f5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/javascript/mastodon/features/compose/components/compose_form.js
  Upstream cleaned up a bit, including on lines in which
  we replaced the hardcoded 500 character limit with a maxChar
  constant. Applied the changes while keeping maxChar instead of 500.
- app/javascript/packs/public.js
  Moved upstream's new animated avatar hover handling in
  app/javascript/core/public.js
- app/javascript/styles/fonts/montserrat.scss
  Upstream fixed local font name, applied those changes.
- app/javascript/styles/fonts/roboto.scss
  Upstream fixed local font name, applied those changes.
- lib/mastodon/version.rb
  Upstream made repo URL configurable, did the same, but
  default to glitch-soc
2019-04-22 20:40:04 +02:00
Ben Lubar be8692b938 Default to the web domain (eg. mastodon.lubar.me) instead of the local domain (eg. lubar.me) for keybase proofs (#10565) 2019-04-21 04:53:24 +02:00
Thibaut Girka 2f422b708f Merge branch 'master' into glitch-soc/merge-upstream 2019-04-10 21:19:21 +02:00
Eugen Rochko a9f130b8d8
Fix Keybase verification using wrong domain for remote accounts (#10547) 2019-04-10 20:28:43 +02:00
Thibaut Girka f5f6d23d55 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/pl.yml
  Conflict caused by new upstream string too close to glitch-specific
  “flavour” string. Took both strings.
2019-04-08 15:57:56 +02:00
Takeshi Umeda a4d5e15327 Fix config serializer regex (#10487) 2019-04-08 07:55:38 +02:00
Thibaut Girka 4444d8f9b3 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-30 12:36:24 +01:00
Eugen Rochko 1714ea5978
Add ActivityPub representation for identity proofs (#10414)
* Add ActivityPub representation for identity proofs

* Add tests
2019-03-30 02:12:06 +01:00
Thibaut Girka ce7d055d3c Merge branch 'master' into glitch-soc/merge-upstream 2019-03-28 18:35:25 +01:00
Alex Gessner 69141dca26 squashed identity proof updates (#10375) 2019-03-28 18:01:09 +01:00
Thibaut Girka 050efbc126 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/workers/activitypub/distribute_poll_update_worker.rb
- config/locales/pl.yml
2019-03-28 13:01:33 +01:00
Eugen Rochko f1bc90ab50
Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
Eugen Rochko e86663b1da
Fix alternative relay support regression (#10398)
Fix #10324
2019-03-27 19:58:24 +01:00
Eugen Rochko 11fe293e1b
Remove unused ActivityPub `@context` values depending on response (#10378)
Fix #8078
2019-03-27 15:55:23 +01:00
Thibaut Girka b9f314bba7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/es.yml
- config/locales/pl.yml
- config/locales/pt-BR.yml
2019-03-26 16:02:54 +01:00
Eugen Rochko e6cfa7ab89
Change language detector threshold from 140 characters to 4 words (#10376)
Add `lang` attribute to statuses in web UI
2019-03-26 01:23:59 +01:00
Thibaut Girka edd5441112 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-22 13:05:17 +01:00
Eugen Rochko 3411fbef19
Improve config serializer for Keybase (#10338)
- Regex must no longer be surrounded by `/`
- Description must be short and cannot contain HTML tags
2019-03-21 23:33:28 +01:00
Thibaut Girka cbf1d711ba Merge branch 'master' into glitch-soc/merge-upstream 2019-03-20 17:32:39 +01:00
ThibG 80f0910e21 Add support for custom emojis in poll options (#10322)
* Backend changes for custom emoji support in poll options

* Serialize poll emojis in REST API

* Render custom emojis in poll options

* Render custom emoji in poll options on public pages
2019-03-20 17:29:12 +01:00
Thibaut Girka 1d6152f440 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/en.yml
  Conflict caused by the glitch-soc-specific “flavour” string being too close
  to the newly introduced “identity_proofs” string. Just included both.
2019-03-20 13:54:00 +01:00
Eugen Rochko 9c4cbdbafb
Add Keybase integration (#10297)
* create account_identity_proofs table

* add endpoint for keybase to check local proofs

* add async task to update validity and liveness of proofs from keybase

* first pass keybase proof CRUD

* second pass keybase proof creation

* clean up proof list and add badges

* add avatar url to keybase api

* Always highlight the “Identity Proofs” navigation item when interacting with proofs.

* Update translations.

* Add profile URL.

* Reorder proofs.

* Add proofs to bio.

* Update settings/identity_proofs front-end.

* Use `link_to`.

* Only encode query params if they exist.

URLs without params had a trailing `?`.

* Only show live proofs.

* change valid to active in proof list and update liveness before displaying

* minor fixes

* add keybase config at well-known path

* extremely naive feature flagging off the identity proof UI

* fixes for rubocop

* make identity proofs page resilient to potential keybase issues

* normalize i18n

* tweaks for brakeman

* remove two unused translations

* cleanup and add more localizations

* make keybase_contacts an admin setting

* fix ExternalProofService my_domain

* use Addressable::URI in identity proofs

* use active model serializer for keybase proof config

* more cleanup of keybase proof config

* rename proof is_valid and is_live to proof_valid and proof_live

* cleanup

* assorted tweaks for more robust communication with keybase

* Clean up

* Small fixes

* Display verified identity identically to verified links

* Clean up unused CSS

* Add caching for Keybase avatar URLs

* Remove keybase_contacts setting
2019-03-18 21:00:55 +01:00
Thibaut Girka 291012ec0c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/accounts_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/statuses_controller.rb

All conflicts caused by the additional `use_pack` used for glitch-soc's theming
system.
2019-03-18 18:03:27 +01:00
ThibG a20354a20b Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
Thibaut Girka 4df3b7cb06 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/settings/follower_domains_controller.rb
  Removed upstream. Did the same here. Maybe we should not have?
- config/locales/en.yml
  Upstream removed the “Authorized followers” page and associated
  translations. This is too close in the file to our glitch-soc-specific
  “flavour” string. No actual conflict.
- config/locales/ja.yml
  Same as above.
- config/locales/pl.yml
  Same as above.
- config/navigation.rb
  No real conflict. New route added too close to the glitch-soc-specific
  “flavours” one.
- config/webpack/configuration.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- config/webpack/loaders/babel.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
  The contents of this file have been moved to package.json.
- config/webpack/shared.js
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- config/webpacker.yml
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- jest.config.js
  The contents of this file have been moved to package.json.
- package.json
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
- yarn.lock
  Upstream refactored the webpack(er) configuration quite a bit.
  Tried to keep up.
2019-03-16 14:14:20 +01:00
Eugen Rochko 1b167707c2
Fix language detection of non-latin alphabets even at few characters (#10276) 2019-03-15 05:07:09 +01:00
Thibaut Girka 02062aab55 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/about_controller.rb
- app/controllers/tags_controller.rb
- app/views/about/show.html.haml
- spec/views/about/show.html.haml_spec.rb
2019-03-13 15:16:02 +01:00
Eugen Rochko 9f5b55ad4f
Fix poll update handler calling method was that was not available (#10246)
* Fix poll update handler calling method was that was not available

Fix regression from #10209

* Refactor VoteService

* Refactor ActivityPub::DistributePollUpdateWorker and optimize it

* Fix typo

* Fix typo
2019-03-12 22:58:59 +01:00
Thibaut Girka f3e7634156 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-11 17:31:02 +01:00
ThibG 79b00d294d Increase DNS timeouts (#10238) 2019-03-11 13:27:57 +01:00
Thibaut Girka 89bee860cd Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/services/post_status_service.rb
  Small conflict due to handling of instance-local toots.
  A subsequent change is required to ensure instance-local polls are not leaked
  through Update.
2019-03-11 11:23:50 +01:00
ThibG 3a92885a86 Support pushing and receiving updates to poll tallies (#10209)
* Process incoming poll tallies update

* Send Update on poll vote

* Do not send Updates for a poll more often than once every 3 minutes

* Include voters in people to notify of results update

* Schedule closing poll worker on poll creation

* Add new notification type for ending polls

* Add front-end support for ended poll notifications

* Fix UpdatePollSerializer

* Fix Updates not being triggered by local votes

* Fix tests failure

* Fix web push notifications for closing polls

* Minor cleanup

* Notify voters of both remote and local polls when those close

* Fix delivery of poll updates to mentioned accounts and voters
2019-03-11 00:49:31 +01:00
Thibaut Girka 5528719bc9 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-08 00:57:13 +01:00
ThibG 3aaac4f134 Do not allow adding votes to expired polls (#10214)
* Do not allow adding votes to expired polls

* Only validate expires_at on create
2019-03-08 00:54:50 +01:00
Thibaut Girka 9c5e7805cc Merge branch 'master' into glitch-soc/merge-upstream 2019-03-07 13:24:30 +01:00
ThibG 34f261e6af Avoid unnecessarily fetching the replies collection when it is empty (#10201) 2019-03-07 01:50:37 +01:00
Thibaut Girka f108c6a042 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-06 00:03:46 +01:00
ThibG efd0fb8088 Fix newlines in OStatus and RSS serializations (#10183) 2019-03-05 23:58:58 +01:00
Thibaut Girka 866496ac16 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-05 21:40:28 +01:00
ThibG 636db1f54f When serializing polls over OStatus, serialize poll options to text (#10160)
* When serializing polls over OStatus, serialize poll options to text

* Do the same for RSS feeds

* Use “[ ] ” as a prefix for poll options instead of “- ”
2019-03-05 21:09:18 +01:00
Thibaut Girka f513317ba2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- db/schema.rb

Both conflicts are caused by us having extra database columns.
2019-03-05 19:23:16 +01:00
ThibG 833ffce2df Store remote votes URI (#10158)
* Store remote votes URI

* Add spec for accepting remote votes

* Make poll vote id generation work the same way as follows
2019-03-04 22:51:23 +01:00
Eugen Rochko 0e6998da3c
Add tests for ActivityPub poll processing (#10143) 2019-03-04 01:13:42 +01:00
Eugen Rochko ae1b9cf70a
Fix remote poll expiration time (#10144) 2019-03-04 00:44:34 +01:00
Eugen Rochko 230a012f00
Add polls (#10111)
* Add polls

Fix #1629

* Add tests

* Fixes

* Change API for creating polls

* Use name instead of content for votes

* Remove poll validation for remote polls

* Add polls to public pages

* When updating the poll, update options just in case they were changed

* Fix public pages showing both poll and other media
2019-03-03 22:18:23 +01:00
Thibaut Girka 2267eee356 Merge branch 'master' into glitch-soc/merge-upstream 2019-02-28 21:35:53 +01:00
ThibG d8498b3983 Give the `replies` collection an identifier and enable pagination (#10128) 2019-02-28 18:16:34 +01:00
ThibG 9d3c6f1849 Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
ThibG 9edab46368 Fix mention processing for unknwon accounts on incoming ActivityPub Notes (#10125)
`::FetchRemoteAccountService` is not `ActivityPub::FetchRemoteAccountService`,
its second argument is the pre-fetched body. Passing `id: false` actually passed
a `Hash` as the prefetched body, instead of properly resolving unknown remote
accounts.
2019-02-27 14:57:14 +01:00
Thibaut Girka f6a4493a58 Merge branch 'master' into glitch-soc/merge-upstream 2019-02-17 15:43:12 +01:00
Eugen Rochko 1a1b8170bb
Fix Announce activities of unknown statuses not fetching those statuses (#10065)
Regression from #9998
2019-02-17 15:16:36 +01:00
Thibaut Girka 369a475a70 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/serializers/rest/instance_serializer.rb
- app/views/layouts/error.html.haml
- public/oops.png
  Took our version (same as upstream but without the things that only
  make sense in an animation).

Additional changes:
- app/javascript/flavours/vanilla/theme.yml
  Include upstream's javascript in error pages.
2019-02-17 12:26:55 +01:00
Eugen Rochko 147b4c2c3a
Add logging for rejected ActivityPub payloads and add tests (#10062) 2019-02-17 03:38:25 +01:00
Eugen Rochko c417e8c198
Filter incoming Announce activities by relation to local activity (#10041)
* Filter incoming Announce activities by relation to local activity

Reject if announcer is not followed by local accounts, and is not
from an enabled relay, and the object is not a local status

Follow-up to #10005

* Fix tests
2019-02-15 18:19:45 +01:00
Thibaut Girka 06cc04fd23 Merge branch 'master' into glitch-soc/merge-upstream 2019-02-15 18:02:45 +01:00
Eugen Rochko dad339da6d
Filter incoming Create activities by relation to local activity (#10005)
Reject those from accounts with no local followers, from relays
that are not enabled, which do not address local accounts and are
not replies to accounts that do have local followers
2019-02-13 18:42:47 +01:00
ThibG 6a5307a573 Alternative handling of private self-boosts (#9998)
* When self-boosting, embed original toot into Announce serialization

* Process unknown self-boosts from Announce object if it is more than an URI

* Add some self-boost specs

* Only serialize private toots in self-Announces
2019-02-13 18:36:23 +01:00
Thibaut Girka bf94a43496 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/oauth/authorized_applications_controller.rb
  Two changes too close to each other
- app/controllers/settings/sessions_controller.rb
- app/lib/user_settings_decorator.rb
  Two changes too close to each other
- app/models/media_attachment.rb
  New changes too close to glitch-soc only changes.
- app/models/user.rb
  Two changes too close to each other.
- app/services/remove_status_service.rb
  Kept direct timeline code which had been removed upstream.
- app/views/settings/preferences/show.html.haml
  Two changes too close to each other.
- config/locales/en.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/ja.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/pl.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/simple_form.en.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/locales/simple_form.pl.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/settings.yml
  Reverted upstream's decision of enabling posting application by default.
2019-02-10 21:10:09 +01:00
Eugen Rochko 016ad37bc8
Fix URL linkifier grabbing full-width spaces and quotations (#9997)
Fix #9993
Fix #5654
2019-02-09 20:13:11 +01:00
Hinaloe 157d3af46c Only URLs extract with pre-escaped text (#9991)
* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( https://github.com/tootsuite/mastodon/issues/9989 )
2019-02-09 03:39:38 +01:00
ThibG ed30110618 Make displaying application used to toot opt-in (#9897)
* Make storing and displaying application used to toot opt-in

* Revert to storing application info, and display it to the author via API
2019-02-02 19:18:15 +01:00
ysksn bcfff65195 Create Redisable#redis (#9633)
* Create Redisable

* Use #redis instead of Redis.current
2019-02-02 19:11:38 +01:00
Jakub Mendyk 6a5e3da6b0 Allow most kinds of characters in URL query (fixes #8408) (#8447)
* Allow unicode characters in URL query strings

Fixes #8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.
2019-02-02 19:01:18 +01:00
Thibaut Girka 05415fa5d3 Make storing and displaying application used to toot opt-in 2019-01-27 13:56:41 +01:00
Thibaut Girka 5e0cf92fd1 Merge branch 'master' into glitch-soc/merge-upstream
No conflicts.
2019-01-19 18:28:37 +01:00
ThibG 75b1488cf4 Add tombstones for remote statuses (#9830)
* Add Tombstone model to remember object deletion

* Do not recreate a status if it has been deleted

* Record Tombstone for remote deleted items

Also, only record deleted items from same-host actors

* Clear an user's tombstones when their key change
2019-01-18 15:56:55 +01:00
Eugen Rochko 31f396b57d
Add support for non-public reblogs from ActivityPub (#9841)
Fix #9838
2019-01-18 15:56:21 +01:00
Eugen Rochko 4ab42287c0
Use summary as summary for converted ActivityPub objects (#9823)
Fix #8609
2019-01-16 18:36:17 +01:00
ThibG a4f07bad95 Reduce chances of race conditions when processing deleted toots (#9815)
* Reduce chances of race conditions when processing deleted toots

* Prevent race condition when processing deleted toots
2019-01-16 15:42:00 +01:00
Thibaut Girka fb0c906c71 Revert "Revert "Add handler for Move activity (#9629)""
This reverts commit bb96a74637.
2019-01-10 18:46:17 +01:00
Thibaut Girka bb96a74637 Revert "Add handler for Move activity (#9629)"
This reverts commit 0f938ff29c.
2019-01-02 15:36:59 +01:00
Thibaut Girka 571d219bb9 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts manually resolved:
- app/services/post_status_service.rb
- config/locales/simple_form.pl.yml
- config/routes.rb
- config/webpack/loaders/sass.js
- config/webpack/shared.js
- package.json
- yarn.lock
2019-01-02 15:36:53 +01:00
ThibG 70be301d69 Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order (#9687)
* Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order

* Add specs for Block causing unfollow and for out-of-order Block + Undo
2019-01-02 01:12:02 +01:00
ThibG b2f4114550 Do not ignore federated reports targetting already-reported accounts (#9534) 2018-12-30 18:58:51 +01:00
ThibG 290932602b Reduce usage of LD signatures (#9659)
* Do not LDS-sign Follow, Accept, Reject, Undo, Block

* Do not use LDS for Create activities of private toots

* Minor cleanup

* Ignore unsigned activities instead of misattributing them

* Use status.distributable? instead of querying visibility directly
2018-12-30 09:48:59 +01:00
Eugen Rochko 0f938ff29c
Add handler for Move activity (#9629) 2018-12-29 02:24:36 +01:00
Eugen Rochko aa9a20cde0
Fix ThreadResolveWorker getting queued with invalid URLs (#9628) 2018-12-26 19:15:53 +01:00
Thibaut Girka bd23288a52 Add a per-user setting to hide followers count
This is only available if the instance-wide setting isn't set and allows
people to hide their own followers count. This does not hide others' to
them.
2018-12-20 14:51:12 +01:00
Thibaut Girka e7f1bfdc2d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/javascript/packs/public.js
- app/models/user.rb
- config/settings.yml
- db/schema.rb

Moved public.js changes to settings.js.
2018-12-09 16:08:04 +01:00
ThibG 81bda7d67c Add setting to not aggregate reblogs (#9248)
* Add setting to not aggregate reblogs

Fixes #9222

* Handle cases where user is nil in add_to_home and add_to_list

* Add hint for setting_aggregate_reblogs option

* Reword setting_aggregate_reblogs label
2018-12-09 13:03:01 +01:00
Thibaut Girka ad7a3974cc Merge branch 'master' into glitch-soc/merge-upstream 2018-11-29 19:22:52 +01:00
Thibaut Girka 53d0293d25 Add database support for list show-reply preferences 2018-11-28 14:46:07 +01:00
Eugen Rochko c39d7e7b2b
Fix TLS handshake timeout not being enforced (#9381)
Follow-up to #9329
2018-11-27 19:46:05 +01:00
Eugen Rochko 43c311b3a1
Fix nil error when no DNS addresses are found for host (#9379) 2018-11-27 18:13:36 +01:00
Thibaut Girka f8e07ca5cd Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb

Resolved by taking both changes (not a real conflict, just changes too close
to each other).
2018-11-27 13:23:02 +01:00
Eugen Rochko fd8145d232
Fix connect timeout not being enforced (#9329)
* Fix connect timeout not being enforced

The loop was catching the timeout exception that should stop execution, so the next IP would no longer be within a timed block, which led to requests taking much longer than 10 seconds.

* Use timeout on each IP attempt, but limit to 2 attempts

* Fix code style issue

* Do not break Request#perform if no block given

* Update method stub in spec for Request

* Move timeout inside the begin/rescue block

* Use Resolv::DNS with timeout of 1 to get IP addresses

* Update Request spec to stub Resolv::DNS instead of Addrinfo

* Fix Resolve::DNS stubs in Request spec
2018-11-22 20:12:04 +01:00
ThibG 466e3d710c Include replies to list owner and replies to list members in list statuses (#9324) 2018-11-21 17:02:58 +01:00
Eugen Rochko 384e953b75
Revert connect timeout from 1s to 10s (#9319)
The failure rate in Sidekiq is too high
2018-11-21 17:00:56 +01:00
Eugen Rochko 9311430ed7
Prevent multiple handlers for Delete of Actor from running (#9292) 2018-11-16 19:46:23 +01:00
Eugen Rochko 8069fd636b
Remove intermediary arrays when creating hash maps from results (#9291) 2018-11-16 15:02:18 +01:00
Thibaut Girka bfe2b9cc50 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-13 16:28:07 +01:00
ThibG 7f5a4be580 Fix emoji update date processing (#9255) 2018-11-10 23:59:51 +01:00
Thibaut Girka 9201398507 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-09 14:56:31 +01:00
Eugen Rochko 4615512285
Reduce connect timeout limit and limit signature failures by source IP (#9236)
* Reduce connect timeout from 10s to 1s

* Limit failing signature verifications per source IP
2018-11-08 21:35:58 +01:00
Thibaut Girka 1c448d6474 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb

Upstream reverted something we partially reverted already.
Reverted the rest to match upstream.
2018-10-30 17:52:08 +01:00
Eugen Rochko be202f9377
Accept the same payload in multiple inboxes and deliver (#9150) 2018-10-30 15:03:55 +01:00
Thibaut Girka a942ad3062 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/simple_form.ja.yml

Not a real conflict, one new string was introduced in
glitch and one in upstream.
2018-10-29 14:00:41 +01:00
m.b 2b18f5f85d Add Page AP type support (#9121) 2018-10-29 13:23:29 +01:00
Thibaut Girka c13cfeac09 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-27 18:54:26 +02:00
ThibG e53cc673e7 Ignore invalid hashtags on remote statuses instead of rejecting them (#9118)
Fixes #9115
2018-10-26 22:48:35 +02:00
Thibaut Girka 5bb8563f6c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/base_controller.rb
- app/controllers/filters_controller.rb
- app/controllers/invites_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/exports_controller.rb
- app/controllers/settings/follower_domains_controller.rb
- app/controllers/settings/migrations_controller.rb
- app/controllers/settings/notifications_controller.rb
- app/controllers/settings/preferences_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
- app/javascript/packs/public.js
- app/views/settings/profiles/show.html.haml

Conflicts were mostly due to the addition of body classes to the settings page,
this was caused by rejecting upstream changes for most of those files and
modifying Settings::BaseController instead.

Another cause of conflicts was the deletion of client-side checking of
display name / bio length, this was modified in app/javascript/core/settings.js
instead.
2018-10-26 20:41:43 +02:00
ThibG 82e7988afc Fix missing `mention` argument when processing incoming Create activities (#9114)
* Fix missing `mention` argument when processing incoming Create activities

* Fix typo (param → params)
2018-10-26 12:59:59 +02:00
Eugen Rochko d4cf963749
Allow inbox owner to view implicitly targeted ActivityPub payload (#9093)
Fix #9091
2018-10-25 18:12:22 +02:00
Thibaut Girka dcded13a99 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .github/ISSUE_TEMPLATE/bug_report.md
  Took our version.
- CONTRIBUTING.md
  Updated the embedded copy of upstream's version.
- README.md
  Took our version.
- app/policies/status_policy.rb
  Not a real conflict, took code from both.
- app/views/layouts/embedded.html.haml
  Added upstream's changes (dns-prefetch) and fixed
  `%body.embed`
- app/views/settings/preferences/show.html.haml
  Reverted some of upstream changes, as we have a
  page dedicated for flavours and skins.
- config/initializers/content_security_policy.rb
  Kept our version of the CSP.
- config/initializers/doorkeeper.rb
  Not a real conflict, took code from both.
2018-10-22 17:51:38 +02:00
Eugen Rochko fd5285658f
Add option to block reports from domain (#8830) 2018-10-20 08:02:44 +02:00
Eugen Rochko ddd30f331c
Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Thibaut Girka 3d5d899094 Allow selecting both default flavour and theme
Fixes #672
2018-10-12 19:06:35 +02:00
Eugen Rochko fac529975b Improve signature verification safeguards (#8959)
* Downcase signed_headers string before building the signed string

The HTTP Signatures draft does not mandate the “headers” field to be downcased,
but mandates the header field names to be downcased in the signed string, which
means that prior to this patch, Mastodon could fail to process signatures from
some compliant clients. It also means that it would not actually check the
Digest of non-compliant clients that wouldn't use a lowercased Digest field
name.

Thankfully, I don't know of any such client.

* Revert "Remove dead code (#8919)"

This reverts commit a00ce8c92c.

* Restore time window checking, change it to 12 hours

By checking the Date header, we can prevent replaying old vulnerable
signatures. The focus is to prevent replaying old vulnerable requests
from software that has been fixed in the meantime, so a somewhat long
window should be fine and accounts for timezone misconfiguration.

* Escape users' URLs when formatting them

Fixes possible HTML injection

* Escape all string interpolations in Formatter class

Slightly improve performance by reducing class allocations
from repeated Formatter#encode calls

* Fix code style issues
2018-10-12 07:00:41 +02:00
Eugen Rochko 21ad21cb50
Improve signature verification safeguards (#8959)
* Downcase signed_headers string before building the signed string

The HTTP Signatures draft does not mandate the “headers” field to be downcased,
but mandates the header field names to be downcased in the signed string, which
means that prior to this patch, Mastodon could fail to process signatures from
some compliant clients. It also means that it would not actually check the
Digest of non-compliant clients that wouldn't use a lowercased Digest field
name.

Thankfully, I don't know of any such client.

* Revert "Remove dead code (#8919)"

This reverts commit a00ce8c92c.

* Restore time window checking, change it to 12 hours

By checking the Date header, we can prevent replaying old vulnerable
signatures. The focus is to prevent replaying old vulnerable requests
from software that has been fixed in the meantime, so a somewhat long
window should be fine and accounts for timezone misconfiguration.

* Escape users' URLs when formatting them

Fixes possible HTML injection

* Escape all string interpolations in Formatter class

Slightly improve performance by reducing class allocations
from repeated Formatter#encode calls

* Fix code style issues
2018-10-12 00:15:55 +02:00
Eugen Rochko 61d44dd11f
Fix typo in ActivityPub Create handler (#8952)
Regression from #8951
2018-10-11 02:10:15 +02:00
Eugen Rochko 790d3bc637
Move network calls out of transaction in ActivityPub handler (#8951)
Mention and emoji code may perform network calls, but does not need
to do that inside the database transaction. This may improve availability
of database connections when using pgBouncer in transaction mode.
2018-10-11 00:50:18 +02:00
Thibaut Girka d17844e6d1 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-08 13:51:33 +02:00
Eugen Rochko 774ac47373
Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
Thibaut Girka 98bb6815a7 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-05 21:48:25 +02:00
Jeong Arm 144d73730d Leave unknown language as nil if account is remote (#8861)
* Force use language detector if account is remote

* Set unknown remote toot's language as nil
2018-10-05 19:17:46 +02:00
Thibaut Girka 00fcdebed7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	db/migrate/20170716191202_add_hide_notifications_to_mute.rb
	spec/controllers/application_controller_spec.rb

Took our version, upstream changes were only minor style linting.
2018-10-05 15:23:57 +02:00
ThibG ba444797d2 Fix handling of ActivityPub activities lacking some attributes (#8864) 2018-10-03 23:44:13 +02:00
Thibaut Girka c91d9b7389 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/lib/user_settings_decorator.rb
	app/models/user.rb
	app/serializers/initial_state_serializer.rb
	app/views/stream_entries/_simple_status.html.haml
	config/locales/simple_form.en.yml
	config/locales/simple_form.ja.yml
	config/locales/simple_form.pl.yml
	config/routes.rb
2018-10-01 12:43:20 +02:00
Eugen Rochko ef69c655cc
Fix class autoloading issue in ActivityPub::Activity::Create (#8820) 2018-09-28 17:02:53 +02:00
cbayerlein 4b78546135 Exclude replies from list timelines (#8683)
* Changed list behaviour

I added the following line to the FeedManager (app/lib/feed_manager.rb) in the push_to_list function:

`return false if status.reply?`

Now all posts that are replies are filtered out, so that now only "genuine" posts are displayed in the list.

This is a first approach to solve issue #5916

* Update feed_manager.rb

As suggested by @Gargron
2018-09-28 00:37:21 +02:00
ふぁぼ原 f7a6f9489d Add a new preference to always hide all media (#8569) 2018-09-25 05:09:35 +02:00
Matt Sweetman 1889526e23 Add user preference to always expand toots marked with content warnings (#8762) 2018-09-24 05:44:01 +02:00
Thibaut Girka 91bef4759f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	Vagrantfile
	app/javascript/packs/public.js
	app/views/admin/settings/edit.html.haml
	app/views/settings/preferences/show.html.haml
	app/views/settings/profiles/show.html.haml
	config/locales/es.yml
	config/locales/simple_form.en.yml
	config/webpack/configuration.js
	config/webpack/loaders/babel.js
	package.json
	yarn.lock

Split new additions to app/javascript/packs/public.js to
app/javascript/core/settings.js
2018-09-19 21:46:01 +02:00
Eugen Rochko f4d549d300
Redesign forms, verify link ownership with rel="me" (#8703)
* Verify link ownership with rel="me"

* Add explanation about verification to UI

* Perform link verifications

* Add click-to-copy widget for verification HTML

* Redesign edit profile page

* Redesign forms

* Improve responsive design of settings pages

* Restore landing page sign-up form

* Fix typo

* Support <link> tags, add spec

* Fix links not being verified on first discovery and passive updates
2018-09-18 16:45:58 +02:00
Thibaut Girka 6f75a9001f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/stream_entries/_simple_status.html.haml
- config/locales/nl.yml

Deleted unused translation strings (themes) and adapted minor changes to
_simple_status.html.haml
2018-08-31 19:10:57 +02:00
Renato "Lond" Cerqueira fe56d26f7b Fix autoplay issue with spoiler tag (#8540)
Add tests to avoid similar issues in the future
2018-08-31 15:16:59 +02:00
Eugen Rochko e3764bdb52
Do not sign useless User-Agent or Accept-Encoding headers (#8533)
Fix #8080
2018-08-31 04:22:52 +02:00
Renato "Lond" Cerqueira 11658d8653 Add animate custom emoji param to embed pages (#8507)
* Add animate custom emoji param to embed pages

* Rename param, use it for avatars and gifs

* Fix issues pointed by codeclimate and breaking test

* Ignore brakeman warning
2018-08-30 23:14:01 +02:00
Thibaut Girka 53b7bb9d04 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/api/v1/mutes_controller.rb
	config/locales/simple_form.pl.yml
2018-08-27 18:09:28 +02:00
Eugen Rochko cabdbb7f9c
Add CLI task for rotating keys (#8466)
* If an Update is signed with known key, skip re-following procedure

Because it means the remote actor did *not* lose their database

* Add CLI method for rotating keys

    bin/tootctl accounts rotate [USERNAME]

Generates a new RSA key per account and sends out an Update activity
signed with the old key.

* Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts

* Skip suspended accounts in key rotation
2018-08-26 20:21:03 +02:00
Quint Guvernator da13fa5021 Fix low-hanging rubocop gripes (#8458)
* rubocop: quit being so picky

* rubocop: miscellany

* rubocop: prefer present to blank
2018-08-26 19:22:46 +02:00
Thibaut Girka 36393e1d2b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/views/layouts/application.html.haml

Edited:
        app/helpers/application_helper.rb
        app/views/admin/domain_blocks/new.html.haml

Conflict wasn't really one, just two changes too close to one another.
Edition was to adapt the class names for themes to class names for
skins and flavours.

Also edited app/views/admin/domain_blocks/new.html.haml to strip the
duplicate admin pack inclusion thing.
2018-08-26 14:23:24 +02:00
Eugen Rochko b4ba4b1b5d
Spread out crawling randomly to avoid DDoSing the link (#8445)
* Spread out crawling randomly to avoid DDoSing the link

Fix #4486

* Remove trailing whitespace
2018-08-26 00:33:57 +02:00
M Somerville 95bd0d4528 Support ActivityStreams’ summaryMap. (#8422)
In the same way as contentMap and nameMap.
2018-08-25 13:27:34 +02:00
Thibaut Girka 246c313d45 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/application_controller.rb

Changed instance theme selection by instance flavour selection.
2018-08-24 15:10:34 +02:00
Jakub Mendyk 6cb3514d64 Add ability to change an instance default theme from the administration panel (#7092) (#8381)
* Add default_settings class method to ScopedSettings

ScopedSettings was extended to use value of unscoped setting instead of
only using defaults set in config/settings.yml for selected settings.
This adds possibility for admins to set default values of users' settings,
for example default theme (as requested in #7092).

* Add ability to change an instance default theme

Closes #7092
2018-08-23 14:17:35 +02:00
ThibG 42573b76f1 Do not crash if remote custom emoji does not define updated date (fixes #8376) (#8377) 2018-08-23 00:27:58 +02:00
Thibaut Girka e70fc059a9 Merge branch 'master' into glitch-soc/merge-upstream 2018-08-22 20:56:32 +02:00
Eugen Rochko 802cf6a4c5
Improve federated ID validation (#8372)
* Fix URI not being sufficiently validated with prefetched JSON

* Add additional id validation to OStatus documents, when possible
2018-08-22 20:55:14 +02:00
Thibaut Girka 280d7b1df8 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/status.rb
	db/migrate/20180528141303_fix_accounts_unique_index.rb
	db/schema.rb

Resolved by taking upstream changes (no real conflicts, just glitch-soc
specific code too close to actual changes).
2018-08-17 17:43:54 +02:00
ThibG 59f7f4c923 Implement Undo { Accept { Follow } } (fixes #8234) (#8245)
* Add Follow#revoke_request!

* Implement Undo { Accept { Follow } } (fixes #8234)
2018-08-17 16:24:56 +02:00
ThibG 1ee675d68b Use correct activity id in Accept when receiving duplicate Follow (fixes #8218) (#8244) 2018-08-17 14:08:17 +02:00
Eugen Rochko 434628d98f Expect relays to answer with accept/reject (#8179) 2018-08-13 18:17:20 +02:00
Eugen Rochko 39e361a56d
Expect relays to answer with accept/reject (#8179) 2018-08-12 18:16:26 +02:00
Thibaut Girka e5a603206d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/controllers/accounts_controller.rb
	app/javascript/mastodon/locales/pl.json
	app/views/about/more.html.haml

Conflicts in `accounts_controller.rb` resolved by taking upstream's
version + our `use_pack`.

Conflicts in `pl.json` resolved by taking upstream's changes.

Conflicts in `aboute/more.html.haml` resolved by taking upstream's changes.
2018-08-10 16:22:31 +02:00
ThibG c52bcc0331 Serialize text-less statuses as '.' over OStatus (fixes #7856) (#8126) 2018-08-05 12:22:23 +02:00
Thibaut Girka c1c514ca70 Merge branch 'master' into glitch-soc/merge-upstream
Manually-resolved conflicts:
	.circleci/config.yml
	app/controllers/accounts_controller.rb
	app/controllers/auth/passwords_controller.rb
	app/controllers/statuses_controller.rb
	app/javascript/packs/public.js
	app/models/media_attachment.rb
	app/views/stream_entries/_content_spoiler.html.haml
	app/views/stream_entries/_media.html.haml
	config/locales/en.yml
	config/locales/ja.yml
	config/locales/pl.yml
	lib/mastodon/version.rb

Some content from app/javascript/packs/public.js has been split to
app/javascript/core/settings.js.

Translation strings for glitch-soc's keyword mutes were dropped.

Everything else was mostly “take both”.
2018-08-01 00:18:13 +02:00
Thibaut Girka 88b593a63f Drop glitch-style account bio fields
Upstream's implementation has been merged a while ago and is the preferred
way to set fields, as it is the only one compatible with upstream and is
more user-friendly.

This commit deletes the legacy glitch-soc FrontMatter functionality in order
to clean up the code and make it easier to maintain.
2018-07-31 16:26:36 +02:00
Eugen Rochko 07b799468d
Fix incorrect context definition for the "featured" keyword (#8090)
* Fix incorrect context definition for the "featured" keyword

Fix #8077

* Adjust context definition for movedTo
2018-07-30 19:33:37 +02:00
Thibaut Girka 7dbed4dab9 Merge branch 'master' into glitch-soc/merge-upstream 2018-07-17 22:05:25 +02:00
Eugen Rochko a23ac107e4
Fix activity:interactions counter to count all interactions (#8037) 2018-07-16 18:35:43 +02:00
Eugen Rochko 8f64b17d98 Whitelist dat/ipfs/gopher links in sanitizer (#8034)
Fix #7994
2018-07-16 22:17:43 +09:00
Thibaut Girka f26f1145ac Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	Dockerfile
	app/javascript/packs/common.js
	config/webpack/loaders/sass.js
	config/webpack/shared.js
	db/schema.rb
	package.json
	yarn.lock

A lot of the conflicts come from updating webpack.

Even though upstream deleted app/javascript/packs/common.js, I kept
glitch-soc's version as it unifies JS/CSS packs behavior across flavours.

Ported glitch changes to webpack 4.x
2018-07-16 14:50:42 +02:00
Eugen Rochko 2354b10eb5
Add admin dashboard (#8029) 2018-07-16 01:11:53 +02:00
Eugen Rochko 38e9662d78
Disable language detection for texts shorter than 140 characters (#8010)
If the input text is blank after preparation (only mention, or
only URL, or empty as in a media post), then use nil as language,
since it's OK to show to everyone.

Otherwise, always fall back to the server's default locale
2018-07-14 04:05:36 +02:00
Eugen Rochko 47c59591e0 Do not count self in potential friendships (#8014) 2018-07-14 10:44:23 +09:00
Thibaut Girka 6f0f583abe Merge branch 'master' into glitch-soc/merge-upstream 2018-07-11 17:57:08 +02:00
Eugen Rochko 20fefdb714
Make whole-word filter regex consistent between Ruby and JS (#7987) 2018-07-10 03:01:50 +02:00
Thibaut Girka 402da8065c Migrate glitch-soc keyword mutes to Mastodon's
Completely remove glitch-soc's Keyword Mutes, migrate
existing database records to CustomFilters.

Handling of client-side filters is still not implemented
in the glitch-soc front-end.
2018-07-09 19:19:23 +02:00
Thibaut Girka d392020da6 Merge branch 'master' into glitch-soc/tentative-merge
Conflicts:
	README.md
	app/controllers/statuses_controller.rb
	app/lib/feed_manager.rb
	config/navigation.rb
	spec/lib/feed_manager_spec.rb

Conflicts were resolved by taking both versions for each change.
This means the two filter systems (glitch-soc's keyword mutes and tootsuite's
custom filters) are in place, which will be changed in a follow-up commit.
2018-07-09 07:13:59 +02:00
ThibG 1ca4e51eb3 Add option to not consider word boundaries when processing keyword filtering (#7975)
* Add option to not consider word boundaries when filtering phrases

* Add a few tests for keyword/phrase filtering
2018-07-09 02:22:09 +02:00
Thibaut Girka c699b2d141 Remove duplicate code for handling muted users
The mute check was indeed streamlined in f62539ce5c
and we somehow kept some of the old code when merging.
2018-07-07 19:10:13 +02:00
Eugen Rochko 404c7702ec
In keyword filter, account for reblogs, HTML and whole-words (#7960)
* In keyword filter, account for reblogs, HTML and whole-words

* Match whole words in JS filter, too

* Fix typo
2018-07-06 02:15:44 +02:00
Eugen Rochko 4b198b172d
Check reblogged status for blocked/muted mentions (#7957) 2018-07-05 18:43:37 +02:00
Eugen Rochko da8fe8079e
Re-add follow recommendations API (#7918)
* Re-add follow recommendations API

    GET /api/v1/suggestions

Removed in 8efa081f21 due to Neo4J
dependency. The algorithm uses triadic closures, takes into account
suspensions, blocks, mutes, domain blocks, excludes locked and moved
accounts, and prefers more recently updated accounts.

* Track interactions with people you don't follow

Replying to, favouriting and reblogging someone you're not following
will make them show up in follow recommendations. The interactions
have different weights:

- Replying is 1
- Favouriting is 10 (decidedly positive interaction, but private)
- Reblogging is 20

Following them, muting or blocking will remove them from the list,
obviously.

* Remove triadic closures, ensure potential friendships are trimmed
2018-07-03 01:47:56 +02:00
MIYAGI Hikaru ddd0bb69e1 Merge `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into `ALLOW_ACCESS_TO_HIDDEN_SERVICE` (#7901)
If Mastodon accesses to the hidden service via transparent proxy, it's needed to avoid checking whether it's a private address, since `.onion` is resolved to a private address.
I was previously using the `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` to provide that function. However, I realized that using `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` is redundant, since this specification is always used with `ALLOW_ACCESS_TO_HIDDEN_SERVICE`. Therefore, I decided to integrate the setting of `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into` ALLOW_ACCESS_TO_HIDDEN_SERVICE`.
2018-06-29 15:36:02 +02:00
Eugen Rochko cdb101340a
Keyword/phrase filtering (#7905)
* Add keyword filtering

    GET|POST       /api/v1/filters
    GET|PUT|DELETE /api/v1/filters/:id

- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration

* Make sure expired filters don't get applied client-side

* Add missing API methods

* Remove "regex filter" from column settings

* Add tests

* Add test for FeedManager

* Add CustomFilter test

* Add UI for managing filters

* Add streaming API event to allow syncing filters

* Fix tests
2018-06-29 15:34:36 +02:00
Thibaut Girka ab5f450700 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/user.rb

Resolved by adding :default_language to user settings fields
2018-06-21 20:49:57 +02:00
Eugen Rochko a58ec29631
Allow selecting default posting language instead of auto-detect (#7828)
* Allow selecting default posting language instead of auto-detect

* Enable default language setting in credentials API

* Fix form saving
2018-06-17 18:57:31 +02:00
David Yip 5cff053944
Merge branch 'master' into 454-allow-keyword-mutes-to-skip-mentions 2018-06-12 18:13:30 -05:00
David Yip 908a770d2b
keyword mute: use mentions scope in home feed filtering (#454)
If a status shows up in mentions because all keyword mutes that might
apply to it are marked as "don't apply to mentions", then it ought to
show up in the home feed also.
2018-06-12 17:14:35 -05:00
Jenkins 4d8d1fa129 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-06-12 13:17:21 +00:00
Eugen Rochko 6ba7d9d0d9
Do not accept ActivityPub follow requests from blocked user (#7756)
* Do not accept ActivityPub follow requests from blocked user

Fix #7745

* Deliver auto-rejection immediately when follow-requested by blocked account

* Fix trailing whitespace
2018-06-08 00:14:06 +02:00
David Yip 26573ad7e6
Thread scopes through #matches?. #454.
Also add an apply_to_mentions attribute on Glitch::KeywordMute, which is
used to calculate scope.  Next up: additions to the test suite to
demonstrate how scoping works.
2018-06-03 23:00:50 -05:00
David Yip 3550470c18
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/mastodon/locales/en.json
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json

The above conflicts appear to be a text conflict introduced by
glitch-soc's additional level of columns (i.e. moving a bunch of columns
under the Misc option).  They were resolved via accept-ours.
2018-06-02 16:15:36 -05:00
Eugen Rochko 1e938b966e
Exclude unlisted, private and direct toots from affecting trends (#7686) 2018-05-31 16:47:28 +02:00
Thibaut Girka c087738270 Merge branch 'master' into glitch-soc/merge-upstream 2018-05-29 13:52:26 +02:00
abcang 90908fc24b Fix N+1 on AtomSerializer (#7669) 2018-05-29 13:34:02 +02:00
Eugen Rochko ab36e0ef72 Record trending tags from ActivityPub, too (#7647) 2018-05-28 12:21:04 +09:00
Thibaut Girka 11cc2e099a Merge branch 'master' into glitch-soc/merge-master
Conflicts:
	app/javascript/styles/mastodon-light.scss
	config/themes.yml

Removed config/themes.yml, took upstream's mastodon-light.scss
2018-05-25 18:59:02 +02:00
ThibG cdbdf7f98b Ignore multiple occurrences of a hashtag within a status (fixes #7585) (#7606) 2018-05-25 14:26:45 +02:00
David Yip e0eebba461
Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/controllers/settings/preferences_controller.rb
 	app/lib/user_settings_decorator.rb
 	app/models/user.rb
 	config/locales/simple_form.en.yml
2018-05-18 08:47:10 -05:00
Eugen Rochko 57b503d4ef
Resolve unknown status from Add activity, skip Remove if unknown (#7526)
Fix #7518
2018-05-18 11:33:56 +02:00
Eugen Rochko 1e02dc8715
Add preference to hide following/followers lists (#7532)
* Add preference to hide following/followers lists

- Public pages
- ActivityPub collections (does not return pages but does give total)
- REST API (unless it's your own) (does not federate)

Fix #6901

* Add preference

* Add delegation

* Fix issue

* Fix issue
2018-05-18 02:26:51 +02:00
Jenkins 4c0770d6c8 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-05-18 00:17:23 +00:00
MIYAGI Hikaru 919eef3098 User agent for WebFinger (#7531)
* User agent for WebFinger

* local_domain → web_domain

* 'http' is away accidentally...
2018-05-18 01:47:22 +02:00
Thibaut Girka 78ac246b87 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	Gemfile

Solved conflicts by using upstream's line for posix-spawn
2018-05-16 21:52:38 +02:00
Akihiko Odaki 55fd55714a Raise Mastodon::RaceConditionError if Redis lock failed (#7511)
An explicit error allows user agents to know the error and Sidekiq to
retry.
2018-05-16 12:29:45 +02:00
Thibaut Girka ba7ee67498 Merge branch 'master' into glitch-soc/merge 2018-05-14 20:51:50 +02:00
ThibG 7467361d70 Fetch boosted statuses on behalf of a follower (fixes #7426) (#7459)
When an ActivityPub Announce is processed and the boosted toot is not known,
fetch it on behalf of one of the booster's followers. This is to allow
fetching self-boosts of previously-unknown private toots.

If fetching on behalf of a user fails, try fetching it anonymously: the
selected follower of a boosting user may be banned by the boosted toot's
author.
2018-05-12 16:48:32 +02:00
Thibaut Girka 45fce0e496 Merge branch 'master' into glitch-soc/merge
Conflicts:
	app/controllers/invites_controller.rb
	app/serializers/initial_state_serializer.rb
	config/locales/ko.yml
2018-05-11 18:12:42 +02:00
Yamagishi Kazutoshi 6588f6a0a9 Disable inflate gzip on head method (#7432) 2018-05-10 14:36:12 +02:00
Thibaut Girka 1c9c0167b7 Merge branch 'master' into glitch-soc/master
Conflicts:
	app/models/account.rb
	app/views/accounts/_header.html.haml
2018-05-10 00:03:28 +02:00
Yamagishi Kazutoshi edf882320a Support gzip encoding on Request (#7425) 2018-05-09 16:59:39 +02:00
Eugen Rochko 6208ea5a53
If an OStatus message contains nsfw hashtag, mark it as sensitive (#7398)
* If an OStatus message contains nsfw hashtag, mark it as sensitive

Undo parts of #7048

* Put nsfw hashtag on OStatus messages if they have any media

* Fix code style issues
2018-05-07 14:49:13 +02:00
Eugen Rochko 61a9018607
Enable custom emojis in profiles (notes, field values, display names) (#7374)
Follow-up to #6124
2018-05-06 11:48:51 +02:00
Eugen Rochko c947e2e4c5
Fix handling of malformed ActivityPub payloads when URIs are nil (#7370)
* Fix handling of malformed ActivityPub payloads when URIs are nil

* Gracefully handle JSON-LD canonicalization failures
2018-05-05 18:22:34 +02:00
David Yip 7600067a30
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
    	db/schema.rb
2018-05-04 14:16:30 -05:00
Eugen Rochko 6793bec4c6
Store URIs of follows, follow requests and blocks for ActivityPub (#7160)
Same URI passed between follow request and follow, since they are
the same thing in ActivityPub. Local URIs are generated during
creation using UUIDs and are passed to serializers.
2018-05-04 21:14:34 +02:00
David Yip da70208b45
Merge remote-tracking branch 'origin/master' into merge-upstream 2018-05-04 03:23:57 -05:00
David Yip c816701550
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	.travis.yml
 	Gemfile.lock
 	README.md
 	app/controllers/settings/follower_domains_controller.rb
 	app/controllers/statuses_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/lib/feed_manager.rb
 	app/models/media_attachment.rb
 	app/models/mute.rb
 	app/models/status.rb
 	app/services/mute_service.rb
 	app/views/home/index.html.haml
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/ca.yml
 	config/locales/en.yml
 	config/locales/es.yml
 	config/locales/fr.yml
 	config/locales/nl.yml
 	config/locales/pl.yml
 	config/locales/pt-BR.yml
 	config/themes.yml
2018-05-03 17:23:44 -05:00
ThibG a24605961a Fixes/do not override timestamps (#7336)
* Revert "Fixes/do not override timestamps (#7331)"

This reverts commit 581a5c9d29.

* Document Snowflake ID corner-case a bit more

Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.

Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.

* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
2018-05-03 23:02:46 +02:00
ThibG 581a5c9d29 Fixes/do not override timestamps (#7331)
* Do not override timestamps for incoming toots

* Remove every reference to override_timestamps

Statuses are now created with the announced publishing date
and are only pushed to timelines if that date is at most
6 hours earlier than the time at which it is processed.
2018-05-03 13:33:08 +02:00
Eugen Rochko a5293fdf61
Fix n+1 queries in StatusThreadingConcern (#7321) 2018-05-03 10:41:58 +02:00
Eugen Rochko cb5b5cb5f7
Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
ThibG d0cdd5cf94 Accept actor object updates from all supported actor types (#7312) 2018-05-02 16:08:16 +02:00
Eugen Rochko f62539ce5c
Remove most behaviour disparities between blocks and mutes (#7231)
* Remove most behaviour disparities between blocks and mutes

The only differences between block and mute should be:

- Mutes can optionally NOT affect notifications
- Mutes should not be visible to the muted

Fix #7230
Fix #5713

* Do not allow boosting someone you blocked

Fix #7248

* Do not allow favouriting someone you blocked

* Fix nil error in StatusPolicy
2018-05-02 15:50:20 +02:00
Eugen Rochko 965345316f
Guard against nil URLs in Request class (#7284)
Fix #7265
2018-05-02 15:44:22 +02:00
Surinna Curtis dc786c0cf4 Support Actors/Statuses with multiple types (#7305)
* Add equals_or_includes_any? helper in JsonLdHelper

* Support arrays in JSON-LD type fields for actors/tags/objects.

* Spec for resolving accounts with extension types

* Style tweaks for codeclimate
2018-05-02 12:40:24 +02:00
Eugen Rochko 705f1d7bf1
Fix missing updated_at attribute on emoji EntityCache (#7297)
Just don't try to save space by only selecting few attributes. If
anyone is wondering, this is needed because the emoji entity cache
is not really only used for entities, it's accessed again to
generate Emoji tags in ActivityPub/OStatus, so a lot more properties
are used than what is needed in HTML alone...
2018-04-30 22:49:33 +02:00
Eugen Rochko 295e3ef02b
Fix missing domain attribute in EntityCache for emoji (#7290) 2018-04-30 09:12:55 +02:00
Eugen Rochko a872392cd9
Add entity cache (#7271)
* Add entity cache

Use a caching layer for mentions and custom emojis that are
dynamically extracted from text.

Reduce duplicate text extractions

* Fix code style issue
2018-04-27 01:38:10 +02:00
MIYAGI Hikaru f58dcbc981 HTTP proxy support for outgoing request, manage access to hidden service (#7134)
* Add support for HTTP client proxy

* Add access control for darknet

Supress error when access to darknet via transparent proxy

* Fix the codes pointed out

* Lint

* Fix an omission + lint

* any? -> include?

* Change detection method to regexp to avoid test fail
2018-04-25 02:14:49 +02:00
Eugen Rochko 9d4710ed00
Add RSS feeds for end-users (#7259)
* Add RSS feed for accounts

* Add RSS feeds for hashtags

* Fix code style issues

* Fix code style issues
2018-04-25 02:10:02 +02:00
Eugen Rochko 7db7d68136
Detect and prevent image bombs, max. processable dimension 4096^2 (#7229) 2018-04-23 09:16:38 +02:00
Eugen Rochko 75c4ab9d12
Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048)
Fix #7011
2018-04-22 22:09:03 +02:00
David Yip 813da6788e
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	db/schema.rb
2018-04-14 09:12:50 -05:00
Eugen Rochko 78ed4ab75f
Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
David Yip 67e77f230e
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/serializers/initial_state_serializer.rb

The glitch flavour isn't yet pulling custom emoji data on its own (see
https://github.com/tootsuite/mastodon/pull/7047).  Once that gets into
the glitch flavour, we can eliminate the custom_emojis load.
2018-04-08 19:05:02 -05:00
Levi Bard cd0eaa349c Enable updating additional account information from user preferences via rest api (#6789)
* Enable updating additional account information from user preferences via rest api
Resolves #6553

* Pacify rubocop

* Decoerce incoming settings in UserSettingsDecorator

* Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters

* Clean up user preferences update

* Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence
2018-04-08 13:43:10 +02:00
Eugen Rochko b5726def55
Forward deletes on the same path as reply forwarding (#7058)
* Forward deletes on the same path as reply forwarding

* Remove trailing whitespace
2018-04-07 18:54:46 +02:00
David Yip eb39db7f93
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/mastodon/locales/en.json
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json
 	app/views/accounts/_header.html.haml
2018-04-01 20:24:50 -05:00
David Underwood 123a343d11 [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
Jenkins f0bb3ff533 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-30 19:17:17 +00:00
ThibG e573bb0990 Fix compatibility with PeerTube (#6968)
* Support fetching objects of convertible types by URL (fixes #6924)

* Ignore invalid hashtags
2018-03-30 15:44:54 +02:00
ThibG 9ed5eebd7c Do not ignore unknown media attachments, only skip them (#6948)
That way, they are displayed in a list below the corresponding toot.
2018-03-29 00:52:24 +02:00
David Yip 1ed9500b52
Merge pull request #400 from vulpineclub/dev-gs-392-frontmatter
Remove ignored 'u' parameter on regexp (#392)
2018-03-27 13:06:28 -05:00
Rey Tucker 21b586436e Remove ignored 'u' parameter on regexp (#392) 2018-03-27 11:30:49 -04:00
David Yip f61aa8e0f7
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/styles/mastodon/components.scss
 	app/models/media_attachment.rb
2018-03-27 10:26:47 -05:00
Akihiko Odaki 40e5d2303b Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Jenkins 837b3804bf Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-25 15:17:21 +00:00
Akihiko Odaki 54b273bf99 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Jenkins e8aeec4fea Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-20 19:17:20 +00:00
Rey Tucker 36b5703796 request: in the event of failure, try other IPs (#6761) (#6813)
* request: in the event of failure, try other IPs (#6761)

In the case where a name has multiple A/AAAA records, we should
try subsequent records instead of immediately failing when we have a
failure on the first IP address.

This significantly improves delivery success when there are network
connectivity problems affecting only IPv4 or IPv6.

* fix method call style

* request_spec: adjust test case to use Addrinfo

* request: Request/open: move private addr check to within begin/rescue

* request_spec: add case to test failover, fix exception check

* Double Addrinfo.foreach so that it correctly yields instances
2018-03-20 09:06:08 +01:00
Jenkins df70571bfe Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-19 20:17:18 +00:00
ThibG ff6b8a6443 Serialize mentions in the order they are added (#6836)
Up until now, the order seemed to be in the *opposite* order,
which caused the WebUI to populate mentions in reversed order
when replying to toots local to one's instance.
2018-03-19 20:19:35 +01:00
David Yip ab04be2f84
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	config/i18n-tasks.yml
2018-03-18 20:21:17 -05:00
Eugen Rochko f02411da40
Ignore media validation when attaching to status during processing (#6822)
Fix #6821
2018-03-19 01:51:19 +01:00
Jenkins 447d7e6127 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-09 00:17:17 +00:00
MitarashiDango 6dcf96271e fix validation error (media only status) (#6684)
* fix validation error (media only status)

* Incorporating review suggestions

* Reflect similar fix to OStatus side

* Fix not to include media in transaction

* Restore the limit of the number of media

* Fix not to return nil
2018-03-08 01:22:47 +01:00
Eugen Rochko cfa9b6e13a
Remove text requirement when media attached from statuses (#6672) 2018-03-07 08:28:52 +01:00
Eugen Rochko e6520c0270
Fix #6657 - Use target instead of origin in Remove activity (#6664) 2018-03-07 03:54:46 +01:00
Eugen Rochko 9110db41c5
Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
David Yip 1b8fcd4df5
Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	README.md
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/serializers/rest/instance_serializer.rb
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/simple_form.ja.yml
2018-03-02 21:46:44 -06:00
Eugen Rochko e852872846
Fix #5708: Reject->Follow will remove the follow if it exists (#6571) 2018-02-28 06:55:06 +01:00
Eugen Rochko 41a01bec23
Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Akihiko Odaki 2e8a492e88 Raise Mastodon::HostValidationError when host for HTTP request is private (#6410) 2018-02-24 19:16:11 +01:00
Jenkins 2edf7128f2 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-24 05:17:10 +00:00
Akihiko Odaki f8f0572ee0 Do not push status to feed if its reblog is already inserted (#6488)
A complemental change for precompute_feed_service_spec.rb also fixes its
random failure which is caused by the Snowlake randomization of the order
of an original status and its reblog.
2018-02-24 05:40:18 +01:00
imncls bb6988a7ac
Merge branch 'master' of https://github.com/tootsuite/mastodon
# Conflicts:
#	app/controllers/settings/exports_controller.rb
#	app/models/media_attachment.rb
#	app/models/status.rb
#	app/views/about/show.html.haml
#	docker_entrypoint.sh
#	spec/views/about/show.html.haml_spec.rb
2018-02-23 23:28:31 +09:00
Eugen Rochko 90f12f2e5a
Focal points (#6520)
* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments
2018-02-22 00:35:46 +01:00
Eugen Rochko a7171af0a3
Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Jenkins 1445ba1703 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-19 15:17:14 +00:00
Eugen Rochko a71af98401
Push discovered status through streaming API within a time window (#6484)
Time window of 6 hours
2018-02-17 14:28:48 +01:00
Jenkins efc9ffcb80 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 04:10:16 +00:00
David Yip b6159bcb0a
Merge remote-tracking branch 'origin/merge-upstream' 2018-02-10 22:08:45 -06:00
David Yip 53c86b29f0
Glitch::FilterHelper -> Glitch::KeywordMuteHelper. #234.
The class helps out with keyword mutes, not just some general concept of
"filtering".
2018-02-10 10:36:15 -06:00
David Yip 29b5b46c87
Strip HTML from keyword mute input. #234. 2018-02-10 02:40:27 -06:00
Eugen Rochko 3ebc0ad4d3
Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
David Yip 020b40efdb
Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	.travis.yml
	app/lib/user_settings_decorator.rb
	app/models/user.rb
	app/serializers/initial_state_serializer.rb
	app/views/stream_entries/_detailed_status.html.haml
	app/views/stream_entries/_simple_status.html.haml
	config/locales/simple_form.en.yml
2018-02-09 09:25:53 -06:00