- this change contains the removal of the `promise.allsettled` package,
as this is not needed on Node 12+, which removes 75 further dependencies
in production mode
ref https://linear.app/tryghost/issue/ENG-1254
- we currently only store a cancellation reason when a member cancels
manually in Portal
- we now also store "Payment failed" when the cancellation is automatic
due to several payment failures
refs
[ENG-709](https://linear.app/tryghost/issue/ENG-709/%F0%9F%90%9B-bad-redirects-causing-container-tear-down)
Added validation to prevent RegEx's susceptible to ReDoS from being used
with custom redirects. Also moved error details out of `context` and
into `errorDetails` to be consistent with error logging elsewhere as
well as fix issue in admin-x where blank screen would be shown when an
error occurred during redirects upload (due to logic not accounting for
`context` being an object)
fixes https://linear.app/tryghost/issue/ENG-642
- When a subscription is in the `canceled` state the corresponding
Member has no access to the Ghost site. The only time a Member will
continue to have access if their subscription is due to cancel at the
period end is if it is still in an active state, which is one of
`active` `trialing` `unpaid` or `past_due`
- When a subscription is canceled immediately (i.e. before the end of
the current billing period), we now render "Ended" without a date,
because we don't store the cancellation date in the subscription object.
We previously used "Ended {current_period_end}" which would sometimes
lead to dates in the future
- Bonus: refactored code and added unit tests
---------
Co-authored-by: Sag <guptazy@gmail.com>
- the existing code creates a new moment instance, takes away some days
and then formats the result
- this is run for every entry of the member attribution stats, which
means dashboards for big sites with a lot of attribution data become
slow
- this value doesn't change across each iteration of the filter, so we
can just extract it out and calculate it once
- this commit removes this code block from the flamegraph completely
fix https://linear.app/tryghost/issue/ENG-779/%F0%9F%90%9B-cmds-does-not-save-member-profile-changes
- previously, pressing Cmd+S on a member profile would save the profile,
but the dirty attributes weren't being cleaned, so the application
would trigger the leave confirmation when exiting
- now, we've fixed the code to keep a dynamic scratch member,
- long term, we should get rid of the scratch model, but this still
allows us to fix the bug for now
closes https://linear.app/tryghost/issue/ENG-1255
- updated Koenig packages including:
- addition of `/preview` for public preview card
- fix for HTML import from Google Docs
- fix for embed thumbnails being cut off in email
- fix for wide image card width on medium screens
- multiple fixes for unhandled (but non user-visible) errors causing noise in console and error logging
ref https://linear.app/tryghost/issue/ONC-111
- changed posts fetching/display behavior to be client-side instead of server-side
- admin will issue (potentially multiple) requests based on the desired status(es)
- updated admin acceptance test for missing coverage
I've pulled the sort from the database query as this triple sort
performs very poorly at scale (taking ~4s+ past ~20k posts sometimes).
Instead, we now split up the fetch to grab only one status at a time and
use the front-end logic to handle displaying scheduled, then drafts,
then published. This should result in a much more responsive view.
We will separately change the default sort on the Admin API as that was the ultimate intent for this change.
- `faker` was the original dependency but the maintainer ended up
deleting the repo, so development continued in `@faker-js/faker`
- we're already using that dependency, so we can make a few simple
changes and remove the old dependency from our repo
ref https://linear.app/tryghost/issue/SLO-173/removed-distinct-from-member-count-query
Performance of GET /members API can be improved by dropping the distinct from the total members count query.
select count(distinct members.id) as aggregate from `members`; // 275ms
select count(*) as aggregate from `members`; // 30ms
In this case we know that the result set will always be unique.
ref https://linear.app/tryghost/issue/ONC-111
- added composite index to posts_tags for post_id,tag_id for faster
lookup
- added composite index to posts for updated_at; this is commonly used
by get helpers on the front end to display data like the latest posts
In testing, this provided a very dramatic improvement for simple get
helper requests like 'filter="id:-{{post.id}}+tag:sampleTag" limit="3"'
which are by default sorted by updated_at desc. I'm not entirely clear
why when sorting by published_at we do not need a composite index - so
far it doesn't seem to be necessary. This should cover the primary cases
for get helpers - the latest posts with a given tag or set of tags.
fix https://linear.app/tryghost/issue/SLO-168/rangeerror-maximum-call-stack-size-exceeded
- this code takes the API output and reduces it down to collect together
stats per date
- the current code is recursive, and we've seen errors with the
recursion hitting a `RangeError: Maximum call stack size exceeded`
error
- as well as that, we're doing a lot of array concat'ing and cloning,
which burns memory and CPU time
- instead, we can just use `.reduce`
- the new implementation is much faster than the existing one (1ms vs
85ms) and uses no recursion, so those errors should go away
- I've also verified that the output is the same between the two
functions
ref ONC-109
- Attempt to fix flaky Admin test, "Publish flow members enabled can
schedule publish+send"
- Adjusted the time calculation to the nearest minute to avoid
off-by-one minute errors
- Added `waitFor` to ensure elements are present and stable before
making assertions.
- Rounded the new scheduled date and time to the nearest minute to
maintain consistency
- Included extra `waitFor` and `settled` calls to allow time for UI
elements to fully load and reflect changes before assertions.
REF https://linear.app/tryghost/issue/MOM-238
- Updated feature image action button styles
- Aligned button and tooltip styles with the rest of the editor
- Updated `koenig-lexical` version to pull in new toolbar design
fixes https://linear.app/tryghost/issue/SLO-156
- we have reached our 10k replays per month quota in 20 days, by using a
100% error sampling rate
- we would need a sampling rate < 0.64% to stay under the quota
- from now on, we will be using a 50% error sampling rate to have a bit
of margin, and have a rounder number that is easier to reason about (1
out of 2 error sessions are recorded in Sentry)
ref https://linear.app/tryghost/issue/CFR-43/
ref 9d9a421
We recently stopped `select *` from posts when making Content API
requests. This is now being applied to the pages endpoint to help
improve performance. These fields were already being stripped out in the
output serializer, and they will now no longer be returned from the db
at all, reducing the amount of data transferred.
ref https://linear.app/tryghost/issue/SLO-127
- problem: contributors see an empty list of labels in the Signup card,
even if some exist
- cause: contributors do not have permission to browse labels
- solution: hide the label input entirely for contributors in the Signup
card, based on the new `renderLabels` config parameter
ref https://linear.app/tryghost/issue/CFR-35
- performance improvement intended for the content api/get helpers
The posts table is shared by posts and pages and seldom is queried for
both. It makes sense to add an index on type, and from the perspective
of the content API, also on status as you're almost only ever querying
for published posts or published pages.
https://github.com/TryGhost/Ghost/issues/20445
- after switching to a browser-like user-agent, YouTube started responding with a "supported browsers" message rather than the actual video meaning bookmark creation failed
- when trying other user-agents it was discovered that nytimes.com (why the user-agent was originally changed) had a problem with the github.com address in the user-agent. By switching to using https://ghost.org instead the request was allowed through fixing both YouTube and NYTimes embeds
closes https://linear.app/tryghost/issue/ONC-115
- OpenTelemetry was throwing errors when viewing posts
- disabled the instrumentation in development mode so it requires explicit config to enable
closes https://linear.app/tryghost/issue/ENG-762
- nytimes.com and other sites return 403 responses when requests do not match typical browser user-agents
- our bookmark fetching requests were using `Ghost(https://github.com/TryGhost/Ghost)` meaning bookmark creation failed for these user-agent-blocking sites
- switched to using a standard browser user-agent string to avoid such blocks
closes https://github.com/TryGhost/Ghost/issues/20028
It's fairly common practice for oembed providers to skip some of the "required" fields from the oembed spec such as `height` when it doesn't make sense for the embeddable content, this was the case with Bluesky embeds which return `height: null`
- removed validation for `height` being present in the response for it to be recognised as an embed because we don't use it anywhere and the validation is blocking otherwise valid embeds
no issue
Full details coming soon to https://ghost.org/changelog
- Link toolbar and bookmark cards now let you search your existing posts/pages/tags/authors in addition to manually entering the URL
- Typing "@" inside your content lets you quickly search and add a text link
- Typing "@" on a blank paragraph provides a quick way to search and add a bookmark
ref https://linear.app/tryghost/issue/SLO-127
- problem: when using a card with a button (Button, Email CTA, Header,
Product), the Button URL suggestions fail to load for Contributors,
Authors, and Editors
- cause: Contributors, Authors and Editors don’t have permission to
fetch offers, and this causes the entire list of button url suggestions
to break
- solution: if offers fail to fetch for any reason, the rest of the url
suggestions for cards with a button is now still populated (i.e. offers
URLs are ignored)
no issue
- removed labs flag
- removed labs flag conditionals
- removed code related to old setup/done screen
- fixed tests that weren't correctly running against the GA flag code
ref
https://linear.app/tryghost/issue/ENG-1240/race-condition-when-updating-members-last-seen-at-timestamp
When members click a link in an email, Ghost updates the member's
`last_seen_at` timestamp, but it should only update the timestamp if the
member hasn't yet been seen in the current day (based on the
publication's timezone).
Currently there is a race condition present where multiple simultaneous
requests from the same member (if e.g. an email link checker is
following all links in an email) can cause the `last_seen_at` timestamp
to be updated multiple times in the same day for the same member. These
additional queries add a significant load on Ghost and its database,
which can contribute to the exhaustion of the connection pool and
eventually requests may time out.
The primary motivation for this change is to avoid that race condition
by adding a lock to the member row, checking if `last_seen_at` has
already been updated in the current day, and only updating it if it
hasn't.
Another beneficial side-effect of this change is that it avoids locking
the `labels` and `newsletters` tables, which are locked when we update
the `last_seen_at` timestamp in the `members` table currently. This
should improve Ghost's ability to handle a large influx of requests to
redirect endpoints (confirmed with load tests), which tend to happen
immediately after a publisher sends an email.
This commit adds OpenTelemetry instrumentation to Ghost's backend, which
allows us to view traces similar to what we see in Sentry Performance
locally.
OpenTelemetry is enabled if `NODE_ENV === 'development'` or if it is
explicitly enabled via config with `opentelemetry:enabled`.
It also adds a [Jaeger](https://www.jaegertracing.io/) container to
Ghost's docker-compose file for viewing the traces. There's no setup
required (beyond running `yarn docker:reset` to pickup the changes in
the docker-compose file the first time — but this will also reset your
DB so be careful). This will launch the Jaeger container, and you can
view the UI to see the traces at `http://localhost:16686/search`.
no issue
Typing "@" in the editor will immediately trigger an internal link search to make it faster to link to one of your articles. After typing "@" continue typing to search, results can be selected using Up/Down arrow keys or the mouse, then pressing Enter or clicking will insert the selected result's title pre-linked. Pressing Escape or moving the cursor out of the search box will cancel the search.
- removed labs flag
- updated Koenig feature flag for at-linking to use the same flag as our internal linking beta
DES-459
The font size of subtitles/excerpts in newsletters was similar to the
body font size which doesn't reflect the content hierarchy
appropriately. Also, the spacing should be adjusted to represent that
the title and the subtitle belong together.
ref
https://linear.app/tryghost/issue/ENG-1240/race-condition-when-updating-members-last-seen-at-timestamp
When members click a link in an email, Ghost updates the member's
`last_seen_at` timestamp, but it should only update the timestamp if the
member hasn't yet been seen in the current day (based on the
publication's timezone).
Currently there is a race condition present where multiple simultaneous
requests from the same member (if e.g. an email link checker is
following all links in an email) can cause the `last_seen_at` timestamp
to be updated multiple times in the same day for the same member. These
additional queries add a significant load on Ghost and its database,
which can contribute to the exhaustion of the connection pool and
eventually requests may time out.
The primary motivation for this change is to avoid that race condition
by adding a lock to the member row, checking if `last_seen_at` has
already been updated in the current day, and only updating it if it
hasn't.
Another beneficial side-effect of this change is that it avoids locking
the `labels` and `newsletters` tables, which are locked when we update
the `last_seen_at` timestamp in the `members` table currently. This
should improve Ghost's ability to handle a large influx of requests to
redirect endpoints (confirmed with load tests), which tend to happen
immediately after a publisher sends an email.
ref https://linear.app/tryghost/issue/ONC-94
- adds Sentry logging for the unsaved changes modal in Admin (tags,
members, collections)
This will let us monitor for spikes in rates so we know if a change to
Ghost has caused faulty logic in detecting dirty models.
Fixes
https://linear.app/tryghost/issue/DES-4/image-caption-size-in-email-newsletter.
There were no styles defined for captions for cards beyond the featured
image (bookmark, gallery, video), and we had no way of targeting those
captions with CSS. They are now wrapped in a div with a specific class,
which allows for more selective styling, and are styled similarly to the
caption of the featured image.
ref https://linear.app/tryghost/issue/ENG-1211
- the post slug now re-generates based on the post title for draft posts
unless manually set
- updated unit tests to be a bit more comprehensive
It's frequently the case that draft posts might have WIP titles. We
would generate a post slug based on the title and never change it, so
unless writers remembered to come back in to update it for their final
post, it could look off to readers. This should make that a bit more
intelligent.
Going forward, we will change the slug unless we expect it to be a custom slug (user-set in the side panel). If the title is cleared out and saved, we will also reset it. We will only ever automatically generate & change the slug for draft posts.
closes https://linear.app/tryghost/issue/ONC-96
- better matches Ghost's current usage
- avoids mismatch in expected data when `post_revisions` is populated in future
ref https://linear.app/tryghost/issue/ENG-661
- added a dirty check to ignore the `direction` field from the lexical
object; this is set dynamically and shouldn't be serialized, see
facebook/lexical/issues/4998
- fixed a bug where the modal wouldn't display on leaving the editor if
the post had no revisions (e.g. import); this could result in content
being saved over published content with no user action
- added Sentry logging for the modal
We would sometimes see the "Are you sure?" modal pop up when opening a
post in the editor and attempting to navigate away immediately, without
any changes to the post. This appears to be an issue with the serialized
Lexical data, which would change after loading into the editor,
resulting in the scratch and model's lexical values to differ, making
Admin think the user changed the content.
Ideally we'll see a fix upstream (or fix it ourselves). We may need to
revisit this if we experience other such situations. It's awfully
difficult to be able to set a flag saying 'the editor is done loading',
so this seems to be the best path for the moment.
Testing is difficult because we don't actually load the new Lexical editor into e2e/acceptance tests. I've added a unit test that can at least simulate the editor state changing on editor load.
no issue
- we weren't adding an `order` param to our posts/pages requests used to populate the search index which meant the default Admin API ordering was applied which isn't optimal for this use-case
- switched to ordering by `updated_at` to use a simple order that has an optimised index in the database
fixes https://linear.app/tryghost/issue/SLO-126
fixes https://linear.app/tryghost/issue/SLO-141
fixes https://linear.app/tryghost/issue/SLO-150
- during a session, posthog-js' rrweb extension can start throwing a lot
of errors. These errors do not affect the application
- similarly, ember-concurrency's task cancellation errors do not affect
the application. Task in ember-concurrency are expected to be canceled.
However, if they're cast to a Promise, they show up as unhandled Promise
rejections as Promises do not have a concept of cancellation
fixes https://linear.app/tryghost/issue/SLO-143
- in the editor, if there is a validation error on a post (e.g. the excerpt is longer than 300 chars), a validation error is rendered as a red banner error. However, when clicking on Preview, this error was bypassed
- additionally, we were throwing an undefined error when a validation error happened. This was unnecessary and caused hundreds of unhandled errors per week
We have noticed some attempts at Greek translation with no follow-up
from contributors. This is a complete translation with accurate file
naming conventions. We are native speakers and professionals.
Co-authored-by: Ryan Feigenbaum <48868107+royalfig@users.noreply.github.com>
Comprehensive, context aware, Bosnian translations for all available
strings.
Informal addressing of users (as is widely accepted in Bosnia and
Herzegovina).
Co-authored-by: Ryan Feigenbaum <48868107+royalfig@users.noreply.github.com>
ref https://linear.app/tryghost/issue/MOM-117
- `url` was missing in the results objects that we generate from the underlying search results
- updated service integration test with check for url presence
- updated service integration test to also run against the beta search
- added missing page factory to mirage setup
- updated mirage post serializer to include a uniquely identifiable URL for unpublished posts
ref https://linear.app/tryghost/issue/MOM-117
ref https://linear.app/tryghost/issue/MOM-70
- moved current search into new `search-provider` service and updated `search` service to use the provider service internally
- added `search-provider-beta` service
- uses `flexsearch` as the underlying index for each document so we have better indexing and matching compared to the naive exact-match search we had previously
- adds `excerpt` matching for posts and pages
- keeps results output the same as the original search provider
- added `internalLinkingSearchImprovements` labs flag so we can test this internally before reaching our internal linking beta testers
- updated `search` service to switch between providers based on labs flag
fixes https://linear.app/tryghost/issue/SLO-147
- removed Sentry message when the Lexical save is slow but successful
- bumped Sentry message to error level when the Lexical is slow and
fails
ref DES-347
- adjusted title and excerpt length of latest posts in emails
- as the layout is same (horizontal) on both desktop and mobile, truncateHtml() needed some update
- now maxLength is expected to be larger than maxLengthMobile, because the mobile layout isn't stacked anymore
- some spacing adjustment has been made as well
no issue
- during the subtitle->excerpt rename some instances were missed resulting in the excerpt field element not being registered correctly
- fixed mismatched action name and renamed remaining uses of "subtitle"
closes https://linear.app/tryghost/issue/MOM-174
- bumps @tryghost/koenig-lexical to version that includes a spinner in search popups whilst a search is in progress
ref
https://linear.app/tryghost/issue/KTLO-58/dont-send-ghost-acess-cookies-if-no-member-is-logged-in
- Currently when member's caching is enabled, but no member is logged
in, we always send `ghost-access=null;` and `ghost-access-hmac=null;`
cookies in the requests to `/members/api/member/`. This is done to clear
the cookies, but an unintended consequence is that these requests can
never be cached since there is a cookie in the response.
- This PR removes the cookies from the requests when no member is logged
in, the cookies will not be sent, allowing the requests to be cached
- It also unsets the cookies when deleting a member's session, so that
the cookies are not sent in the requests after the member logs out
- This should improve the cache hit ratio with members caching enabled
no issue
- No code changes, only documentation
- Added detailed overview of everything that happens when a member
clicks on a redirect link in an email, along with a `mermaid.js`
sequence diagram
no issue
- following on from the removal of the `post_revisions.custom_excerpt` column population it was possible in some circumstances to lose your excerpt when restoring an old version
- this change means when no custom_excerpt exists on a revision we both preview and restore the current custom_excerpt to avoid any data loss
no issue
- the query can take a very long time to run on large sites causing problems during the upgrade process
- impact from not populating the column:
- only has an effect when the inline excerpt beta is enabled
- when beta enabled, if a revision created before the upgrade is restored then the excerpt will be removed (will be visibly empty in preview before restoring, if any edit has occurred on the post after upgrading then it can still be recovered by restoring the later version or copy/pasting from the preview)
no issue
Full details coming soon to https://ghost.org/changelog/
- when enabled in newsletter design settings a post's custom excerpt will be displayed as a subtitle in the email
ref https://linear.app/tryghost/issue/CFR-31/
- when relations were included, erroneous logic resulted in the model's
default sort being applied
- the model default sort is not intended for the content API and
needlessly slowed down responses
- there's a change for users here that should be incredibly unlikely to
be hit; default sort is `published_at desc` which will be secondarily
sorted by `id desc` instead of `published_at desc, updated_at desc, id
desc`
This is a very significant performance improvement for content API
requests with includes for sites with a significant amount of data,
which will primarily impact those using Ghost as a CMS or theme {{#get}}
helpers.
no issue
We've settled on using "excerpt" naming in place of "subtitle" to better reflect the underlying property name and tie in with themes and historical usage.
- added migration to rename the `show_subtitle` newsletter setting to `show_excerpt`
- renamed all places in the codebase that referenced subtitle
closes https://linear.app/tryghost/issue/MOM-170
When the subtitle field is included in the editor it creates a disconnect with post revisions if the underlying custom excerpt data is not included so we'd like to both preview and restore the subtitle when the in-editor subtitle field is enabled.
- added `post_revisions.custom_excerpt` column to schema
- added migration to add `post_revisions.custom_excerpt` to existing databases
- added migration to populate `post_revisions.custom_excerpt` with the current `post.custom_excerpt` value from the associated record
- ensures no data is inadvertently lost when restoring an old version
- using current data matches what would have happened previously where custom_excerpt was never overwritten when restoring an old version
- updated post revisions handling to accept the `custom_excerpt` field
- updated Admin's revision preview and restoration to display and set the `custom_excerpt` field
ref https://linear.app/tryghost/issue/ENG-1078
- adds required setup for acceptance testing post revisions
- adds happy-path test for listing, previewing, and restoring a post revision
MOM-190
MOM-192
- The subtitle style (serif/sans) was tied to the body style which was
misleading. It makes more sense to connect it to the title style both
from the UX and the output POV.
- Newsletter design preview was not updated according to subtitle
styles.
---------
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
closes https://linear.app/tryghost/issue/MOM-194
- whilst working on the feature our naming changed from "Subhead" to "Subtitle"
- this rename of the newsletter design setting column brings naming back into a consistent state before public release
closes https://linear.app/tryghost/issue/MOM-173
- updated email renderer to add `post.customExcerpt` data
- updated template to skip rendering subtitle when no custom excerpt is present
- updated template to use actual custom excerpt
closes https://linear.app/tryghost/issue/MOM-182
- we were seeing some odd behaviour with the validation engine when validating individual properties that meant our tracked property wasn't auto-updating on subtitle change after a body edit/autosave event
- switched to a manually tracked property that's updated based on the validate call status
closes https://linear.app/tryghost/issue/MOM-149
- our styles set all `textarea` elements to have a min-height of 100px which wasn't being overridden for our subtitle element meaning our auto-expand code was never able to fully collapse it to a single line
closes https://linear.app/tryghost/issue/MOM-175
- matches cursor behaviour on Up/Down/Left/Right/Tab/Enter to our previous behaviour when we only had the title and editor
closes https://linear.app/tryghost/issue/MOM-150
- use our validation engine to display an error state when >300 characters have been typed in the subtitle input field
---------
Co-authored-by: Sanne de Vries <sannedv@protonmail.com>
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
closes https://linear.app/tryghost/issue/MOM-160
- return `undefined` early from `searchLinks` when the underlying task gets cancelled
- bump `@tryghost/koenig-lexical` so it properly handles cancelled search promises
closes https://linear.app/tryghost/issue/MOM-172
- staff users with no posts won't have a front-end URL so it can be confusing for them to appear in the internal link search results within the editor
- added filtering for `/404/` in staff URLs so we don't list unlinkable staff members
REF MOM-147
- Updated newsletter preview in settings to match the newsletter
template
- Updated spacing for a few edge-case newsletter template combinations
closes https://linear.app/tryghost/issue/MOM-83
- added additional labs flag to allow internal testing prior to private beta release
- bumped Koenig packages containing support for @-link feature
REF MOM-119
- Split subhead feature flag into two: editorSubtitle and
newsletterSubtitle
- Updated UI copy, feature flag names and class names from subhead to
subtitle
refs MOM-152 MOM-148 MOM-151
- Added Subheads behind a flag + toggle in settings.
- Removes Excerpt fields from post settings if flag is enabled.
- Added subhead toggle in newsletter settings.
- Loads of styling
---------
Co-authored-by: Sanne de Vries <sannedv@protonmail.com>
ref https://linear.app/tryghost/issue/SLO-128
- Sentry will record session replays when errors happen in Admin (Settings, Editor) to facilitate debugging
- The recorded sessions mask sensitive information (input fields, media items, content in the editor, metrics in the dashboard)
REF MOM-146
- These styling changes are a general newsletter template improvement,
and are also laying the groundwork for including a subhead in the
newsletter header. Both the newsletter template as well as the
newsletter preview in settings have been updated.
Got some code for us? Awesome 🎊!
Please include a description of your change & check your PR against this
list, thanks!
- [ ] There's a clear use-case for this code change, explained below
- [ ] Commit message has a short title & references relevant issues
- [ ] The build will pass (run `yarn test:all` and `yarn lint`)
We appreciate your contribution!
---------
Co-authored-by: Ryan Feigenbaum <48868107+royalfig@users.noreply.github.com>
We want to use a randomly generated 64 byte secret for the hmac, and
utf8 encoding isn't nice to work with for this, so we're going to use a
base64 string and decode it into a buffer for the secret.
- we don't need this in `ghost/core` as it's not used in there
- we need to declare this dependency for the apps, as they use it for
running tests
- this doesn't change the lockfile but it means we're declaring the
dependency in the right places now
ref
https://linear.app/tryghost/issue/KTLO-45/deploy-members-caching-solution-to-a-single-site-to-validate-and-test
Currently we only cache publicly available content. Any content that is
accessed by a logged in member is only cached for that specific member
based on their cookie. As a result, almost all requests from logged in
members bypass our caching layer and reach Ghost, which adds unnecessary
load to Ghost and its database.
This change adds experimental headers that allow our CDN to understand
which tier to cache the content against, and securely tell the CDN which
tier a logged in member has access to. With these changes, we can cache
the member content against the tier, rather than the individual member,
which should result in a higher cache HIT ratio and reduce the load on
Ghost.
For requests to the frontend of the site, Ghost will set a custom
`X-Member-Cache-Tier` header to the ID of the tier of the member who is
accessing the content. This tells the CDN which tier to cache the
content against.
For requests to either `/members/?token=...` endpoint (the magic link
endpoint) or `/members/api/member`, Ghost will set a `ghost-access` and
`ghost-access-hmac` cookie with the ID of the tier of the logged in
member. With these two pieces of information, our CDN can serve cached
content to logged in members.
These headers are experimental, and can only be enabled via Ghost's
config. To enable these headers, set `cacheMembersContent:enabled` to
`true` and provide an HMAC key in `cacheMembersContent:hmacSecret`.
DES-192
We often hear that people are not aware of the new features we ship.
Ways in which people can find out are social media/changelog/dashboard –
all of these are easy to miss. We'd like to introduce a template for a
simple notification in the sidebar that can be used any time a new and noteworthy feature has
shipped. The purpose of this is simply to notify and will
disappear forever after it's been dismissed.
ref https://linear.app/tryghost/issue/CFR-29
- Removed the mobiledoc and lexical columns from the posts input
serializer, meaning they will no longer be queried for.
Get helpers are essentially a gateway to the Content API. We already
strip out the mobiledoc and lexical fields in the output
serializer/returned response, but this means we're passing the mobiledoc
and lexical fields back from the db. This is pointless and these fields
are substantial in size - by far the largest fields in the whole ghost
db - leading to slowed performance.
I've updated the posts input serializer to strip out the lexical and mobiledoc
columns so we stop doing a `select *` with every query.
ref ENG-824
- the bug is causing resize prefixes being added to images served from
outside of Ghost.
- this now would only append the prefex to images served by Ghost and
other images urls' would get served as is.
- we can determine that by checking whether imageName doesn't exist,
meaning the source is a third party.
- this mostly affect edge case users, eg where a feature image url was
passed in via the API and doesn't get served by Ghost.
refs CFR-21
Reorganised middleware execution so that member data is not redundantly loaded for static assets or the sitemap.
---------
Co-authored-by: Michael Barrett <mike@ghost.org>
The use of Article and Actor in Activity meant that we got way more data in the
JSONLD representation, but it wasn't be picked up when reconstructing from data
over the wire. This makes sure that we can recreate the object from the JSONLD.
ref https://linear.app/tryghost/issue/MOM-126
Similar to using the Article object for object in Activity, this allows us to
more easily pull out all of the extra data for Actors. At the moment its the
full JSONLD representation, but we can slim that down in future.
ref https://linear.app/tryghost/issue/MOM-128
We want to render more than just the content, so we need to bulk out the
Article objects with metadata like feature images etc...
ref https://linear.app/tryghost/issue/MOM-127
This will save us on network fetches when trying to hydrate the inbox, instead
we can include all of the data we want/need. I had to improve the typing a bit
here to work properly which meant ensuring that we have a `type` property.
ref https://linear.app/tryghost/issue/MOM-126
Now that we're setting the recipient of our Create Activites to the Followers
Collection, we need to actually dereference it and pull out all the inboxes.
This is all done over the network at the moment, but we'll start storing this
information locally when we've got the DB wired up.
ref https://linear.app/tryghost/issue/MOM-126
This is the first step of handling delivery of Activities to our Followers as
we can dereference the Collection and get a list of all our Followers Inboxes
Now that we have the URI object, we don't need to convert the Actor to JSONLD
to get its resource ids. Instead we can have shared getters that expose the ids
as a URI, that can be realized as/when they're needed with the base URL.
This makes the code a little cleaner and more performant.
refs #20197
- adds a jackspeak resolution to Ghost core so we can try and ensure the compatible version of jackspeak/string-width is used when the lockfile is regenerated
ref https://linear.app/tryghost/issue/MOM-108
Apologies to my future self and maintainers if you come across this commit.
This is a bit of a mega commit because we need to cut corners somewhere and it
came down to commit atomicity or tests/code quality.
The main changes here are a bunch of tests, as well as some scaffolding for
Inbox handling of Activities and delivery of Activities. The structure is not
final at all - and we have logic split across services which isn't ideal - but
thsi will do for now as we play around and discover the structure through
building.
With TypeScript, when creating mock instances, it's preferable to maintain a
reference to the params, even if they're not used. This allows us to have
unused variables as long as they're prefixed with an underscore.
ref https://linear.app/tryghost/issue/MOM-73
We've made it easier to use by adding defaults for required header, as well as
adding support for signing POST requests.
ref https://linear.app/tryghost/issue/MOM-73
We need to add body parsing middleware here, so that NestJS has access to it.
We also attach the rawBody which is used to validate the HTTP Signatures
closes https://linear.app/tryghost/issue/MOM-80
- updated internal linking search results items
- removed visibility text from meta data
- added additional icon for paid/specific tier visibility
- added titles to icons
- bumped `@tryghost/koenig-lexical` to include support for meta icon titles
- bumped other Koenig packages due to sub-dependency updates
ref DES-228
This PR updates messaging and error handling in order to make Ghost calmer and friendlier. High level summary of the changes:
- Removed all onBlur validation in Settings -> now it’s possible to just click around without being warned to fill mandatory fields
- Removed lot of technical errors like `ValidationError: Validation (isEmpty) failed for locale`
- Completely removed the red background toast notifications, it was aggressive and raw esp. on the top
- Removed some unnecessary notifications (e.g. when removing a webhook, the removal already communicates the result)
- Now we show field errors on submitting forms, and in case of an error we show a “Retry” button in Settings too. This allowed to remove a lot of unnecessary error messages, like the big error message on the top, plus it’s consistent with the patterns outside Settings.
- Notification style is white now with filled color icons which makes everything much calmer and more refined.
- Removes redundant copy (e.g. "successful(ly)") from notifications
---------
Co-authored-by: Sodbileg Gansukh <sodbileg.gansukh@gmail.com>
ref https://linear.app/tryghost/issue/CFR-27
- updated packages to include performance improvement for NQL filter
strings including multiple neq filters for the same resource
- bumped `bookshelf-plugins`
- bumped NQL versions
We identified a performance fix that allows us to combine not equal
(neq) filters for the same resource in a logically-equivalent way that
also has far more performant resulting SQL.
We're effectively automatically combining strings like
'tag:-tag1+tag:-tag2` into 'tag:-[tag1,tag2]'.
- it appears as though we only accept `err` when it's in the constructor
of the IncorrectUsageError, so in its current form, it is ignored
- this commit performs a minor refactor to switch to constructing a new
IncorrectUsageError and then throwing it
- detected by tsserver complaining about the `err` property not existing
on the error
Data generator uses CSV imports for a massive speed increase, but
can't be used in some environments where SQL admin isn't
available. This allows us to set a flag to use the original
insert-based importer.
fix https://linear.app/tryghost/issue/SLO-104/cannot-read-properties-of-undefined-reading-0-an-unexpected-error
- if the request body didn't contain the correct keys, it'd just HTTP
500 out of there
- this adds some optional chaining so we end up with undefined if
anything isn't as expected, and the following if-statement does the
rest of the check for us
- this also adds a breaking test (the first E2E test for authentication, yay!)
closes https://linear.app/tryghost/issue/MOM-80
- bumps @tryghost/koenig-lexical to add support for search result metadata in internal links as well as some improvements to the internal linking UI/UX
- updates search service to fetch and expose additional `visibility` and `published_at` fields for post/page resources
- updates `searchLinks` method passed to editor to decorate the search results with appropriate meta text and icon based on publish date, post visibility and member settings
closes https://linear.app/tryghost/issue/MOM-106
- the search results can hide any matching authors/tags due to them appearing after matching posts which is typically a longer list that needs scrolling through
- changed the order to list matched authors and tags before posts, this matches the behaviour in our front-end search
refs https://docs.sentry.io/platforms/javascript/configuration/filtering/#using--1%20
- this simplifies our logic to determine whether we should send events
by moving the code to `beforeSend`
- `errorHandler` is going away in Sentry v8 so this results in a shorter
diff in the future
- the logic should be the same, always send non-Ghost errors, and only
send HTTP 500 Ghost errors
- due to the structure of our API controllers, the docName and methods
are under the same structure
- this code loops over the keys of the controller and forms the method
map
- however, it currently also loops over every character of the docName,
so the resulting map contains a weird structure of chars
- we don't need the docName for this, so we can just exclude it from the
keys
- this doesn't change any functionality
fix https://linear.app/tryghost/issue/SLO-101/http-500-with-invalid-multipart-data
- previously, busboy would error out if we supplied a body that was
invalid (such as an empty FormData)
- we would then return a HTTP 500 to the user, which causes all manner
of problems
- now we catch errors from busboy and return a nice BadRequestError
fix https://linear.app/tryghost/issue/SLO-85/fix-http-500-on-contentposts
- in the event we give the incorrect format in a filter, MySQL will
throw an error and we'll throw a HTTP 500 error
- we can capture this error and return a more useful error to the user
- ideally we'd do this in a validation step before attempting the query,
but parsing this out of NQL and detecting which columns are DATETIME
could be quite tricky
- this updates a bunch of places where we're just using Object to cheat
the system
- doing this means editor autocomplete and basic type checking is better
because we now have proper types in place
- functionality should not change, these are just comments
