ref https://linear.app/tryghost/issue/ONC-154
- the query params did not carry through on portal sign up links because
of the hash creating an ignored fragment
(/#/portal/signup?ref=something)
Now when we check link attribution, we'll attempt to run the same logic
for the referrer source after stripping out `#/portal` from the URL.
Otherwise we should continue to treat these fragments as fragments to be
ignored by the client.
NOTE: We do not have e2e tests that cover member signup on the front end
and the data entered in the back end. The tests we have mock only the
server side of things. The test added here only covers the data that is
generated from the front end request (at this time), *not* the front end
request itself, meaning it's fragile.
ref
https://linear.app/tryghost/issue/ENG-1240/race-condition-when-updating-members-last-seen-at-timestamp
When members click a link in an email, Ghost updates the member's
`last_seen_at` timestamp, but it should only update the timestamp if the
member hasn't yet been seen in the current day (based on the
publication's timezone).
Currently there is a race condition present where multiple simultaneous
requests from the same member (if e.g. an email link checker is
following all links in an email) can cause the `last_seen_at` timestamp
to be updated multiple times in the same day for the same member. These
additional queries add a significant load on Ghost and its database,
which can contribute to the exhaustion of the connection pool and
eventually requests may time out.
The primary motivation for this change is to avoid that race condition
by adding a lock to the member row, checking if `last_seen_at` has
already been updated in the current day, and only updating it if it
hasn't.
Another beneficial side-effect of this change is that it avoids locking
the `labels` and `newsletters` tables, which are locked when we update
the `last_seen_at` timestamp in the `members` table currently. This
should improve Ghost's ability to handle a large influx of requests to
redirect endpoints (confirmed with load tests), which tend to happen
immediately after a publisher sends an email.
ref https://linear.app/tryghost/issue/ENG-721
- when changing the response to a `204` for requests with no cookie we'd lost the `Cache-Control: public, max-age: 0` header which meant some cache systems weren't caching as efficiently as possible
closes https://linear.app/tryghost/issue/ENG-721
ref https://linear.app/tryghost/issue/ENG-708
Comments-UI loads `/ghost/admin-frame/` in an iframe to check if a Staff User is authenticated in order to show moderation options. That iframe request loads a HTML page which in turn contains a script that fires off an API request that attempts to fetch the logged-in user details, resulting in a 403 "error" showing up when not authenticated. In the vast majority of cases there will be no staff user authenticated so lots of extra requests and "errors" are seen unnecessarily.
- adjusted the `/ghost/auth-frame/` endpoint to check if the request contains an Admin session cookie
- if it does, continue as before with rendering the HTML page so the script is loaded
- if it doesn't, return an empty 204 response avoiding the script request and subsequent 403-generating API request
- eliminates the 403 error being generated for all typical visitor traffic, the error should only be seen when an Admin was previously logged in but their cookie is no longer valid (either from logging out, or going past the 6month validity period)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69
When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.
- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address
A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`
Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).
Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI
Fixed settings test that set the site title to an array - bug tracked in
GRO-68
fixes GRO-34
fixes GRO-33
This is a revision of a previous commit, that broke the browser tests
because changes in the data generator (requiring bookshelf had side
effects).
This adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So
they surface tiny bugs in our codebase.
You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e
Removed some defensive logic that could be explained by this discovered
issue.
fixes GRO-34
fixes GRO-33
This also adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So they
surface tiny bugs in our codebase.
You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e
Also removed some defensive logic that could be explained by unquoted ids.
refs https://github.com/TryGhost/Product/issues/4095
- Removes `min-height` of the card in order to make the template more
flexible half-empty states look better (missing description, short
description, featured image...)
- Fixes wrong variables used in Outlook-specific template
fixes https://github.com/TryGhost/Product/issues/4005
We no longer use the 'reason' of a recommendation, but allow a flexible
description instead. Because this is a breaking change in the API, we do
this before making this feature GA.
- Added new database utils for renaming a column
- Added new migration to rename the column
- Updated all references in code
no issue
- Removed redundant Admin X flags from the email service since we are preparing for release.
- updated snapshots
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at fb0df07</samp>
Simplified staff URL generation for emails by removing unused feature
flag and code. This affects the `StaffServiceEmails.js` file and the
`adminXSettings` flag.
refs https://github.com/TryGhost/Product/issues/3957
This changes how we fetch recommendations:
- Relations can be included in one query instead of extra queries
- Sorting is now possible by click or subscriber counts
refs https://github.com/TryGhost/Product/issues/3938
- Updated "Recommendation received" email design
- Slightly improved how dark favicons show in dark mode
- Added TabView with counter variant to Storybook
refs https://github.com/TryGhost/Product/issues/3958
- Disabled automatic network retries for external site lookups (=> timed
out to 5s in every situation because it returned 404 when a site doesn't
implement the Ghost api)
- Disabled representing a modal when it is already present on hash
changes
- Added support for search params in modals
- Handle `?url` search param in the addRecommendationModal
refs https://ghost.slack.com/archives/C02G9E68C/p1695296293667689
- We block all outgoing networking by default in tests. When editing a
post/page, Ghost tries to send a webmention. Because of my earlier
changes
(1e3232cf82)
it tries 3 times - all network requests fail instantly when networking
is disabled - but it adds a delay in between those retries. So my change
disables retries during tests.
- Adds a warning when afterEach hook tooks longer than 2s due to
awaiting jobs/events
- We reduce the amount of webmentions that are send, by only sending
webmentions for added or removed urls, no longer when a post html is
changed (unless post is published/removed).
refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs https://github.com/TryGhost/Toolbox/issues/592
- heads up, I'm not really sure about this fix
- when we're wrapping `setTimeout`, time stops and mysql2 starts doing
weird things because we then shift time and it hits timeouts
- apparently `shouldAdvanceTime` should fix this by automatically
incrementing time along with the system clock
- given the problem is quite difficult to hit, I could just be seeing a
lack of this due to some other factor
- also removed unnecessary sinon sandbox creation as this is superfluous
refs https://github.com/TryGhost/Team/issues/3234
Added an e2e for the update check script to detect potential breakages
in the script due to uninitialised dependencies in the isolated
execution environment
no issue
This change waits for domain events and jobs before continuing with the
next test. This prevents issues where background tasks in tests are
executed when the next test is running and the configurations have
changed, causing random error logs and test failures.
It also includes a change in Stripe mocking in one E2E test to make use
of the new StripeMocker instead of custom mocking in each test (also to
reduce error logs).
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA✨ Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4
- When milestones will be activated we would send out emails to users
that are way above the achieved milestone, as we didn't record
milestones before
- The plan is to implement a 0 milestone and don't send an email for
achieving those and also add all achieved milestones in the first run
until a first milestone is stored in the DB, then increment from there.
- This change takes care of two cases:
1. Milestones gets enabled and runs initially. We don't want to send
emails unless there's already at least one milestone achieved. For that
we add a 0 milestone helper and add a `initial` reason to the meta
object for the milestone event, so we can choose not to ping Slack and
also disable email sending for all milestones achieved in this initial
run.
2. All achieved milestones will be stored in the DB, even when that
means we skip some. This introduces the `skipped` reason which also
doesn't send emails for the skipped milestones, but will do for
correctly achieved milestones (always the highest one).
- Added handling for slack notifications to not attempt sending when
reason is `skipped` or `initial`
- this cleans up all imports or variables that aren't currently being used
- this really helps keep the tests clean by only allowing what is needed
- I've left `should` as an exemption for now because we need to clean up
how it is used
no issue
- Nock doesn't support multiple calls to enableNetConnect -> only the last one counts. This fixes that issue.
- Some tests interacted directly with nock instead of using the mockManager to restore everything.
fixes https://github.com/TryGhost/Team/issues/2611
The old email flow is no longer used since we introduced the email stability flow. This commit removes the related code and tests. The general test coverage decreased a bit as a result, because the old email flow probably had a high test coverage. The new flow is in separate packages, so it couldn't contribute to a higher test coverage (but it does have 100% unit test coverage).
refs https://github.com/TryGhost/Team/issues/2667
Some tests still accessed the internet. Now network access is disabled
by default. This change also introduces two helper methods related to
networking (mocking Slack and Mailgun).
This fixes two unreliable tests:
- Staff service was accessing a Slack test API -> timeout possible
- MentionSendingService was trying to send webmentions for every post
publish/change -> possible timeouts and job issues
refs: https://github.com/TryGhost/Toolbox/issues/389
This removes many error logs when the end-to-end test suite is run with the log-level set to error. Many errors are intentional, so the resolution is typically to stub the error log function and assert that it would have been called.
no issue
- Switches to used newly added config values throughout the services
- Updated the `shouldSendEmail` fn to check if actual value is too far from achieved milestone as determined by the percentage setting (e. g. 998 members should not accidentally receive an email for achieving 100 members)
no issue
- Instead of running milestone service directly on boot, set a random
timeout of 0-4 days to run after boot
- Updated tests
- Service is still behind a beta flag
[Added initial mentions-jobs
service](3656190114)
This is the result of running `cp -r jobs mentions-jobs` in the services
directory.
[Waited for mentions-jobs queue before
shutdown](2bb1a12a89)
This matches the functionality of the existing jobs service where we
will wait
for jobs to complete before closing the process.
[Used mentions-jobs service in the mentions
service](4e4f9fdd00)
This ensures that any delays in the mentions jobs queue does not effect
other
parts of the application.
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4
- Added a `slack-notifications` repository which handles sending Slack
messages to a URL as defined in our Ghost(Pro) config (also includes a
global switch to disable the feature if needed) and listens to
`MilestoneCreatedEvents`.
- Added a `slack-notification` service which listens to the events on
boot.
- In order to have access to further information such as the reason why
a Milestone email hasn't been sent, or the current ARR or Member value
as comparison to the achieved milestone, I added a `meta` object to the
`MilestoneCreatedEvent` which then gets accessible by the event
subscriber. This avoid doing further requests to the DB as we need to
have this information in relation to the event occurred.
---------
Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
