Commit Graph

103 Commits

Author SHA1 Message Date
Steve Larson
dac2561252
🔒 Added uuid verification to member endpoints not requiring a session
ref https://linear.app/tryghost/issue/ENG-1364
ref https://linear.app/tryghost/issue/ENG-1464

- credits to https://github.com/1337Nerd
- added a hashed value to endpoints that do not require a member sign in in order to verify the source of the link and resulting request
- added redirect to sign in page when trying to access newsletter
management
2024-08-20 16:24:02 +02:00
Sag
83b1603202
Cleaned up "List Unsubscribe Header" GA feature flag (#20573)
no issue

- "List Unsubscribe Header" feature was added in Ghost release
[v5.74.0](https://github.com/TryGhost/Ghost/releases/tag/v5.74.0)
(commit: 69ee4a5)
- [Project
details](https://www.notion.so/ghost/One-click-unsubscribe-from-gmail-2b5cdc81e49f462287e9894c9c368aad?pvs=4)
2024-07-10 09:52:13 +00:00
Kevin Ansfield
e6df014f84 Cleaned up newsletterExcerpt flag
no issue

- feature is GA so the flag and related conditionals are no longer required
2024-07-03 18:22:39 +01:00
Peter Zimon
73a88d0c13
Refined newsletter typography (#20406)
DES-459

The font size of subtitles/excerpts in newsletters was similar to the
body font size which doesn't reflect the content hierarchy
appropriately. Also, the spacing should be adjusted to represent that
the title and the subtitle belong together.
2024-06-19 11:57:29 +02:00
Sodbileg Gansukh
9ef92035f6
Adjusted spacing and amount of text of newsletter latest posts (#20285)
ref DES-347

- adjusted title and excerpt length of latest posts in emails
- as the layout is same (horizontal) on both desktop and mobile, truncateHtml() needed some update
- now maxLength is expected to be larger than maxLengthMobile, because the mobile layout isn't stacked anymore
- some spacing adjustment has been made as well
2024-06-11 12:30:32 +08:00
Kevin Ansfield
9ca1f3ce24
Renamed subtitle to excerpt (#20334)
no issue

We've settled on using "excerpt" naming in place of "subtitle" to better reflect the underlying property name and tie in with themes and historical usage.

- added migration to rename the `show_subtitle` newsletter setting to `show_excerpt`
- renamed all places in the codebase that referenced subtitle
2024-06-05 17:59:30 +01:00
Peter Zimon
0efec254ec
Refine subtitle typography in newsletters (#20321)
MOM-190
MOM-192

- The subtitle style (serif/sans) was tied to the body style which was
misleading. It makes more sense to connect it to the title style both
from the UX and the output POV.
- Newsletter design preview was not updated according to subtitle
styles.

---------

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2024-06-04 15:39:14 +02:00
Kevin Ansfield
952a89f673
Renamed show_subhead to show_subtitle (#20320)
closes https://linear.app/tryghost/issue/MOM-194

- whilst working on the feature our naming changed from "Subhead" to "Subtitle"
- this rename of the newsletter design setting column brings naming back into a consistent state before public release
2024-06-04 10:57:14 +00:00
Kevin Ansfield
72a00b5fcc Wired up subtitle in newsletter template
closes https://linear.app/tryghost/issue/MOM-173

- updated email renderer to add `post.customExcerpt` data
- updated template to skip rendering subtitle when no custom excerpt is present
- updated template to use actual custom excerpt
2024-06-03 17:58:53 +01:00
Sanne de Vries
acac7801f4
Split subhead feature into two feature flags (#20279)
REF MOM-119
- Split subhead feature flag into two: editorSubtitle and
newsletterSubtitle
- Updated UI copy, feature flag names and class names from subhead to
subtitle
2024-05-29 17:59:25 +02:00
Sanne de Vries
b0940f1b5a
Added subhead to newsletter template (#20276)
REF MOM-146
- Added subhead to email template behind a flag and and wired it up to
the subhead newsletter setting.
2024-05-29 16:22:42 +02:00
Ronald Langeveld
fb465e4704
Added hasFeature email template helper (#20271)
refs
https://ghost.slack.com/archives/C06TQR9SHSM/p1716816764653789?thread_ts=1716795330.737919&cid=C06TQR9SHSM MOM-158

- adds a new `hasFeature` helper for adding content behind a flag within
Email Templates.
- Usage: `{{#hasFeature 'flagname'}} <p>html behind flag</p>
{{/hasFeature}}`
- Added additional testing for testing the helpers
2024-05-29 11:43:46 +07:00
Sag
5c4a4e812c
Removed Powered by Ghost clicks in publisher analytics (#19926)
fixes https://linear.app/tryghost/issue/TRI-65/add-powered-by-ghost-badge-tracking

- clicks on the "Powered by Ghost" badge were unintentionally surfaced
in publisher analytics, under Newsletter Clicks
2024-03-26 17:51:23 +01:00
Sag
5477d70a0c
Revert "Added referral tracking to the powered-by-ghost newsletter badge" (#19899)
refs https://ghost.slack.com/archives/CTH5NDJMS/p1710976281912809

- this reverts commit 9869d9adb6
- the referral query parameter is unintentionally surfacing in publisher
analytics
2024-03-21 10:02:17 +01:00
Simon Backx
b1c60d20d1
Updated email error button text in case of partial email errors (#19877)
fixes DES-66

In case some batches succeeded sending, the button text will be
different if the email sending was partially successful.

For now this uses text matching with a warning in our E2E tests because
we don't have a straightforward way to check if an error is partial or
not yet.
2024-03-19 10:31:21 +01:00
Sag
9869d9adb6
Added referral tracking to the powered-by-ghost newsletter badge (#19850)
ref https://linear.app/tryghost/issue/TRI-65

In the context of referrals, we want to understand how useful our
“Powered by Ghost” badges are.

There are currently a few places where the “Powered by Ghost” badge can
be rendered:
- in newsletters (enabled/disabled by publisher, on a newsletter basis)
- in Portal popups, e.g. member signup/signin/account settings
- in the footer of some themes, including Source & Casper

We're adding the query param ?via to evaluate the usage of the badge in
newsletters.
2024-03-13 16:03:13 +01:00
Sag
21397a4518
Added [test] prefix to test emails (#19431)
fixes GRO-157

- the [TEST] prefix was removed unintentionally during the rewrite of
the email sending service
2024-01-03 16:08:56 +00:00
Chris Raible
24910a6ef2
Added Sentry logging in BatchSendingService (#19321)
refs ARCH-25

- Added a log message if the email_count on an email differs from the
totalCount calculated while creating batches by more than 1%, so we can
investigate further.
2023-12-12 11:28:43 -08:00
Simon Backx
ab21b8ae1d
Allowed custom/empty replyTo for newsletters with managed sending domain (#19183)
fixes GRO-75
fixes GRO-100

And allow them to be empty
2023-11-30 09:16:03 +00:00
Simon Backx
17ec1e8937
Added email address alignment protections (#19094)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69

When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.

- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address

A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`

Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).

Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI

Fixed settings test that set the site title to an array - bug tracked in
GRO-68
2023-11-23 10:25:30 +01:00
Simon Backx
b6519e0f1f
Removed usage of unquoted ids in filter strings (#19070)
fixes GRO-34
fixes GRO-33

This is a revision of a previous commit, that broke the browser tests
because changes in the data generator (requiring bookshelf had side
effects).

This adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So
they surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Removed some defensive logic that could be explained by this discovered
issue.
2023-11-21 09:45:36 +01:00
Simon Backx
d5492bd63c
Revert "Removed usage of unquoted ids in filter strings" (#19052)
Reverts TryGhost/Ghost#19031

Browser tests are failing with an unknown cause
2023-11-20 13:50:07 +00:00
Simon Backx
320eaac4c4
Removed usage of unquoted ids in filter strings (#19031)
fixes GRO-34
fixes GRO-33

This also adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So they
surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Also removed some defensive logic that could be explained by unquoted ids.
2023-11-20 14:00:31 +01:00
Sag
6db7cc8156
Removed email from Mailgun's suppression list on unsubscribe (#18922)
closes https://github.com/TryGhost/Product/issues/4075

- when a member clicks on "Unsubscribe from that list" from Apple Mail,
the member's email is put into Mailgun's Unsubscribe suppression list.
Ghost listens for "Unsubscribe" events from Mailgun, and unsubscribes
the member from all the newsletters
- now, the member is only unsubscribed from the newsletter they
unsubscribe to (not all of them)
- now, the email is also deleted from Mailgun's suppression list, so
that it doesn't affect any other membership
2023-11-13 16:56:37 -03:00
Simon Backx
14927ee24b
Added quotes to NQL filters with ids (#18958)
refs https://github.com/TryGhost/Product/issues/4120

Updated some places where we don't add quotes around ids in NQL filters,
which can be an issue when the id is a number
2023-11-13 12:00:20 +01:00
Steve Larson
342b5512fc
🐛 Fix edge case resulting in duplicate emails for some recipients (#18941)
refs https://ghost.slack.com/archives/CTH5NDJMS/p1699359241142969

It's possible for `ObjectIDs` to have only numeric characters. We were
previously letting the type be inferred, which created a very rare but
possible edge case where the last recipient of an email batch had a
numeric ObjectID, resulting in a numeric comparison against alphanumeric
`ObjectIDs` in the database.
- updated the filter to add `'`'s around the `lastId` parameter
- updated tests to check for the type of the id filter parameter value
- can't fully test for numeric object IDs using what we have because
javascript cannot handle numerics of that size; may be able to look at
using fixture data loaded directly into the db
2023-11-10 01:24:56 +00:00
Simon Backx
6cc19e1851
Added List-Unsubscribe https endpoint (#18758)
refs TryGhost/Product#4052
2023-10-25 16:16:31 +02:00
Simon Backx
c8f71e8504
Added list-unsubscribe feature flag and header (#18736)
refs https://github.com/TryGhost/Product/issues/4053

This adds the feature flag. If enabled, the list-unsubscribe header
should be set. The value currently is only for testing purposes and
probably won't work yet.
2023-10-24 10:35:47 +00:00
Simon Backx
4c8179312d
🎨 Added support for relative links in emails (#17630)
fixes https://github.com/TryGhost/Product/issues/3687

After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.

Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
2023-08-08 13:22:56 +02:00
Ronald Langeveld
0029c444ad
Added test email rate limiting (#17505)
refs https://github.com/TryGhost/Product/issues/3651

- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
2023-07-27 08:46:50 +02:00
Ghost CI
fc50d1e92c Merged v5.55.1 into main 2023-07-25 02:46:04 +00:00
Chris Raible
e50ad7561c
🐛 Fixed image rendering in Outlook email client (#17475)
refs TryGhost/Product#3647

- The latest version of juice (which Ghost uses to inline css in email
newsletters) included new functionality to add height="auto" and
width="auto" for any images with dimensions set to auto in css
- This was causing rendering issues in Outlook, which would render the
image at full width, which often added a horizontal scroll and generally
messed up the flow of the document
- This change prevents juice from modifying the height or width of `<img
/>` tags
2023-07-24 18:33:56 -07:00
Michael Barrett
184c6ae951
Retain newsletter subscriptions on suppression (#17373)
refs https://github.com/TryGhost/Product/issues/2610
2023-07-24 10:47:57 +01:00
Steve Larson
22441fe730
Made lexical rendering async (#17438)
-moved lexical rendering to async
-includes rendering for front end and email
-necessary to pull dynamic data into render method, e.g. collections
2023-07-20 17:48:48 -05:00
hemri
afaf4965db
Fixed timezone configuration in package tests (#17087) 2023-07-20 07:46:27 +00:00
Daniel Lockyer
8a4ccff942 Updated html-validate to v8
refs https://gitlab.com/html-validate/html-validate/blob/HEAD/CHANGELOG.md#800-2023-06-04

- includes one small change as per their breaking changes list
2023-07-11 10:17:18 +02:00
Simon Backx
e6dbc0bc4c
🐛 Fixed repeating text in plaintext version of emails (#17162)
fixes https://github.com/TryGhost/Team/issues/3541

The email preheader, which is only present in the html version of an
email, is also included in the plaintext version of all emails. This
results in all text being duplicated twice in plaintext emails.
2023-06-29 12:47:17 +02:00
Michael Barrett
576fba0568
🐛 Fixed members-only content incorrectly showing in plaintext email (#17137)
fixes https://github.com/TryGhost/Ghost/issues/16131

Members only content was incorrectly being shown in a plaintext email
due to the email `preheader` using the post model `plaintext` field
directly (which contained the members-only content). This changes this
behaviour so that the post html content is utilised for the `preheader`
but has all members-only content (post-preview content + segmented
content) removed
2023-06-29 09:40:04 +01:00
Kuba
9601285c3d
Added bulkEmail.batchSize option to configure batch size
refs https://github.com/TryGhost/Ghost/issues/15725

This pull request adds a new configuration option for the Mailgun email
provider that allows the user to set the maximum number of recipients
per email batch via a new config option `bulkEmail.batchSize`
2023-06-26 12:57:44 +02:00
Hannah Wolfe
6161f94910
Updated to use assert/strict everywhere (#17047)
refs: https://github.com/TryGhost/Toolbox/issues/595

We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.

This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing,  there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a

Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
2023-06-21 09:56:59 +01:00
Naz
c507ea9600 Improved oversized batch correction logic
refs 551532f874
refs https://github.com/TryGhost/Team/issues/3324

- After analyzing data dumps, the data revealed that we have extra data from a stray batch. The filtering logic manually filters out the data to the recipients that belong to a "current batch".
- Hunting down the root cause of the data mixup proved to be too expensive of an investigation, so this is a "good enough patch" to deal with the problem.
- Most likely cause is the concurrent batch sending, but reducing the concurrency would be too expensive of a performance price to pay instead of filtering the data rarely.
2023-06-09 14:40:12 +07:00
Naz
efac36c36c
Removed data dump during email batching
refs 551532f874
refs https://github.com/TryGhost/Team/issues/3324

- Having a data dumped in production was a one-off trick to allow us debug the error. Now that we have data it's no longer needed.
2023-06-09 10:44:13 +07:00
Naz
1605198a1a
Added debug data dump to email service
closes https://github.com/TryGhost/Team/issues/3324

- When the recipients batch size is larger than the limit in addition to logging the error we need extra data to figure out what exactly is inside those `2000` or `3000` records causing faulty behavior.
- This change grabs all available models and dumps them into a file inside of the `content/data` folder. The code is temporary and should be removed once the problem is narrowed down
2023-05-30 21:32:01 +07:00
naz
320c659a1e
Added forced debug output for EmailRecipients fetching (#16823)
refs TryGhost/Team#3229

- The issue we are observing that even though the returned amount of email recipients should not ever accede the max batch size (1000 in case of MailGun), there are rare glitches when this number is doubled and we fetch 2000 records instead.
- The fix takes it's best guess in de-duping data in the batch and then truncates it if the amount of records is still above the threshold. This ensures we at least end up sending the emails out to some of the recipients instead of none.
2023-05-19 17:57:24 +07:00
Fabien "egg" O'Carroll
104f84f252 Added eslint rule for file naming convention
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.

This will help find classes faster, and should push better naming for them too.

Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
2023-05-09 12:34:34 -04:00
Chris Raible
0b0e3f8e85
Added error handling for email analytics unsubscribe event (#16613)
refs TryGhost/Team#2974

- currently the unsubscribeFromNewsletters event is failing with 'member
not found' in elastic
- this change catches the error and logs it, which should allow the rest
of the event(s) to be processed
2023-04-11 13:13:34 -07:00
Simon Backx
ca00a3d682 Fixed duplicate images in Outlook for dark/light mode
refs https://github.com/TryGhost/Team/issues/2671

The inline style display: none; isn't applied to the images in Outlook for some reason. This change manually removes the images in the backend.
2023-04-05 12:58:11 +02:00
Simon Backx
33237c4df7 Added newsletter auto border color and dynamic color picker
fixes https://github.com/TryGhost/Team/issues/2879
fixes https://github.com/TryGhost/Team/issues/2880

- Replaced black border color with 'auto' based on background color.
- When a color is 'auto', the color that are visible in the UI (color pciker) will be dynamic based on the background color.
2023-04-03 11:27:57 +02:00
Fabien "egg" O'Carroll
f23b869499 Added newsletter design customisation data to template
refs https://github.com/TryGhost/Team/issues/2845

Ideally the calculation of these values would be handled by a Newsletter entity
but we don't have one yet, we can look to fix this if we have time. For now
we're calculating them in separate methods to make it easier to extract in
future.
2023-03-30 12:21:58 +07:00
Simon Backx
109cdeb492 Fixed exporting post metrics without email
no issue

Bookshelf by default returns an empty model when requesting .related('email') for a post without an email. So we need to be a bit smarter to know if a post has an email or not. This fixed an issue where we always showed 'published and emailed' instead of 'published only'.

Since this change also included some changes to test helpers, it also made some changes to the email service because coverage dropped below 100% as a result of fixing the .related method mocking. Ideally we want to move test test helpers to a seperate package in the future.
2023-03-28 12:41:32 +02:00