refs ARCH-25
- Added a log message if the email_count on an email differs from the
totalCount calculated while creating batches by more than 1%, so we can
investigate further.
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69
When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.
- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address
A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`
Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).
Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI
Fixed settings test that set the site title to an array - bug tracked in
GRO-68
fixes GRO-34
fixes GRO-33
This is a revision of a previous commit, that broke the browser tests
because changes in the data generator (requiring bookshelf had side
effects).
This adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So
they surface tiny bugs in our codebase.
You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e
Removed some defensive logic that could be explained by this discovered
issue.
fixes GRO-34
fixes GRO-33
This also adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So they
surface tiny bugs in our codebase.
You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e
Also removed some defensive logic that could be explained by unquoted ids.
closes https://github.com/TryGhost/Product/issues/4075
- when a member clicks on "Unsubscribe from that list" from Apple Mail,
the member's email is put into Mailgun's Unsubscribe suppression list.
Ghost listens for "Unsubscribe" events from Mailgun, and unsubscribes
the member from all the newsletters
- now, the member is only unsubscribed from the newsletter they
unsubscribe to (not all of them)
- now, the email is also deleted from Mailgun's suppression list, so
that it doesn't affect any other membership
refs https://ghost.slack.com/archives/CTH5NDJMS/p1699359241142969
It's possible for `ObjectIDs` to have only numeric characters. We were
previously letting the type be inferred, which created a very rare but
possible edge case where the last recipient of an email batch had a
numeric ObjectID, resulting in a numeric comparison against alphanumeric
`ObjectIDs` in the database.
- updated the filter to add `'`'s around the `lastId` parameter
- updated tests to check for the type of the id filter parameter value
- can't fully test for numeric object IDs using what we have because
javascript cannot handle numerics of that size; may be able to look at
using fixture data loaded directly into the db
refs https://github.com/TryGhost/Product/issues/4053
This adds the feature flag. If enabled, the list-unsubscribe header
should be set. The value currently is only for testing purposes and
probably won't work yet.
fixes https://github.com/TryGhost/Product/issues/3687
After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.
Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
refs https://github.com/TryGhost/Product/issues/3651
- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
refs TryGhost/Product#3647
- The latest version of juice (which Ghost uses to inline css in email
newsletters) included new functionality to add height="auto" and
width="auto" for any images with dimensions set to auto in css
- This was causing rendering issues in Outlook, which would render the
image at full width, which often added a horizontal scroll and generally
messed up the flow of the document
- This change prevents juice from modifying the height or width of `<img
/>` tags
fixes https://github.com/TryGhost/Team/issues/3541
The email preheader, which is only present in the html version of an
email, is also included in the plaintext version of all emails. This
results in all text being duplicated twice in plaintext emails.
fixes https://github.com/TryGhost/Ghost/issues/16131
Members only content was incorrectly being shown in a plaintext email
due to the email `preheader` using the post model `plaintext` field
directly (which contained the members-only content). This changes this
behaviour so that the post html content is utilised for the `preheader`
but has all members-only content (post-preview content + segmented
content) removed
refs https://github.com/TryGhost/Ghost/issues/15725
This pull request adds a new configuration option for the Mailgun email
provider that allows the user to set the maximum number of recipients
per email batch via a new config option `bulkEmail.batchSize`
refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs 551532f874
refs https://github.com/TryGhost/Team/issues/3324
- After analyzing data dumps, the data revealed that we have extra data from a stray batch. The filtering logic manually filters out the data to the recipients that belong to a "current batch".
- Hunting down the root cause of the data mixup proved to be too expensive of an investigation, so this is a "good enough patch" to deal with the problem.
- Most likely cause is the concurrent batch sending, but reducing the concurrency would be too expensive of a performance price to pay instead of filtering the data rarely.
closes https://github.com/TryGhost/Team/issues/3324
- When the recipients batch size is larger than the limit in addition to logging the error we need extra data to figure out what exactly is inside those `2000` or `3000` records causing faulty behavior.
- This change grabs all available models and dumps them into a file inside of the `content/data` folder. The code is temporary and should be removed once the problem is narrowed down
refs TryGhost/Team#3229
- The issue we are observing that even though the returned amount of email recipients should not ever accede the max batch size (1000 in case of MailGun), there are rare glitches when this number is doubled and we fetch 2000 records instead.
- The fix takes it's best guess in de-duping data in the batch and then truncates it if the amount of records is still above the threshold. This ensures we at least end up sending the emails out to some of the recipients instead of none.
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.
This will help find classes faster, and should push better naming for them too.
Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
refs TryGhost/Team#2974
- currently the unsubscribeFromNewsletters event is failing with 'member
not found' in elastic
- this change catches the error and logs it, which should allow the rest
of the event(s) to be processed
refs https://github.com/TryGhost/Team/issues/2671
The inline style display: none; isn't applied to the images in Outlook for some reason. This change manually removes the images in the backend.
refs https://github.com/TryGhost/Team/issues/2845
Ideally the calculation of these values would be handled by a Newsletter entity
but we don't have one yet, we can look to fix this if we have time. For now
we're calculating them in separate methods to make it easier to extract in
future.
no issue
Bookshelf by default returns an empty model when requesting .related('email') for a post without an email. So we need to be a bit smarter to know if a post has an email or not. This fixed an issue where we always showed 'published and emailed' instead of 'published only'.
Since this change also included some changes to test helpers, it also made some changes to the email service because coverage dropped below 100% as a result of fixing the .related method mocking. Ideally we want to move test test helpers to a seperate package in the future.
Refs https://github.com/TryGhost/Team/issues/2801
- It was not possible to click latest post links in Outlook due to <a>
tag wrapping around a table
- The post meta data wouldn't display properly when centered in Outlook
---------
Co-authored-by: Simon Backx <simon@ghost.org>
refs https://github.com/TryGhost/Team/issues/2805
When we render mobiledoc to HTML, it automatically escapes HTML entities in the process, so a button or directly pasted link with href="https://example.com?code=test" will be rendered as href="https://example.com?code=test" as the url is encoded in the rendered HTML. Our link tracking was using the encoded URL as the redirect URL in newsletters, causing certain links to break.
This change updates the link tracking to decode the URL with `entities.decode(url)` so we store the correct redirect URL in our DB and ensure link tracking redirects to the correct url from newsletters.
---------
Co-authored-by: Rishabh <zrishabhgarg@gmail.com>
refs https://github.com/TryGhost/Team/issues/2674
- The segment detection doesn't work outside the main post content. So the data-gh-segment attribute didn't work. It is now replaced with just a simple email replacement that is empty for a free member.
- Fixed that a trialing member was shown as 'paid'. This is now replaced with 'trialing'.
This commit also includes E2E tests for a couple of member statusses.
refs https://github.com/TryGhost/Team/issues/2736
If the name is not known for a member, we'll hide the name row in the subscription details in an email. This method is supported in most email clients, and requires the support of `<style>` in `<head>`.
fixes https://github.com/TryGhost/Team/issues/2672
- Moves the feature from behind the flag
- Also hides the comment CTA for email only posts
Co-authored-by: Sanne de Vries <sannedv@protonmail.com>
fixes https://github.com/TryGhost/Team/issues/2725
- Added to emails if labs flag enabled, comments enabled and comment CTA button enabled
- Links to comment section
- Design and styling not added yet
fixes https://github.com/TryGhost/Team/issues/2705
- Added showPostTitleSection to newsletter model in admin
- Wired up UI to admin model so it saves to the database
- Implemented showPostTitleSection in newsletter preview and added some
minor temporary css styling
- Implemented showPostTitleSection in newsletter template in backend,
and added some extra CSS styling to fix spacing
