no issue
- Nock doesn't support multiple calls to enableNetConnect -> only the last one counts. This fixes that issue.
- Some tests interacted directly with nock instead of using the mockManager to restore everything.
https://github.com/TryGhost/Toolbox/issues/523
- During import process of content files the files from the root directory were also copied over. This is causing chaos in the root of content folder with files that only needed for data import. For example, the csv files needed for Revue import were also copied over by "file importer" even though those do not belong to any content.
- Any content import files - images, media, files, should be in according folders in the imported zip file. The root files in the base zip directory are for data-related imports
fixes https://github.com/TryGhost/Team/issues/2611
The old email flow is no longer used since we introduced the email stability flow. This commit removes the related code and tests. The general test coverage decreased a bit as a result, because the old email flow probably had a high test coverage. The new flow is in separate packages, so it couldn't contribute to a higher test coverage (but it does have 100% unit test coverage).
fixes https://github.com/TryGhost/Team/issues/2683
When sending a newsletter with a replacement that has a fallback, the
replacement only happens in the HTML version of the newsletter. The
plaintext version isn't replaced.
This commit fixes the issue and adds some tests to make sure it doesn't
happen again.
The cause of the issue was that we used the original matched Regex text
to replace. But that was calculated on the HTML version, so double
quotes were encoded. This change updates the generated 'token' regex to
also match on both a double quote as the escaped double quote.
refs https://github.com/TryGhost/Team/issues/2667
Some tests still accessed the internet. Now network access is disabled
by default. This change also introduces two helper methods related to
networking (mocking Slack and Mailgun).
This fixes two unreliable tests:
- Staff service was accessing a Slack test API -> timeout possible
- MentionSendingService was trying to send webmentions for every post
publish/change -> possible timeouts and job issues
refs 6460522352
- these were both bumped around the time that Playwright browser tests
started going awry, so they may be the cause of some random failures
refs https://github.com/TryGhost/Toolbox/issues/524
- We need to be able to inline external media in all internal resources: tags, users, post's meta fields.
- This change adds media inlining logic to all these resources
- these models should be as lightweight as possible to require, so we
need to measure how long they take
- this has already found a couple of models which need optimizing
refs https://github.com/TryGhost/Toolbox/issues/523
- The test is useful for future iterations of the response format and as a quick reference on which parameters the media inlining endpoint accepts.
refs https://github.com/TryGhost/Toolbox/issues/523
- Media inlining is a long running piece of logic that suits perfectly for a job. It is not as critical to be offloaded into a separate thread at this point, so leaving it as "inline" while the feature is in experimental stage.
refs https://github.com/TryGhost/Toolbox/issues/523
- This is a first pass media inliner going through all posts and checking to inline media from specified domains
- As a working copy the inliner looks for image content from Revue and Substack
refs: https://github.com/TryGhost/Toolbox/issues/389
Instead of logging errors, this will warn when adding a duplicate URL in the test environment.
At the moment, this is happening a lot in the test suite. While we also need to fix the root cause of this so we're not erroring in the product, it's a massive amount of spam in the logs when running the test suite which could prevent us from finding other errors which are causing issues.
refs: https://github.com/TryGhost/Toolbox/issues/389
This removes many error logs when the end-to-end test suite is run with the log-level set to error. Many errors are intentional, so the resolution is typically to stub the error log function and assert that it would have been called.
refs: https://github.com/TryGhost/Toolbox/issues/389
The newsletter fixtures no longer errors when accessing the test image, and tests which intentionally error now stub the logging call
refs https://github.com/TryGhost/Toolbox/issues/523
- When importing a zip with Revue data and no "files" or "media" the importer blew up. This is due to a bug where the files from the root directory are still being copied over (the should not be!)
- Added protective extra ".data" check to make sure code doest not blow up when there's no data property to be processed
refs https://github.com/TryGhost/Toolbox/issues/524
- This is groundwork to build up on when adding media inlining functionality. This service will be used by the API to trigger post's content processing and media inlining.
refs https://github.com/TryGhost/Toolbox/issues/524
- This flag will be used to test the trigger to external media inliner job
- The Admin UI is still unclear so sticking it behind the flag for some experimentation before shipping the feature
refs https://github.com/TryGhost/Toolbox/issues/523
- When migrating or importing ZIP files into Ghost there's often a need to include document files.
- When document files are present in the imported zip file they are now copied across and processed along with the rest of import files: json, images, csvs, etc.
- The importer also searches for use of the document files in the imported "posts" substituting the links with local ones
- The document files importer recognizes media files inside of "files" or "content/files" folders present in the zip.
- The supported media file extensions are same as for file upload widget:
".pdf",".json",".jsonld",".odp",".ods",".odt",".ppt",".pptx",".rtf",".txt",".xls",".xlsx",".xml"
with following content-types:
"application/pdf", "application/json", "application/ld+json", "application/vnd.oasis.opendocument.presentation", "application/vnd.oasis.opendocument.spreadsheet", "application/vnd.oasis.opendocument.text", "application/vnd.ms-powerpoint", "application/vnd.openxmlformats-officedocument.presentationml.presentation", "application/rtf", "text/plain", "application/vnd.ms-excel", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "application/xml", "application/atom+xml"
refs https://github.com/TryGhost/Toolbox/issues/523
refs c2534e3c86/packages/mg-assetscraper/lib/AssetScraper.js (L14-L16)
refs https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types
- Importer needs to process and recognize document files like pdfs, presentations etc to be able to import them into sites file storage.
- The handler allows a new root directory "files" to place imported documents
- The handler adds validation and processing for following file extensions:
".pdf",
".json",
".jsonld",
".odp",
".ods",
".odt",
".ppt",
".pptx",
".rtf",
".txt",
".xls",
".xlsx",
".xml"
- With following content types:
"application/pdf",
"application/json",
"application/ld+json",
"application/vnd.oasis.opendocument.presentation",
"application/vnd.oasis.opendocument.spreadsheet",
"application/vnd.oasis.opendocument.text",
"application/vnd.ms-powerpoint",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/rtf",
"text/plain",
"application/vnd.ms-excel",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/xml",
"application/atom+xml"
refs https://github.com/TryGhost/Toolbox/issues/523
- To support generic "file" import handling the content file handler needs more configuration options for properties like type, extensions, content path etc. This refactor makes the handler configurable and reusable for any type of file import
refs https://github.com/TryGhost/Toolbox/issues/523
- We need to process generic files like .pdf, .md, etc. on the importer "handler" stage.
- The media handler can process more than just media files after few refactorings. Renaming it to a generic content file handler indicates it can process any type of content file.
- In future we can substitute the built-in "images" import handler with this generic content file handler.
refs https://github.com/TryGhost/Toolbox/issues/523
- When migrating or importing ZIP files into Ghost there's often a need to include more than just post content and images.
- When media files are present in the imported zip file they are now copied across and processed along with the rest of import files: json, images, csvs, etc.
- The importer also searches for use of the media files in the imported "posts" substituting the links with local ones
- The media importer recognizes media files inside of "media" or "content/media" folders present in the zip. The supported media file extensions are same as for media upload widget:
".mp4", ".webm", ".ogv", ".mp3", ".wav", ".ogg", ".m4a"
with following content-types:
"video/mp4", "video/webm", "video/ogg", "audio/mpeg", "audio/vnd.wav", "audio/wave", "audio/wav", "audio/x-wav", "audio/ogg", "audio/mp4", "audio/x-m4a"
refs https://github.com/TryGhost/Toolbox/issues/523
- This is ground work before introducing a "media" content type importer
- Previous "image" file name was not describing well what the importer was capable of doing
