Commit Graph

541 Commits

Author SHA1 Message Date
Fabien 'egg' O'Carroll
d94859f2e5
Added /stats/subscriptions API (#14547)
refs https://github.com/TryGhost/Team/issues/1505
refs https://github.com/TryGhost/Team/issues/1466

Exposes an API for historical counts broken down by tier and cadence.

Counts backwards from the current stats like MRR to minimize inaccruate
data due to missing/superfluous events.
2022-04-27 14:53:32 +01:00
Kevin Ansfield
a8687b35b9 Added newsletter from address verification (#14491)
refs https://github.com/TryGhost/Team/issues/1498
refs https://github.com/TryGhost/Team/issues/584

- Added newsletter `from` address verification

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-04-26 12:31:34 +01:00
Naz
cc71bbfd61 Hooked up api version compatibility middleware
refs https://github.com/TryGhost/Toolbox/issues/280

- ctd of putting pieces together to allow Ghost notifying owner and admin users about version mismatch errors
-  The `@tryghost/mw-api-version-mismatch` in a combination with api version compatibility service make the whole notification process play nicely :)
- The flow of the logic from the request to a sent notification email is following:
1. Request comes is with an Accept-Version header that's behind current Ghost version and is not supported
2. mw-error-handler middleware's 'resourceNotFound' detects such request and returns a 406 with a special 'code' identifying if the version of the client is ahead or behind
3. mw-api-version-mismatch intercepts the 406 request with "code === 'UPDATE_CLIENT'` and calls up APIVersionCompatibilityService
4. emails are sent out to active owner and admin users

- The above flow is also illustratd in the e2e tests that come with the changeset
2022-04-22 16:31:56 +08:00
Thibaut Patel
52a855f314
Added the newsletter read endpoint (#14520)
refs https://github.com/TryGhost/Team/issues/1533

- Retrieves one newsletter
- Makes the newsletter resource consistent with the other resources
- Solves an issue with the admin expecting the route to exist
2022-04-21 09:59:44 +02:00
Hannah Wolfe
1d121c52f4
Replaced white/black list terminology
refs 92986b77e3

- I thought we did this a while ago, but uses in comments and elsewhere in the codebase were missed
2022-04-19 11:19:59 +01:00
Simon Backx
132726fe20
Added MRR stats service and endpoint (#14427)
refs https://github.com/TryGhost/Team/issues/1470

Instead of counting the MRR by resolving all the deltas from the past until now, we should start with the current calculated MRR and resolve it until the first event. That would give a more accurate recent MRR (in exchange for a less accurate MRR for older data) and allows us to limit the amount of returned days in the future.

- Includes MRR stats service that can fetch the current MRR per currency
- The service can return a history of the MRR for every day and currency
- New admin API endpoint /stats/mrr that returns the MRR history
- Includes tests for these new service and endpoint
2022-04-08 09:18:04 +02:00
Simon Backx
759e362a30 Fixed snakeCase in stats member count endpoint
refs 31c1d4f513

Still had the camel case 🙈
2022-03-31 17:09:28 +02:00
Simon Backx
31c1d4f513 Renamed stats endpoint to /stats/member-count and added totals metadata
refs https://ghost.slack.com/archives/C02G9E68C/p1648737467414789?thread_ts=1648644801.253699&cid=C02G9E68C

- Anything in the API should use snakeCase
- Reduce amount of nesting in endpoint name
- Added totals metadata
2022-03-31 17:05:08 +02:00
Simon Backx
ae54352a29
Created new stats service and stats API to get member count history (#14391)
refs TryGhost/Team#1458
refs TryGhost/Team#1459
refs TryGhost/Team#1372

- Added a new stats service, which is divided into several categories. Currently only the 'members' category for member related stats.
- When there are missing or corrupt members status events in the DB, the totals returned by the old member stats endpoint (`/members/stats/count`) were wrong. This is fixed in the new service by counting in reverse order and starting with the actual totals.
- New Stats API, with the new `/stats/members/count-history` endpoint.
- This new endpoint also returns the paid deltas -> dashboard 5.0 will show subscribed and canceled paid members for each day
- Includes tests for the new stats service and endpoint
2022-03-31 16:01:11 +02:00
Thibaut Patel
9e4401d9f6 Added the newsletters endpoint
refs https://github.com/TryGhost/Team/issues/1463

- This enables listing, creating and editing newsletters
- The tests are commented out as the permissions will be added in a follow-up commit
2022-03-31 15:19:58 +02:00
Hannah Wolfe
4def34ef21
Removed unused members/stats endpoints
refs: https://github.com/TryGhost/Team/issues/1446

- These endpoints are unused, so they are safe to remove
- We're starting to remove as much unused & unnecessary code as possible to try to reduce the codebase and increase test coverage
2022-03-29 14:00:04 +01:00
Hannah Wolfe
470ed05620
Removed unnecessary email_preview.read serializer
refs: https://github.com/TryGhost/Toolbox/issues/245
refs: https://github.com/TryGhost/Ghost/commit/73f91a524

- we don't need this serializer because the default serializer will do the same thing
- commit 73f91a524 fixes the logic so that the default serializer is called as a fallback
  even though the email_preview serialzier exists, as there's no matching method name
- sadly the route name here is wrong, it should be email_previews plural, but the response format is correct
  to make this work we have to fix the docName and rename the serializer
2022-03-29 13:44:51 +01:00
Sam Lord
de3a45a805
Fixed theme error handler (#14363)
no issue

Prevents errors from being uploaded to Sentry when a 404 happens in Ghost Admin. At the moment, 404s in Ghost Admin create an ENOENT error in express' static library. Our generic 404 handler at the end will only intercept requests that don't have any errors in the context, so a simple middleware can strip out 404 errors just before we add in our own.

The Ghost-specific error that we attach to requests does not get uploaded to Sentry :)
2022-03-22 15:32:55 +00:00
Naz
25e5cb46de 🐛 Fixed /canary/ API endpoints 404s
refs https://github.com/TryGhost/Toolbox/issues/169
refs 7becf0a2b2

- The referenced commit has dropped existance of Content and Admin APIs under `/canary/` prefix, which made a breaking change and clients that are still relying on "canary" started to break.
- The `/canary/` prefix should be used up untill the introduction of Ghost v5, otherwise we run the risk of accidentally breaking API clients.
2022-03-21 19:11:13 +08:00
Rishabh Garg
f9aa18a534
Enabled Admin integration for tiers and offers API (#14325)
As multiple tiers is now GA, we want to allow devs to be able to work with Tiers and offers via content/Admin API. This change -

- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on tiers and offers API for integrations
2022-03-16 16:46:26 +05:30
Naz
7becf0a2b2 Aliased canary endpoints to point to non-versioned URLs
refs https://github.com/TryGhost/Toolbox/issues/169

- Before releasing Ghost v5 we would like to move all canary-related URLs to a non-versioned format, which will become a default in v5.
- 'canary' is by definition unstable, so breaking any unprepared client explicitly using the canary is expected
- Removed the aliased /content/ and /admin/ apps from app.js because with updated configuration they become duplicates of 'canary' endpoints
2022-03-14 21:22:54 +13:00
Naz
9c64d7af81 Centralized base API path value across server codebase
refs https://github.com/TryGhost/Team/issues/1420

- This changeset makes the "/ghost/api" base path for the APIs centralized in one place and reused by dependent modules. There are couple benefits this refactor brings: easy way to spot where the API base path is used (was hard to find it in regexp) and makes it easy to change the hardcoded path to a configurable one in the future (e.g. host all APIs under `domain.tld/custom-path/awesome-apis/posts`)
- I hear that scream from the back of your head: "But hey! This introduced coupling to url-utils!". To that my unswer is: "No. This change only makes the coupling explicit, it's been there already and now can be addressed if we need to!".
- A neat thing about his change, making the API work on a custom path is one line away, by moving the hardcoded `/ghost/api` to a config ;)
2022-03-09 17:15:51 +08:00
Naz
da9f018c70 🐛 Fixed theme activation with capitalized names
closes https://github.com/TryGhost/Team/issues/1420
refs da0dee548c
refs https://github.com/TryGhost/Toolbox/issues/169

- After introducing non-versioned API urls the "isAPI" regex failed to pass the test as it was expecting a `canary/vX` in the API URL. This caused "uncapitalization" to stop working for API requests.
- Regex visualizer for quick reference: https://jex.im/regulex/#!flags=&re=%5E(.*%5C%2Fghost%5C%2Fapi(%5C%2F(v%5B%5Cd.%5D%2B%7Ccanary))%3F%5C%2F.*%3F%5C%2F)
2022-03-09 20:07:04 +13:00
Matt Hanley
01a6d576b3 Removed public cache-control header for member site endpoint
- Caching is causing issues with Portal preview
- Changes to tiers and Portal settings are taking too long to display on the site, causing confusion for users
- This reverts commit 5d8f491823.
2022-03-03 15:25:49 +00:00
Fabien "egg" O'Carroll
1b96ce2794 Added /tiers endpoint to Content API
refs https://github.com/TryGhost/Team/issues/1313

When adding the tiers endpoint the Content API was missed, this is
needed so that themes can access Tiers via the `{{#get}}` helper.
2022-03-03 15:15:43 +02:00
Naz
da0dee548c Added alias for non-versioned API endpoints
refs https://github.com/TryGhost/Toolbox/issues/169

- Before next major version release we need to prepare for removal of API versioning.
- This change allows unversioned API requests to work under following endpoints:
- /ghost/api/admin
- /ghost/api/content
- This change should allow further preparation of the API clients (SDKs, Integrations, etc.) to non-versioned APIs in Ghost instances in ^5.0.0
- Changed default e2e test targets to non-versioned API. It's a trial, to have working examples. In the future all tests should switch to use only non-versioned endpoints.
2022-03-03 20:21:22 +13:00
Naz
1cc38733ba Added Media and Files APIs to API key allowlist
refs https://github.com/TryGhost/Toolbox/issues/219

- These two APIs similarly to Images API should be accessible by the Admin-API SDK. Opens up a way to write custom scripts uploading files and media
2022-03-02 16:33:56 +07:00
Fabien 'egg' O'Carroll
694721cbea
Added /tiers API to Admin API (#14200)
refs https://github.com/TryGhost/Team/issues/1313

Rather than removing the /products API we're adding a /tiers API as
a first step towards renaming "products" to "tiers". The initial idea was
to alias the URL's but out API framework doesn't easily allow for this so
we've duplicated it instead.
2022-02-23 17:00:18 +02:00
Fabien 'egg' O'Carroll
a565da06b2
🐛 Fixed Offer Redemptions being over counted (#13988)
refs https://github.com/TryGhost/Team/issues/1257

Offer Redemptions were being overcounted due to the way we were updating
Stripe configuration for the Members service. We would create a new
instance of the members-api, which would have event handlers for
creating Offer Redemptions - by creating a new instance each time Stripe
config changed, we would overcount them.

Here we've pulled out Stripe related logic into the Stripe service, and
updated it internally - rather than creating a new instance. This means
that we've been able to remove all of the logic for re-instantiating the
members-api.

- Bumped members-api & stripe-service
- Removed reinstantiation of members-api
- Used stripe service to execute migrations
- Updated Stripe Service to handle webhooks & migrations
- Used webhook controller from stripe service
- Used disconnect method from stripe service
- Removed unused stripe dependency
- Removed Stripe webhook config from members-api
2022-01-18 17:56:47 +02:00
Fabien "egg" O'Carroll
5d8f491823 Updated caching policy for /members/api/site
refs https://github.com/TryGhost/Team/issues/1240

As this endpoint is hit on every page load when Portal is enabled, it
can cause a lot of traffic to Ghost. The data does not change very
frequently so we've added a 30s cache policy to alleviate load on servers.
2022-01-14 14:04:23 +02:00
Fabien "egg" O'Carroll
76b33b6e77 Added JSDoc to cache control middleware
no-issue
2022-01-14 14:04:23 +02:00
Kevin Ansfield
9b210f9add Fixed linter error
refs https://github.com/TryGhost/Team/issues/1258

- all uses of `labs` service in canary routes have been removed but the require had been left in place
2022-01-04 15:26:17 +00:00
Kevin Ansfield
0de2c7e261 Removed filesAPI GA labs flag
refs https://github.com/TryGhost/Team/issues/1258

- files API is GA so labs flag and conditional access to files API routes is no longer necessary
2022-01-04 14:56:07 +00:00
Kevin Ansfield
2773e82e3e Removed mediaAPI GA labs flag
refs https://github.com/TryGhost/Team/issues/1258

- media API is GA so labs flag and conditional access to media API routes is no longer necessary
2022-01-04 13:11:23 +00:00
Kevin Ansfield
cabf78e938 Cleaned up customThemeSettings labs flag
closes https://github.com/TryGhost/Team/issues/1164

- `customThemeSettings` feature is GA so any conditionals can be cleaned up
- removed conditional loading of custom theme settings and associated API routes
- removed event trigger for reloading custom theme settings when the feature flag is toggled
- removed flag from labs GA list
2022-01-03 17:45:25 +00:00
Sam Lord
97c68dd388
Replace error handler middleware with @tryghost/mw-error-handler (#13879)
refs: https://github.com/TryGhost/Toolbox/issues/137

Extract error handling middleware and replace with a package.
2021-12-14 15:18:46 +00:00
Naz
b19424acb3 Moved vhost mounts into boot file
refs https://github.com/TryGhost/Toolbox/issues/152

- This stops the mounting of the admin and frontend from being buried deep in express initialisation
- Instead it's explicit, which makes two things almost possible:
   1. we can potentially boot the frontend or backend independently
   2. we can pass services and settings loaded during boot into the frontend
- This needs more work, but we can start to group all the frontend code together
- Meanwhile we also need to rip apart the routing and url services to decouple the frontend from the backend fully
- BABY STEPS!

Co-authored-by: Hannah Wolfe <erisds@gmail.com>
2021-12-06 21:28:53 +13:00
Hannah Wolfe
004371be36
Revert "Removed overly-specific handling of handlebars errors"
This reverts commit 303ea87897.

- Although gscan catches these now, we have a number of sites that have slipped through the net
- Reverting until we get them all cleaned up
2021-12-01 12:04:36 +00:00
Sam Lord
2887e416da
Switch to @tryghost/errors from ignition errors package (#13807)
refs: TryGhost/Toolbox#147

* Replaces all references to isIgnitionError with isGhostError
* Switches use of GhostError to InternalServerError - as GhostError is no longer public
There are places where InternalServerError is not the valid error, and new errors should be added to the @tryghost/errors package to ensure that we can use semantically correct errors in those cases.
2021-12-01 10:22:01 +00:00
Hannah Wolfe
ad9eb35746
Moved theme-specific error handling to frontend
- our themeErrorRenderer is only used in the frontend.. move it there
- this required exposing prepareError as shared middleware
- TODO: move these shared compontents to @tryghost/error
2021-11-29 17:04:01 +00:00
Hannah Wolfe
d7c4168452
Removed HTMLErrorRenderer completely
refs: 0799f02e80
refs: 5e931e2e37

- with the referenced two commits I replaced our old HTML renderer with some code borrowed heavily from finalHandler
- I had intended to modify this further to out put our message, context and help error messages
- However, I ended up doing this in prepareError so it's done for all error renderers
- There's now very little point keeping duplicated code from finalHandler just to output the status code
- If we remove this code, express will fall back to finalHandler anyway, so the output is near identical
2021-11-29 17:00:47 +00:00
Hannah Wolfe
3b069b544f
Cleaned code patterns in error handler
- got rid of old _private & variable pattern in favour of const and module.exports
- changed weird capitalisation naming conventions to be camelCase
- removed some very old TODOs that we're never gonna get TODONE
   - these are mostly old ideas that never made it, and it's been so long they're clearly not important
2021-11-29 16:53:58 +00:00
Hannah Wolfe
8c93bdff41
Removed unnecessary BasicErrorRenderer
refs: 4474ca1a1d
refs: 0799f02e80

The BasicErrorRenderer was created as a fallback for when we needed to not render templates, which is
chiefly when we're trying to render a 404 for an image. Using a template puts us at risk of an infinite 404 loop
if the missing image is referenced in the 404 template.

As of 0799f02e, the HTMLErrorRenderer no longer uses templates - instead we serve a very simple HTML page.
This can be used instead of the BasicErrorRenderer, as it results in a properly formatted error.
Even when sending responses in plain text, the content type is returned as HTML and therefore having an
unformatted error makes no sense - if we really need a non-html format I guess there should be no body at all.
2021-11-29 15:40:55 +00:00
Hannah Wolfe
303ea87897
Removed overly-specific handling of handlebars errors
refs: https://github.com/TryGhost/Ghost/issues/10496
refs: https://github.com/handlebars-lang/handlebars.js/issues/1548

- For a while, handlebars would and could generate quite a lot of obscure sytnax error messages for malformed helpers that we couldn't catch in gscan
- This was fixed in https://github.com/handlebars-lang/handlebars.js/issues/1548
- We have also since added AST parsing to gscan, so if there are more obscure syntax errors, we should be able to add rules to catch them
- Therefore, this additional, overly specific code block isn't needed anymore, so let's clean it up!
2021-11-25 14:53:36 +00:00
Hannah Wolfe
621cfd9866
Moved admin redirect middleware to named file
- moving this middleware because we're about to add a second piece of middleware
- it's easier to see what we have when each middleware is in its own file rather than in one big middleware.js file
2021-11-25 14:29:17 +00:00
Hannah Wolfe
5e931e2e37
Added err.context & err.help to stack traces
- When we handle errors in Ghost, we are supposed to use a pattern of supplying 3 messages:
   - message: what went wrong
   - context: details about why how or where the error happened
   - help: where the user can go to get help with this error
- We do this in many places and our JSON error handler and CLI error logging tools are designed to output this extra information
- However, stack traces, which start with message as the first line and then output the stack are totally missing this
- By injecting the additional messages into the stack once an error has been "ghostified" we should get clearer messages everywhere

Notes:
- I've additionally injected a "Stack Trace:" line that makes it easier to read the error vs the stack
- This code looks a little weird because the lines are inserted backwards, but that allows us to always to the insert at position 1 as per the comment,
   so we don't have to keep track of whether we already injected something or not
2021-11-24 13:01:06 +00:00
Hannah Wolfe
0799f02e80
Rewritten HTMLErrorRenderer w/o asset helper & template
refs: 2af9e2e12

- This new HTMLErrorRenderer is borrowed heavily from finalHandler
   - This is the module that express uses to render errors if there is no custom errorhandler
- It just renders a really simple html page wrapping err.stack in a <pre>
   - This results in a nicely formatted, but unstyled error page
- I also updated BasicErrorRenderer to use the same res.statusCode + err.stack pattern rather than err.message

Note: This error renderer is _only_ used for renderering errors on the `/ghost/` route
 - In almost all cases, errors here are rendered by Ember
 - The only error that can be rendered here is a missing template error see: 2af9e2e12
2021-11-24 13:01:06 +00:00
Hannah Wolfe
2af9e2e125
Fixed error handling for missing admin templates
- If the admin templates default.html or default-prod.html are missing, don't throw a 500
- Instead throw a well considered 400 error with extra help for what to do to fix it
2021-11-24 12:30:41 +00:00
Hannah Wolfe
0c4c9ce553
Moved JSON error handlers next to each other
- helps for comparing and contrasting
- context is largely lost on why we have these two, need to figure it out and if possible, simplify
2021-11-24 11:42:53 +00:00
Hannah Wolfe
f417c4c732
Merged our two maintenance middleware into one
- Reduced our maintenance middleware code down to the bare minimum!
  - We have an old maintenance middleware in place to handle when a site is forcibly put into maintenance mode, or the urlService hasn't finished booting
    - This maintenance middleware was mounted on every sub app, instead of globally for reasons I no longer remember
  - Recently, we introduced a new, static version of maintenence middleware to show during the boot process so we can get the server started earlier & not drop requests
    - This version has its own HTML template and doesn't depend on any of Ghost's error rendering code
  - To simplify and help with decoupling, this commit merges the two middleware, so that the new independent & static middleware renders its template for any one of the 3 possible maintenance modes
    - It only needs to exist in the top level app 🙌

TODO: move the maintenance middleware to its own file/package so it's not part of the app.js as that is weird
2021-11-24 11:27:18 +00:00
Naz
bbc93ff996 Renamed withBackend/withFrotned to backend/frontend
refs https://github.com/TryGhost/Toolbox/issues/135

- Shorter name still makes sense and the "with" might be causing confusion understanding what's gonig on during the boot time
2021-11-22 21:56:32 +13:00
Naz
149c100b4c Fixed frontend initialization in "mock-express" tests
refs https://github.com/TryGhost/Toolbox/issues/135

- Without sensible defaults the web app was not initializing either the backend nor the frontned parts of the application. Fixed the defaults so the problem doesn't happen again and optimized mock-express-style initialization to only initialize the frontend routing
2021-11-22 21:56:32 +13:00
Naz
c12ae81ece Introduced "withFrontend" and "withBackend" flags in boot
refs https://github.com/TryGhost/Toolbox/issues/135

- These flags are meant to control initialization of sections of the boot sequence depending on the needs - with or without bakend (API)/frontend (public handlebars site)
- Ideally these flags should not be passed deep into the components, and if the are (like in the web/parent/app case) it's a smell that we need to move things up into the boot process!
2021-11-22 21:56:32 +13:00
Sam Lord
0692ddf57d Revert "Replace update user last seen middleware with package version"
This reverts commit 07f8c6efbb.

Revert "Remove unused unit test file"

This reverts commit f1798119d3.
2021-11-17 11:00:55 +00:00
Hannah Wolfe
332beaaf90
Moved "vhost-utils" to config helpers
- These are simple functions that get data from config in a specific format
- They are also used by the topmost part of the application
- Config helpers seems like a reasonable fit to get them out of the web folder
- Functions have also been renamed to try to get them to make more sense
2021-11-17 08:37:08 +00:00