Background:
- Ghosts existing boot process is split across multiple files, has affordances for outdated ways of running Ghost and is generally non-linear making it nigh-impossible to follow
- The web of dependencies that are loaded on boot are also impossible to unpick, which makes it really hard to decouple Ghost
- With 4.0 we want to introduce a new, linear, simpler, clearer way to boot up Ghost to unlock decoupling Ghost into much smaller pieces
This commit:
- adds a new ghost.js file which switches between boot mode with `node index` or `node index old` so that if we find bugs we can work around them this week
- Note: the old boot process will go away very soon, but ghost.js will remain as the interface between the command to start Ghost and the application code
- reworks the database migration process into a standalone utility, so that the DB is handled as one simple step of the boot process, decoupled from everything else
- is missing tests for this new db utility
- leaves a lot of work to do around loading core code, services, express apps in a sensible order, as work to fix this would start to break the old boot process
- doesn't use the new maintenance app because we aren't restarting the server here, instead we have the concept of a "core app" that starts in maintenance mode - need to think about how apps will be decoupled in the near future
- Represents that logging is shared across all parts of Ghost at present
* moved core/server/lib/common/logging to core/shared/logging
* updated logging path for generic imports
* updated migration and schema imports of logging
* updated tests and index logging import
* 🔥 removed logging from common module
* fixed tests
- Added a wrapper around express.Router to our shared/express util
- Also export static and _express
- Use this shared util everywhre, meaning express is only used directly in this one file
- ATM this file is mostly an experiment / debug helper, it might be removed again later
- The aim is to have a minimal framework wrapping express that allows us to:
- reduce our usage of express() in favour of Router()
- unify some of our duplicated logic
- fix some structural issues e.g. Sentry
- make it easier to understand the codebase
- Use array destructuring
- Use @tryghost/errors
- Part of the big move towards decoupling, this gives visibility on what's being used where
- Biting off manageable chunks / fixing bits of code I'm refactoring for other reasons
- added core/shared to watched folders in grunt
- moved sentry to shared
- moved express initialisation to a shared file
- always set trust proxy + sentry error handler
- use this new express init everywhere, and remove duplicate trust proxy and sentry error handler code
- renamed the parentApp in index.js to ghostApp, to reduce confusion with the layer that is named parentApp
- renamed the adminApp inside of parentApp to backendApp to reflect the fact it's both admin+api
- renamed a bunch more variables there to be backend, rather than admin
- renamed the api index.js file to app.js and created a new index which is an actual index
- updated various files I noticed were outdated on my travels around the codebase
- doesn't make any more advanced ES6 changes, this is mostly in the persuit of getting rid of var x = y, z = a; lists at the top of files
This reverts commit 6e024331eb.
Temporarily reverting whilst we investigate an issue with Sentry and running Ghost via Ghost-CLI.
Ghost-CLI initiated boot was failing when Sentry was installed due to what appears to be `process.cwd()` returning `undefined` here https://github.com/TryGhost/Ignition/blob/master/lib/config/index.js#L26
refs #10790
- Moved /core/apps into core/frontend
- Moved /core/server/helpers to /core/frontend/helpers along with /core/server/services/themes
- Changed helper location in overrides
- Moved /core/server/services/routing to /core/frontend/services
- Moved /core/server/services/url to /core/frontend/services
- Moved /core/server/data/meta to /core/frontend/meta
- Moved /core/server/services/rss to /core/frontend/services
- Moved /core/server/data/xml to /core/frontend/services
closes#9802
- we have to trigger both functions within Ghost core, otherwise people who are using Ghost as NPM module have to call these functions
- this is internal logic
- plus: this logic is conditional, because of our internal maintenance flag
- make it backwards compatible in case you call announceServerStart or announceServerStopped twice
- tested with "Ghost as NPM module" and with the CLI on production
no issue
- IPC communication happens before we trigger process.exit
- this is a timing issue
- we should give IPC a little time to finish sending the message over to the CLI
- wait 100ms before exiting the process
no issue
- this is a fix when starting Ghost and it cannot start because of a native error
- it could happen that a third library returns a native error, which will then be wrapped into an Ignition error
- we should tell Ignition to inherit from the error message, otherwise the error message is for example
not directly visible in the CLI
no issue
- if we trigger the IPC message to the CLI and the process manager is systemd, systemd will restart Ghost too early (same for local process manager) - this is a timing issue
- the consequence is that the error log won't happen in Ghost (`content/logs/[domain].error.log` won't contain that error)
- let's reorder both executions
[See](https://github.com/TryGhost/Ghost-CLI/pull/612/files#r165817172) this comment on the CLI PR.
refs #9178
- we have to take care that we don't end up in circular dependencies
- e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
no issue
- Upgraded ghost-ignition
- Use debug from ghost-ignition everywhere in the code base
- Remove debug dependency
- Fixed random typo in Gruntfile.js
refs #2182
* ⏱ Add boot timer - improve visibility of boot time
I've been playing around with Ghost start times a lot recently.
Every time I do, I add a console.time output for boot, which is annoying.
This commit adds that change permanently. We can always revert later before shipping 1.0 😁
* ⏱ Add debug call before main requires
- this demonstrates that the majority of boot time is spent on requires
- had to rejig the var pattern because of the linter... 💩
* 🐷💄 Special debug mode for config
- I ❤️ being able to output the config, but this is not useful when trying to debug / optimise timings.
- This change makes it so we can see how long it takes to do config work by default
- If we want to output config specifically, we do `DEBUG=ghost:*,ghost-config npm start`
- This also prevents nconf.get() from being called unnecessarily
refs #7489
- as we are now using a different migration approach (knex-migrator), we don't need to remember the database version anymore
- it was once used to check the state of a database and based on it we decided to migrate or not
- with knex-migrator everything depends on the migration table entries and the current ghost version you are on
- on current master the leftover usage is to add the db version when exporting the database, which can be replaced by reading the ghost version
- removing this solves also an interesting migration case with knex-migrator:
- you are on 1.0
- you update to 1.1, but 1.1 has no migrations
- the db version would remain in 1.0
- because the db version was only updated when knex migrator executed a migration
refs #7116, refs #2001
- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.
Summary of changes:
* 🐛 set NODE_ENV in config handler
* ✨ add GhostError implementation (core/server/errors.js)
- register all errors in one file
- inheritance from GhostError
- option pattern
* 🔥 remove all error files
* ✨ wrap all errors into GhostError in case of HTTP
* 🎨 adaptions
- option pattern for errors
- use GhostError when needed
* 🎨 revert debug deletion and add TODO for error id's
- 🛠 add bunyan and prettyjson, remove morgan
- ✨ add logging module
- GhostLogger class that handles setup of bunyan
- PrettyStream for stdout
- ✨ config for logging
- @TODO: testing level fatal?
- ✨ log each request via GhostLogger (express middleware)
- @TODO: add errors to output
- 🔥 remove errors.updateActiveTheme
- we can read the value from config
- 🔥 remove 15 helper functions in core/server/errors/index.js
- all these functions get replaced by modules:
1. logging
2. error middleware handling for html/json
3. error creation (which will be part of PR #7477)
- ✨ add express error handler for html/json
- one true error handler for express responses
- contains still some TODO's, but they are not high priority for first implementation/integration
- this middleware only takes responsibility of either rendering html responses or return json error responses
- 🎨 use new express error handler in middleware/index
- 404 and 500 handling
- 🎨 return error instead of error message in permissions/index.js
- the rule for error handling should be: if you call a unit, this unit should return a custom Ghost error
- 🎨 wrap serve static module
- rule: if you call a module/unit, you should always wrap this error
- it's always the same rule
- so the caller never has to worry about what comes back
- it's always a clear error instance
- in this case: we return our notfounderror if serve static does not find the resource
- this avoid having checks everywhere
- 🎨 replace usages of errors/index.js functions and adapt tests
- use logging.error, logging.warn
- make tests green
- remove some usages of logging and throwing api errors -> because when a request is involved, logging happens automatically
- 🐛 return errorDetails to Ghost-Admin
- errorDetails is used for Theme error handling
- 🎨 use 500er error for theme is missing error in theme-handler
- 🎨 extend file rotation to 1w
refs #2001, #7116
- added debug and wired it up:
- across several key parts of the boot process
- throughout the middleware loading
- for requests
- at render points for key routes
issues #6406#6399
- all dates are stored as UTC with this commit
- use moment.tz.setDefault('UTC')
- add migration file to recalculate local datetimes to UTC
- store all dates in same format into our three supported databases
- add option to remeber migrations inside settings (core)
- support DST offset for migration
- ensure we force UTC in test env
- run whole migration as transaction
- extend: Settings.findOne function
- Updated all dev dependencies in the root package.json
- Added `--colors` to mocha when called from grunt test:... so that works better
- Removed non-unit tests from coverage, because they're just not useful
- Updated docker config to generate useful docs again - as a statement of intent, I also updated docs in key files
- Setup grunt watch-docs task for documentation writing easyness
Refs #3864
- Make sure that require() is able to resolve all dependencies
listed in package.json. If packages are missing halt the
bootstrap process and display an error and help message for user.
- Check that contentPath and its subdirectories exist with the correct
permissions.
- Check sqlite3 database file is set for read/write access.
refs #827
- Moves ./index to use Ghost in a similar manner to how someone uses
Ghost as an npm module.
- Allows Ghost to be cleanly mounted on another express application
on any arbitrary endpoint, all you need to customize is the mount path.
closes#3789
- Create a GhostServer class to manage state
- index.js now calls start on the exported server
- Alter tests to expect a GhostServer instance
hooking into when ghost has finished loading
addresses item 9 in #2078
and makes progress on #2182
- has files that startup ghost return a promise
that is resolved once ghost has finished loading
- moves getSocket into config file
- removes models.reset() as it's not used anywhere
- update functions in server startup
- remove unused version hash variable
Extracts all express-server-related code in index.js to core/server.js, leaving index.js purely for booting up Ghost's core components in a sensible order.
Aside from the project's tidiness, this means that we can perform asynchronous configuration loading/checks before requiring any modules that read the config.
