fix https://linear.app/tryghost/issue/ENG-805/
refs https://owasp.org/www-community/attacks/CSV_Injection
- it's possible for certain fields in a member CSV export to be executed
by software that opens the CSVs
- we can protect against this for the user by escaping any forumulae in
the CSV fields
- papaparse provides this option natively, so it's just a case of
providing the field to the unparse method
- credits to Harvey Spec (phulelouch) for reporting
fixes ENG-611
- Previously, if an existing member with newsletter subscriptions was
imported, and `subscribe_to_emails` was blank/empty, the member would be
unsubscribed from all newsletters, which is not the expected behavior.
- This PR changes the behavior so if `subscribe_to_emails` is blank, it
will not unsubscribe existing members.
refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs TryGhost/Team#2605
-updated unparse to look at both subscribed and subscribed_to_emails
-subscribed is for backwards compatibility
-may want to retire subscribed since we can't set from front-end
refs https://github.com/TryGhost/Team/issues/1076
refs 70229e4fd3 (diff-b67ecda91b5bd79c598e5c5a9ec2ccf28dbfab6a924b21352273865e07cd7ceaR57)
- The "products" column has not been doing any logic anything since at least 5.20.0 (see refed commit). The concept of columns in the export file was mostly there for analytical/data filtering reasons - so the user could analyze their exports. CSV was never a good suite for relational data that "products" (or now tiers) represent
- The "tiers" column will still be present in the exported CSV file, but there is not going to be any logic attached to it.
- The only columns that can effect the "tiers" state of the member are: "complimentary_plan" (assign default tier to the member) and "stripe_customer_id" (pulls in subscription/tier data from Stripe)
refs https://github.com/TryGhost/Team/issues/1076
- The 'should' assertion library is deprecated. Native 'assert' is the recommended lib to use
- Migrating this bit of code allows to remove the should's "utils" folder. Less code, yey!
refs https://github.com/TryGhost/Toolbox/issues/430
- Previously the CSV parser had "map whatever you can and pass on unknown properties further" approach to CSV parsing. This logic has led to unwanted fields leaking through CSV imports - messy, dangerous.
- The strict mapping rules act as a "validator" to the user input, only passing through the fields we expect explicitly - safer clean cut solution with no unintended side-effects.
refs https://github.com/TryGhost/Toolbox/issues/430
refs https://github.com/TryGhost/Ghost/issues/14882
- Having an explicit mappings passed into the members CSV parser makes it easier to control and understand the transforms for package clients
- Eventually the parser will receive a strict map with the fields it should parse - skipping all unknown & unmapped fields
refs https://github.com/TryGhost/Toolbox/issues/430
- Not having any extra logic in the mapper will allow to have a generalized "mapping" concept for CSV input serialization
- This is groundwork for stricter header value filtering on the parsing stage
refs https://github.com/TryGhost/Toolbox/issues/430
- The 'readCSV' method was only exposed to be used in the unit tests. To keep the module code to the minimum moved readCSV to the unit test itself - the only place where it's used and belongs to.
no-issue
We have a special mapping for subscribed_to_emails -> subscribed in the
parse method, but were not mapping it in the unparse method, which meant
we were losing information during CSV imports.
refs https://github.com/TryGhost/Team/issues/765
Support for multiple products means we can no longer map a members state
to a csv row using just the `complimentary_plan` option. Instead we must
include the product(s) that a member has. This ensures that we can read
and write this data from/to csv files.
no-issue
- Was not used by the importer and removed for simplicity.
- Updates the header mapping to happen in place, rather than in a loop
- Updates the parsing of values to give correct types
no issue
- When items are parsed from CSV empty values were interpreted as empty strings - ''. Empty strings are always transformed into 'null' values in Ghost's model layer and are much more problematic to validate comparing to plain `null`. Specifically validation was imossible for 'format: date-time' with JSON schema validation through ajv when the value of date property was an empty string
- This behavior resemples one present in Ghost's model layer - 95880dddeb
- When testing performance overhead for this change did not spot any statistically significant change in performance (tested set was 50K rows)
refs 5c46786ebc
- This is continuation of work removing csv-parser as main CSV handling library with more suitable papaparse library
- Referenced commit introduced papaparse as a library to serialize JSON to CSV, this changeset takes it a step further and replaces CSV to JSON seriazliation logic
no issue
- When processing CSV files `parse` function now allows for the client to specify "mapping" parameter in format of a hash as follows:
{ destination_property_name: 'source_column_name'}
e.g.:
{
name: 'weird_name_column',
email: 'email_column'
}
- It is done so to allow for the end user to provide exact mapping of the fields to be transformed into JSON.
no issue
- This fixes a problem where files are skiped form the @tryghost/members-csv package
- Also this follows the file structure convention with other packages
refs 8185b42d9e5fd9e9051f08ce3395a648ec02f3a4
- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
