Commit Graph

7 Commits

Author SHA1 Message Date
Fabien O'Carroll
0a5f91f3d0 Refactored members-csv to remove unused columnsToExtract
no-issue

 - Was not used by the importer and removed for simplicity.
 - Updates the header mapping to happen in place, rather than in a loop
 - Updates the parsing of values to give correct types
2020-12-04 10:10:59 +00:00
Nazar Gargol
b8c1aeee35 Added empty string '' to null transform when parsing CSVs
no issue

- When items are parsed from CSV empty values were interpreted as empty strings - ''. Empty strings are always transformed into 'null' values in Ghost's model layer and are much more problematic to validate comparing to plain `null`. Specifically validation was imossible for 'format: date-time' with JSON schema validation through ajv when the value of date property was an empty string
- This behavior resemples one present in Ghost's model layer  - 95880dddeb
- When testing performance overhead for this change did not spot any statistically significant change in performance (tested set was 50K rows)
2020-08-17 17:57:49 +12:00
Nazar Gargol
3a97d39d72 Substituted csv-parser with papaparse as CSV parsing package
refs 5c46786ebc

- This is continuation of work removing csv-parser as main CSV handling library with more suitable papaparse library
- Referenced commit introduced papaparse as a library to serialize JSON to CSV, this changeset takes it a step further and replaces CSV to JSON seriazliation logic
2020-07-08 19:39:37 +12:00
Nazar Gargol
fd982fef73 Added mapping support to CSV parser
no issue

- When processing CSV files `parse` function now allows for the client to specify "mapping" parameter in format of a hash as follows:
{ destination_property_name: 'source_column_name'}
e.g.:
{
  name: 'weird_name_column',
  email: 'email_column'
}
- It is done so to allow for the end user to provide exact mapping of the fields to be transformed into JSON.
2020-07-02 14:53:56 +12:00
Nazar Gargol
93ab8ae946 Moved modules into lib folder
no issue

- This fixes a problem where files are skiped form the @tryghost/members-csv package
- Also this follows the file structure convention with other packages
2020-06-19 19:49:39 +12:00
Nazar Gargol
5c47d725cb Fixed tests and added fixtures from Ghost repo 2020-06-19 19:33:43 +12:00
Nazar Gargol
5c46786ebc Colocated readCSV util with members input serializer
refs 8185b42d9e5fd9e9051f08ce3395a648ec02f3a4

- Allows to move files and keep the history in one go.
- 'csv-parser' will be upgraded to 'papaparse' lib in a new package
2020-06-19 19:33:43 +12:00