Documents

Read/Write data with the good document format

CSV

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`csv`	`csv`
metadata	meta	Metadata describe the resource	`none`	Metadata
is_flexible	-	If flexible is true, the application try to match the number of header's fields and the number of line's fields	`true`	`true` / `false`
quote_style	-	The quoting style to use when writing CSV	`NOT_NUMERIC`	`NOT_NUMERIC` / `ALWAYS` / `NEVER` / `NECESSARY`
trim	-	Define where you trim the data. The application can trimmed fields, headers or both	`ALL`	`ALL` / `FIELDS` / `HEADERS`

examples:

[
    {
        "type": "write",
        "document": {
            "type": "csv",
            "is_flexible": true,
            "quote_style": "NOT_NUMERIC",
            "trim": "ALL",
            "metadata": {
                "has_headers": true,
                "delimiter": ",",
                "quote": "\"",
                "escape": "\\",
                "comment": "#",
                "terminator": "\n"
            }
        }
    }
]

input/output:

"column1","column2",...
"value1","value2",...
...

Json

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`json`	`json`
metadata	meta	Metadata describe the resource	`none`	Metadata
is_pretty	-	Display json data readable for human	`false`	`false` / `true`
entry_path	-	Use this field if you want target a specific field in the json object	`none`	String in json pointer format

examples:

[
    {
        "type": "read",
        "document": {
            "type": "json",
            "entry_path": "/0"
        }
    },
    {
        "type": "write",
        "document": {
            "type": "json",
            "is_pretty": true
        }
    }
]

input:

[
    {"field1":"value1"},
    {"field1":"value2"},
    ...
]

output:

[
    {
        "field1":"value1"
    }
]

Jsonl

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`jsonl`	`jsonl`
metadata	meta	Metadata describe the resource	`none`	Metadata
is_pretty	-	Display data in readable format for human	`false`	`false` / `true`
entry_path	-	Use this field if you want target a specific field in the object	`none`	String in json pointer format

examples:

[
    {
        "type": "read",
        "document": {
            "type": "jsonl",
            "entry_path": "/field1"
        }
    },
    {
        "type": "write",
        "document": {
            "type": "jsonl",
            "is_pretty": true
        }
    }
]

input:

[
    { "field1":"value1", ... },
    { "field1":"value2", ... },
    ...
]

output:

value1
value2
...

Text

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`text`	`text`
metadata	meta	Metadata describe the resource	`none`	Metadata

examples:

[
    {
        "type": "read",
        "document": {
            "type": "text"
        },
        "connector": {
            "type": "mem",
            "data": "Hello world !!!"
        }
    }
]

input/output:

Hello world !!!

Toml

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`toml`	`toml`
metadata	meta	Metadata describe the resource	`none`	Metadata

examples:

[
    {
        "type": "read",
        "document": {
            "type": "toml"
        }
    }
]

input/output:

[[line]]
field= value
...

Xml

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`xml`	`xml`
metadata	meta	Metadata describe the resource	`none`	Metadata
is_pretty	-	Display data in readable format for human	`false`	`false` / `true`
indent_char	-	Character to use for indentation in pretty mode	`space`	Simple character
indent_size	-	Number of indentation to use for each line in pretty mode	`4`	unsigned number
entry_path	-	Use this field if you want to target a specific field in the object	`/root/*/item`	String in json pointer format

examples:

[
    {
        "type": "read",
        "document": {
            "type": "xml",
            "is_pretty": true,
            "indet_char": " ",
            "indent_size": 4,
            "entry_path": "/root/*/item"
        }
    },
    {
        "type": "w"
    }
]

input:

<root>
    <item field1="value1"/>
    ...
</root>

output:

[{"field1":"value1"},...]

Yaml

key	alias	Description	Default Value	Possible Values
type	-	Required in order to use this document	`yaml`	`yaml`
metadata	meta	Metadata describe the resource	`none`	Metadata

examples:

[
    {
        "type": "read",
        "document": {
            "type": "yaml"
        }
    }
]

input/output:

---
field: value
...

Metadata

By default, the metadata is manage with the document metadata but you can override it if necessary

key	Description	Default Value	Possible Values	Example
has_headers	Define if the document contain a header entry	`none`	`true` \ `false`	Read/Write CSV
delimiter	Special character to seperate fields	`none`	String	Read/Write CSV
quote	Special character used for quote	`none`	String	Read/Write CSV
escape	Special character used for escape	`none`	String	Read/Write CSV
comment	Special character used for comment	`none`	String	Read/Write CSV
terminator	Special character used to define new line	`none`	String	Read/Write CSV
mime_type	Mime type of the document	`none`	`application` / etc...	Read/Write CSV
mime_subtype	Mime sub type of the document	`none`	`plain` / `octet-stream` / etc...	Read/Write CSV
charset	Charset of the document	`none`	`utf-8` / `ISO-8859-1` / etc...	Read/Write CSV
compression	Compression used to compress the document	`none`	String	`gzip`
language	The language link to the document	`none`	String	See language