Package scriptella.driver.csv

CSV Driver for Scriptella.

See:
Description

Class Summary

CsvConnection	Represents a connection to CSV file.
CsvQuery	Query for CSV file.
Driver	Scriptella driver for CSV files.

Exception Summary

CsvProviderException

Thrown to indicate a problem with CSV file processing.

Package scriptella.driver.csv Description

CSV Driver for Scriptella.

The driver to write and query CSV files.

Note: The driver doesn't use SQL syntax

General information

Driver class:	scriptella.driver.csv.Driver
URL:	CSV file URL. URIs are resolved relative to a script file directory. If url has no value the output is read from/printed to the console (System.out).
Runtime dependencies:	None

Driver Specific Properties

Name	Description	Required
encoding	Specifies charset encoding of CSV files.	No, the system default encoding is used.
separator	The delimiter to use for separating entries when reading from or writing to files.	No, the default value is ,.
quote	The character to use for quoted elements when reading from or writing to files. Use empty string to suppress quoting.	No, the default value is ".
headers	Value of true means the first line contains headers. Only valid for <query> elements.	No, the default value is true(first line contains column names).
eol	End-Of-Line suffix. Only valid for <script> elements.	No, the default value is \n.
trim	Value of true specifies that the leading and trailing whitespaces in CSV fields should be omitted.	No, the default value is true.
skip_lines	The number of lines to skip before start reading. Please note that if headers=true, the actual number of skipped lines is skipped_lines+1	No, the default value is 0 (no lines are skipped).
null_string	Specifies value of a string token to be parsed as Java null literal.	No, by default strings are preserved, i.e. empty CSV field is parsed as empty string.

Query Syntax

The driver utilizes query by example approach for CSV content filtering. If you want to read the whole CSV simply use empty query element, otherwise specify a set of lines containing comma-separated case insensitive regular expressions.
Example:

,food
,^Beverages$
    

This query selects rows where the second column equals "Beverages" or contains "food" substring.

The columns of the matched row can be referenced by name in nested queries/scripts. It is also possible to reference columns by an index, i.e. $1, $2, ...

Script Syntax

The script syntax is simple, just specify a comma-separated set of columns.
Example:

<script>
id,priority,summary,status
1,Critical,NullPointerException in Main class,Open
</script>
    

This script writes 2 lines to the output file.

IMPORTANT: Always use commas as a column separator inside both <script> and <query> elements no matter what separator is used in files being parsed or produced. This decision allows switching between different formats like tab or semicolumn while keeping scripts and queries unchanged.

Properties substitution

In CSV script and query elements use ${property} syntax for properties/variables substition.

NOTE:

By default NULL variables are not substituted, because Scriptella substitution engine cannot distinguish null value from undeclared variable which is not always desired.

Use ${etl.text.ifNull(variable)} or ${etl.text.ifNull(variable, '')} to output NULL as empty string.

Examples

<connection id="in" driver="csv" url="data.csv">

</connection>
<connection id="out" driver="csv" url="report.csv">
    #Use empty quote to turn off quoting
    quote=
    separator=;
</connection>

<script connection-id="out">
    ID,Priority,Summary,Status
</script>

<query connection-id="in">
    <!--Empty query means select all-->
    <script connection-id="out">
        $rownum,$priority,$summary,$status
    </script>
</query>

Copies rows from data.csv file to report.csv, additionally the ID column is added. The result file will be semicolon separated.