Filters

Use filters in conjunction with the FilterStep.

DateTimeThresholdFilter

This filter is useful if you want to do incremental imports. Specify a threshold DateTime instance, a column name (defaults to updated_at), and a DateTimeValueConverter that will be used to convert values read from the filtered items. The item strictly older than the threshold will be discarded.

<?php

use Port\Filter\DateTimeThresholdFilter;
use Port\ValueConverter\DateTimeValueConverter;

new DateTimeThresholdFilter(
    new DateTimeValueConverter(),
    new \DateTime('yesterday')
);

OffsetFilter

OffsetFilter allows you to

  • skip a certain amount of items from the beginning
  • process only specified amount of items (and skip the rest)

You can combine these two parameters to process a slice from the middle of the data, like rows 5-7 of a CSV file with ten rows.

OffsetFilter is configured by its constructor: new OffsetFilter($offset = 0, $limit = null). Note: $offset is a 0-based index.

<?php

use Port\Filter\OffsetFilter;

// Default implementation is to start from the beginning without maximum count
$filter = new OffsetFilter(0, null);
$filter = new OffsetFilter(); // You can omit both parameters

// Start from the third item, process to the end
$filter = new OffsetFilter(2, null);
$filter = new OffsetFilter(2); // You can omit the second parameter

// Start from the first item, process max three items
$filter = new OffsetFilter(0, 3);

// Start from the third item, process max five items (items 3 - 7)
$filter = new OffsetFilter(2, 5);

ValidatorFilter

It’s a common use case to validate the data before you save it to the database. This is exactly what the ValidatorFilter does. To use it, include Symfony’s Validator component in your project:

$ compose require symfony/validator

The ValidatorFilter works as follows:

<?php

use Port\Filter\ValidatorFilter;
use Symfony\Component\Validator\Constraints as Assert;

// $validator is a Symfony Validator
$filter = new ValidatorFilter($validator);
$filter->add('email', new Assert\Email());
$filter->add('sku', new Assert\NotBlank());

Here we add the validation assertions manually. Alternatively, you can choose to read assertions from annotations on the objects that you want to validate (or any other metadata source). To do so, use a validator that is pre-configured with metadata resources.

So, for instance in a Symfony project:

<?php

use Port\Filter\ValidatorFilter;

// Pre-configured for looking at object annotations etc.
$validator = $container->get('validator'); 
$filter = new ValidatorFilter($validator);

// No need to manually add assertions now

The default behaviour for the validator is to collect all violations and skip each invalid row. If you want to stop on the first failing row you can call ValidatorFilter::throwExceptions(), which throws a ValidationException containing the line number and the violation list.