synchro_import.php

DRAFT! 🚧

Purpose

This service is used for data synchronization.

Invoke this service to import data and trigger the synchronization in one single step.

In case you have chosen to import the data directly into the synchronization table, then you just need to trigger the synchronization. Use the service synchro_exec.php.

Arguments

Argument	Description	Defaut value
param_file	Parameters file - see Parameters file	-
auth_login	User login - CLI mode only	-
auth_pwd	User password - CLI mode only	-
data_source_id	Synchro data source id	Mandatory!
csvdata	In HTTP mode, this argument must contain the data	n/a
csvfile	In CLI mode, this arguments must point to the file containing the input data	n/a
synchronize	If set to 1, then the synchronization will be executed right after the data load	1
charset	Character set encoding of the CSV data: UTF-8, ISO-8859-1, WINDOWS-1251, WINDOWS-1252, ISO-8859-15	UTF-8
date_format	Input date format (used both for dates and datetimes) - Examples: %Y-%m-%d, %d/%m/%Y (Europe) - no transformation is applied if the argument is omitted
separator	column separator in CSV data (1 char, or tab)	;
qualifier	test qualifier in CSV data	“
output	`retcode` to return the count of lines in error, `summary` to return a concise report, `details` to get a detailed report (each line listed)	summary
max_chunk_size	CLI mode only, Limit on the count of records that can be loaded at once while performing the synchronization	0
~~simulate~~	~~If set to 1, then the load will not be executed, but the expected report will be produced~~	0
comment	Comment to be added into the change log
no_stop_on_import_error	Do not stop the import in case of SQL import error. By default the import will stop at the first error (and rollback all changes). If this flag is set to 1 the import will continue anyway	0

Some issues have been reported with the simulate mode. This feature remains experimental.

Excel often generates CSV using this charset=ISO-8859-1

Example

php -q /var/www/html/itop/synchro/synchro_import.php --auth_user=john --auth_pwd=trust,no1 --data_source_id=2 --csvfile=mydata.csv

Data Source

Create a Data Source for loading with:

target class = Person
reconciliation ? = email
Update = Yes for name,first_name,org_id,email
org_id (reconcialiation key) = Full name

Source file

Loading those data on an iTop with the sample data

primary_key;name;org_id;first_name;email
1;nom1;Demo;person1;person1.nom1@gmail.com
2;nom2;Demo;person2;person2.nom2@gmail.com
3;nom3;Demo;person3;person3.nom3@gmail.com
4;nom4;Demo;person4;person4.nom4@gmail.com
5;nom5;Demo;person5;person5.nom5@gmail.com
6;nom6;Demo;person6;person6.nom6@gmail.com
7;nom7;Demo;person7;person7.nom7@gmail.com
8;nom8;Demo;person8;person8.nom8@gmail.com
9;nom9;Demo;person9;person9.nom9@gmail.com
10;nom10;Demo;person10;person10.nom10@gmail.com

Normal execution

#------------------------------------------------------------
# Import phase summary
#------------------------------------------------------------
#Data Source: 2
#Synchronize: 1
#Class: Person
#Separator: ;
#Qualifier: "
#Charset Encoding:UTF-8
#Date and time format: 'Y-m-d H:i:s' 
#Date only format: 'Y-m-d' 
#Data Size: 484
#Data Lines: 10
#Columns: primary_key, name, org_id, first_name, email
#Output format: summary
#Simulate: 0
#Change tracking comment: 
#Issues (before synchro): 0
#Created (before synchro): 0
#Updated (before synchro): 10
#------------------------------------------------------------
# Synchronization phase summary
#------------------------------------------------------------
#Replicas: 10
#Replicas touched since last synchro: 10
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 0
#Objects updated: 2 (0 warnings)
#Objects update errors: 2
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

2nd load of the modified source with error

Modified source file with some errors

primary_key;name;org_id;first_name;email
1;nom1;InvalidOrgName;person1;person1.nom1@gmail.com
2;nom2;2;changed-person2;person2.nom2@gmail.com
3;nom3;Demo;changed-person3;person3.nom3@gmail.com
4;nom4;IT Department;person4;person4.nom4@gmail.com
5;nom5;Demo;person5;person5.nom5@gmail.com
6;nom6;Demo;person6;person6.nom6@gmail.com

Only 6 rows are provided instead of 10
The 2 first rows contain invalid Organization full name generating: #Objects errors in update: 2
The rows 3 & 4 have been modified with allowed/valid values generating: #Objects updated: 2

Output with some Errors

#------------------------------------------------------------
# Synchronization phase summary
#------------------------------------------------------------
#Replicas: 10
#Replicas touched since last synchro: 6
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 0
#Objects updated: 2 (0 warnings)
#Objects update errors: 2
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

As visible in iTop

Allowed users

Administrators will always be allowed to execute this page.

Non administrators will be allowed if they are marked as being owner of the data source.

History

Available since 1.1

Sidebar