Data Synchronization: How to synchronize a Persons from a CSV file

In this How to, a person's details extracted from a CSV file (which might be an LDAP extract or any other source), will be synchronized with iTop.

What you will need to:

• Know how to extract information about a person from your LDAP server and save this information to a .CSV file
• Have an iTop system (with the demo data installed), with access to an admin account
• Know how to connect to your iTop web server using the command line

In this tutorial, you will learn how to:

• Create and configure a Synchro Data Source for synchronizing one class of objects into iTop
• Feed and synchronize the external data into this Synchro Data Source
• Understand the synchronization status report

Here, you will create the Synchro Data Source.

In the Admin tools menu, click on Synchronization Data Sources

Click on Create a new Synchro Data Source.

In the following screen, enter the Name (Persons from LDAP), Target class (Person) and User (admin) in the appropriate fields.

Click on the Create button to create the Data Source.

Once the Data Source has been created, click on Modify in the top right corner and edit the Attributes tab as shown below:

Then create the text file “persons.csv” in the directory “data” under the root of iTop, with the following content:

persons.csv

primary_key;name;first_name;org_id;email;status;function;phone;notify
1;Monet;Claude;Demo;claude.monet@demo.com;Active;Manager;+33123456789;Yes
2;Flaubert;Gustave;IT Department;gustave.flaubert@demo.com;Active;;+33987654321;No

From command line in the iTop root directory, launch the following command:

php synchro/synchro_import.php --auth_user=admin --auth_pwd=admin --csvfile=./data/persons.csv --data_source_id=1 --output=summary

Expected output:

#Load--------------
#------------------
#Data Source: 1
#Synchronize: 1
#Class: Person
#Separator: ;
#Qualifier: "
#Charset Encoding:UTF-8
#Date format: <none>
#Data Size: 228
#Data Lines: 2
#Columns: primary_key, name, first_name, org_id, email, status, function, phone, notify
#Output format: summary
#Simulate: 0
#Change tracking comment: 
#Issues (before synchro): 0
#Created (before synchro): 2
#Updated (before synchro): 0
#Synchronization---
#------------------
#Replicas: 2
#Replicas touched since last synchro: 2
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 0
#Objects updated: 0 (0 warnings)
#Objects update errors: 0
#Objects reconciled (updated): 2 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

Check the status of the Synchro Data Source (press the “Refresh” icon if needed)

You can see that 2 Persons were updated.

Navigate to the details of the contact “Claude Monet”, you can see that this object is now synchronized:

Check the “History tab” for recent changes:

If you try to modify the Person, you can see that most of its attributes are locked:

Modify the file “persons.csv” to change the function of Claude Monet from “Manager” to “Famous painter” Modify the text file “persons.csv” in the directory data under the root of iTop, with the following content:

persons.csv

primary_key;name;first_name;org_id;email;status;function;phone;notify
1;Monet;Claude;Demo;claude.monet@demo.com;Active;Famous painter;+33123456789;Yes
2;Flaubert;Gustave;IT Department;gustave.flaubert@demo.com;Active;;+33987654321;No

Launch the import and synchronization again using the same command:

php synchro/synchro_import.php --auth_user=admin --auth_pwd=admin --csvfile=./data/persons.csv --data_source_id=1 --output=summary

The output is the following:

#Load--------------
#------------------
#Data Source: 1
#Synchronize: 1
#Class: Person
#Separator: ;
#Qualifier: "
#Charset Encoding:UTF-8
#Date format: <none>
#Data Size: 235
#Data Lines: 2
#Columns: primary_key, name, first_name, org_id, email, status, function, phone, notify
#Output format: summary
#Simulate: 0
#Change tracking comment: 
#Issues (before synchro): 0
#Created (before synchro): 0
#Updated (before synchro): 2
#Synchronization---
#------------------
#Replicas: 2
#Replicas touched since last synchro: 2
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 0
#Objects updated: 1 (0 warnings)
#Objects update errors: 0
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

the output of the synchro states that only 1 object was updated. This can be viewed also on the status tab of the Synchro Data Source:

Then navigate to the details of the Person “Claude Monet”:

Next, modify the text file “persons.csv” in the directory data under the root of iTop, by adding one line at the end:

persons.csv

primary_key;name;first_name;org_id;email;status;function;phone;notify
1;Monet;Claude;Demo;claude.monet@demo.com;Active;Famous painter;+33123456789;Yes
2;Flaubert;Gustave;IT Department;gustave.flaubert@demo.com;Active;;+33987654321;No
3;Mondrian;Piet;Demo;p.m@demo.com;Active;;+334433221100;No

Launch the import and synchronization again using the same command:

php synchro/synchro_import.php --auth_user=admin --auth_pwd=admin --csvfile=./data/persons.csv --data_source_id=1 --output=summary

The output is the following:

#Load--------------
#------------------
#Data Source: 1
#Synchronize: 1
#Class: Person
#Separator: ;
#Qualifier: "
#Charset Encoding:UTF-8
#Date format: <none>
#Data Size: 294
#Data Lines: 3
#Columns: primary_key, name, first_name, org_id, email, status, function, phone, notify
#Output format: summary
#Simulate: 0
#Change tracking comment: 
#Issues (before synchro): 0
#Created (before synchro): 1
#Updated (before synchro): 2
#Synchronization---
#------------------
#Replicas: 3
#Replicas touched since last synchro: 3
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 1 (0 warnings)
#Objects creation errors: 0
#Objects updated: 0 (0 warnings)
#Objects update errors: 0
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

The output of the synchro states that only 1 object was updated. This can be also viewed on the Status tab of the Synchro Data Source:

Use the global search at the top-left of the page to search for “Contact:Piet”, then click on the name of the object to display its details.

Then modify the text file “persons.csv” in the directory “data” under the root of iTop, by adding one more line at the end:

persons.csv

primary_key;name;first_name;org_id;email;status;function;phone;notify
1;Monet;Claude;Demo;claude.monet@demo.com;Active;Famous painter;+33123456789;Yes
2;Flaubert;Gustave;IT Department;gustave.flaubert@demo.com;Active;;+33987654321;No
3;Mondrian;Piet;Demo;p.m@demo.com;Active;;+334433221100;No
4;Klimt;Gurstav;Test;gustav.klimt@demo.com;Active;;+335566778899;No

Launch the import and synchronization again using the same command:

php synchro/synchro_import.php --auth_user=admin --auth_pwd=admin --csvfile=./data/persons.csv --data_source_id=1 --output=summary

The output is the following:

#Load--------------
#------------------
#Data Source: 5
#Synchronize: 1
#Class: Person
#Separator: ;
#Qualifier: "
#Charset Encoding:UTF-8
#Date format: <none>
#Data Size: 361
#Data Lines: 4
#Columns: primary_key, name, first_name, org_id, email, status, function, phone, notify
#Output format: summary
#Simulate: 0
#Change tracking comment: 
#Issues (before synchro): 0
#Created (before synchro): 1
#Updated (before synchro): 3
#Synchronization---
#------------------
#Replicas: 4
#Replicas touched since last synchro: 4
#Objects deleted: 0
#Objects deletion errors: 0
#Objects obsoleted: 0
#Objects obsolescence errors: 0
#Objects created: 0 (0 warnings)
#Objects creation errors: 1
#Objects updated: 0 (0 warnings)
#Objects update errors: 0
#Objects reconciled (updated): 0 (0 warnings)
#Objects reconciled (unchanged): 0 (0 warnings)
#Objects reconciliation errors: 0
#Replica disappeared, no action taken: 0

the output of the synchro states that there is one error

Here add screenshot of the details of the replica in error

Then modify the text file “persons.csv” in the directory “data” under the root of iTop, by fixing the last line at the end (replace Test by Demo):

persons.csv

primary_key;name;first_name;org_id;email;status;function;phone;notify
1;Monet;Claude;Demo;claude.monet@demo.com;Active;Famous painter;+33123456789;Yes
2;Flaubert;Gustave;IT Department;gustave.flaubert@demo.com;Active;;+33987654321;No
3;Mondrian;Piet;Demo;p.m@demo.com;Active;;+334433221100;No
4;Klimt;Gurstav;Demo;gustav.klimt@demo.com;Active;;+335566778899;No

Launch the import and synchronization again using the same command:

php synchro/synchro_import.php --auth_user=admin --auth_pwd=admin --csvfile=./data/persons.csv --data_source_id=1 --output=summary

There should be no error and the contact Gustav Klimt is created in the Demo organization.