Import data via CLI

Purpose

iTop provides a generic web service to import data in itop based on a source csv file. This service can be used locally on the server via the command line interface or remotely via an HTTP post request.

Using this service, you can easily import data in iTop via a batch script that extract data from a third party tool and generate a csv formatted file. For instance a script data extract a list of contact from your Active Directory.

When it comes to import a real huge amount of data, the CLI mode will be a must because you will not reach the limitations of the web server (timeout and memory limits).

As opposed to the interactive CVS import, the CLI import lets you specify any attribute to reconcile the external keys. This is a workaround when some reconciliation keys are missing. Example: the class Software has only the name as reconciliation key (default setting in the datamode). You need to import DBServer objects with their respective software information (an external key pointing to the software object). The interactive CSV import will issue errors whenever the same software has two versions (same name, ambiguous!). In the CLI import you can specify both Software→Name and Software→Version and resolve the ambiguity issue.

Usage

Command line interface

All the parameters of the commande line are listed in the section “Arguments” here after. The following line is an example of the mandatory argument. All arguments starts with –

php /var/www/itop/webservices/import.php –auth_login=login –auth_pwd=password –class=Organization –csvfile=file –reconciliationkeys=“list of keys”

The reconciliation keys are very important to identify uniquely the element to be imported in iTop. The reconciliation keys must be present in the data file. As opposed to the interactive CSV import, you are not limited to the list of reconciliation keys defined by default for the class.

When the reconciliation can be performed on an external key, you can specify any of the target class attributes. Example: you are importing Person objects and need to reconcile on the location. You can specify Location→postal_code as one of the reconciliation keys (weird but…).

Moreover, the external key will be uniquely identified by all its attributes specified in the reconciliation scheme. In our example, if you consider that Name/Country uniquely identifies the locations, the use both Location→Name and Location→Country in your reconciliation scheme.

Arguments

Argument	Description	Defaut value
param_file	Parameters file - see Parameters file	-
auth_login	User login - CLI mode only	-
auth_pwd	User password - CLI mode only	-
class	Class of the loaded objects	Mandatory!
csvdata	CSV contents (HTTP mode only, data may be posted through this page argument)	Mandatory in HTTP mode
csvfile	Path and name of the source file	Mandatory in CLI mode
charset	Character set encoding of the CSV data: UTF-8, ISO-8859-1, WINDOWS-1251, WINDOWS-1252, ISO-8859-15	UTF-8
date_format	Input date format (used both for dates and datetimes) - Examples: %Y-%m-%d, %d/%m/%Y (Europe) - no transformation is applied if the argument is omitted	<blank>
separator	Column separator in CSV data	, (coma)
qualifier	Qualifier in CSV data	“ (double quote)
output	retcode to return the count of lines in error, “summary” to return a concise report, “details” to get a detailed report (each line listed)	summary
reconciliationkeys	name of the columns used to identify existing objects and update them, or create a new one	<blank>
simulate	If set to 1, then the load will not be executed, but the expected report will be produced	0
comment	Comment to be added into the change log	<blank>
no_localize	If set to 0, then header and values are supposed to be localized in the language of the logged in user. Set to 1 to use internal attribute codes and values (enums)	0

Output

The commande line will return a summary report (see below) summarizing the argument used for the import and the result of the import:

Number of elements created
Number of elements updated
Number of elements in error

In case you have specified the option –result=details you will get one line for each element imported with the status of the import (see example below)

Result

#Output format: details
#Class: Organization
#Separator: ;
#Qualifier: \"
#Charset Encoding:UTF-8
#Data Size: 26
#Data Lines: 1
#Simulate: 0
#Columns: name, code
#Reconciliation Keys: name
#Change tracking comment: 
#Issues: 0
#Warnings: 0
#Created: 1
#Updated: 0
#Unchanged: 0
Line;Status;Object Class;Object Id;name (Name);code (Code)
0;created;Organization;3;\"WorldCompany\";\"WCY\"

Possible errors

Missing argument

ERROR: Missing argument 'class'

Such error is displayed when a mandatory argument is missing

Wrong argument

#Unknown class: 'toto'

This error occurs if the class specified does not exists in iTop. Be careful the class name is case sensitive.

#Unknown reconciliationkeys: 'attribute'

This error occurs if at least one column in the reconciliation keys list is not known

#Unknown column: 'Last Nme'

This error occurs if a column is unknown in the csv file

Allowed users

Administrators will always be allowed to execute this page.

Non administrators will be allowed if they are allowed to modify objects of the given class.

Use cases (importing person and corresponding login into iTop)

This use case describes how to import persons and their login in iTop. This is one way to load those information from an Active Directory. In that case you just have to write a script that create the two csv files person.csv and login.csv from your Active Directory.

Import persons

We would like to create two contacts:

Claude Monet who belongs to organization Demo
Gustave Flaubert who belongs to organization IT Department

In this example the two organizations should already exists in iTop

The class to import is UserLocal, and the data in csv file (login.csv) are:

Last Name,Status,Organization->Name,Email,Phone,Notification,Function,First Name
"Monet","Active","Demo","monet@demo.com","","yes","","Claude"
"Flaubert","Active","IT Department","flaubert@it.com","","yes","","Gustave"

then run the following command line:

php webservices/import.php --auth_user=admin --auth_pwd=admin --class=Person --csvfile=person.csv --reconciliationkeys="Email"

Import user accounts

We would like to create two user account:

support is configuration manager and change supervisor, related to the contact flaubert@it.com
portal is a Porta user related to the contact monet@demo.com

In this example the login are linked to a contact via the email and we import the Profiles that are linked to the contact. Look at the particulare syntaxe for importing the profiles. More information about importing link set in Import a link set

The class to import is UserLocal:

login.csv

Contact (person)->Email,Login,Language,Password,Profiles
"flaubert@it.com","support","EN US",1234,profileid->name:Configuration Manager|profileid->name:Change Supervisor
"monet@demo.com","portal","EN US",1234,profileid->name:Portal user