Import data via CLI

iTop provides a generic web service to import data in itop based on a source csv file. This service can be used locally on the server via the command line interface or remotely via an HTTP post request.

Using this service, you can easily import data in iTop via a batch script that extract data from a third party tool and generate a csv formatted file. For instance a script data extract a list of contact from your Active Directory.

When it comes to import a real huge amount of data, the CLI mode will be a must because you will not reach the limitations of the web server (timeout and memory limits).

All the parameters of the command line are listed in the section “Arguments” here after. The following line is an example of the mandatory argument. All arguments starts with --

 php /var/www/itop/webservices/import.php --auth_user=login --auth_pwd=password --class=Organization --csvfile=file --reconciliationkeys="list of keys"

The reconciliation keys are very important to identify uniquely the element to be imported in iTop. The reconciliation keys must be present in the data file. As opposed to the interactive CSV import, you are not limited to the list of reconciliation keys defined by default for the class.

The command line will return a summary report (see below) summarizing the argument used for the import and the result of the import:

• Number of elements created
• Number of elements updated
• Number of elements in error

In case you have specified the option details value for the output parameter you will get one line for each element imported with the status of the import (see example below)

 php /var/www/itop/webservices/import.php --auth_user=login --auth_pwd=password --class=Organization --csvfile=file --reconciliationkeys="list of keys" --output=details

#Output format: details
#Class: Organization
#Separator: ;
#Qualifier: \"
#Charset Encoding:UTF-8
#Data Size: 26
#Data Lines: 1
#Simulate: 0
#Columns: name, code
#Reconciliation Keys: name
#Change tracking comment: 
#Issues: 0
#Warnings: 0
#Created: 1
#Updated: 0
#Unchanged: 0
Line;Status;Object Class;Object Id;name (Name);code (Code)
0;created;Organization;3;\"WorldCompany\";\"WCY\"

ERROR: Missing argument 'class'

Such error is displayed when a mandatory argument is missing

#Unknown class: 'toto'

This error occurs if the class specified does not exists in iTop. Be careful the class name is case sensitive.

#Unknown reconciliationkeys: 'attribute'

This error occurs if at least one column in the reconciliation keys list is not known.
The reason can be:

• a code instead of a label was provided or vice-versa
• the reconcialiation key is a External Field
• the reconcialiation key is not present in the csv file

#Unknown column: 'Last Nme'

This error occurs if a column is unknown in the csv file

Administrators will always be allowed to execute this page.

Non administrators will be allowed if they are allowed to modify objects of the given class.

Some classes contain an image or a file document as one of their field. For example, the Person class, contains a Picture field which holds the picture of this person. When editing a Person interactively, the user can upload an image from her computer to provide the picture of the person. However when performing a CSV import, it is not possible to “upload” such a file or to put its content inside the CSV file itself, since the CSV format does not support this.

In such a case the CSV file must provide - in the appropriate CSV column - an URL to upload the file from. This URL must be accessible from the iTop server and return the expected image (or file document). If the upload fails or if the format of the uploaded document does not match the expected format for the field (for example if the URL does not return an image for the picture field), the import/update will fail for this field (and the whole line of the CSV will be rejected).

Exemple of referencing an image already stored in iTop on another object. Syntaxe used:
https://<your-itop>/pages/ajax.document.php?operation=download_document&class=<ObjectClass>&id=<ObjectId>&field=<FieldCode>

This use case describes how to import persons and their login in iTop. This is one way to load those information from an Active Directory. In that case you just have to write a script that create the two csv files person.csv and login.csv from your Active Directory.

We would like to create two contacts:

• Claude Monet who belongs to organization Demo
• Gustave Flaubert who belongs to organization IT Department

In this example the two organizations already exist in iTop

The class to import is UserLocal, and the data in csv file (login.csv) are:

person.csv

Last Name,Status,Organization->Name,Email,Phone,Notification,Function,First Name
"Monet","Active","Demo","monet@demo.com","","yes","","Claude"
"Flaubert","Active","IT Department","flaubert@it.com","","yes","","Gustave"

then run the following command line:

php webservices/import.php --auth_user=admin --auth_pwd=admin --csvfile="person.csv"
 --class="Person" --reconciliationkeys="Email"

We would like to create two user account:

• support is configuration manager and change supervisor, related to the contact flaubert@it.com
• portal is a Porta user related to the contact monet@demo.com

In this example the login are linked to a contact via the email and we import the Profiles that are linked to the contact. Look at the particular syntax for importing the profiles. For more information about importing link sets refer to Import a linkset

The class to import is UserLocal:

login.csv

Person->Email,Login,Language,Password,Profile
"flaubert@it.com","support","EN US",1234,profileid->name:Configuration Manager|profileid->name:Change Supervisor
"monet@demo.com","portal","EN US",1234,profileid->name:Portal user

then run the following command line:

php webservices/import.php --auth_user=login--auth_pwd=password --csvfile="login.csv"
  --class="UserLocal" --reconcialiationkeys="Login"

Example for importing allowed organizations,

• Let's imagine that I want to allow 3 organisations to the same user.
• Each of the Organizations in the source file can be identified by their id, name or code.
• The fields which are expected when loading a AttributeLinkedSetIndirect like Allowed Organizations, are those of the n:n relation class, (here URP_UserOrg), which means that you could only provide allowed_org_id.
• But in general, with n:n relationships, what you know about the remote object, are some of its fields values but maybe not its iTop id. In this case, you can specify a code from the remote object and there is a special syntax for this:

In this example, I am combining different means to identify organizations in a many to many relationship, for the same user, that works, also you probably won't use it every day.

login-with-organizations.csv

Person->Email,Login,Language,Password,Profiles,Allowed Organizations
"monet@demo.com","portal","EN US",1234,"profileid->name:Portal user","allowed_org_id->code:IT|allowed_org_id->name:Demo|allowed_org_id:1"

location.csv

"name","org_id->name","country","city"
"Bordeaux","Demo","France","Bordeaux"
"Grenoble","Demo","France","Eybens"
"Paris","Demo","France",""

then run the following command line:

php webservices/import.php --auth_user=login --auth_pwd=password --csvfile="location.csv"
   --charset="UTF-8" --no_localize=1 --class="Location" --reconcialiationkeys="name,org_id->name"

• External Fields (such as such as virtualhost_id->org_name) cannot be used as reconciliation keys.
• Workaround: org_name can be replaced by org_id->name
• Limitation: the arrow operator (->) cannot be used mutiple times (i.e. virtualhost_id->org_id->name is not supported).

• 0.9: first implementation of this feature
• 2.0: added the flag no_localize