Data collector for OCS Inventory
- name:
- Data collector for OCS Inventory
- description:
- Data Collector for OCS Inventory NG
- version:
- 2.0.0
- release:
- 2023-03-28
- itop-version-min:
- 2.7
- download:
- https://store.itophub.io/en_US/products/ocsng-data-collector
- code:
- ocsng-data-collector
- state:
- stable
- alternate-name:
- Data collector for OCS Inventory NG
- diffusion:
- iTop Hub,Combodo site
- php-version-max:
- PHP 8.3
This stand-alone application collects information from a single OCS Inventory NG server in order to populate the iTop CMDB with the collected Servers, PCs and Virtual machines. The collector uses the iTop data synchronization engine to synchronize the information between OCS Inventory and iTop.
An additional extension Data model for OCS Inventory is mandatory in iTop. It adds some fields needed for synchronization and displays the contents of the OCS inventory pages directly from iTop, for each synchronized object (Server, PC, Mobile phone or Virtual Machine):
Features
-
Automated inventory of Brand, Model, OS Family, OS Version, OS Licence, Physical Server, PC, Virtual Machine, Mobile Phone, Printer, Software from OCS Inventory server
-
Automated inventory of Network Interfaces
-
The collector can reside on any system with web access to iTop and mysql access to the OCS Inventory server
-
Automatic creation and update of the Synchronization Data Sources in iTop.
-
It is necessary to install the small extension Data model for OCS Inventory which adds some fields and an “OCS Inventory” link on synchronized PCs, Servers, Virtual Machines, Mobile Phones and software. This link opens a new tab with the OCS page of the device.
Revision History
Date | Version | Description |
---|---|---|
2023-01-20 | 2.0.0 | - Add new fields and classes to collector |
2022-01-05 | 1.2.0 | - New version with softwares collector |
2021-01-05 | 1.0.7 | - Fix compatibility with SSO set as default
connection mode - Configurable timestamp added in the logs - New option for usage: –help - The path to the configuration file can be specified via the option --config_file on the command line.- The location where to store the collected data is now a parameter in the configuration file: data_path .- Better checking of Data Source definitions to catch missing reconciliation keys |
2019-11-13 | 1.0.6 | - Reject invalid characters for
database_table_name - Added the specific class MySQLCollector which forces the DB connection to use UTF-8 characters - Change all OCS SQLCollectors into MySQLCollectors |
2019-10-29 | 1.0.5 | Handles TeemIp : - Automatically detects if TeemIp (as an iTop module or as a standalone application) is present. - Optional synchronization of IPv4 addresses - Optional synchronisation of IP interfaces |
2018-09-04 | 1.0.4 | First public version. No longer requires an alteration of the data model. |
2018-06-26 | 1.0.3 | New debug trace |
2015-05-20 | 1.0.1 | A few bug fixes: support of Windows file names (i.e. c:\Program Files\…). Conversion of SPEED from '100 Mb/s' to '100'. Forcing a NON case-sensitive comparison for 'Laptop' et al. |
2015-02-16 | 1.0.0 | First version |
Limitations
-
The collector is not normalizing Model, OS Family and OS Version
-
The collector synchronizes only PCs and Servers which have a valid serial number in OCS
-
If IP collection is enabled, only IPv4 addresses are collected, not IPv6
-
There is no single sign-on between iTop and OCS, so you will need to log in to OCS when you display the OCS tab
Requirements
-
PHP Version 7.2.0
-
An access to the OCS Inventory NG mysql database
-
An access to the iTop web services (REST + synchro_import.php and synchro_exec.php). Note: starting with iTop 2.5.0, the user must have the profile
REST Services User
in iTop. -
+ Data collector Base requirements.
-
If you have ocs collector v1, you need go to database in order to rename some tables (XX replace id in corresponding datasource in iTop):
-
synchro_data_pc_XX to synchro_data_pc_ocsng
-
synchro_data_server_XX to synchro_data_server_ocsng
-
synchro_data_virtualmachine_XX to synchro_data_virtualmachine_ocsng
-
Later, if you have some synchronization problem due to table name, you need to rename the table which starts with synchro_data_ and ends with synchro data source id with name seen in synchro data source object.
By example, in this case :
you have to rename the table synchro_data_physicalinterface_171 to synchro_data_pcphysicalinterface_ocsng
Installation
-
Expand the content of the zip archive “ocsng-data-collector” in a folder on the machine that will run the collector application. This machine must have an SQL access to the OCS NG database and a web access to the iTop server.
-
Create the file
conf/params.local.xml
according to your installation, providing the appropriate credentials to connect to OCS NG and iTop.
By default this file should contain the values used to connect to the iTop server and the OCS NG server:
<parameters> <itop_url>http://localhost/</itop_url> <itop_login>admin</itop_login> <itop_password>admin</itop_password> <contact_to_notify>john.doe@demo.com</contact_to_notify> <synchro_user>admin</synchro_user> <sql_host>localhost</sql_host> <sql_database>ocsweb;charset=UTF8</sql_database> <sql_login>root</sql_login> <sql_password>root</sql_password> </parameters>
Parameter | Meaning | Sample value |
---|---|---|
itop_url | iTop Application URL | https://localhost/ |
itop_login | Login (user account) for connection to iTop. Must have administrator rights to run the data sync. | admin |
itop_password | Password for the iTop account. | |
contact_to_notify | The email address of an existing contact in iTop, who will be informed of the results of the synchronisation. | john.doe@demo.com |
synchro_user | iTop user used for synchronization web service | admin |
sql_host | The address to connect to the OCS server database | default:localhost |
sql_database | The database to connect to. You can use ;charset to force the character set | default ocsweb |
sql_login | Login to connect to the OCS database | |
sql_password | Password to connect to the OCS database |
Configuration
By default the data collection configuration is defined in the
file collectors/params.distrib.xml
. Do not modify this
file! If you need to adapt the configuration, create a file named
params.local.xml
in the conf
directory
and copy/paste the needed definitions into it (the structure of
both XMl files is the same). This configuration defines the SQL
queries that must be executed on the OCS NG server to retrieve data
and synchronise it with iTop.
<!-- default values --> <default_org_id>Demo</default_org_id> <default_status>production</default_status> <default_active_status>active</default_active_status> <!--Filter type for synchronization ( categories or default filters ) --> <use_asset_categories>yes</use_asset_categories> <use_software_categories>yes</use_software_categories> <!-- Class to collect --> <CategoryCollection>yes</CategoryCollection> <PCCollection>yes</PCCollection> <ServerCollection>yes</ServerCollection> <VMCollection>yes</VMCollection> <MobilePhoneCollection>yes</MobilePhoneCollection> <PrinterCollection>yes</PrinterCollection> <SoftwareCollection>yes</SoftwareCollection>
This parameters allow to activate or desactivate a group of collectors. If you want to disable one collector in a group, you can do so in collectors_launch_sequence (cf Data collector Base mechanism).
Principles
-
For each class of object to import into iTop (PC, Server, VM, Brand, Model, OSVersion OSFamily), a Synchro Data Source is created by the collector application.
-
Each collector is associated with an SQL query (via the parameter
<name_of_the_collector>_query
). -
This query must return column where aliases (i.e. “name” of the columns) correspond to the expected fields for the corresponding Synchro Data Source.
-
These queries can be adapted to suit your needs (for example to better differentiate PCs and Servers or to change/set some default values for some columns)
-
Since the synchronization of PCs, Servers and Virtual Machines all rely on their own SQL query, it is important to make sure that the results of these 3 queries do no overlap. Otherwise the same “system” will be imported several times in itop (for example as both a PC and a Server).
-
You can use OCS categories in order to synchronize assets and softwares. In this case, you must create categories in OCS and create corresponding objects AssetCategories and SoftwareCategories in iTop. You can use synchronisation in order to help you to create this objects.
Global Parameter | Meaning |
---|---|
default_org_id | Define the default organization for the synchronized Cis (name of default organization) |
default_status | Define the default status for the synchronized Cis (implementation, obsolete, production, stock) |
default_active_status | Define the default status for the synchronized Software instances (active, inactive) |
use_asset_categories | Define if asset categories defined in iTop are used to get assets in OCS |
use_software_categories | Define if software categories defined in iTop are used to get softwares in OCS |
CategoryCollection | Define whether Categories collection is active (yes) or not (no). It is useful if use_asset_categories or use_software_categories is set to yes : Be consistent with these 2 parameters otherwise, you will collect and create Datasources for categories even if you don't use them. |
PCCollection | Define whether PC collection is active (yes) or not (no) |
ServerCollection | Define whether Server collection is active (yes) or not (no) |
VMCollection | Define whether Virtual machine collection is active (yes) or not (no) |
MobilePhoneCollection | Define whether Mobile phone collection is active (yes) or not (no) |
PrinterCollection | Define whether Printer collection is active (yes)
or not (no) The “Printer” extension must be installed in OCS |
SoftwareCollection | Define whether Software collection is active (yes) or not (no) |
During the process, some of the following queries are executed depending on your parameters. If use_asset_categories = yes only queries with “with_categories” are executed. If use_asset_categories = no only corresponding queries without “with_categories” are executed.
These queries can be redefined in the file conf/params.local.xml in order to take into account your specific needs (For instance change status of created Server, PC, Virtual Machine, as well as default organisation).
params.distrib.xml
contains the default values for the
parameters. Both files (params.distrib.xml
and
params.local.xml
) use exactly the same format. But
params.distrib.xml
is considered as the reference and
should remain unmodified. Should you need to change the value of a
parameter, copy and modify its definition in
params.local.xml
. The values in
params.local.xml
have precedence over the ones in
params.distrib.xml
Configure Asset collection
Query Parameter | Launch if | Meaning |
---|---|---|
OCSAssetCategoryCollector_query | CategoryCollection= yes | SQL query to retrieve from OCS NG server the
assets categories Usefull only if use_asset_categories is set to yes |
OCSBrandCollector_query | SQL query to retrieve from OCS NG server the list of known Brands | |
OCSOSFamilyCollector_query | SQL query to retrieve from OCS NG server the list of known OS Families | |
OCSOSVersionCollector_query | SQL query to retrieve from OCS NG server the list of known OS Versions | |
OCSOSLicenceCollector_query | SQL query to retrieve from OCS NG server the list of known Windows Licences | |
OCSServerModelCollector_query | ServerCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of known Physical server models. Servers are hardware with different type than Mobile, Notebook and Laptop and with different manufacturer than WMWare |
OCSServerModelCollector_with_categories_query | ServerCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of known Physical server models. Servers are found using OCS categories. iTop settings are required. |
OCSServerCollector_query | ServerCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of physical servers Servers are hardware with different type than Mobile, Notebook and Laptop and with different manufacturer than WMWare |
OCSServerCollector_with_categories_query | ServerCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of physical servers Servers are found using OCS categories. iTop settings are required. |
OCSServerPhysicalInterfaceCollector_query | ServerCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of physical network interfaces for servers Servers are hardware with different type than Mobile, Notebook and Laptop and with different manufacturer than WMWare |
OCSServerPhysicalInterfaceCollector_with_categories_query | ServerCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of physical network interfaces for servers Servers are found using OCS categories. iTop settings are required. |
OCSPCModelCollector_query | PCCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of known PC models PCs are hardware with type Notebook or Laptop and with different manufacturer than WMWare |
OCSPCModelCollector_with_categories_query | PCCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of known PC models PCs are found using OCS categories. iTop settings are required. |
OCSPCCollector_query | PCCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of PCs PCs are hardware with type Notebook or Laptop and with different manufacturer than WMWare |
OCSPCCollector_with_categories_query | PCCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of PCs PCs are found using OCS categories. iTop settings are required. |
OCSPCPhysicalInterfaceCollector_query | PCCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of physical interfaces for PCs PCs are hardware with type Notebook or Laptop and with different manufacturer than WMWare |
OCSPCPhysicalInterfaceCollector_with_categories_query | PCCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of physical interfaces for PCs PCs are found using OCS categories. iTop settings are required. |
OCSMobilePhoneCollector_query | MobilePhoneCollection = yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of Mobile phones Mobile phones are hardware with type Mobile |
OCSMobilePhoneCollector_with_categories_query | MobilePhoneCollection = yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of physical interfaces for Mobile phones Mobile phones are found using OCS categories. iTop settings are required |
OCSVirtualMachineCollector_query | VMCollection= yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of virtual machines Virtual machines are hardware with VMWare as manufacturer |
OCSVirtualMachineCollector_with_categories_query | VMCollection= yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of virtual machines VMs are found using OCS categories. iTop settings are required |
OCSLogicalInterfaceCollector_query | VMCollection= yes use_asset_categories = no |
SQL query to retrieve from OCS NG server the list
of logical network interfaces for virtual machines Virtual machines are hardware with VMWare as manufacturer |
OCSLogicalInterfaceCollector_with_categories_query | VMCollection= yes use_asset_categories = yes |
SQL query to retrieve from OCS NG server the list
of logical network interfaces for virtual machines VMs are found using OCS categories. iTop settings are required |
OCSPrinterCollector_query | PrinterCollection= yes | SQL query to retrieve from OCS NG server the list of printers |
Collect Assets without using OCS categorization
-
Collector will discover all Cis resulting of SQL queries not ending by _with_categories_query defined in config file.
-
Indicate in configuration file
use_asset_categories = no
<CI>Collection= yes (replace CI by required collected
class)
Collect Assets with OCS categorization
If you use synchronisation with assets categories, you have to configure the same assets categories both in OCS and in iTop. You can have many categories for one asset type.
Configure Assets categories in OCS
First configure assets categories in OCS thanks to search menu below Inventory choice
Verify that assets correspond to your request and create your category thanks to this search.
You can see your assets categories in menu:
In order to have the same results as with default params, you can create this categories:
Configure Assets categories in iTop
Asset categories are in typologie menu.
You have to create categories exactly with the same names as in OCS and parameter in which class there are collected. If you use categories collector, you only have to indicate the target class.
Finaly you have the same list in the both environment
Configure Software collection
Query Parameter | Launch if | Meaning |
---|---|---|
OCSSoftwareCategoryCollector_query | CategoryCollection= yes | SQL query to retrieve from OCS NG server the
softwarescategories Usefull only if use_software_categories is set to yes |
OCSSoftware_getListFromItop | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from iTop the list of existing software |
OCSSoftwareCollector_query | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from OCS NG server the
softwares with their differents versions Only software already in iTop are requested. |
OCSSoftwareCollector_with_categories_query | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from OCS NG server the
softwares with their differents versions. Software are found using OCS categories. iTop settings are required |
OCSPCSoftware_getListFromItop | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from iTop the list of existing software on PC |
OCSPCSoftwareCollector_query | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from OCS NG server the PC
softwares with their version Only software already in iTop are requested |
OCSPCSoftwareCollector_with_categories_getListFromItop | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from iTop the list of software categories with type PC Software |
OCSPCSoftwareCollector_with_categories_query | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from OCS NG server the PC
softwares with their version. Software are found using OCS categories. iTop settings are required |
OCSWebServerCollector_getListFromItop | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from iTop the list of existing software with type WebServer |
OCSWebServerCollector_query | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from OCS NG server the
WebServer with their version Only software already in iTop are requested |
OCSWebServerCollector_with_categories_getListFromItop | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from iTop the list of software categories with type Web Server |
OCSWebServerCollector_with_categories_query | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from OCS NG server the
WebServer with their version WebServerare found using OCS categories. iTop settings are required |
OCSOtherSoftwareCollector_getListFromItop | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from iTop the list of existing software with type Other Software |
OCSOtherSoftwareCollector_query | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from OCS NG server the Other
softwares with their version Only software already in iTop are requested |
OCSOtherSoftwareCollector_with_categories_getListFromItop | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from iTop the list of software categories with type Other Software |
OCSOtherSoftwareCollector_with_categories_query | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from OCS NG server the Other
Softwares with their version WebServerare found using OCS categories. iTop settings are required |
OCSMiddlewareCollector_getListFromItop | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from iTop the list of existing software with type Middleware |
OCSMiddlewareCollector_query | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from OCS NG server the
Middlewares with their version Only software already in iTop are requested |
OCSMiddlewareCollector_with_categories_getListFromItop | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from iTop the list of software categories with type Middleware |
OCSMiddlewareCollector_with_categories_query | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from OCS NG server the
Middleware with their version Software are found using OCS categories. iTop settings are required |
OCSDBServerCollector_getListFromItop | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from iTop the list of existing software with type DBServer |
OCSDBServerCollector_query | SoftwareCollection= yes use_software_categories= no |
SQL query to retrieve from OCS NG server the
DBServer with their version Only software already in iTop are requested |
OCSDBServerCollector_with_categories_getListFromItop | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from iTop the list of software categories with type DBServer |
OCSDBServerCollector_with_categories_query | SoftwareCollection= yes use_software_categories= yes |
SQL query to retrieve from OCS NG server DBServer
with their version Software are found using OCS categories. iTop settings are required |
Collect Software without using OCS categorization
-
Create softwares in iTop. Collector will try to discover all defined softwares in iTop.
-
Indicate in configuration file
SoftwareCollection= yes
use_software_categories= no
Collector will collect from OCS software's names defined in iTop
Collect Software using OCS categorization
If you use synchronisation with software categories, you have to configure the same software categories both in OCS and in iTop. You can have many categories for one software type. You will need to
-
define in OCS, categories on all sofwtares you want to collect
-
execute collector to have OCS catgeories synchronised in iTop
-
map target class to each categorie synchronised
-
execute again collector to receive categorised OCS CIs.
Configure Software categories in OCS
First configure software categories in OCS thanks to Software Category menu below Manage
Click on New Category and give a name to your category
Then click on Add software. Select your category and add a software name or a regexp
You can add many regexp on a software category. You can seen the list in List of regexp
The list of all software in a category can be seen in List of categories
Configure Software categories in iTop
Software categories are in typologie menu.
You have to create categories exactly with the same names as in OCS and parameter in which class there are collected. If you use categories collector, you only have to parameters the corresponding class.
IPs and IP Interfaces collection
The collector automatically detects if TeemIp is present on the remote iTop application (as a module or even as a stand-alone application) and if TeemIp Zone Management extension is installed. Should that be the case, then the following parameters may trigger IP addresses and IP interfaces collection.
<?xml version="1.0" encoding="UTF-8"?> <parameters> ... <!-- TeemIp options --> <IPCollection>yes</IPCollection> <default_ip_status>allocated</default_ip_status> <manage_ipv6>no</manage_ipv6> <default_view_name></default_view_name> ... </parameters>
Global Parameter | Meaning | Sample value |
---|---|---|
IPCollection | Triggers IP addresses collection | yes |
default_ip_status | Satus of newly created IP addresses | allocated |
manage_ipv6 | Triggers IPv6 collection - not working yet | no |
default_view_name | View of the newly created IP addresses | <empty string> |
Query Parameter | Meaning |
---|---|
OCSIPv4AddressCollector_query | SQL query to retrieve from OCS NG server the list of IPv4 address |
OCSIPv6AddressCollector_query | SQL query to retrieve from OCS NG server the list of IPv6 address |
OCSlnkIPInterfaceToIPAddressCollector_query | SQL query to retrieve from OCS NG server the list of link between IP interface and IP address |
Usage
To launch the data collection and synchronization with iTop, run the following command (from the root directory where the data collector application is installed):
php exec.php
The following (optional) command line options are available:
Option | Meaning | default value |
---|---|---|
--console_log_level=<level> | Level of output to the console. From -1 (none) to 9 (debug). | 6 (info) |
--collect_only | Run only the data collection, but do not synchronize the data with iTop | false |
--synchro_only | Synchronizes the data previously collected (stored
in the data directory) with iTop. Do not run the
collection. |
false |
--configure_only | Check (and update if necessary) the synchronization data sources in iTop and exit. Do NOT run the collection or the synchronization | |
--max_chunk_size=<size> | Maximum number of items to process in one pass, for preserving the memory of the system. If there are more items to process, the application will iterate. | 1000 |
The execution of the command line will:
-
Connect to iTop to create the Synchronization Data Sources (or check their definition if they already exist, updating them if needed)
-
Connect to OCS NG to collect the information about the Servers, PC, Virtual Machines
-
Upload the collected data into iTop (this is the configuration parameter
<name_of_the_collector>_query
-
Synchronize the collected data with the existing iTop CIs.
Scheduling
Once you've run the data collector interactively, the next step is to schedule its execution so that the collection and import occurs automatically at regular intervals.
The data collector does not provide any specific scheduling
mechanism, but the simple command line php exec.php
can be scheduled with either cron (on
Linux systems) or using the Task Scheduler on Windows.
full_load_interval
to make it consistent with the
frequency of the scheduling.Steps of synchronization
Questions & Answers
Question: how can I add OCS classes to the collection
plan
Answer: Other classes than the ones listed by default in the
collector can be added to it. For that purpose, we highly recommend
to use the extensibility mechanism that is available since version
2 of the collector (version 1.3.0 of iTop Data Collector Base).
-
Make sure the class (MyNewClass) has been modelized in iTop (Through the Data model for OCS inventory extension for instance).
-
Within the <your_collector>/collectors/extensions/ directory:
-
Create a params.distrib.xml file
-
Add a class entry in it, like what is done in the default section,
-
Add a extensions_collectors_launch_sequence similarly to what is done in the Class Collection Sequence section,
-
-
Create the json/MyNewClassOCSCollector.json file that describes the synchro data source of the new class,
-
Create the src/MyNewClassOCSCollector.class.inc.php file that contains the collector specificities for the class (REST API URL, dependency parameters…),
-
-
And that's it