In the 2.7.8 release, the function get_config_parameter() is no more available in Twig files (portal or console)

The version 2.7.0 of iTop is roughly backward compatible with previous 2.x versions, with a few significative changes.

This document highlights issues which can occur while migrating your iTop to this version.

• For a focus on new features check What's New
• For an exhaustive list of changes check the Change Log

• OQL queries returning multiple classes of objects, may return less data after the upgrade, to users having “Allowed organizations” set on their user. This is due to a fix in query generation.

• External key display in forms are now clickable, if you rather keep the previous behavior replace the xxx_id attribute code in your form with xxx_id_friendlyname.
• We have update Font Awesome from v4 to v5.12.0, check that your icons in portal are still displayed as expected. Some icons names have changed between the 2 versions, old name will still work except if you have used aliases from the version 4, which will need to be changed.
• For security reasons, LinkedSet displayed in forms, such as for example contacts associated to a User Request, are now by default limited to the scope of the portal user. A typical example there, was that portal user could see on his User Requests, IT department associated contacts, even if in theory his Contact scope was limiting him to contacts of his own organization. If you want to keep the previous behavior, you need to modify the forms in your portal displaying those linksets as explained in the 2.7 new features in the Portal XML reference

• Admin tools has been modified and some of its entries moved to new Configuration and System menus.
• Labels of Persons fields in User objects have been changed. This will change what you get in CSV and excel export of Users:

• Ensure that you have php-gd installed on your iTop server. It is now enforced at setup, as it is mandatory for handling correctly images within caselogs and description.

iTop 2.7.0 brings a new log rotation functionality, which is enabled by default.

If you use logrotate or another similar system on the iTop log files, then you should either :

• disable your rotation system to use the new iTop rotation system
• or keep using your rotation system and disable iTop rotation mechanism

To disable iTop log rotation, add in the iTop configuration file the line below :

   'log_filename_builder_impl' => '',

Also, if you choose to keep using your own rotation system, for the files to be visible in the Log Management extension (included in iTop Professional and Essential) you should :

• do not compress rotated files (zip, tar.gz, etc)
• make the rotated files accessible by the webserver's user

• Starting from iTop 2.7.0, due to an OQL optimisation, some columns in the views were renamed.
• In that case, deploy this extension to restore the views as they were in 2.6 and before.

With the following versions of MariaDB, iTop backup won't work:

The symptoms

When you perform this query:

MariaDB [sbacmdb]> select * from view_DBServer;

You get this error:

Error: Failed to execute mysqldump: mysqldump: Couldn't execute 'show create table `view_DBServer`': You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '= 0 and `SoftwareLicence_softwarelicence_id_Licence`.`end_date` < date_format...' at line 1 (1064)

Root cause: This is a MariaDB issue, providing the versions in which the bug was fixed by them.

Strategy: avoid those buggy MariaDB versions. Downgrade to a not impacted version or upgrade to fixed versions when they are available:

Workaround

• Drop Views before doing Backup, Views are not required by iTop itself.
• Cautious, a Setup will recreate the Views, so drop them again after that.

Views will be totally removed in 3.0, an optional free extension will be proposed for those still using them

• This release will increase the size of your database
• History tables and event notification tables will increase by 1KB per row
• For instance if you have 1 million history entries, then you need 1GB of additional disk space just for the History

 SELECT COUNT(*) FROM priv_changeop;  /* History */
 SELECT COUNT(*) FROM priv_event;  /* Notification */

The upgrade to the 2.7 product version, will generate database changes, which can take time if you have a lot of data.

Finalclass on intermediate classes

To increase query performance, we are adding on each intermediate table/class within a class hierarchy, a new column to store the final class (Only on intermediate classes as on a root class this column exist already and on a final class, this column is useless as this is the table name).

As a result during Setup, for each of those intermediate classes, iTop runs an ALTER TABLE

Example

ALTER TABLE `priv_changeop_setatt` 
   ADD `optype` VARCHAR(255) CHARACTER 
      SET utf8mb4 COLLATE utf8mb4_unicode_ci 
      DEFAULT 'CMDBChangeOpSetAttribute', 
   ADD INDEX `optype` (`optype` (95))

and just after an UPDATE table

UPDATE `priv_changeop_setatt`,`priv_changeop` 
   SET  `priv_changeop_setatt`.`optype` = `priv_changeop`.`optype` 
   WHERE `priv_changeop_setatt`.`id` = `priv_changeop`.`id` 

Attachments

To store creation date and user having uploaded an attachment, iTop Setup automatically updates the DB schema, like this:

ALTER TABLE `attachment` ADD `creation_date` DATETIME, 
                         ADD `user_id` INT(11) DEFAULT 0, 
                         ADD INDEX `user_id` (`user_id`);

Warning, if you're using the table prefix option (db_subname iTop config file parameter) the table name will vary. For example with db_subname='myprefix' then the table name would be : myprefix_attachment.

This operation can take a long time if you have a big number of attachments and/or a huge history on your objects.

• ALTER TABLE execution time is roughly proportional to the table size
• UPDATE TABLE execution time is mainly impacted by the number of rows and a bit by the table size.

Examples of execution time

Here are examples of the time it took with table size and count:

longqueriesmigration27.xlsx

 ALTER TABLE `priv_changeop_setatt` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttribute', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_changeop_setatt_scalar` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttributeScalar', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_changeop_setatt_longtext` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttributeLongText', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_changeop_links` ADD `optype` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'CMDBChangeOpSetAttributeLinks', ADD INDEX `optype` (`optype` (95))  
 ALTER TABLE `priv_event_notification` ADD `realclass` VARCHAR(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'EventNotification', ADD INDEX `realclass` (`realclass` (95))  
 ALTER TABLE `attachment` ADD `creation_date` DATETIME, ADD `user_id` INT(11) DEFAULT 0, ADD INDEX `user_id` (`user_id`)  
 UPDATE `priv_changeop_setatt`,`priv_changeop` SET  `priv_changeop_setatt`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_changeop_setatt_scalar`,`priv_changeop` SET  `priv_changeop_setatt_scalar`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt_scalar`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_changeop_setatt_longtext`,`priv_changeop` SET  `priv_changeop_setatt_longtext`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_setatt_longtext`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_changeop_links`,`priv_changeop` SET  `priv_changeop_links`.`optype` = `priv_changeop`.`optype` WHERE `priv_changeop_links`.`id` = `priv_changeop`.`id`  
 UPDATE `priv_event_notification`,`priv_event` SET  `priv_event_notification`.`realclass` = `priv_event`.`realclass` WHERE `priv_event_notification`.`id` = `priv_event`.`id`

To identify tables which can take time, you can use this query, modified with your true itop db name

SELECT TABLE_NAME AS "Tables", 
    round(((data_length + index_length) / 1024 / 1024), 2) AS SIZE 
  FROM information_schema.tables 
  WHERE table_schema =  'itop db name'
  ORDER BY SIZE

Then run the upgrade process on a test environment on a copy of your production database. The setup.log file will provide you the exact queries which run.

You can run those SQL queries on your production database before upgrading ! Apart from the performance degradation it can induce during the execution, there is no risk to break iTop as it will just ignore the added columns, until you upgrade it to version 2.7

You can connect to mysql to check the queries which are on-going, you will see the time already spent on the current query, if you run the command again, the time will increase or show the next query.

php>mysql -u root -p xxxx
mysql>show processlist;

As long as queries are ongoing, the setup is still going on. Then run

tail -f setup.log
 
2020-03-02 13:49:52 | Ok      | <---- Exiting read only mode | SetupLog

to ensure that the setup is ended.

If your browser timeout, then the Setup is not completed. Once you have ensured that the database upgrade is finished, you must run the Setup again, also this time it will be faster as the database upgrade will not be performed again.

If you upgrade your production instance of iTop to 2.7 version, then later for whatever reason, you downgrade your iTop to any previous version. Everything will work smoothly, but the day you upgrade again to 2.7.0 or 2.7.1, we have a known issue in that case. To fix it you must run the Database integrity menu and fix manually in SQL directly on your MySQL or MariaDB database, the missing finalclass values for objects created in your iTop during the rollback interval.

Because of the Silex to Symphony migration, the following extensions are not compatible with iTop 2.7 Portal. A new version will be published mid of April 2020 and if you have deployed them on your iTop, you must upgrade those extensions before upgrating your iTop.

• Customized request forms
• Approval process Automation
• Approval process light
• Communications to the Customers
• Customer Survey

If you have custom extensions bringing new bricks in your portal, you will need to get/write a new version of those extensions.

Because, in order to ensure better security, support and sustainability; we migrated the portal's framework from Silex 2.x to Symfony 3.4. Even though we managed to keep a backward compatibility for most of the code, if you made a custom extension for the portal, you are most likely to rework part of it.

Read this page to check if your extension is concerned.

As announced, the Legacy portal is no more available in iTop 2.7

The XML default datamodel of iTop 2.7, contains a few changes which can collide with changes you may have done already on your iTop datamodel though extensions (or using the ITSM Designer if your are a Combodo customer).

• The default theme is now included under /itop_design/branding/themes/theme[@id=“light-grey”], so if you altered the branding node, make sure to put the alteration flag (eg. _delta=“define”) on the subnodes.

itop_design iTop 2.7.0

  <branding>
    <themes>
      <theme id="light-grey">
        ...

No more working:

itop-design in MyExtension

  <branding _delta="define">
    <main_logo>
      <fileref ref="logo-combodo_f737e922334a6d51b0d98e9f590d2295"/>
    </main_logo>
    <login_logo>
      ...
  </branding>

Compatible with 2.7:

itop-design in MyExtension

  <branding>
    <main_logo _delta="define">
      <fileref ref="logo-combodo_f737e922334a6d51b0d98e9f590d2295"/>
    </main_logo>
    <login_logo _delta="define">
      ...
  </branding>

The legacy customers portal was deprecated for a few releases and has now been completly removed. If you had redefined some of its constants or the routing to it, you need to correct your XML.

Constants: Remove all constant nodes starting with “PORTAL_” as they don't exist anymore.

itop_design iTop 2.6 and earlier

<constants>
    <constant id="PORTAL_POWER_USER_PROFILE">...</constant>
    <constant id="PORTAL_SERVICECATEGORY_QUERY">...</constant>
    ...
</constants>

Routing: Remove “legacy_portal” from the portal nodes.

itop_design iTop 2.6 and earlier

<portals>
    <portal id="legacy_portal">
        ...
    </portal>
</portals>

Navigation rules have been added under /itop_design/module_designs/module_design[@id=“itop-portal”]/forms/form[@id=“<FORM_ID>”]/properties for the “ticket-create”, “ticket-edit”, “ticket-reopen” and “ticket-apply-stimulus” forms, so if you altered the properties node, make sure to put the alteration flag (eg. _delta=“define”) on the subnodes.

itop_design / module_designs / module_design / forms iTop 2.7.0

  <form id="ticket-create">
    <properties>
      <navigation_rules>
        <submit>
          <default>go-to-open-requests</default>
        </submit>
        ...

No more working:

itop-design / module_designs / module_design / forms in MyExtension

  <form id="ticket-create">
    <properties _delta="define">
      <always_show_submit>true</always_show_submit>
    </properties>
      ...
  </form>

Compatible with 2.7:

itop-design / module_designs / module_design / forms in MyExtension

  <form id="ticket-create">
    <properties>
      <always_show_submit _delta="define">true</always_show_submit>
    </properties>
      ...
  </form>

The Admin tools sub-menus have been moved under the newly created group-menus Configuration and System. If you have redefined those menus, it may not work as expected.

• A new GetTicketRefFormat() method is now part of the Ticket|UserRequest|Incident|Problem|Change classes, so if you added a custom method with the same name, you'll have to adjust your code.

While these are not breaking changes, you should avoid using deprecated code. This is a summary of those introduced in the 2.7 version:

• You should verify if you make use of the deprecated MetaModel::GetNextKey, such use is generally done into /itop_design/classes/class/methods/method[@id='DBInsertNoReload']. In the same logic this method CMDBSource::GetNextInsertId is deprecated, use method on ItopCounter class instead.
• Portal: Action rules of type goto should be replaced by navigation rules. This means that instead of having an action rule in your brick, you should have a navigation rule in the corresponding form. Search for the following XPath:
  • /itop_design/module_designs/module_design/action_rules/action_rule/submit
  • /itop_design/module_designs/module_design/action_rules/action_rule/cancel

Previously PHP code getting User object instances were querying database regardless of the current user rights. This was changed in 2.7.0.

However, some code still needs to access the User class even if the current user doesn't have rights on this class. To do so, you will need to change your code to use the AllowAllData option.

Two examples of “no more working code” and the “appropriate fix”:

// before 2.7.0 this was working for non admin users
$oUser = MetaModel::GetObject('User', $iCurrentUserId);
 
// After 2.7.0 to search for objects of the User class 
// You need to explicitly specify to ignore current user rights on the User class
// Forcing the 4st parameter to true, bypass "current user rights"
$oUser = MetaModel::GetObject('User', $iCurrentUserId, true, true); 

Reference: MetaModel::GetObject

// before 2.7.0 this was working for non admin users
$oFilter = DBSearch::FromOQL('SELECT User WHERE id = :id');
$oSet = new DBObjectSet($oFilter, array(), array('id' => $iCurrentUserId));
$oCurrentUser = $oSet->Fetch();
 
// After 2.7.0 you should add this to get Users objects
// Despite current user doesn't have rights on the User class :
$oFilter = DBSearch::FromOQL('SELECT User WHERE id = :id');
$oFilter->AllowAllData(); // this method call will bypass user access rights
$oSet = new DBObjectSet($oFilter, array(), array('id' => $iCurrentUserId));
$oCurrentUser = $oSet->Fetch();

Reference: DBSearch::AllowAllData

If the stimulus is not applicable to the object in its current state, the method ApplyStimulus() now returns false, instead of true before

Many images have been replaced by Font Awesome icons in the admin. console. Those that were not used anymore have been removed from the iTop package (/images folder). This means that if you were using them in your extensions, you should change your code accordingly.

images/actions_bkg.png
images/actions_left.png
images/asc.gif
images/help.png
images/home.png
images/menu.png
images/mini-arrow-orange-open.gif
images/mini-arrow-orange.gif
images/mini_add.gif
images/mini_search.gif
images/mini_tree.gif
images/newsroom-message.svg
images/newsroom_menu.png
images/on-off-menu.png
images/onOffBtn.png
images/pencil-menu.png
images/printableversion.png
images/reload.png
images/searchBtn.png
images/settings.gif
images/zoom.gif

Since iTop 2.7.10 / 3.0.4 / 3.1.2 / 3.2.0, the X-Content-Type-Options HTTP header is sent with the nosniff value. This could trigger CORB protection for certain resources, preventing them from loading (see https://chromium.googlesource.com/chromium/src/+/master/services/network/cross_origin_read_blocking_explainer.md#determining-whether-a-response-is-corb_protected).

To mitigate the issue, sending this HTTP header is disabled in ajax_page and XMLPage.
Also, the security_header_xcontent_type_options config parameter can be set to false to prevent the header to be sent.