Introduction

The DataImporter program (ESI.Octopus.DataImporterApp.exe) can be used to import and/or synchronize data from an external source that is compatible with OLEDB/ODBC. 

DataImporter can be used to perform the initial import of data or to synchronize on a regular basis data contained in another system.

This article explains how DataImporter works and the different types of import available.


 

References

Links to the articles related to the import sources and the XML file. 

• XML Configuration File
• To the Import Source Specification list

Prerequisites

• Create an Octopus system account that will be used to run the task. See Octopus System Account information.

• Presence on the workstation/server running the DataImporter task of an OLEDB provider compatible with the data source to be imported (the ODBC provider is generally pre-installed).

• If the data source is an Excel worksheet or an Access database, presence of an installation of Microsoft Office (2016, 2019, 2021 or 365) on the workstation/server running the DataImporter.

• XML file containing the information on the source of the data to be imported. See article DataImporter - XML Configuration File .
• Data source configured as specified in the section Data Source Specifications for Import

OLEDB Provider

If the data source is an Excel worksheet or an Access database, you will need to deploy the Microsoft Access Database Engine 2016 Redistributable package on the workstation/server running the DataImporter.

You can download this package 

• from Microsoft web site : https://microsoft.com . It is important to download / install the 32 bit version
• from our web site : https://updates.octopus-itsm.com

How it works

DataImporter makes it possible to import different elements in Octopus (Users, CI, Suppliers, etc.) from a OLEDB/ODBC data source (CSV, Excel, Access, SQL Server, Oracle, MySQL, etc.).

• DataImporter can replace the ADSIReader program when the users are not in Active Directory or when the a data source is more complete elsewhere.
• DataImporter can be used to complete the information gathered by WMIUpdater (Order Number, Purchase Price, Date of Purchase, etc.) or to import information from non-Windows devices (Printer, Switch, Projector, etc.) or even from Linux servers or Mac computers. 

The 3 essential elements to operating the DataImporter program are:

• Data Source (Excel file, Access table, SQL view, etc.)
• XML File; this file indicates the data source, where it can be found and the view/table to import.
• Command Line; to run the program..

How to Use DataImporter

DataImporter is a Command Line (DOS) type of program. It is conceived to be easy to automate.

To start ESI.Octopus.DataImporterApp.exe, you need to open a command prompt (DOS) and change the directory to the folder where is installed the Octopus Windows client. The working directory must be the folder where is installed the Octopus Windows client.

Octopus is normally installed in the following folder %homepath%\AppData\Local\Octopus. For instance, for a user having jsmith as logon, the folder will be something like C:\Users\jsmith\AppData\Local\Octopus.

Parameters

There are multiple available parameters, of which 4 are mandatory, to execute DataImporter:

Mandatory Parameters

• /Login
  • ​Octopus Username.
• /Password
  • ​Octopus Password.
• /ConfigFilePath
  • ​Path to the DataImporter configuration file.

• /Team

  • The number of the team to be updated.

  • By default, IT is team number 1.

  • How to find the team number?

Example: ESI.Octopus.DataImporterApp.exe /Login:system /Password:octo /ConfigFilePath:c:\Import\CI.xml /team:1

Optional Parameters

• /LogFilePath
  • ​To move the location of the log file. By default, the file is named DataImporter_AAAAMMJJ_HHMMSS.log and will be located in the Octopus folder.
  • If you move the file to another folder, make sure this folder exists (the system cannot create the folder).
  • Furthermore the path must contain the name of the file.
  • To learn more about the location of Octopus files, see the Octopus Installation Folder article.
    Example : /LogFilePath:C:\LogFiles\ImportJournal.log.
• /Debug
  • ​Our Service Centre may ask you to execute the command line with the /Debug parameter in cases of problems with DataImporter.
  • This parameter generates a binary file that enables us to analyze the errors when we do not have access to the imported data.
  • The generated file can then be used with DataImporter as follows:/UseDebugOutput:FileName.
• /ExtractionTimeout TimeOutInSeconds
  • This parameter allows to specify a longer TIMEOUT than the one configured by default before DataImporter abandons the import. 
  • ​This can hapen in a context where the network is a bit slow or the data quantity too large; in such cases the risks are higher and you may want to increase the response time.
    Example: 5 minutes = 300 seconds
• /OutputImportResultToIT
  • Automatically creates an event in the IT team when there is a problem with an import. 
• /ValidateOnly
  • No import is done, it will only validate the content of the import file with Octopus.
• /WaitKeyPress
  • Allows to force the activation of a key press after the import is executed. Meaning the DOS window will be shown until a key is pressed. 
  • This option is useful when doing import tests to make sure there are no problems with the command line.
  • Note that this parameter should not be used in a recurring task. 
• /PurgeLogs
  • Can automatically erase old log files
  • The parameter is used to determine how many days of log files that you want to keep.
  • For more information, see the Octopus Tools | Maintenance of log files.

Examples

• Import command line with basic parameters.

ESI.Octopus.DataImporterApp.exe /Login:system /Password:octo /ConfigFilePath:c:\Import\CI.xml /team:1

• Import command line with optional parameters.

ESI.Octopus.DataImporterApp.exe /Login:system /Password:octo /ConfigFilePath:c:\Import\CI.xml /Team:1 /Debug /LogFilePath:C:\LogFiles\JournalImport.log /ValidateOnly /ExtractionTimeout:300 /OutputImportResultToIT /WaitKeyPress /PurgeLogs:2

Automate DataImporter

Refer to the Windows Scheduled Task article to find out how to automate imports with DataImporter.

Configuration File (XML)

The configuration file is an XML format text file that contains all the information about the data source to import. Refer to the following article explaining how to program the references of your data source. 

• XML Configuration File

Troubleshooting

This section describes the ways to help you troubleshoot issues.

Operation Log File

During execution, DataImporter creates a log file in addition to the results returned on screen. This log is saved in the DataImporter_AAAAMMJJ_HHMMSS.log file and it is located in the folder in which DataImporter is run, the Octopus folder.

Consult this file after each import, as it lists all the error messages of the data that was not imported.
 

Error Message Analysis

When the import is successful, the following lines will be in the log file:

• The source <XXXXX> Represents the name of the view or of the tab being imported
• n record(s) indicates the number of recordings successfully imported

However, when there are one of more errors, the following lines will be in the log file:

Here is a list of possible error message and how to correct them.

• ServiceProxy : unknown service infrastructure exception error occurred

  • ​DataImporter is unable to connect to the server.

  • Make sure you are able to start the Octopus client to confirm it is not corrupted. If required, download a fresh version of the client.

• An unexpected error has occurred: The 'xxxxxx' provider is not registered on the local machine.

  • The OLEDB provider associated with the data source is not installed on the machine running DataImporter.

  • Make sure you have a 32-bits version of Microsoft Office.

• ​An unexpected error has occurred: Invalid character in the given encoding. Line X, position Y.

  • ​This error is related to a mistake in the XML file. Please refer to the next section to configure the XML file. Make sure there are no accents in the name of the source.

• ​An unexpected error has occurred: '.', hexadecimal value 0x00, is an invalid character. Line 2, position 1.

  • ​The XML file has been saved in Unicode instead of ANSI.

• ​Cannot import this record because of an irresolvable naming conflict.

  • ​The record that is being imported or updated is not unique. The system does not know which one to use.

• ​XX$ : XXXX : RequiredColumn

  • ​Indicates that a mandatory column is missing.

• ​The data modified is in read-only mode

  • ​The Octopus User used in the command line does not have all the required permissions to run this type of import.

• The specified main contact does not exist

  • Make sure that the Main Contact used exists and that the correct Windows Username is used during the import.

• ​Some fields contain invalid values (CI Import)

  • ​TypeName: The value specified in the 'Type' field is invalid.

  • PurchasePrice: The value in the 'PurchasePrice' field must be numerical.

  • Status: The value specified in the 'Status' field is invalid.

• ​There are no attributes called XXX for this CI type (CI Import)

  • ​When the system does not recognize the name of a column, it will suppose that it is the name of a non-existing or misspelled attribute. 

  • If the name really exist, make sure that it is properly spelled and that there are no spaces in the name in the Excel cell.

• ​The attribute XXXXX is required

  • ​A mandatory column is missing in the data source.

• ​There are no attributes called 'Test#1'' for this CI type.

  • ​If the name of an attribute contains a period (.), the system cannot interpret it and will replace it with the pound sign (#) in the error message of the log. ​So there should not be any periods in the names of attributes.

• An unexpected error has occurred: String was not recognized as a valid TimeSpan

  • When importing the Effort field in requests, the information must be in the following format: 03:45 or 00:30. 

• The value specified for field 'XXXX' is invalid.

  • The value that is being imported is not part of the accepted values for this field type. 

• ​An unexpected error has occurred: Could not find file 'C:\Import\XYZ.xml'

  • ​The colon (:) was missing from between the parameter and it's value in the command line.

​If you encounter an unknown error message, please Contact us.