Français

DataImporter : Specifications

Modified: 2010/07/30 17:36 by cdutremble - Categorized as: Technical

Edit

Overview

The program "ESI.Octopus.DataImporterApp.exe" allows you to import CI, user and incident data contained in other systems.

DataImporter can be used to perform an initial import of data into Octopus or to synchronize on a regular basis the data contained in another system as well. For instance, DataImporter can be used to synchronize computers and installed software information from SMS.

DataImporter can be used to import data coming from any ODBC source.

Edit

Quick overview on how DataImporter works

DataImporter is a program that can read one or more sources of ODBC/OLEDB data and then synchronize them with Octopus data. Field names are usually different from one database to another. The configuration of DataImporter is like a mapping work in which you map the structure of the external database with Octopus structure, mapping one field at a time.

The mapping between Octopus and the external system is done through a view. This view is used to transform the source data in the appropriate format to match Octopus fields. To create the view, you have two (2) options:

  1. Use the external database system (including database management tools) to define the view. Here are some systems that offer the possibility to create views: Oracle, Microsoft SQL Server, MySql, DB2, DBase, Fox Pro.
  2. Use MS-Access as a “bridge” between Octopus and the external system. This option is useful when the external system has an ODBC/OLEDB compatible database but do not include a tool to create views. In that case, we use MS-Access to link tables with the external system and we create the view in MS-Access. With this option, the Access database becomes the data source of DataImporter.

Once data sources are defined, you only have to schedule an automated task that will launch DataImporter on a regular basis (once a day, once a week, etc.). Windows scheduled tasks are then used to automate DataImporter execution.

Edit

Steps to define a view that can be used by DataImporter:

To demonstrate how this works, ‘USER’ specification has been used.

  1. Create a MS-Access database named db1.mdb.
  2. Link one or more tables from the external system.
  3. Create a query (view) named “UsersSysFile” in the MS-Access database. The view name can be different.
  4. Analyze the external system table structure to discover how you can match it with Octopus specifications. See herein below the section describing specifications.
  5. Edit “UsersSysFile” query in order to match the specifications (field names, required fields, etc.).
  6. Save MS-Access database including the appropriate changes.

If you want to import different data types (users, requests, etc.), you can define many queries (views) in the same MS-Access file. You can even define many views of the same data type. These views will be then referenced through your DataImporter configuration file.

Edit

Creation of the DataImporter Configuration File

Once one or more data sources are defined, a configuration file for DataImporter must be written. This file will give to DataImporter all the necessary information to reach the data source(s) to be imported as well as the type of data that this source contains.

The DataImporter Configuration File is a XML format text file.

Here is an example of a file used to import Users defined in UsersSysFile view from MS-Access database located in c:\db1.mdb.

ImageThe physical location of your MS-Access database can vary, make sure to apply changes to the configuration file according to its real location.

<?xml version=”1.0” encoding=”utf-8” ?>
<Sources>
     <Source Name=”UserSource” >
          <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
	  <ViewName>UsersSysFile</ViewName>
          <Content>User</Content>
    </Source>
</Sources>

As DataImporter can import from several different sources, we can create one XML file containing a section for each source. For instance, the following document contains three (3) sources. When executed, DataImporter program will import all three (3) sources, one after the other.

<?xml version= »1.0 » encoding= »utf-8 » ?>
<Sources>
     <Source Name=”SourceName” >
          <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
          <ViewName>UsersSysFile</ViewName>
          <Content>User</Content>
     </Source>

     <Source Name=”SourceName2” >
          <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
          <ViewName>Computers</ViewName>
          <Content>CI</Content>
     </Source>

     <Source Name=”SourceName3” >
          <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
          <ViewName>Monitors</ViewName>
          <Content>CI</Content>
     </Source>
</Sources>

You can save as the configuration file under any name. You may decide to use separate files if you want to launch DataImporter at different times, depending on different data type that need to be imported.

Edit

XML file for data sources declaration

When DataImporter is executed, the XML file indicating the location of the data to be imported must be specified.

The following example shows that we want to import computer information (request "Computers") contained in an Access database located in C:\ directory.

<Source Name="Computers">
   <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\\Computers.mdb</ConnectionString>
   <ViewName>Computers</ViewName>
   <Content>CI</Content>
</Source>

  • ConnectionString : Connection String specifies information about a data source and the means of connecting to it. For further information, go to the following site : www.connectionstrings.com.
  • ViewName : Name of the view. If your source is built in Excel, the view name must be your Excel worksheet name followed by a ”$“ and wrapped in brackets (example : [Sheet1$]).
  • Content : Source Declaration, i.e. what needs to be imported in Octopus (CI, Users, etc.). See next section for the list of sources you can import.


Edit

How to use DataImporter Program

DataImporter is a ‘command line’ (DOS) kind of program. Its conception facilitates automation (next section describes how to automate the program execution).

The complete name of the DataImporter Program is:

ESI.Octopus.DataImporterApp.exe

It can be found in the local Octopus directory (C:\Program Files\Octopus).

Edit

Parameters


There are 3 mandatory parameters to be able to launch DataImporter:

  • /Login: Octopus username.
  • /Password: Octopus password.
  • /ConfigFilePath: Complete path of DataImporter Configuration File.

Example :

ESI.Octopus.DataImporterApp.exe /login:system /password:octo /ConfigFilePath:c:\pc.xml

Important : The current directory must be the directory containing ESI.Octopus.DataImporterApp.exe program (C:\Program Files\Octopus).

Edit

Operation Log

During the execution, DataImporter displays an Operation Log.
This log is saved in the DataImporter.log file and is located in the DataImporter Program Directory.

Edit

Importation Sources Specifications

Edit

User Importation

Required fields :

  • Surname - Text(100)
  • Firstname - Text(100)

Optional fields :

  • EmployeeID – Text(50)
    • Must be unique.
  • Language
    • Must be one of the existing reference values in Octopus.
  • Department - Text(50)
    • The system will create the department if it doesn't exist.
  • SubDepartment - Text(50)
    • The system will create the sub-department if it doesn't exist.
    • If a sub-department is specified, then a department must also be specified.
  • Title - Text(100)
  • Site - Text(50)
    • The system will create the site if it doesn't exist.
  • Local - Text(50)
  • OfficePhone - Text(50)
  • PhoneExtension - Text(6)
  • CellPhone - Text(50)
  • Pager - Text(50) « Version 2.15 »
  • Email - Text(50)
  • Note - Text(2000)
  • Active - (1/0 or True/False)
  • WindowsUsername - Text(50)


Identification Method for Users

When users are updated, it is possible to specify the key to be used to synchronize users (instead of using the first and last name fields).
In the file containing the data source declaration, add the IdentificationMethod attribute.

Possible values are :

  • UserByID
  • UserByName : The format is {First name} {Last name} (ex : John Smith).
  • UserByNameAndPhoneExtension
  • UserByWindowsUsername

The source declaration is done by indicating "User" as Content.

Example :

<Source Name="SourceName">
   <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
   <ViewName>ViewName</ViewName>
   <Content>User</Content>
   <IdentificationMethod>UserByID</IdentificationMethod>
</Source>

Edit

CI Importation

Required fields :

  • Name - Text(125)
    • Must be unique
    • We suggest that you concatenate software name with its version to make it as a unique name. For instance, for Octopus version 1.9, we would have: « Octopus (1.9) ».

  • Type - Text(50)
    • This column describes the CI type that we want to import. To make the import working, the returned type must match EXACTLY with one of the CI types of Octopus database. Available Octopus CI types appear in the Octopus main menu under Tools > Reference Data Management.

Optional fields :

  • Manufacturer – Text(500)
    • The system will create a manufacturer if it does not exist.
  • Model - Text(50)
    • The system will create a new model if it does not exist.
  • Version - Text(50)
  • Status
    • Must be one of the CI status in the reference values.
  • Criticality
    • Must be one of the CI criticality in the reference values.
  • SerialNumber - Text(250)
  • InventoryNumber - Text(50)
  • Department - Text(50)
    • The system will create a department if it does not exist.
  • SubDepartment - Text(50)
    • The system will create a sub-department if it does not exist.
    • If a sub-department is specified, a department must be specified also.
  • MainContact
    • See section «Main contact identification method».
  • Site - Text(50)
    • The system will create a site if it does not exist.
  • Local - Text(50)
    • The system will create a room (local) if it does not exist.
    • If a room (local) is specified, a site must also be specified.
  • Category - Text(50)
    • The system will create a category if it does not exist.
    • A CI category is related to its type.
  • Note - Text(5000)
  • Supplier - Text(500)
    • The system will create the supplier if it does not exist.
  • MaintenanceSupplier - Text(500)
  • PurchaseOrderNumber - Text(50)
  • PurchaseDate - Date and Hour
    • The date format must be compatible with the Octopus server settings (AAAA-MM-JJ HH:MM:SS).
  • PurchaseCost - Decimal (ex.: 123456789,12)
  • CostCenter – Text(50)
  • InvoiceNumber – Text(50)
  • WarrantyExpiryDate – Text(50)
  • WarrantyType – Text(100)
  • FundingSource – Text(100)
  • RequiresServiceContract – (1/0 or True/False)
  • Lifetime – Text(50)
  • AmortizationDuration – Integer (ex : 123)
    • The amortization duration expressed as a number of months.

  • Any other custom attribute corresponding to the CI type imported.


CI identification method

In the data source declaration, the CI finding method can be specified. For example, if you want to identify the CIs by inventory number instead of by CI name, the value of the attribute “IdentificationMethod” will be “CIByInventoryNumber” .

Possible values for the attribute “IdentificationMethod” :

  • CIByName : (Default value) Name of the CI
  • CIByInventoryNumber : CI Inventory number
  • CIBySerialNumber : CI Serial number


Main contact identification method (Owner)

In the source declaration, the main contact finding method can be specified. For example, if the field “Main contact” contains employee IDs, the attribute value “UserIdentificationMethod ” will be “UserByID”.

Possible values for “MainContactIdentificationMethod” attribute :

  • UserByID : Main contact Employee number
  • UserByName : Main contact First name and last name (format : Martin Tremblay)
  • UserByWindowsUsername : (Default value) Main contact Network username (Windows Logon)

The source declaration is done by indicating “CI” as the Content.

<Source Name=”SoftwareView” >
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>
     <Content>CI</Content>
     <ManageCIRetirement>false</ManageCIRetirement>
     <IdentificationMethod>CIByName</IdentificationMethod>
     <MainContactIdentificationMethod>UserByWindowsUsername</MainContactIdentificationMethod>
</Source>

Here is an example of a view containing only one record (a monitor).
Note that the columns Name and Type appear in the view, as it is mentioned in the CI specifications.

Monitors
NameManufacturerModelSerialNumberSizeType
NECNECFP2141zz42pz21 po.Monitor


Managing automatic retirement of CIs:

DataImporter has the ability to deactivate or reactivate CIs that are imported, according to whether they are returned or not from the data source. The deactivation of the CI is made by assigning the status “Retired” in Octopus. By default, if the configuration element “ManageCIRetirement” is not specified or its value is “false”, DataImporter will not manage automatic retirement of CIs.

Edit

Import of Relationships Between CI

Minimal columns required:
  • CI1 - Text (125)
  • CI2 - Text (125)
  • Relation - Texte (50)
    • Name of the relationship to be created between 2 CI.

Optional columns:

  • Note – Text (5000)



The declaration of the source is done by indicating “CIRelation” as the Content.

<Source Name=”ComputersMonitorsView”>
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>	
     <Content>CIRelation</Content>
</Source>

Edit

Remove relations that no longer exist between 2 CI

Put the value True to the parameter «ManageRelations».

Exemple :

<?xml version="1.0" encoding="utf-8" ?> 
<Source Name="CIRelation">   
   <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
   <ViewName>NomDeVue</ViewName>	
   <ManageRelations>True</ManageRelations>
   <Content>CIRelation</Content>
</Source>
Edit

CI Users Importation

Minimal columns required:

  • CI - Text (50)
  • User


User identification method

In the declaration of the source, the method to find the user can be specified. For example, if the field “User” contains User numbers, the value of the attribute “UserIdentificationMethod ” will be “UserByID” in the configuration file.

Possible values for the attribute “UserIdentificationMethod” :

  • UserByID : Employee number of the user
  • UserByName : First name and last name of the user (format : John Smith)
  • UserByWindowsUsername : (Default value) Network username of the user (Windows Logon)

The declaration of the source is done by indicating “CIUser” as the Content.

Example:

<Source Name=”UsersOfCIs”>
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>	
     <Content>CIUser</Content>
     <UserIdentificationMethod>UserByWindowsUsername</UserIdentificationMethod>
</Source>

Edit

Service Contract Importation

Minimal columns required:

  • Number - Text (50)
  • Type - Text (100)
    • Type of service contract (Maintenance, Extended warranty, etc.)
  • StartDate - Text (50)
    • The date format must be compatible with Octopus server settings (MM-JJ-YYYY HH :MM :SS)
  • EndDate - Text (50)
    • The date format must be compatible with Octopus server settings (MM-JJ-YYYY HH :MM :SS)
  • Supplier - Text (500)

Optional columns:

  • PurchaseOrderNumber - Text (50)
  • Cost – Text (50)
  • Description - Text (5000)

The declaration of the source is done by indicating “ServiceContract” as the Content.

Example:

<Source Name=”ServiceContracts”>
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>	
     <Content>ServiceContract</Content>
</Source>
Edit

Import of CI supported by a Service Contract

Minimal columns required:

  • Contract - Text (50)
    • Must contain the contract number
  • CI - Text (100)

Optional columns:

  • Cost – Text (50)

The declaration of the source is done by indicating “SupportedCI” as the Content.

Example:

<Source Name=”ServiceContractsCIs”>
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>	
     <Content>SupportedCI</Content>
</Source>

Edit

Lease Contract Importation

Minimal columns required:

  • Number - Text (50)
  • StartDate - Text (50)
    • The date format must be compatible with Octopus server settings (MM-JJ-YYYY HH :MM :SS)
  • EndDate - Text (50)
    • The date format must be compatible with Octopus server settings (MM-JJ-YYYY HH :MM :SS)
  • Supplier - Text (500)

Optional columns:

  • PurchaseOrderNumber - Text (50)
  • Cost – Text (50)
  • Description - Text (5000)

The declaration of the source is done by indicating “LeaseContract” as the Content.

Example:

<Source Name=”LeaseContracts”>
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>	
     <Content>LeaseContract</Content>
</Source>

Edit

Import of CI supported by a Lease Contract

Minimal columns required:

  • Contract - Text (50)
    • Must contain the contract number
  • CI - Text (100)

Optional columns:

  • Cost – Text (50)

The declaration of the source is done by indicating “LeasedCI” as the Content.

Example:

<Source Name=”LeaseContractsCIs”>
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>QueryOrViewNameHere</ViewName>	
     <Content>LeasedCI</Content>
</Source>

Edit

Incident Importation

ImageIt is important to note that this type of record is currently only available in an insertion mode. The repeat importation of a data source will not update existing records but will rather create duplicates.

Minimal columns required

  • Issue - Text(500)
  • Status
    • Must be a valid incident status.
  • User
    • Must be the valid Windows username of an existing user.
  • RequestedBy
    • Must be the valid Windows username of an existing user.

Optional columns:

  • IncidentNumber
    • This number will be prefixed to the subject of the incident.
  • DetailedDescription - Text(5000)
    • Full description.
  • Group
    • Specifies the group to which the incident will be assigned and therefore must be a valid group name to which the specified assignee has access.
  • Assignee
    • Octopus user to whom the incident will be assigned. Must be a valid octopus username.
  • Impact
    • Must be equal to the text value of an impact level. (ex : « 1 – High », « 2 – Medium », « 3 – Low »).
  • Urgency
    • Must be equal to the text value of an urgency level. (ex : « 1 – High », « 2 – Medium », « 3 – Low »).
  • Priority
    • Must be equal to the text value of a priority level (ex : « 1 – Urgent », « 2 – High », « 3 – Normal »).
  • Category
    • Must be equal to the text value of an existing Octopus category.
  • Subcategory
    • Must be equal to the text value of an existing Octopus subcategory.
  • OpenDate
    • Indicates the opening date with the format YYYY-MM-DD HH :MM :SS.
  • ResolutionDate
    • Indicates the resolution date with the format YYYY-MM-DD HH :MM :SS.
    • This date must be greater than the opening date.
  • ClosureDate
    • Indicates the date at which the incident was closed with the format YYYY-MM-DD HH :MM :SS.
    • This date must be greater than the resolution date.
  • OnHoldReason
    • Indicates the reason for which the incident has ben put in the status « On Hold ».
    • Must be equal to the text value of an exiting Ocotopus on hold reason.
  • ResolutionDescription - Text(5000)
    • Required field when the incident has the status « Resolved ».
    • Represents the description of the resolution activity.
  • ResolutionActivity
    • Required field when the incident has the status « Resolved ».
    • Represents the TYPE of the resolution activity.
  • ClosureNote
    • Represents the closure note associated with an incident that has the status « Closed ».
  • CI
    • Represents the CI in cause associated with the incident.
  • Site
    • Site associated with the incident, if incident sites are required.
  • ITService
    • IT service affected by the incident.


The declaration of the source is done by indicating “Incident” as the Content.

Example:

<Source Name="SourceName" >
     <ConnectionString>Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\\db1.mdb</ConnectionString>
     <ViewName>NomDeVue</ViewName>
     <Content>Incident</Content>
</Source>

Administration | This wiki was designed using ScrewTurn.