Data Migration & Integration

Import Sets are the primary mechanism for bringing external data into ServiceNow. They use a staging table and Transform Maps to map source data to target tables.

Import Set Process

1. Load Data — Upload a file (CSV, Excel, XML, JSON) or pull from a JDBC data source into a staging table.
2. Staging Table — An intermediate table (import set table) where raw data is loaded.
3. Transform Map — Defines the mapping between staging table columns and target table fields.
4. Transform — Execute the mapping. Data moves from the staging table to the target table.
5. Coalesce — Determines whether to insert a new record or update an existing one based on matching fields.

Key Import Set Concepts

• Coalesce Fields — Fields used to match incoming data to existing records. If a match is found, the record is updated; if not, a new record is inserted.
• Field Maps — Individual column-to-field mappings within a Transform Map.
• Transform Scripts — Run during the transform to manipulate data (onBefore, onAfter, onStart, onComplete).
• Data Source — Defines where import data comes from (file, JDBC, LDAP, etc.).
• Scheduled Import — Import sets can run on a schedule for recurring data loads.

Update Sets capture configuration changes (customizations) and move them between instances (dev → test → prod). They are the primary mechanism for migrating configurations, NOT data.

Update Set Lifecycle

1. Create — Create a new Update Set in the source instance and set it as your current update set.
2. Develop — Make configuration changes (business rules, UI policies, ACLs, etc.). Changes are automatically tracked.
3. Complete — Mark the update set as Complete when all changes are done.
4. Export — Export the update set as an XML file.
5. Import — Import the XML file into the target instance.
6. Preview — Preview the update set to identify conflicts and issues.
7. Commit — Apply the changes to the target instance.

What Update Sets Capture

• Customizations — Business rules, client scripts, UI policies, ACLs, etc.
• Form layouts and list layouts.
• System properties and scheduled jobs.
• Application menus and modules.

What Update Sets Do NOT Capture

• Data records (incidents, users, etc.) — use Import Sets for data.
• User preferences and personal settings.
• Homepage and dashboard configurations.
• Attachments (unless specifically configured).

ServiceNow provides both REST and SOAP APIs for integrating with external systems. The Table API is the most commonly used REST API.

REST API (Table API)

• URL pattern: /api/now/table/{table_name}
• Supports GET (read), POST (create), PUT (update), PATCH (partial update), DELETE.
• Returns JSON or XML.
• Supports query parameters for filtering, pagination, and field selection.
• Authentication: Basic Auth, OAuth 2.0, or Session-based.

Scripted REST APIs

• Custom REST endpoints built by admins/developers.
• Define your own URL paths, HTTP methods, and response formats.
• More control over input/output than the standard Table API.
• Used when the Table API doesn't meet specific integration requirements.

REST vs. SOAP

LDAP (Lightweight Directory Access Protocol) integration synchronizes user and group data from directory services like Microsoft Active Directory into ServiceNow.

LDAP Integration Components

• LDAP Server — Defines the connection to the directory server (host, port, credentials).
• LDAP OU (Organizational Unit) — Specifies which OUs to query for users/groups.
• Data Source — Links the LDAP server to ServiceNow import set tables.
• Transform Map — Maps LDAP attributes to ServiceNow user/group fields.
• Scheduled Import — Runs periodically to keep user data synchronized.
• MID Server — Required to reach LDAP servers behind a firewall (on-premises).

The MID (Management, Instrumentation, and Discovery) Server is a Java application installed on-premises that acts as a proxy between the ServiceNow instance and resources in your network.

MID Server Use Cases

• Discovery — Scan your network to find and map CIs.
• LDAP Integration — Reach on-premises directory servers.
• JDBC Imports — Connect to on-premises databases.
• Orchestration — Execute commands on remote servers.
• Any integration with resources behind a firewall.

How MID Server Works

• Installed on a server within your network.
• Communicates OUTBOUND to ServiceNow over HTTPS (port 443).
• No inbound firewall rules required — MID Server polls ServiceNow for work.
• Multiple MID Servers can be deployed for redundancy and load balancing.

The Configuration Management Database (CMDB) stores information about IT assets and services as Configuration Items (CIs) and their relationships.

CMDB Core Concepts

• Configuration Item (CI) — Any component that needs to be managed (server, application, router, service).
• CI Class — The type/category of CI (e.g., cmdb_ci_server, cmdb_ci_app_server).
• CI Relationships — Connections between CIs (Runs On, Depends On, Used By, etc.).
• Dependency Map — Visual representation of CI relationships and dependencies.
• Discovery — Automated tool to find and populate CIs in the CMDB.
• Service Mapping — Maps business services to their underlying infrastructure CIs.