Products > CODS (Consumer Operational Data Source)

Business (Functional) Solution Definition: Overview of Context and Scope The purpose of the Consumer ODS project is to build an infrastructure (data structures and processes to maintain) to provide a source of reference for Consumer information. The Consumer sources of information include Consumer contact, Consumer opt, Consumer account, & Consumer preference data.

Process

At a high level, the following processes are required for the Consumer ODS implementation

This document deals with the following processes:

P1.0 - Conversion
P2.0 - Daily
P3.0 - Monthly
P4.0 - Quarterly
P5.0 - Ad Hoc

Holding and processing daily input (P6.0 and P7.0) are one-time efforts specific to implementation and will be addressed in the implementation plan (not this document).

Conversion Processes

The Consumer ODS will be populated initially with data from the existing Consumer Database (GDB), and with data from the OLTP and Old Src. The population of Consumers that will be included in the initial extract efforts will be defined by an Ad-hoc data extract, using criteria defined by the clients.

Business flow of conversion process:

From a technical standpoint conversion is separated into the following processes:

  1. Initial Contact Data Conversion from Consumer DB
  2. Initial contact conversion from OLTP & Old Src

Daily Processes

On a daily basis, Consumer ODS will receive Consumer contact information like name, address, phone, e-mail as well as non-contact information like account, legal ids, preferences, enrollments from different source systems. The following diagram explains the processing requirements of the data received from source systems.

Business flow of daily process:


Technical view of daily process:


Data stage Extraction jobs

Source system Database for Consumer Database, OLTP, Campaign Enrolment

Source System FTP Jobs

Consumer ODS will use FTP jobs to get data from source systems Consumer Database, OLTP, Campaign Enrolment.

Source Data Formatters

Formatter jobs are basically Data stage look up & transfer jobs that read the data from the files provided by source systems, do basic data validation on the source data & create different files like contact file, preference file, account file etc in a common file format, which subsequently will be used by next Data stage job i.e. ODS manager. Source data formatter jobs are specific to source systems like Consumer formatter, OLTP formatter etc.

ODS Manager

Data stage lookup & transfer jobs that read files created by source data formatter jobs. ODS manager will process following file types:

  1. Consumer contact
  2. Consumer e-mail
  3. Legal ids
  4. Preference
  5. Enrollment
  6. Account
  7. Opt
  8. Account Services

On each file ODS Manager will encrypt sensitive data using the encryption routine, will assign ODS_ID & for existing Consumers, will do other lookup & transfer service to Consumer contact data as well as other non-contact data.

Contact data formatter

Contact data formatter is a Data stage lookup & transform job that process Consumer contact files. This job will read Contact files, apply business logic to it & will do other transformation to the input data & will create Consumer contact files, which will be subsequently loaded into ODS.

Non-contact data formatter

Data stage lookup & transfer jobs to process ODS non-contact files. These jobs will apply any business rules for the non-contact files, will do any lookup & transformation require for the data & will create flat files, which will be subsequently loaded into ODS.