PROFESSIONAL · DATA INTEGRATION
Duty-free Incentive Platform
A recoverable Oracle-to-SQL Server ETL and incentive-calculation workflow for roughly 6–7 high-volume retail stores.
- Role
- ETL designer · backend co-design
- Status
- Production delivery
- Scope
- ~6–7 retail stores
- Stack
- Oracle · SQL Server · SSIS
My Role
I designed the ETL pipeline and co-designed the backend calculation model with my manager.
Data Reality
Long-running transfers and source data could produce incomplete records, encoding problems, invalid characters, failed rows, and orphan data. The workflow explicitly classified successful, failed, and orphan records instead of assuming every batch was clean.
Recovery
Operators could identify which data required another attempt and selectively replay it, while retaining the option to rerun a full set when appropriate. Recovery was part of the workflow rather than an ad-hoc manual repair step.
Business Rules
Incentive calculations varied by store, product category, team, and specific products, with multiple conditions needing to hold at the same time. The calculation model had to make those rules repeatable over imported operational data.
Outcome
HR previously cross-referenced and validated many reports manually. The system took over the data processing and calculation work so people could focus primarily on validation.
What this demonstrates
ETL ownership, data-quality handling, recoverable batch processing, complex business rules, and automation of a high-friction human workflow.